X-Git-Url: https://git.tokkee.org/?p=sysdb.git;a=blobdiff_plain;f=ReleaseNotes;h=22950a54c3518e482de128c21c7836e0a1505056;hp=f9e91bc5a2df2f517367429ff664b56b47871735;hb=ed2c9fc3e4ca6840a5a31c735f0cfc02fd21d4fc;hpb=c77afd8880079f6e2d3abd5180c7de7c3e96270d diff --git a/ReleaseNotes b/ReleaseNotes index f9e91bc..22950a5 100644 --- a/ReleaseNotes +++ b/ReleaseNotes @@ -6,6 +6,14 @@ and very detailed list of changes can be obtained from the Git logs for each release. + Compatibility Note: + ~~~~~~~~~~~~~~~~~~~ + + All pre 1.0 development snapshots are not considered stable yet. That is, + all interfaces, including the plugin API, the network protocol, or the query + language, may change in an incompatible way at any time. All changes are + documented here. + About SysDB: ~~~~~~~~~~~~ “System DataBase” (SysDB) is a multi-backend system management and inventory @@ -17,9 +25,288 @@ Object names are canonicalized before they are added to the store to ensure a consistent view of your infrastructure. - + + +Version 0.7.0, Release Date: 2015-02-21 +--------------------------------------- + + This release introduces various features related to running SysDB as an + agent. Objects may now also be submitted by arbitrary applications using the + network protocol. Two new plugins are available to collect local facter fact + values and to mirror all store operations to another instance. Various fixes + and improvements have been applied to the client program (sysdb). + + * build system: + - Added support for C++. + * sysdb: + - Fixed terminal resets on exit. + - Don't exit if the server closed the connection; instead reconnect + automatically on the next user input. + - Fixed string parsing: previously, quoted strings were not handled at + all and might have been terminated by an embedded semicolon. + - Also store failed commands in the history to enable history editing in + such cases. + - Improved program output: don't mix log messages with prompts after + sending a command; print log messages to the standard error channel. + - Fixed lookup of the .sysdb_history file for cases where the SysDB user + does not match the current operating system user. + * sysdbd: + - SSL options can be specified inside a block. + - Unload all plugins on exit to ensure a clean shutdown. + * client: + - Simplified RPC handling by providing a helper function. + - Added support for TCP connections using SSL/TLS and supporting, both, + IPv4 and IPv6. Remote peer verification is required for any such + connection. + - New command line flags -K, -C, and -A to specify the SSL private key, + client certificate, and CA certificates file names. + * core: + - Fully support arrays of all types. + - Fixed (implemented) reloading of time-series fetcher plugins. + - Added new callback type for "store-writers", making that part of the + store pluggable (queries are not supported). + * store: + - Added support for querying an object's siblings. + - Send all stored objects to store-writer plugins in addition to storing + them in the in-memory store. + * frontend: + - Fixed parsing of time values. + - Add support for storing hosts, services, metrics, and attributes using + the new 'STORE' command. + - Fixed handling of subsequent requests on the same connection by + avoiding unnecessary pauses of up to one second, thus, greatly speeding + up request handling. + - Added support for TCP connections supporting, both, IPv4 and IPv6. + SSL/TLS is enforced for any such connection. + - Added support for and require actual client authentication: + UNIX socket connections require peer authentication. + TCP connections require client SSL certificate authentication. + * query language: + - Added support for expressions accessing an object's siblings by using + the '.' syntax. + * JSON format: + - Fixed handling of special characters which are now being escaped. + - Include a flag indicating whether time-series data are available for a + metric. + * utils: + - os: This utility module provides some Operating System related helper + functions: access user information (current user, homedir), manage + directories (create, delete, resolve), I/O helpers (select, read, + write), networking helpers (DNS resolution). + - proto: Added support for marshaling and unmarshaling messages encoding + a datum of any type, hosts, services, metrics, and attributes. + - ssl: This new module provides convenience functions for SSL/TLS + servers, clients, and sessions using OpenSSL. + + Backends: + * facter: Added a plugin collecting local facter fact values using + libcfacter (C++). + * store::network: Added a store-writer plugin sending all data to another + SysDB instance using the client library and low-level STORE commands. + + Plugins: + * syslog: The log-level is now configurable using the "LogLevel" option. + +Version 0.6.0, Release Date: 2014-11-17 +--------------------------------------- + + This release focuses on a clearer query language. The underlying + architecture has been refactored to allow for more powerful and generic + expressions. + + * core: + - Added support for arrays. Elements may be of any other supported + data-type but only integers, decimals, and strings are fully supported. + * store: + - Refactored matchers: all matchers now operate on one or two expressions + or other matchers, replacing the old and less flexible conditional + matchers. + - Attribute values may now be used anywhere in an expression. + - On data-type mismatch when comparing attribute values, the values will + be cast to string (previously only done for regex matches). + - Added support for querying an object's backends (array of strings). + - Added support for iterating child objects and array values. + - All matchers now return false if any operand is NULL (except for IS + NULL). + - All operations now return NULL if any operand is NULL. + - Fixed behavior of not-operators (!=, !~) and let them not match if any + of the operands is NULL. + - Add support for looking up all object types (hosts, services, metrics). + - Refactored the JSON formatter to be more flexible and powerful. + - Record parent objects for each stored object. + * frontend: + - Added a separate parser mode for expressions. + - Added a new queryable field 'name' to be used in place of the rather + magical keyword 'host' when looking up hosts. + - Improved error reporting in the parser and the (newly introduced) + analyzer. + - Apply filters to hosts in the FETCH command. + * query language: + - All operators may now be used in all places if the involved data-types + support the respective operation. For example, regular expressions may + be used to match any value. Non-string values are cast to strings for + that purpose. IS NULL / IS NOT NULL may be applied to arbitrary + expressions. + - Introduced the 'IN' operator to check if a value or an array of values + is included in an array. + - Added support for arrays: [,,...] + - Added support for string and array concatenation using the '||' + operator. + - Attribute names have to be strings instead of identifiers now. + - Access object fields using '' rather than '.'. + - Let the LOOKUP and FETCH commands support services and metrics. + - Introduced 'FETCH service|metric . ' replaces + 'MATCHING service|metric|attribute ' expressions. + +Version 0.5.0, Release Date: 2014-10-06 +--------------------------------------- + + This release focuses on simplifications and more flexibility in the frontend + and the query language. Most of those changes are not backward-compatible + but easy client implementations. -Version 0.2.0, Release Date: 2014-07-01 + * sysdb: + - Include priority when printing log messages. + - Fixed handling of empty queries. + * store: + - When serializing the entire store (LIST command), return an array of + host objects (rather than wrapping it into another object). + - Fixed JSON format of time-series data. + - Fixed error checks of invalid compare expressions and reject them. + - Fixed lookups by back-end name. + - Added support for comparing attributes with different types by + comparing their string values. + * frontend: + - Clarified details about asynchronous messages and authentication + options and made client implementation more robust. + - Introduced a new message type encoding the JSON serialized response of + a query and its data type. + - Include the priority in log messages. + - Include object types in FETCH, LIST, and LOOKUP messages. + - Added support for including services and metrics in LIST responses and + skip hosts without the respective children. + * query language: + - Added support for services and metrics to LIST command. + - Changed syntax for attribute lookup to 'attributes[]'. + - Changed syntax for accessing queryable fields to '.'. + * utils: + - strbuf: Improved memory management. + +Version 0.4.0, Release Date: 2014-09-01 +--------------------------------------- + + This release features support for metrics and transparent access to a + backend's time-series data. Some backward-incompatible changes were applied + to some query commands for more consistency. + + * build system: + - Added checks for format strings and arguments. + - Changed configure option for collectd::unixsock to + --enable-collectd-unixsock. + * core: + - Added support for “metric” objects which may be assigned to a host. + - Introduced support for handling time-series data: added new data + structures describing time-series data and a new type of plugins called + “time-series fetchers” which are used to access time-series + information. + * store: + - Fixed a memory leak. + - Apply (very) simple optimizations to lookup expressions. + - Added support for metrics and their “data-stores” (describing how to + access the actual data referenced by a metric). + - Added support for querying time-series data through the newly + introduced “time-series fetcher” plugins. + * frontend: + - Fixed a bug that might have caused reading too much data from the + connection buffer in certain situations. + * query language: + - The ‘MATCHING’ clause is now optional in ‘LOOKUP’ queries. + - The ‘LIST’ and ‘FETCH’ commands support ‘FILTER’ clauses as well. + - ‘LIST’ / ‘FETCH’ were changed to ‘LIST hosts’ / ‘FETCH host’. + - Added the ‘TIMESERIES’ command to query arbitrary time-series which + fetches time-series data from a backend's data-store for a given period + of time. + - Added support for date/time values. + - Added support for single quotes in string values: two adjacent single + quotes may be used to specify a single quote. + * documentation: + - Documented the frontend protocol in frontend/proto.h. + - Documented changes to the query language and new / updated plugins. + - Documented all supported data types. + + Backends: + * collectd::unixsock: + - The plugin now uses metrics in place of services. + - New config options ‘TimeseriesBackend’ and ‘TimeseriesBaseURL’ to + specify access to time-series, currently limited to ‘rrdtool’ and + ‘rrdcached’ access. + + Plugins: + * timeseries::rrdtool: + - New plugin adding support to fetch time-series from local RRD files. + - Supports flushing of values in RRDCacheD. + +Version 0.3.0, Release Date: 2014-08-01 +--------------------------------------- + + This release includes major enhancements and new features in the SysDB store + which is the core of the database. Most notably, complex expressions are now + supported when looking up information and filters may be used to preselect + the information to be returned. Some backward-incompatible changes were + introduced in the query language to avoid potentially confusing semantics. + + * sysdb: Fixed non-interactive mode when using libedit. + * core: + - Fixed weird behavior when collector callbacks were registered without + any plugin context information (as it would, e.g., happen when using + LoadPlugin instead of LoadBackend in sysdbd). + * store: + - Introduced service attributes -- similar to host attributes but + assigned to a service object. + - Don't log “value too old” messages if an updated object uses the same + timestamp as the old object. + - Switched from linked-lists to AVL trees for storing all object types to + reduce lookup times from O(n) to O(log n). + - Added infrastructure for handling and evaluating arithmetic + expressions and filters (see also the changes to the query language + below). + - Export various fields describing base attributes of all stored object + types: last_update, age, interval, and backend. These fields may be + used when querying the store. + * frontend: Include new service attributes in JSON output. + * query language: + - Added support for ‘IS NULL’ attribute checks matching on non-existent + attributes. + - Changed ‘LOOKUP ... WHERE’ queries to ‘LOOKUP ... MATCHING’ to avoid + confusion with SQL semantics. + - Switched from ‘.name’ to ‘’ in MATCHING clauses, thus, no + longer shadowing attributes called “name”. + - Added support for arithmetic expressions in places that used simple + data values before. Arithmetic expressions support addition, + subtraction, multiplication, division, modulo, and concatenation + operations. The expressions may be based on constant values or + queryable fields of the stored objects (specified as ‘:last_update’, + ‘:age’, ‘:interval’, and ‘:backend’). + - Added ‘FILTER’ support to ‘LOOKUP’ queries. Filters are conditional + expressions which may optionally be used to limit the query and the + result to object (of any type) matching the filter condition. + - Added support to specify date and time values. + * utils: + - avltree: Added an AVL tree (self-balancing binary tree) implementation. + * documentation: Added documentation for the new query language features. + * testing: Run all unit tests through valgrind by default (if available). + + Backends: + * collectd::unixsock: + - Added support for spaces in identifiers. + - Store plugin identifier (host, plugin, plugin_instance, type, + type_instance) as service attributes. + +Version 0.2.0, Release Date: 2014-07-02 --------------------------------------- This is another development snapshot of SysDB. The main focus of this @@ -51,7 +338,6 @@ Version 0.2.0, Release Date: 2014-07-01 * documentation: Added missing files to the distribution tarball. Plugins: - * syslog: Don't log debug messages to syslog. Version 0.1.0, Release Date: 2014-06-01