![[tokkee]](http://tokkee.org/images/avatar.png){kind=avatar}
diff --git a/ReleaseNotes b/ReleaseNotes
index 341f58f1af0bd5db6327f5dd01c2dc8d5df05fa8..22950a54c3518e482de128c21c7836e0a1505056 100644 (file)
--- a/ReleaseNotes
+++ b/ReleaseNotes
and very detailed list of changes can be obtained from the Git logs for each
release.
+ Compatibility Note:
+ ~~~~~~~~~~~~~~~~~~~
+
+ All pre 1.0 development snapshots are not considered stable yet. That is,
+ all interfaces, including the plugin API, the network protocol, or the query
+ language, may change in an incompatible way at any time. All changes are
+ documented here.
+
About SysDB:
~~~~~~~~~~~~
“System DataBase” (SysDB) is a multi-backend system management and inventory
- collection service. It may be used to (continuously) collect information
- about your systems from various backends (inventory services, monitoring
- services, etc.) and provides a unique interface to access the information
- independent of the active backends.
+ collection service. It stores system and inventory information about
+ hardware and software systems. This information is (continuously) collected
+ from various configurable backends (inventory services, monitoring services,
+ etc.) and stored in a graph-like hierarchy of generic objects. The store may
+ be queried through a generic interface independent of the active backends.
+ Object names are canonicalized before they are added to the store to ensure
+ a consistent view of your infrastructure.
+
+ <https://sysdb.io/>
+
+Version 0.7.0, Release Date: 2015-02-21
+---------------------------------------
+
+ This release introduces various features related to running SysDB as an
+ agent. Objects may now also be submitted by arbitrary applications using the
+ network protocol. Two new plugins are available to collect local facter fact
+ values and to mirror all store operations to another instance. Various fixes
+ and improvements have been applied to the client program (sysdb).
+
+ * build system:
+ - Added support for C++.
+ * sysdb:
+ - Fixed terminal resets on exit.
+ - Don't exit if the server closed the connection; instead reconnect
+ automatically on the next user input.
+ - Fixed string parsing: previously, quoted strings were not handled at
+ all and might have been terminated by an embedded semicolon.
+ - Also store failed commands in the history to enable history editing in
+ such cases.
+ - Improved program output: don't mix log messages with prompts after
+ sending a command; print log messages to the standard error channel.
+ - Fixed lookup of the .sysdb_history file for cases where the SysDB user
+ does not match the current operating system user.
+ * sysdbd:
+ - SSL options can be specified inside a <Listen> block.
+ - Unload all plugins on exit to ensure a clean shutdown.
+ * client:
+ - Simplified RPC handling by providing a helper function.
+ - Added support for TCP connections using SSL/TLS and supporting, both,
+ IPv4 and IPv6. Remote peer verification is required for any such
+ connection.
+ - New command line flags -K, -C, and -A to specify the SSL private key,
+ client certificate, and CA certificates file names.
+ * core:
+ - Fully support arrays of all types.
+ - Fixed (implemented) reloading of time-series fetcher plugins.
+ - Added new callback type for "store-writers", making that part of the
+ store pluggable (queries are not supported).
+ * store:
+ - Added support for querying an object's siblings.
+ - Send all stored objects to store-writer plugins in addition to storing
+ them in the in-memory store.
+ * frontend:
+ - Fixed parsing of time values.
+ - Add support for storing hosts, services, metrics, and attributes using
+ the new 'STORE' command.
+ - Fixed handling of subsequent requests on the same connection by
+ avoiding unnecessary pauses of up to one second, thus, greatly speeding
+ up request handling.
+ - Added support for TCP connections supporting, both, IPv4 and IPv6.
+ SSL/TLS is enforced for any such connection.
+ - Added support for and require actual client authentication:
+ UNIX socket connections require peer authentication.
+ TCP connections require client SSL certificate authentication.
+ * query language:
+ - Added support for expressions accessing an object's siblings by using
+ the '<object_type>.<sibling_type>' syntax.
+ * JSON format:
+ - Fixed handling of special characters which are now being escaped.
+ - Include a flag indicating whether time-series data are available for a
+ metric.
+ * utils:
+ - os: This utility module provides some Operating System related helper
+ functions: access user information (current user, homedir), manage
+ directories (create, delete, resolve), I/O helpers (select, read,
+ write), networking helpers (DNS resolution).
+ - proto: Added support for marshaling and unmarshaling messages encoding
+ a datum of any type, hosts, services, metrics, and attributes.
+ - ssl: This new module provides convenience functions for SSL/TLS
+ servers, clients, and sessions using OpenSSL.
+
+ Backends:
+ * facter: Added a plugin collecting local facter fact values using
+ libcfacter (C++).
+ * store::network: Added a store-writer plugin sending all data to another
+ SysDB instance using the client library and low-level STORE commands.
+
+ Plugins:
+ * syslog: The log-level is now configurable using the "LogLevel" option.
+
+Version 0.6.0, Release Date: 2014-11-17
+---------------------------------------
+
+ This release focuses on a clearer query language. The underlying
+ architecture has been refactored to allow for more powerful and generic
+ expressions.
+
+ * core:
+ - Added support for arrays. Elements may be of any other supported
+ data-type but only integers, decimals, and strings are fully supported.
+ * store:
+ - Refactored matchers: all matchers now operate on one or two expressions
+ or other matchers, replacing the old and less flexible conditional
+ matchers.
+ - Attribute values may now be used anywhere in an expression.
+ - On data-type mismatch when comparing attribute values, the values will
+ be cast to string (previously only done for regex matches).
+ - Added support for querying an object's backends (array of strings).
+ - Added support for iterating child objects and array values.
+ - All matchers now return false if any operand is NULL (except for IS
+ NULL).
+ - All operations now return NULL if any operand is NULL.
+ - Fixed behavior of not-operators (!=, !~) and let them not match if any
+ of the operands is NULL.
+ - Add support for looking up all object types (hosts, services, metrics).
+ - Refactored the JSON formatter to be more flexible and powerful.
+ - Record parent objects for each stored object.
+ * frontend:
+ - Added a separate parser mode for expressions.
+ - Added a new queryable field 'name' to be used in place of the rather
+ magical keyword 'host' when looking up hosts.
+ - Improved error reporting in the parser and the (newly introduced)
+ analyzer.
+ - Apply filters to hosts in the FETCH command.
+ * query language:
+ - All operators may now be used in all places if the involved data-types
+ support the respective operation. For example, regular expressions may
+ be used to match any value. Non-string values are cast to strings for
+ that purpose. IS NULL / IS NOT NULL may be applied to arbitrary
+ expressions.
+ - Introduced the 'IN' operator to check if a value or an array of values
+ is included in an array.
+ - Added support for arrays: [<elem1>,<elem2>,...]
+ - Added support for string and array concatenation using the '||'
+ operator.
+ - Attribute names have to be strings instead of identifiers now.
+ - Access object fields using '<field>' rather than '.<field>'.
+ - Let the LOOKUP and FETCH commands support services and metrics.
+ - Introduced 'FETCH service|metric <host>.<name' for fetching services
+ and metrics.
+ - Added support for 'ANY' and 'ALL' operators which apply a matcher to
+ each value emitted by an iterator (a host's services and metrics or
+ arrays). 'MATCHING ANY service|metric|attribute <cmp> <expr>' replaces
+ 'MATCHING service|metric|attribute <cmp> <value>' expressions.
+
+Version 0.5.0, Release Date: 2014-10-06
+---------------------------------------
+
+ This release focuses on simplifications and more flexibility in the frontend
+ and the query language. Most of those changes are not backward-compatible
+ but easy client implementations.
+
+ * sysdb:
+ - Include priority when printing log messages.
+ - Fixed handling of empty queries.
+ * store:
+ - When serializing the entire store (LIST command), return an array of
+ host objects (rather than wrapping it into another object).
+ - Fixed JSON format of time-series data.
+ - Fixed error checks of invalid compare expressions and reject them.
+ - Fixed lookups by back-end name.
+ - Added support for comparing attributes with different types by
+ comparing their string values.
+ * frontend:
+ - Clarified details about asynchronous messages and authentication
+ options and made client implementation more robust.
+ - Introduced a new message type encoding the JSON serialized response of
+ a query and its data type.
+ - Include the priority in log messages.
+ - Include object types in FETCH, LIST, and LOOKUP messages.
+ - Added support for including services and metrics in LIST responses and
+ skip hosts without the respective children.
+ * query language:
+ - Added support for services and metrics to LIST command.
+ - Changed syntax for attribute lookup to 'attributes[<name>]'.
+ - Changed syntax for accessing queryable fields to '.<field>'.
+ * utils:
+ - strbuf: Improved memory management.
- <https://github.com/tokkee/sysdb>
+Version 0.4.0, Release Date: 2014-09-01
+---------------------------------------
+
+ This release features support for metrics and transparent access to a
+ backend's time-series data. Some backward-incompatible changes were applied
+ to some query commands for more consistency.
+
+ * build system:
+ - Added checks for format strings and arguments.
+ - Changed configure option for collectd::unixsock to
+ --enable-collectd-unixsock.
+ * core:
+ - Added support for “metric” objects which may be assigned to a host.
+ - Introduced support for handling time-series data: added new data
+ structures describing time-series data and a new type of plugins called
+ “time-series fetchers” which are used to access time-series
+ information.
+ * store:
+ - Fixed a memory leak.
+ - Apply (very) simple optimizations to lookup expressions.
+ - Added support for metrics and their “data-stores” (describing how to
+ access the actual data referenced by a metric).
+ - Added support for querying time-series data through the newly
+ introduced “time-series fetcher” plugins.
+ * frontend:
+ - Fixed a bug that might have caused reading too much data from the
+ connection buffer in certain situations.
+ * query language:
+ - The ‘MATCHING’ clause is now optional in ‘LOOKUP’ queries.
+ - The ‘LIST’ and ‘FETCH’ commands support ‘FILTER’ clauses as well.
+ - ‘LIST’ / ‘FETCH’ were changed to ‘LIST hosts’ / ‘FETCH host’.
+ - Added the ‘TIMESERIES’ command to query arbitrary time-series which
+ fetches time-series data from a backend's data-store for a given period
+ of time.
+ - Added support for date/time values.
+ - Added support for single quotes in string values: two adjacent single
+ quotes may be used to specify a single quote.
+ * documentation:
+ - Documented the frontend protocol in frontend/proto.h.
+ - Documented changes to the query language and new / updated plugins.
+ - Documented all supported data types.
+
+ Backends:
+ * collectd::unixsock:
+ - The plugin now uses metrics in place of services.
+ - New config options ‘TimeseriesBackend’ and ‘TimeseriesBaseURL’ to
+ specify access to time-series, currently limited to ‘rrdtool’ and
+ ‘rrdcached’ access.
+
+ Plugins:
+ * timeseries::rrdtool:
+ - New plugin adding support to fetch time-series from local RRD files.
+ - Supports flushing of values in RRDCacheD.
+
+Version 0.3.0, Release Date: 2014-08-01
+---------------------------------------
+
+ This release includes major enhancements and new features in the SysDB store
+ which is the core of the database. Most notably, complex expressions are now
+ supported when looking up information and filters may be used to preselect
+ the information to be returned. Some backward-incompatible changes were
+ introduced in the query language to avoid potentially confusing semantics.
+
+ * sysdb: Fixed non-interactive mode when using libedit.
+ * core:
+ - Fixed weird behavior when collector callbacks were registered without
+ any plugin context information (as it would, e.g., happen when using
+ LoadPlugin instead of LoadBackend in sysdbd).
+ * store:
+ - Introduced service attributes -- similar to host attributes but
+ assigned to a service object.
+ - Don't log “value too old” messages if an updated object uses the same
+ timestamp as the old object.
+ - Switched from linked-lists to AVL trees for storing all object types to
+ reduce lookup times from O(n) to O(log n).
+ - Added infrastructure for handling and evaluating arithmetic
+ expressions and filters (see also the changes to the query language
+ below).
+ - Export various fields describing base attributes of all stored object
+ types: last_update, age, interval, and backend. These fields may be
+ used when querying the store.
+ * frontend: Include new service attributes in JSON output.
+ * query language:
+ - Added support for ‘IS NULL’ attribute checks matching on non-existent
+ attributes.
+ - Changed ‘LOOKUP ... WHERE’ queries to ‘LOOKUP ... MATCHING’ to avoid
+ confusion with SQL semantics.
+ - Switched from ‘<obj>.name’ to ‘<obj>’ in MATCHING clauses, thus, no
+ longer shadowing attributes called “name”.
+ - Added support for arithmetic expressions in places that used simple
+ data values before. Arithmetic expressions support addition,
+ subtraction, multiplication, division, modulo, and concatenation
+ operations. The expressions may be based on constant values or
+ queryable fields of the stored objects (specified as ‘:last_update’,
+ ‘:age’, ‘:interval’, and ‘:backend’).
+ - Added ‘FILTER’ support to ‘LOOKUP’ queries. Filters are conditional
+ expressions which may optionally be used to limit the query and the
+ result to object (of any type) matching the filter condition.
+ - Added support to specify date and time values.
+ * utils:
+ - avltree: Added an AVL tree (self-balancing binary tree) implementation.
+ * documentation: Added documentation for the new query language features.
+ * testing: Run all unit tests through valgrind by default (if available).
+
+ Backends:
+ * collectd::unixsock:
+ - Added support for spaces in identifiers.
+ - Store plugin identifier (host, plugin, plugin_instance, type,
+ type_instance) as service attributes.
+
+Version 0.2.0, Release Date: 2014-07-02
+---------------------------------------
+
+ This is another development snapshot of SysDB. The main focus of this
+ release is on stabilizing existing features and extending the query
+ infrastructure to support all attribute types and further operators.
+
+ * build system: Fixed linking of sysdbd to include *all* necessary object
+ files. This bug caused some plugins to be un-loadable.
+ * sysdbd: Fixed shutdown process in case of errors.
+ * sysdb: Fixed a bug causing some server replies to be ignored when exiting
+ from the client.
+ * core:
+ - Always log to the standard output stream if no user-provided logging
+ callbacks had been specified. Previously this was disabled after
+ enabling logging to clients in the frontend.
+ - Simplified plugin naming information and determine configuration
+ callback names automatically to ensure better consistency.
+ * store:
+ - Simplified internal data management.
+ - Store the list of backends along with each stored object which provide
+ the respective object.
+ * frontend:
+ - Simplified and improved the internal architecture matching / looking up
+ objects.
+ - Fixed parser to correctly reject invalid object types.
+ - Automatically create the base directory of a listening UNIX socket.
+ * query language: Added support for numeric constants and added operators
+ for less and greater than or equal attribute comparison.
+ * documentation: Added missing files to the distribution tarball.
+
+ Plugins:
+ * syslog: Don't log debug messages to syslog.
-Version 0.1.0, Release Date: tbd
---------------------------------
+Version 0.1.0, Release Date: 2014-06-01
+---------------------------------------
This is the initial release of SysDB. It provides the following core
features:
listen addresses.
* query language: The store may be queried using the ‘LIST’, ‘LOOKUP’, and
‘FETCH’ commands returning JSON formatted objects.
+ * utils: A rich set of abstract data-types and utility functions for
+ internal use:
+ - generic channels for asynchronous, multi-threaded I/O
+ - DBI wrapper to ease common database access operations
+ - thread-aware error handling and reporting
+ - doubly linked-lists
+ - dynamic sized memory buffers supporting formatted strings and binary
+ data
+ - UNIX socket client implementation
* documentation: Full documentation for the tools, all plugins, and the
query language provided as manpages and HTML pages.
* testing: Unit and integration tests exist for most parts except plugins