![[tokkee]](http://tokkee.org/images/avatar.png){kind=avatar}
diff --git a/ReleaseNotes b/ReleaseNotes
index f9e91bc5a2df2f517367429ff664b56b47871735..57b04e20460b9ff7240ead71fc23613b8eb79993 100644 (file)
--- a/ReleaseNotes
+++ b/ReleaseNotes
and very detailed list of changes can be obtained from the Git logs for each
release.
+ Compatibility Note:
+ ~~~~~~~~~~~~~~~~~~~
+ All pre 1.0 development snapshots are not considered stable yet. That is,
+ all interfaces, including the plugin API, the network protocol, or the query
+ language, may change in an incompatible way at any time. All changes are
+ documented here.
+
About SysDB:
~~~~~~~~~~~~
“System DataBase” (SysDB) is a multi-backend system management and inventory
Object names are canonicalized before they are added to the store to ensure
a consistent view of your infrastructure.
- <http://sysdb.io/>
+ <https://sysdb.io/>
+
+Version 0.8.0, Release Date: 2016-02-25
+---------------------------------------
+
+ This release focuses on a generic and much more powerful store (query)
+ interface and modular store implementations via plugins. The in-memory store
+ has been moved into a plugin on top of that. Various fixes and improvements,
+ mostly related to iterators, have been applied to the parser and query
+ language.
+
+ * build system:
+ - Fix VPATH (out of tree) builds.
+ - Fixed a build issue (configure failure) when using RRDtool 1.5.
+ - Enable silent build rules by default (if supported).
+ - Fix build issues with GCC 5.
+ * parser:
+ - Add a new, standalone parser module based on a generic (public) AST
+ (abstract syntax tree). This replaces the (internal) parser of the
+ frontend module.
+ - Improved error reporting for syntactic and semantic errors by
+ propagating error messages in the parser and analyzer.
+ * core / store:
+ - Fully support array access and comparison (all element types).
+ - Service/metric parent host names are now available via the "hostname"
+ attribute (populated automatically).
+ - Rebase the store query interface on top of the new parser: store
+ implementations prepare actual queries from an AST, thus, allowing
+ multiple implementations of store readers.
+ - Querying the store (through the new interface) will emit data to an
+ arbitrary writer implementation allowing for more flexible post-
+ processing.
+ - Fixed a memory leak in the TIMESERIES implementation.
+ - data: Added support for boolean values; this is a backward incompatible
+ change which breaks the network encoding.
+ - Include nano-second fraction of datatime values (if any) in the string
+ format (including JSON query results).
+ - Add new callback type for store-readers (query plugins) based on the
+ new generic query interface.
+ - memstore: Refactor the in-memory store implementation into a separate
+ data-type. It implements the writer and query/reader APIs.
+ - The main, in-memory store is no longer enabled by default (see
+ store::memory plugin). Instead, all updates go through the plugin API
+ and are distributed to all registered writers.
+ - Handle hostname canonicalization in the generic plugin module instead
+ of requiring each writer plugin to do so.
+ - Do not canonicalize hostnames when fetching time-series.
+ - Determine an object's backends and update interval in the generic
+ plugin module and pass it on to writer plugins which are expected to
+ merge it with existing entries.
+ - Let the JSON formatter implement the writer API to make it more
+ flexible and uncouple it from the memstore module; let it support
+ arbitrary top-level objects (besides hosts).
+ - FETCH (in the core) now supports all object types and child objects as
+ well as fetching the base object without children / siblings.
+ * frontend:
+ - Added the SERVER_VERSION command; this allows clients to check for
+ supported servers.
+ - Fixed a memory leak in failed FETCH commands.
+ - Switch to the new, pluggable query interface: use registered plugins to
+ write to or query the SysDB store.
+ - Move protocol-specific (query) logic out of the core and into the
+ frontend (based on the new query API).
+ - A generic TIMESERIES implementation based on the query API replaces the
+ in-core implementation.
+ * query language:
+ - Check types of comparison operands and arithmetic expression and treat
+ mismatches as errors; previously those comparisons would fail silently.
+ - Added the 'NOT IN' operator.
+ - Support iterators (ANY/ALL) of arbitrary array expressions.
+ - Change child-object iterator syntax to 'ANY/ALL <type>.name', that is,
+ require the field name and add support for accessing arbitrary fields.
+ - Added new field "value" for attribute values.
+ - Added new field "timeseries" (boolean) for metrics indicating whether
+ a time-series store is known to SysDB.
+ - Support attribute and backend iterators in FILTER expressions.
+ - Enforce that TIMESERIES end times are greater than start times.
+ - Added unary 'IS TRUE' / 'IS FALSE' operators.
+ - Iterator expressions may now reference a chain of sibling objects as
+ long as only one such reference is an iterator. For example,
+ `LOOKUP services MATCHING ANY host.backend =~ 'foo'`.
+ * testing:
+ - Use check "loop" tests; this will run all test-cases even if there are
+ failures.
+ - Create separate binaries for each test; this allows to run more in
+ parallel.
+ - Make it possible to run unit and integration tests separately; see
+ README.
+
+ Stores:
+ * store::memory: Added a store-writer/store-reader plugin providing an
+ in-memory store replacing the previous, always-enabled store. It is based
+ on the 'memstore' module.
+
+Version 0.7.0, Release Date: 2015-02-21
+---------------------------------------
+
+ This release introduces various features related to running SysDB as an
+ agent. Objects may now also be submitted by arbitrary applications using the
+ network protocol. Two new plugins are available to collect local facter fact
+ values and to mirror all store operations to another instance. Various fixes
+ and improvements have been applied to the client program (sysdb).
+
+ * build system:
+ - Added support for C++.
+ * sysdb:
+ - Fixed terminal resets on exit.
+ - Don't exit if the server closed the connection; instead reconnect
+ automatically on the next user input.
+ - Fixed string parsing: previously, quoted strings were not handled at
+ all and might have been terminated by an embedded semicolon.
+ - Also store failed commands in the history to enable history editing in
+ such cases.
+ - Improved program output: don't mix log messages with prompts after
+ sending a command; print log messages to the standard error channel.
+ - Fixed lookup of the .sysdb_history file for cases where the SysDB user
+ does not match the current operating system user.
+ * sysdbd:
+ - SSL options can be specified inside a <Listen> block.
+ - Unload all plugins on exit to ensure a clean shutdown.
+ * client:
+ - Simplified RPC handling by providing a helper function.
+ - Added support for TCP connections using SSL/TLS and supporting, both,
+ IPv4 and IPv6. Remote peer verification is required for any such
+ connection.
+ - New command line flags -K, -C, and -A to specify the SSL private key,
+ client certificate, and CA certificates file names.
+ * core:
+ - Fully support arrays of all types.
+ - Fixed (implemented) reloading of time-series fetcher plugins.
+ - Added new callback type for "store-writers", making that part of the
+ store pluggable (queries are not supported).
+ * store:
+ - Added support for querying an object's siblings.
+ - Send all stored objects to store-writer plugins in addition to storing
+ them in the in-memory store.
+ * frontend:
+ - Fixed parsing of time values.
+ - Add support for storing hosts, services, metrics, and attributes using
+ the new 'STORE' command.
+ - Fixed handling of subsequent requests on the same connection by
+ avoiding unnecessary pauses of up to one second, thus, greatly speeding
+ up request handling.
+ - Added support for TCP connections supporting, both, IPv4 and IPv6.
+ SSL/TLS is enforced for any such connection.
+ - Added support for and require actual client authentication:
+ UNIX socket connections require peer authentication.
+ TCP connections require client SSL certificate authentication.
+ * query language:
+ - Added support for expressions accessing an object's siblings by using
+ the '<object_type>.<sibling_type>' syntax.
+ * JSON format:
+ - Fixed handling of special characters which are now being escaped.
+ - Include a flag indicating whether time-series data are available for a
+ metric.
+ * utils:
+ - os: This utility module provides some Operating System related helper
+ functions: access user information (current user, homedir), manage
+ directories (create, delete, resolve), I/O helpers (select, read,
+ write), networking helpers (DNS resolution).
+ - proto: Added support for marshaling and unmarshaling messages encoding
+ a datum of any type, hosts, services, metrics, and attributes.
+ - ssl: This new module provides convenience functions for SSL/TLS
+ servers, clients, and sessions using OpenSSL.
+
+ Backends:
+ * facter: Added a plugin collecting local facter fact values using
+ libcfacter (C++).
+
+ Stores:
+ * store::network: Added a store-writer plugin sending all data to another
+ SysDB instance using the client library and low-level STORE commands.
+
+ Plugins:
+ * syslog: The log-level is now configurable using the "LogLevel" option.
+
+Version 0.6.0, Release Date: 2014-11-17
+---------------------------------------
+
+ This release focuses on a clearer query language. The underlying
+ architecture has been refactored to allow for more powerful and generic
+ expressions.
+
+ * core:
+ - Added support for arrays. Elements may be of any other supported
+ data-type but only integers, decimals, and strings are fully supported.
+ * store:
+ - Refactored matchers: all matchers now operate on one or two expressions
+ or other matchers, replacing the old and less flexible conditional
+ matchers.
+ - Attribute values may now be used anywhere in an expression.
+ - On data-type mismatch when comparing attribute values, the values will
+ be cast to string (previously only done for regex matches).
+ - Added support for querying an object's backends (array of strings).
+ - Added support for iterating child objects and array values.
+ - All matchers now return false if any operand is NULL (except for IS
+ NULL).
+ - All operations now return NULL if any operand is NULL.
+ - Fixed behavior of not-operators (!=, !~) and let them not match if any
+ of the operands is NULL.
+ - Add support for looking up all object types (hosts, services, metrics).
+ - Refactored the JSON formatter to be more flexible and powerful.
+ - Record parent objects for each stored object.
+ * frontend:
+ - Added a separate parser mode for expressions.
+ - Added a new queryable field 'name' to be used in place of the rather
+ magical keyword 'host' when looking up hosts.
+ - Improved error reporting in the parser and the (newly introduced)
+ analyzer.
+ - Apply filters to hosts in the FETCH command.
+ * query language:
+ - All operators may now be used in all places if the involved data-types
+ support the respective operation. For example, regular expressions may
+ be used to match any value. Non-string values are cast to strings for
+ that purpose. IS NULL / IS NOT NULL may be applied to arbitrary
+ expressions.
+ - Introduced the 'IN' operator to check if a value or an array of values
+ is included in an array.
+ - Added support for arrays: [<elem1>,<elem2>,...]
+ - Added support for string and array concatenation using the '||'
+ operator.
+ - Attribute names have to be strings instead of identifiers now.
+ - Access object fields using '<field>' rather than '.<field>'.
+ - Let the LOOKUP and FETCH commands support services and metrics.
+ - Introduced 'FETCH service|metric <host>.<name' for fetching services
+ and metrics.
+ - Added support for 'ANY' and 'ALL' operators which apply a matcher to
+ each value emitted by an iterator (a host's services and metrics or
+ arrays). 'MATCHING ANY service|metric|attribute <cmp> <expr>' replaces
+ 'MATCHING service|metric|attribute <cmp> <value>' expressions.
+
+Version 0.5.0, Release Date: 2014-10-06
+---------------------------------------
+
+ This release focuses on simplifications and more flexibility in the frontend
+ and the query language. Most of those changes are not backward-compatible
+ but easy client implementations.
+
+ * sysdb:
+ - Include priority when printing log messages.
+ - Fixed handling of empty queries.
+ * store:
+ - When serializing the entire store (LIST command), return an array of
+ host objects (rather than wrapping it into another object).
+ - Fixed JSON format of time-series data.
+ - Fixed error checks of invalid compare expressions and reject them.
+ - Fixed lookups by back-end name.
+ - Added support for comparing attributes with different types by
+ comparing their string values.
+ * frontend:
+ - Clarified details about asynchronous messages and authentication
+ options and made client implementation more robust.
+ - Introduced a new message type encoding the JSON serialized response of
+ a query and its data type.
+ - Include the priority in log messages.
+ - Include object types in FETCH, LIST, and LOOKUP messages.
+ - Added support for including services and metrics in LIST responses and
+ skip hosts without the respective children.
+ * query language:
+ - Added support for services and metrics to LIST command.
+ - Changed syntax for attribute lookup to 'attributes[<name>]'.
+ - Changed syntax for accessing queryable fields to '.<field>'.
+ * utils:
+ - strbuf: Improved memory management.
+
+Version 0.4.0, Release Date: 2014-09-01
+---------------------------------------
+
+ This release features support for metrics and transparent access to a
+ backend's time-series data. Some backward-incompatible changes were applied
+ to some query commands for more consistency.
+
+ * build system:
+ - Added checks for format strings and arguments.
+ - Changed configure option for collectd::unixsock to
+ --enable-collectd-unixsock.
+ * core:
+ - Added support for “metric” objects which may be assigned to a host.
+ - Introduced support for handling time-series data: added new data
+ structures describing time-series data and a new type of plugins called
+ “time-series fetchers” which are used to access time-series
+ information.
+ * store:
+ - Fixed a memory leak.
+ - Apply (very) simple optimizations to lookup expressions.
+ - Added support for metrics and their “data-stores” (describing how to
+ access the actual data referenced by a metric).
+ - Added support for querying time-series data through the newly
+ introduced “time-series fetcher” plugins.
+ * frontend:
+ - Fixed a bug that might have caused reading too much data from the
+ connection buffer in certain situations.
+ * query language:
+ - The ‘MATCHING’ clause is now optional in ‘LOOKUP’ queries.
+ - The ‘LIST’ and ‘FETCH’ commands support ‘FILTER’ clauses as well.
+ - ‘LIST’ / ‘FETCH’ were changed to ‘LIST hosts’ / ‘FETCH host’.
+ - Added the ‘TIMESERIES’ command to query arbitrary time-series which
+ fetches time-series data from a backend's data-store for a given period
+ of time.
+ - Added support for date/time values.
+ - Added support for single quotes in string values: two adjacent single
+ quotes may be used to specify a single quote.
+ * documentation:
+ - Documented the frontend protocol in frontend/proto.h.
+ - Documented changes to the query language and new / updated plugins.
+ - Documented all supported data types.
+
+ Backends:
+ * collectd::unixsock:
+ - The plugin now uses metrics in place of services.
+ - New config options ‘TimeseriesBackend’ and ‘TimeseriesBaseURL’ to
+ specify access to time-series, currently limited to ‘rrdtool’ and
+ ‘rrdcached’ access.
+
+ Plugins:
+ * timeseries::rrdtool:
+ - New plugin adding support to fetch time-series from local RRD files.
+ - Supports flushing of values in RRDCacheD.
-Version 0.2.0, Release Date: 2014-07-01
+Version 0.3.0, Release Date: 2014-08-01
+---------------------------------------
+
+ This release includes major enhancements and new features in the SysDB store
+ which is the core of the database. Most notably, complex expressions are now
+ supported when looking up information and filters may be used to preselect
+ the information to be returned. Some backward-incompatible changes were
+ introduced in the query language to avoid potentially confusing semantics.
+
+ * sysdb: Fixed non-interactive mode when using libedit.
+ * core:
+ - Fixed weird behavior when collector callbacks were registered without
+ any plugin context information (as it would, e.g., happen when using
+ LoadPlugin instead of LoadBackend in sysdbd).
+ * store:
+ - Introduced service attributes -- similar to host attributes but
+ assigned to a service object.
+ - Don't log “value too old” messages if an updated object uses the same
+ timestamp as the old object.
+ - Switched from linked-lists to AVL trees for storing all object types to
+ reduce lookup times from O(n) to O(log n).
+ - Added infrastructure for handling and evaluating arithmetic
+ expressions and filters (see also the changes to the query language
+ below).
+ - Export various fields describing base attributes of all stored object
+ types: last_update, age, interval, and backend. These fields may be
+ used when querying the store.
+ * frontend: Include new service attributes in JSON output.
+ * query language:
+ - Added support for ‘IS NULL’ attribute checks matching on non-existent
+ attributes.
+ - Changed ‘LOOKUP ... WHERE’ queries to ‘LOOKUP ... MATCHING’ to avoid
+ confusion with SQL semantics.
+ - Switched from ‘<obj>.name’ to ‘<obj>’ in MATCHING clauses, thus, no
+ longer shadowing attributes called “name”.
+ - Added support for arithmetic expressions in places that used simple
+ data values before. Arithmetic expressions support addition,
+ subtraction, multiplication, division, modulo, and concatenation
+ operations. The expressions may be based on constant values or
+ queryable fields of the stored objects (specified as ‘:last_update’,
+ ‘:age’, ‘:interval’, and ‘:backend’).
+ - Added ‘FILTER’ support to ‘LOOKUP’ queries. Filters are conditional
+ expressions which may optionally be used to limit the query and the
+ result to object (of any type) matching the filter condition.
+ - Added support to specify date and time values.
+ * utils:
+ - avltree: Added an AVL tree (self-balancing binary tree) implementation.
+ * documentation: Added documentation for the new query language features.
+ * testing: Run all unit tests through valgrind by default (if available).
+
+ Backends:
+ * collectd::unixsock:
+ - Added support for spaces in identifiers.
+ - Store plugin identifier (host, plugin, plugin_instance, type,
+ type_instance) as service attributes.
+
+Version 0.2.0, Release Date: 2014-07-02
---------------------------------------
This is another development snapshot of SysDB. The main focus of this
* documentation: Added missing files to the distribution tarball.
Plugins:
-
* syslog: Don't log debug messages to syslog.
Version 0.1.0, Release Date: 2014-06-01