d37d4207016337332743efa22f948bd6bb30a80d
1 SysDB Release Notes
2 =====================
4 The release notes contain noteworthy changes in each System DataBase (SysDB)
5 release, with a summary of the most important changes at the top. A complete
6 and very detailed list of changes can be obtained from the Git logs for each
7 release.
9 About SysDB:
10 ~~~~~~~~~~~~
11 “System DataBase” (SysDB) is a multi-backend system management and inventory
12 collection service. It stores system and inventory information about
13 hardware and software systems. This information is (continuously) collected
14 from various configurable backends (inventory services, monitoring services,
15 etc.) and stored in a graph-like hierarchy of generic objects. The store may
16 be queried through a generic interface independent of the active backends.
17 Object names are canonicalized before they are added to the store to ensure
18 a consistent view of your infrastructure.
20 <https://sysdb.io/>
22 Version 0.4.0, Release Date: 2014-09-01
23 ---------------------------------------
25 This release features support for metrics and transparent access to a
26 backend's time-series data. Some backward-incompatible changes were applied
27 to some query commands for more consistency.
29 * build system:
30 - Added checks for format strings and arguments.
31 - Changed configure option for collectd::unixsock to
32 --enable-collectd-unixsock.
33 * core:
34 - Added support for “metric” objects which may be assigned to a host.
35 - Introduced support for handling time-series data: added new data
36 structures describing time-series data and a new type of plugins called
37 “time-series fetchers” which are used to access time-series
38 information.
39 * store:
40 - Fixed a memory leak.
41 - Apply (very) simple optimizations to lookup expressions.
42 - Added support for metrics and their “data-stores” (describing how to
43 access the actual data references by a metric).
44 - Added support for querying time-series data through the newly
45 introduced “time-series fetcher” plugins.
46 * frontend:
47 - Fixed a bug that might have caused reading too much data from the
48 connection buffer in certain situations.
49 * query language:
50 - The ‘MATCHING’ clause is now optional in ‘LOOKUP’ queries.
51 - The ‘LIST’ and ‘FETCH’ commands support ‘FILTER’ clauses as well.
52 - ‘LIST’ / ‘FETCH’ were changed to ‘LIST hosts’ / ‘FETCH host’.
53 - Added the ‘TIMESERIES’ command to query arbitrary time-series which
54 fetches time-series data from a backend's data-store for a given period
55 of time.
56 - Added support for date/time values.
57 - Added support for single quotes in string values: two adjacent single
58 quotes may be used to specify a single quote.
59 * documentation:
60 - Documented the frontend protocol in frontend/proto.h.
61 - Documented changes to the query language and new / updated plugins.
62 - Documented all supported data types.
64 Backends:
65 * collectd::unixsock:
66 - The plugin now uses metrics in place of services.
67 - New config options ‘TimeseriesBackend’ and ‘TimeseriesBaseURL’ to
68 specify access to time-series, currently limited to ‘rrdtool’ and
69 ‘rrdcached’ access.
71 Plugins:
72 * timeseries::rrdtool:
73 - New plugin adding support to fetch time-series from local RRD files.
74 - Supports flushing of values in RRDCacheD.
76 Version 0.3.0, Release Date: 2014-08-01
77 ---------------------------------------
79 This release includes major enhancements and new features in the SysDB store
80 which is the core of the database. Most notably, complex expressions are now
81 supported when looking up information and filters may be used to preselect
82 the information to be returned. Some backward-incompatible changes were
83 introduced in the query language to avoid potentially confusing semantics.
85 * sysdb: Fixed non-interactive mode when using libedit.
86 * core:
87 - Fixed weird behavior when collector callbacks were registered without
88 any plugin context information (as it would, e.g., happen when using
89 LoadPlugin instead of LoadBackend in sysdbd).
90 * store:
91 - Introduced service attributes -- similar to host attributes but
92 assigned to a service object.
93 - Don't log “value too old” messages if an updated object uses the same
94 timestamp as the old object.
95 - Switched from linked-lists to AVL trees for storing all object types to
96 reduce lookup times from O(n) to O(log n).
97 - Added infrastructure for handling and evaluating arithmetic
98 expressions and filters (see also the changes to the query language
99 below).
100 - Export various fields describing base attributes of all stored object
101 types: last_update, age, interval, and backend. These fields may be
102 used when querying the store.
103 * frontend: Include new service attributes in JSON output.
104 * query language:
105 - Added support for ‘IS NULL’ attribute checks matching on non-existent
106 attributes.
107 - Changed ‘LOOKUP ... WHERE’ queries to ‘LOOKUP ... MATCHING’ to avoid
108 confusion with SQL semantics.
109 - Switched from ‘<obj>.name’ to ‘<obj>’ in MATCHING clauses, thus, no
110 longer shadowing attributes called “name”.
111 - Added support for arithmetic expressions in places that used simple
112 data values before. Arithmetic expressions support addition,
113 subtraction, multiplication, division, modulo, and concatenation
114 operations. The expressions may be based on constant values or
115 queryable fields of the stored objects (specified as ‘:last_update’,
116 ‘:age’, ‘:interval’, and ‘:backend’).
117 - Added ‘FILTER’ support to ‘LOOKUP’ queries. Filters are conditional
118 expressions which may optionally be used to limit the query and the
119 result to object (of any type) matching the filter condition.
120 - Added support to specify date and time values.
121 * utils:
122 - avltree: Added an AVL tree (self-balancing binary tree) implementation.
123 * documentation: Added documentation for the new query language features.
124 * testing: Run all unit tests through valgrind by default (if available).
126 Backends:
127 * collectd::unixsock:
128 - Added support for spaces in identifiers.
129 - Store plugin identifier (host, plugin, plugin_instance, type,
130 type_instance) as service attributes.
132 Version 0.2.0, Release Date: 2014-07-01
133 ---------------------------------------
135 This is another development snapshot of SysDB. The main focus of this
136 release is on stabilizing existing features and extending the query
137 infrastructure to support all attribute types and further operators.
139 * build system: Fixed linking of sysdbd to include *all* necessary object
140 files. This bug caused some plugins to be un-loadable.
141 * sysdbd: Fixed shutdown process in case of errors.
142 * sysdb: Fixed a bug causing some server replies to be ignored when exiting
143 from the client.
144 * core:
145 - Always log to the standard output stream if no user-provided logging
146 callbacks had been specified. Previously this was disabled after
147 enabling logging to clients in the frontend.
148 - Simplified plugin naming information and determine configuration
149 callback names automatically to ensure better consistency.
150 * store:
151 - Simplified internal data management.
152 - Store the list of backends along with each stored object which provide
153 the respective object.
154 * frontend:
155 - Simplified and improved the internal architecture matching / looking up
156 objects.
157 - Fixed parser to correctly reject invalid object types.
158 - Automatically create the base directory of a listening UNIX socket.
159 * query language: Added support for numeric constants and added operators
160 for less and greater than or equal attribute comparison.
161 * documentation: Added missing files to the distribution tarball.
163 Plugins:
164 * syslog: Don't log debug messages to syslog.
166 Version 0.1.0, Release Date: 2014-06-01
167 ---------------------------------------
169 This is the initial release of SysDB. It provides the following core
170 features:
172 * sysdb: A terminal-based, interactive client program. Supports
173 line-editing and history based on libreadline or libedit and uses an
174 asynchronous, event-driven architecture to handle user input and server
175 replies simultaneously.
176 * sysdbd: The database backend service configurable through a configuration
177 file. The daemon supports online reconfiguration.
178 * core: Flexible, multi-threaded core architecture providing a configurable
179 plugin infrastructure for querying external data-sources and extending
180 core functionality. The following plugin types are supported:
181 - data collection
182 - hostname canonicalization
183 - logging
184 * store: Central, in-memory object store managing host and service objects
185 and their attributes. Host names may be canonicalized through respective
186 plugins. Each object is automatically attributed with last update and
187 interval information using nano-seconds resolution. Attributes may have
188 various types (integer, decimal, string, date-time, binary).
189 * frontend: The frontend handling client connections uses a multi-threaded,
190 asynchronous, event-driven architecture capable of handling multiple
191 listen addresses.
192 * query language: The store may be queried using the ‘LIST’, ‘LOOKUP’, and
193 ‘FETCH’ commands returning JSON formatted objects.
194 * utils: A rich set of abstract data-types and utility functions for
195 internal use:
196 - generic channels for asynchronous, multi-threaded I/O
197 - DBI wrapper to ease common database access operations
198 - thread-aware error handling and reporting
199 - doubly linked-lists
200 - dynamic sized memory buffers supporting formatted strings and binary
201 data
202 - UNIX socket client implementation
203 * documentation: Full documentation for the tools, all plugins, and the
204 query language provided as manpages and HTML pages.
205 * testing: Unit and integration tests exist for most parts except plugins
206 covering 77% of all functions. Valgrind and Clang's address sanitzer are
207 used to detect memory leaks and programming errors.
209 The following plugins are shipped with this version of SysDB:
211 Backends:
212 * collectd::unixsock: Query collectd through its ‘unixsock’ interface
213 * mk-livestatus: Query arbitrary (monitoring) systems using the Check_MK
214 Livestatus interface.
215 * puppet::store-configs: Query Puppet through its “stored configuration”
216 database.
218 Plugins:
219 * cname::dns: Canonicalize hostnames by querying DNS information.
220 * syslog: Send log messages to the system log service.