1 collectd - System information collection daemon
2 =================================================
3 https://collectd.org/
5 About
6 -----
8 collectd is a small daemon which collects system information periodically
9 and provides mechanisms to store and monitor the values in a variety of
10 ways.
13 Features
14 --------
16 * collectd is able to collect the following data:
18 - apache
19 Apache server utilization: Number of bytes transferred, number of
20 requests handled and detailed scoreboard statistics
22 - apcups
23 APC UPS Daemon: UPS charge, load, input/output/battery voltage, etc.
25 - apple_sensors
26 Sensors in Macs running Mac OS X / Darwin: Temperature, fan speed and
27 voltage sensors.
29 - aquaero
30 Various sensors in the Aquaero 5 water cooling board made by Aquacomputer.
32 - ascent
33 Statistics about Ascent, a free server for the game `World of Warcraft'.
35 - barometer
36 Reads absolute barometric pressure, air pressure reduced to sea level and
37 temperature. Supported sensors are MPL115A2 and MPL3115 from Freescale
38 and BMP085 from Bosch.
40 - battery
41 Batterycharge, -current and voltage of ACPI and PMU based laptop
42 batteries.
44 - bind
45 Name server and resolver statistics from the `statistics-channel'
46 interface of BIND 9.5, 9,6 and later.
48 - ceph
49 Statistics from the Ceph distributed storage system.
51 - cgroups
52 CPU accounting information for process groups under Linux.
54 - chrony
55 Chrony daemon statistics: Local clock drift, offset to peers, etc.
57 - conntrack
58 Number of nf_conntrack entries.
60 - contextswitch
61 Number of context switches done by the operating system.
63 - cpu
64 CPU utilization: Time spent in the system, user, nice, idle, and related
65 states.
67 - cpufreq
68 CPU frequency (For laptops with speed step or a similar technology)
70 - cpusleep
71 CPU sleep: Time spent in suspend (For mobile devices which enter suspend automatically)
73 - curl
74 Parse statistics from websites using regular expressions.
76 - curl_json
77 Retrieves JSON data via cURL and parses it according to user
78 configuration.
80 - curl_xml
81 Retrieves XML data via cURL and parses it according to user
82 configuration.
84 - dbi
85 Executes SQL statements on various databases and interprets the returned
86 data.
88 - df
89 Mountpoint usage (Basically the values `df(1)' delivers)
91 - disk
92 Disk utilization: Sectors read/written, number of read/write actions,
93 average time an IO-operation took to complete.
95 - dns
96 DNS traffic: Query types, response codes, opcodes and traffic/octets
97 transferred.
99 - dpdk
100 Collect DPDK interface statistics.
102 - drbd
103 Collect individual drbd resource statistics.
105 - email
106 Email statistics: Count, traffic, spam scores and checks.
107 See collectd-email(5).
109 - entropy
110 Amount of entropy available to the system.
112 - ethstat
113 Network interface card statistics.
115 - exec
116 Values gathered by a custom program or script.
117 See collectd-exec(5).
119 - fhcount
120 File handles statistics.
122 - filecount
123 Count the number of files in directories.
125 - fscache
126 Linux file-system based caching framework statistics.
128 - gmond
129 Receive multicast traffic from Ganglia instances.
131 - gps
132 Monitor gps related data through gpsd.
134 - grpc
135 Send and receive values over the network using the gRPC framework.
137 - hddtemp
138 Hard disk temperatures using hddtempd.
140 - hugepages
141 Report the number of used and free hugepages. More info on
142 hugepages can be found here:
143 https://www.kernel.org/doc/Documentation/vm/hugetlbpage.txt.
145 - intel_rdt
146 The intel_rdt plugin collects information provided by monitoring features
147 of Intel Resource Director Technology (Intel(R) RDT) like Cache Monitoring
148 Technology (CMT), Memory Bandwidth Monitoring (MBM). These features
149 provide information about utilization of shared resources like last level
150 cache occupancy, local memory bandwidth usage, remote memory bandwidth
151 usage, instructions per clock.
152 <https://01.org/packet-processing/cache-monitoring-technology-memory-bandwidth-monitoring-cache-allocation-technology-code-and-data>
154 - interface
155 Interface traffic: Number of octets, packets and errors for each
156 interface.
158 - ipc
159 IPC counters: semaphores used, number of allocated segments in shared
160 memory and more.
162 - ipmi
163 IPMI (Intelligent Platform Management Interface) sensors information.
165 - iptables
166 Iptables' counters: Number of bytes that were matched by a certain
167 iptables rule.
169 - ipvs
170 IPVS connection statistics (number of connections, octets and packets
171 for each service and destination).
172 See http://www.linuxvirtualserver.org/software/index.html.
174 - irq
175 IRQ counters: Frequency in which certain interrupts occur.
177 - java
178 Integrates a `Java Virtual Machine' (JVM) to execute plugins in Java
179 bytecode. See “Configuring with libjvm” below.
181 - load
182 System load average over the last 1, 5 and 15 minutes.
184 - lpar
185 Detailed CPU statistics of the “Logical Partitions” virtualization
186 technique built into IBM's POWER processors.
188 - lua
189 The Lua plugin implements a Lua interpreter into collectd. This
190 makes it possible to write plugins in Lua which are executed by
191 collectd without the need to start a heavy interpreter every interval.
192 See collectd-lua(5) for details.
194 - lvm
195 Size of “Logical Volumes” (LV) and “Volume Groups” (VG) of Linux'
196 “Logical Volume Manager” (LVM).
198 - madwifi
199 Queries very detailed usage statistics from wireless LAN adapters and
200 interfaces that use the Atheros chipset and the MadWifi driver.
202 - mbmon
203 Motherboard sensors: temperature, fan speed and voltage information,
204 using mbmon(1).
206 - md
207 Linux software-RAID device information (number of active, failed, spare
208 and missing disks).
210 - memcachec
211 Query and parse data from a memcache daemon (memcached).
213 - memcached
214 Statistics of the memcached distributed caching system.
215 <http://www.danga.com/memcached/>
217 - memory
218 Memory utilization: Memory occupied by running processes, page cache,
219 buffer cache and free.
221 - mic
222 Collects CPU usage, memory usage, temperatures and power consumption from
223 Intel Many Integrated Core (MIC) CPUs.
225 - modbus
226 Reads values from Modbus/TCP enabled devices. Supports reading values
227 from multiple "slaves" so gateway devices can be used.
229 - mqtt
230 Publishes and subscribes to MQTT topics.
232 - multimeter
233 Information provided by serial multimeters, such as the `Metex
234 M-4650CR'.
236 - mysql
237 MySQL server statistics: Commands issued, handlers triggered, thread
238 usage, query cache utilization and traffic/octets sent and received.
240 - netapp
241 Plugin to query performance values from a NetApp storage system using the
242 “Manage ONTAP” SDK provided by NetApp.
244 - netlink
245 Very detailed Linux network interface and routing statistics. You can get
246 (detailed) information on interfaces, qdiscs, classes, and, if you can
247 make use of it, filters.
249 - network
250 Receive values that were collected by other hosts. Large setups will
251 want to collect the data on one dedicated machine, and this is the
252 plugin of choice for that.
254 - nfs
255 NFS Procedures: Which NFS command were called how often. Only NFSv2 and
256 NFSv3 right now.
258 - nginx
259 Collects statistics from `nginx' (speak: engine X), a HTTP and mail
260 server/proxy.
262 - ntpd
263 NTP daemon statistics: Local clock drift, offset to peers, etc.
265 - numa
266 Information about Non-Uniform Memory Access (NUMA).
268 - nut
269 Network UPS tools: UPS current, voltage, power, charge, utilisation,
270 temperature, etc. See upsd(8).
272 - olsrd
273 Queries routing information from the “Optimized Link State Routing”
274 daemon.
276 - onewire (EXPERIMENTAL!)
277 Read onewire sensors using the owcapu library of the owfs project.
278 Please read in collectd.conf(5) why this plugin is experimental.
280 - openldap
281 Read monitoring information from OpenLDAP's cn=Monitor subtree.
283 - openvpn
284 RX and TX of each client in openvpn-status.log (status-version 2).
285 <http://openvpn.net/index.php/documentation/howto.html>
287 - oracle
288 Query data from an Oracle database.
290 - perl
291 The perl plugin implements a Perl-interpreter into collectd. You can
292 write your own plugins in Perl and return arbitrary values using this
293 API. See collectd-perl(5).
295 - pf
296 Query statistics from BSD's packet filter "pf".
298 - pinba
299 Receive and dispatch timing values from Pinba, a profiling extension for
300 PHP.
302 - ping
303 Network latency: Time to reach the default gateway or another given
304 host.
306 - postgresql
307 PostgreSQL database statistics: active server connections, transaction
308 numbers, block IO, table row manipulations.
310 - powerdns
311 PowerDNS name server statistics.
313 - processes
314 Process counts: Number of running, sleeping, zombie, ... processes.
316 - protocols
317 Counts various aspects of network protocols such as IP, TCP, UDP, etc.
319 - python
320 The python plugin implements a Python interpreter into collectd. This
321 makes it possible to write plugins in Python which are executed by
322 collectd without the need to start a heavy interpreter every interval.
323 See collectd-python(5) for details.
325 - redis
326 The redis plugin gathers information from a Redis server, including:
327 uptime, used memory, total connections etc.
329 - routeros
330 Query interface and wireless registration statistics from RouterOS.
332 - rrdcached
333 RRDtool caching daemon (RRDcacheD) statistics.
335 - sensors
336 System sensors, accessed using lm_sensors: Voltages, temperatures and
337 fan rotation speeds.
339 - serial
340 RX and TX of serial interfaces. Linux only; needs root privileges.
342 - sigrok
343 Uses libsigrok as a backend, allowing any sigrok-supported device
344 to have its measurements fed to collectd. This includes multimeters,
345 sound level meters, thermometers, and much more.
347 - smart
348 Collect SMART statistics, notably load cycle count, temperature
349 and bad sectors.
351 - snmp
352 Read values from SNMP (Simple Network Management Protocol) enabled
353 network devices such as switches, routers, thermometers, rack monitoring
354 servers, etc. See collectd-snmp(5).
356 - statsd
357 Acts as a StatsD server, reading values sent over the network from StatsD
358 clients and calculating rates and other aggregates out of these values.
360 - swap
361 Pages swapped out onto hard disk or whatever is called `swap' by the OS..
363 - table
364 Parse table-like structured files.
366 - tail
367 Follows (tails) log files, parses them by lines and submits matched
368 values.
370 - tail_csv
371 Follows (tails) files in CSV format, parses each line and submits
372 extracted values.
374 - tape
375 Bytes and operations read and written on tape devices. Solaris only.
377 - tcpconns
378 Number of TCP connections to specific local and remote ports.
380 - teamspeak2
381 TeamSpeak2 server statistics.
383 - ted
384 Plugin to read values from `The Energy Detective' (TED).
386 - thermal
387 Linux ACPI thermal zone information.
389 - tokyotyrant
390 Reads the number of records and file size from a running Tokyo Tyrant
391 server.
393 - turbostat
394 Reads CPU frequency and C-state residency on modern Intel
395 turbo-capable processors.
397 - uptime
398 System uptime statistics.
400 - users
401 Users currently logged in.
403 - varnish
404 Various statistics from Varnish, an HTTP accelerator.
406 - virt
407 CPU, memory, disk and network I/O statistics from virtual machines.
409 - vmem
410 Virtual memory statistics, e. g. the number of page-ins/-outs or the
411 number of pagefaults.
413 - vserver
414 System resources used by Linux VServers.
415 See <http://linux-vserver.org/>.
417 - wireless
418 Link quality of wireless cards. Linux only.
420 - xencpu
421 XEN Hypervisor CPU stats.
423 - xmms
424 Bitrate and frequency of music played with XMMS.
426 - zfs_arc
427 Statistics for ZFS' “Adaptive Replacement Cache” (ARC).
429 - zone
430 Measures the percentage of cpu load per container (zone) under Solaris 10
431 and higher
433 - zookeeper
434 Read data from Zookeeper's MNTR command.
436 * Output can be written or sent to various destinations by the following
437 plugins:
439 - amqp
440 Sends JSON-encoded data to an Advanced Message Queuing Protocol (AMQP)
441 server, such as RabbitMQ.
443 - csv
444 Write to comma separated values (CSV) files. This needs lots of
445 diskspace but is extremely portable and can be analysed with almost
446 every program that can analyse anything. Even Microsoft's Excel..
448 - lua
449 It's possible to implement write plugins in Lua using the Lua
450 plugin. See collectd-lua(5) for details.
452 - network
453 Send the data to a remote host to save the data somehow. This is useful
454 for large setups where the data should be saved by a dedicated machine.
456 - perl
457 Of course the values are propagated to plugins written in Perl, too, so
458 you can easily do weird stuff with the plugins we didn't dare think of
459 ;) See collectd-perl(5).
461 - python
462 It's possible to implement write plugins in Python using the python
463 plugin. See collectd-python(5) for details.
465 - rrdcached
466 Output to round-robin-database (RRD) files using the RRDtool caching
467 daemon (RRDcacheD) - see rrdcached(1). That daemon provides a general
468 implementation of the caching done by the `rrdtool' plugin.
470 - rrdtool
471 Output to round-robin-database (RRD) files using librrd. See rrdtool(1).
472 This is likely the most popular destination for such values. Since
473 updates to RRD-files are somewhat expensive this plugin can cache
474 updates to the files and write a bunch of updates at once, which lessens
475 system load a lot.
477 - unixsock
478 One can query the values from the unixsock plugin whenever they're
479 needed. Please read collectd-unixsock(5) for a description on how that's
480 done.
482 - write_graphite
483 Sends data to Carbon, the storage layer of Graphite using TCP or UDP. It
484 can be configured to avoid logging send errors (especially useful when
485 using UDP).
487 - write_http
488 Sends the values collected by collectd to a web-server using HTTP POST
489 requests. The transmitted data is either in a form understood by the
490 Exec plugin or formatted in JSON.
492 - write_kafka
493 Sends data to Apache Kafka, a distributed queue.
495 - write_log
496 Writes data to the log
498 - write_mongodb
499 Sends data to MongoDB, a NoSQL database.
501 - write_redis
502 Sends the values to a Redis key-value database server.
504 - write_riemann
505 Sends data to Riemann, a stream processing and monitoring system.
507 - write_sensu
508 Sends data to Sensu, a stream processing and monitoring system, via the
509 Sensu client local TCP socket.
511 - write_tsdb
512 Sends data OpenTSDB, a scalable no master, no shared state time series
513 database.
515 * Logging is, as everything in collectd, provided by plugins. The following
516 plugins keep us informed about what's going on:
518 - logfile
519 Writes log messages to a file or STDOUT/STDERR.
521 - perl
522 Log messages are propagated to plugins written in Perl as well.
523 See collectd-perl(5).
525 - python
526 It's possible to implement log plugins in Python using the python plugin.
527 See collectd-python(5) for details.
529 - syslog
530 Logs to the standard UNIX logging mechanism, syslog.
532 - log_logstash
533 Writes log messages formatted as logstash JSON events.
535 * Notifications can be handled by the following plugins:
537 - notify_desktop
538 Send a desktop notification to a notification daemon, as defined in
539 the Desktop Notification Specification. To actually display the
540 notifications, notification-daemon is required.
541 See http://www.galago-project.org/specs/notification/.
543 - notify_email
544 Send an E-mail with the notification message to the configured
545 recipients.
547 - notify_nagios
548 Submit notifications as passive check results to a local nagios instance.
550 - exec
551 Execute a program or script to handle the notification.
552 See collectd-exec(5).
554 - logfile
555 Writes the notification message to a file or STDOUT/STDERR.
557 - network
558 Send the notification to a remote host to handle it somehow.
560 - perl
561 Notifications are propagated to plugins written in Perl as well.
562 See collectd-perl(5).
564 - python
565 It's possible to implement notification plugins in Python using the
566 python plugin. See collectd-python(5) for details.
568 * Value processing can be controlled using the "filter chain" infrastructure
569 and "matches" and "targets". The following plugins are available:
571 - match_empty_counter
572 Match counter values which are currently zero.
574 - match_hashed
575 Match values using a hash function of the hostname.
577 - match_regex
578 Match values by their identifier based on regular expressions.
580 - match_timediff
581 Match values with an invalid timestamp.
583 - match_value
584 Select values by their data sources' values.
586 - target_notification
587 Create and dispatch a notification.
589 - target_replace
590 Replace parts of an identifier using regular expressions.
592 - target_scale
593 Scale (multiply) values by an arbitrary value.
595 - target_set
596 Set (overwrite) entire parts of an identifier.
598 * Miscellaneous plugins:
600 - aggregation
601 Selects multiple value lists based on patterns or regular expressions
602 and creates new aggregated values lists from those.
604 - threshold
605 Checks values against configured thresholds and creates notifications if
606 values are out of bounds. See collectd-threshold(5) for details.
608 - uuid
609 Sets the hostname to a unique identifier. This is meant for setups
610 where each client may migrate to another physical host, possibly going
611 through one or more name changes in the process.
613 * Performance: Since collectd is running as a daemon it doesn't spend much
614 time starting up again and again. With the exception of the exec plugin no
615 processes are forked. Caching in output plugins, such as the rrdtool and
616 network plugins, makes sure your resources are used efficiently. Also,
617 since collectd is programmed multithreaded it benefits from hyper-threading
618 and multicore processors and makes sure that the daemon isn't idle if only
619 one plugin waits for an IO-operation to complete.
621 * Once set up, hardly any maintenance is necessary. Setup is kept as easy
622 as possible and the default values should be okay for most users.
625 Operation
626 ---------
628 * collectd's configuration file can be found at `sysconfdir'/collectd.conf.
629 Run `collectd -h' for a list of built-in defaults. See `collectd.conf(5)'
630 for a list of options and a syntax description.
632 * When the `csv' or `rrdtool' plugins are loaded they'll write the values to
633 files. The usual place for these files is beneath `/var/lib/collectd'.
635 * When using some of the plugins, collectd needs to run as user root, since
636 only root can do certain things, such as craft ICMP packages needed to ping
637 other hosts. collectd should NOT be installed setuid root since it can be
638 used to overwrite valuable files!
640 * Sample scripts to generate graphs reside in `contrib/' in the source
641 package or somewhere near `/usr/share/doc/collectd' in most distributions.
642 Please be aware that those script are meant as a starting point for your
643 own experiments.. Some of them require the `RRDs' Perl module.
644 (`librrds-perl' on Debian) If you have written a more sophisticated
645 solution please share it with us.
647 * The RRAs of the automatically created RRD files depend on the `step'
648 and `heartbeat' settings given. If change these settings you may need to
649 re-create the files, losing all data. Please be aware of that when changing
650 the values and read the rrdtool(1) manpage thoroughly.
653 collectd and chkrootkit
654 -----------------------
656 If you are using the `dns' plugin chkrootkit(1) will report collectd as a
657 packet sniffer ("<iface>: PACKET SNIFFER(/usr/sbin/collectd[<pid>])"). The
658 plugin captures all UDP packets on port 53 to analyze the DNS traffic. In
659 this case, collectd is a legitimate sniffer and the report should be
660 considered to be a false positive. However, you might want to check that
661 this really is collectd and not some other, illegitimate sniffer.
664 Prerequisites
665 -------------
667 To compile collectd from source you will need:
669 * Usual suspects: C compiler, linker, preprocessor, make, ...
671 * A POSIX-threads (pthread) implementation.
672 Since gathering some statistics is slow (network connections, slow devices,
673 etc) collectd is parallelized. The POSIX threads interface is being
674 used and should be found in various implementations for hopefully all
675 platforms.
677 * aerotools-ng (optional)
678 Used by the `aquaero' plugin. Currently, the `libaquaero5' library, which
679 is used by the `aerotools-ng' toolkit, is not compiled as a shared object
680 nor does it feature an installation routine. Therefore, you need to point
681 collectd's configure script at the source directory of the `aerotools-ng'
682 project.
683 <https://github.com/lynix/aerotools-ng>
685 * CoreFoundation.framework and IOKit.framework (optional)
686 For compiling on Darwin in general and the `apple_sensors' plugin in
687 particular.
688 <http://developer.apple.com/corefoundation/>
690 * libatasmart (optional)
691 Used by the `smart' plugin.
692 <http://git.0pointer.de/?p=libatasmart.git>
694 * libcap (optional)
695 The `turbostat' plugin can optionally build Linux Capabilities support,
696 which avoids full privileges requirement (aka. running as root) to read
697 values.
698 <http://sites.google.com/site/fullycapable/>
700 * libclntsh (optional)
701 Used by the `oracle' plugin.
703 * libhiredis (optional)
704 Used by the redis plugin. Please note that you require a 0.10.0 version
705 or higher. <https://github.com/redis/hiredis>
707 * libcurl (optional)
708 If you want to use the `apache', `ascent', `bind', `curl', `curl_json',
709 `curl_xml', `nginx', or `write_http' plugin.
710 <http://curl.haxx.se/>
712 * libdbi (optional)
713 Used by the `dbi' plugin to connect to various databases.
714 <http://libdbi.sourceforge.net/>
716 * libesmtp (optional)
717 For the `notify_email' plugin.
718 <http://www.stafford.uklinux.net/libesmtp/>
720 * libganglia (optional)
721 Used by the `gmond' plugin to process data received from Ganglia.
722 <http://ganglia.info/>
724 * libgrpc (optional)
725 Used by the `grpc' plugin. gRPC requires a C++ compiler supporting the
726 C++11 standard.
727 <https://grpc.io/>
729 * libgcrypt (optional)
730 Used by the `network' plugin for encryption and authentication.
731 <http://www.gnupg.org/>
733 * libgps (optional)
734 Used by the `gps' plugin.
735 <http://developer.berlios.de/projects/gpsd/>
737 * libhal (optional)
738 If present, the `uuid' plugin will check for UUID from HAL.
739 <http://hal.freedesktop.org/>
741 * libi2c-dev (optional)
742 Used for the plugin `barometer', provides just the i2c-dev.h header file
743 for user space i2c development.
745 * libiptc (optional)
746 For querying iptables counters.
747 <http://netfilter.org/>
749 * libjvm (optional)
750 Library that encapsulates the `Java Virtual Machine' (JVM). This library is
751 used by the `java' plugin to execute Java bytecode. See “Configuring with
752 libjvm” below.
753 <http://openjdk.java.net/> (and others)
755 * libldap (optional)
756 Used by the `openldap' plugin.
757 <http://www.openldap.org/>
759 * liblua (optional)
760 Used by the `lua' plugin. Currently, Lua 5.1 and later are supported.
761 <https://www.lua.org/>
763 * liblvm2 (optional)
764 Used by the `lvm' plugin.
765 <ftp://sources.redhat.com/pub/lvm2/>
767 * libmemcached (optional)
768 Used by the `memcachec' plugin to connect to a memcache daemon.
769 <http://tangent.org/552/libmemcached.html>
771 * libmnl (optional)
772 Used by the `netlink' plugin.
773 <http://www.netfilter.org/projects/libmnl/>
775 * libmodbus (optional)
776 Used by the `modbus' plugin to communicate with Modbus/TCP devices. The
777 `modbus' plugin works with version 2.0.3 of the library – due to frequent
778 API changes other versions may or may not compile cleanly.
779 <http://www.libmodbus.org/>
781 * libmysqlclient (optional)
782 Unsurprisingly used by the `mysql' plugin.
783 <http://dev.mysql.com/>
785 * libnetapp (optional)
786 Required for the `netapp' plugin.
787 This library is part of the “Manage ONTAP SDK” published by NetApp.
789 * libnetsnmp (optional)
790 For the `snmp' plugin.
791 <http://www.net-snmp.org/>
793 * libnotify (optional)
794 For the `notify_desktop' plugin.
795 <http://www.galago-project.org/>
797 * libopenipmi (optional)
798 Used by the `ipmi' plugin to prove IPMI devices.
799 <http://openipmi.sourceforge.net/>
801 * liboping (optional)
802 Used by the `ping' plugin to send and receive ICMP packets.
803 <http://octo.it/liboping/>
805 * libowcapi (optional)
806 Used by the `onewire' plugin to read values from onewire sensors (or the
807 owserver(1) daemon).
808 <http://www.owfs.org/>
810 * libpcap (optional)
811 Used to capture packets by the `dns' plugin.
812 <http://www.tcpdump.org/>
814 * libperfstat (optional)
815 Used by various plugins to gather statistics under AIX.
817 * libperl (optional)
818 Obviously used by the `perl' plugin. The library has to be compiled with
819 ithread support (introduced in Perl 5.6.0).
820 <http://www.perl.org/>
822 * libpq (optional)
823 The PostgreSQL C client library used by the `postgresql' plugin.
824 <http://www.postgresql.org/>
826 * libpqos (optional)
827 The PQoS library for Intel(R) Resource Director Technology used by the
828 `intel_rdt' plugin.
829 <https://github.com/01org/intel-cmt-cat>
831 * libprotobuf, protoc 3.0+ (optional)
832 Used by the `grpc' plugin to generate service stubs and code to handle
833 network packets of collectd's protobuf-based network protocol.
834 <https://developers.google.com/protocol-buffers/>
836 * libprotobuf-c, protoc-c (optional)
837 Used by the `pinba' plugin to generate a parser for the network packets
838 sent by the Pinba PHP extension.
839 <http://code.google.com/p/protobuf-c/>
841 * libpython (optional)
842 Used by the `python' plugin. Currently, Python 2.6 and later and Python 3
843 are supported.
844 <http://www.python.org/>
846 * librabbitmq (optional; also called “rabbitmq-c”)
847 Used by the `amqp' plugin for AMQP connections, for example to RabbitMQ.
848 <http://hg.rabbitmq.com/rabbitmq-c/>
850 * librdkafka (optional; also called “rdkafka”)
851 Used by the `write_kafka' plugin for producing messages and sending them
852 to a Kafka broker.
853 <https://github.com/edenhill/librdkafka>
855 * librouteros (optional)
856 Used by the `routeros' plugin to connect to a device running `RouterOS'.
857 <http://octo.it/librouteros/>
859 * librrd (optional)
860 Used by the `rrdtool' and `rrdcached' plugins. The latter requires RRDtool
861 client support which was added after version 1.3 of RRDtool. Versions 1.0,
862 1.2 and 1.3 are known to work with the `rrdtool' plugin.
863 <http://oss.oetiker.ch/rrdtool/>
865 * librt, libsocket, libkstat, libdevinfo (optional)
866 Various standard Solaris libraries which provide system functions.
867 <http://developers.sun.com/solaris/>
869 * libsensors (optional)
870 To read from `lm_sensors', see the `sensors' plugin.
871 <http://www.lm-sensors.org/>
873 * libsigrok (optional)
874 Used by the `sigrok' plugin. In addition, libsigrok depends on glib,
875 libzip, and optionally (depending on which drivers are enabled) on
876 libusb, libftdi and libudev.
878 * libstatgrab (optional)
879 Used by various plugins to collect statistics on systems other than Linux
880 and/or Solaris.
881 <http://www.i-scream.org/libstatgrab/>
883 * libtokyotyrant (optional)
884 Used by the `tokyotyrant' plugin.
885 <http://1978th.net/tokyotyrant/>
887 * libupsclient/nut (optional)
888 For the `nut' plugin which queries nut's `upsd'.
889 <http://networkupstools.org/>
891 * libvirt (optional)
892 Collect statistics from virtual machines.
893 <http://libvirt.org/>
895 * libxml2 (optional)
896 Parse XML data. This is needed for the `ascent', `bind', `curl_xml' and
897 `virt' plugins.
898 <http://xmlsoft.org/>
900 * libxen (optional)
901 Used by the `xencpu' plugin.
902 <http://xenbits.xensource.com/>
904 * libxmms (optional)
905 <http://www.xmms.org/>
907 * libyajl (optional)
908 Parse JSON data. This is needed for the `ceph', `curl_json' and
909 `log_logstash' plugins.
910 <http://github.com/lloyd/yajl>
912 * libvarnish (optional)
913 Fetches statistics from a Varnish instance. This is needed for the
914 `varnish' plugin.
915 <http://varnish-cache.org>
917 * riemann-c-client (optional)
918 For the `write_riemann' plugin.
919 <https://github.com/algernon/riemann-c-client>
921 Configuring / Compiling / Installing
922 ------------------------------------
924 To configure, build and install collectd with the default settings, run
925 `./configure && make && make install'. For detailed, generic instructions
926 see INSTALL. For a complete list of configure options and their description,
927 run `./configure --help'.
929 By default, the configure script will check for all build dependencies and
930 disable all plugins whose requirements cannot be fulfilled (any other plugin
931 will be enabled). To enable a plugin, install missing dependencies (see
932 section `Prerequisites' above) and rerun `configure'. If you specify the
933 `--enable-<plugin>' configure option, the script will fail if the depen-
934 dencies for the specified plugin are not met. In that case you can force the
935 plugin to be built using the `--enable-<plugin>=force' configure option.
936 This will most likely fail though unless you're working in a very unusual
937 setup and you really know what you're doing. If you specify the
938 `--disable-<plugin>' configure option, the plugin will not be built. If you
939 specify the `--enable-all-plugins' or `--disable-all-plugins' configure
940 options, all plugins will be enabled or disabled respectively by default.
941 Explicitly enabling or disabling a plugin overwrites the default for the
942 specified plugin. These options are meant for package maintainers and should
943 not be used in everyday situations.
945 By default, collectd will be installed into `/opt/collectd'. You can adjust
946 this setting by specifying the `--prefix' configure option - see INSTALL for
947 details. If you pass DESTDIR=<path> to `make install', <path> will be
948 prefixed to all installation directories. This might be useful when creating
949 packages for collectd.
951 Configuring with libjvm
952 -----------------------
954 To determine the location of the required files of a Java installation is not
955 an easy task, because the locations vary with your kernel (Linux, SunOS, …)
956 and with your architecture (x86, SPARC, …) and there is no ‘java-config’
957 script we could use. Configuration of the JVM library is therefore a bit
958 tricky.
960 The easiest way to use the `--with-java=$JAVA_HOME' option, where
961 `$JAVA_HOME' is usually something like:
962 /usr/lib/jvm/java-1.5.0-sun-1.5.0.14
964 The configure script will then use find(1) to look for the following files:
966 - jni.h
967 - jni_md.h
968 - libjvm.so
970 If found, appropriate CPP-flags and LD-flags are set and the following
971 library checks succeed.
973 If this doesn't work for you, you have the possibility to specify CPP-flags,
974 C-flags, LD-flags and LIBS for the ‘Java’ plugin by hand, using the
975 following environment variables:
977 - JAVA_CPPFLAGS
978 - JAVA_CFLAGS
979 - JAVA_LDFLAGS
980 - JAVA_LIBS
982 For example (shortened for demonstration purposes):
984 ./configure JAVA_CPPFLAGS="-I$JAVA_HOME/include -I$JAVA_HOME/include/linux"
986 Adding "-ljvm" to JAVA_LIBS is done automatically, you don't have to
987 do that.
989 Generating the configure script
990 -------------------------------
992 Collectd ships with a `build.sh' script to generate the `configure'
993 script shipped with releases.
995 To generate the `configure` script, you'll need the following dependencies:
997 - autoconf
998 - automake
999 - flex
1000 - bison
1001 - libtool
1002 - libtool-ltdl
1003 - pkg-config
1005 The `build.sh' script takes no arguments.
1007 Crosscompiling
1008 --------------
1010 To compile correctly collectd needs to be able to initialize static
1011 variables to NAN (Not A Number). Some C libraries, especially the GNU
1012 libc, have a problem with that.
1014 Luckily, with GCC it's possible to work around that problem: One can define
1015 NAN as being (0.0 / 0.0) and `isnan' as `f != f'. However, to test this
1016 ``implementation'' the configure script needs to compile and run a short
1017 test program. Obviously running a test program when doing a cross-
1018 compilation is, well, challenging.
1020 If you run into this problem, you can use the `--with-nan-emulation'
1021 configure option to force the use of this implementation. We can't promise
1022 that the compiled binary actually behaves as it should, but since NANs
1023 are likely never passed to the libm you have a good chance to be lucky.
1025 Likewise, collectd needs to know the layout of doubles in memory, in order
1026 to craft uniform network packets over different architectures. For this, it
1027 needs to know how to convert doubles into the memory layout used by x86. The
1028 configure script tries to figure this out by compiling and running a few
1029 small test programs. This is of course not possible when cross-compiling.
1030 You can use the `--with-fp-layout' option to tell the configure script which
1031 conversion method to assume. Valid arguments are:
1033 * `nothing' (12345678 -> 12345678)
1034 * `endianflip' (12345678 -> 87654321)
1035 * `intswap' (12345678 -> 56781234)
1037 Configuring with DPDK
1038 ---------------------
1040 Note: DPDK 16.04 is the minimum version and currently supported version of DPDK
1041 required for the dpdkstat plugin. This is to allow the plugin to take advantage
1042 of functions added to detect if the DPDK primary process is alive.
1044 Note: For Ubuntu, GCC 4.9 is the minimum version required to build collectd
1045 with DPDK. Ubuntu 14.04, for example, has GCC 4.8 by default and will require
1046 an upgrade:
1047 $ sudo add-apt-repository ppa:ubuntu-toolchain-r/test
1048 $ sudo apt-get update
1049 $ sudo apt-get install gcc-4.9
1050 Alternatively, if you know that the platform that you wish to run collectd
1051 on supports the SSSE3 instruction set, you can run make as follows:
1052 $ make -j CFLAGS+='-mssse3'
1054 Build DPDK for use with collectd:
1055 To compile DPDK for use with collectd dpdkstat start by:
1056 - Clone DPDK:
1057 $ git clone git://dpdk.org/dpdk
1058 - Checkout the system requirements at
1059 http://dpdk.org/doc/guides/linux_gsg/sys_reqs.html and make sure you have
1060 the required tools and hugepage setup as specified there.
1061 NOTE: It's recommended to use the 1GB hugepage setup for best performance,
1062 please follow the instruction for "Reserving Hugepages for DPDK Use"
1063 in the link above.
1064 However if you plan on configuring 2MB hugepages on the fly please ensure
1065 to add appropriate commands to reserve hugepages in a system startup script
1066 if collectd is booted at system startup time. These commands include:
1067 mkdir -p /mnt/huge
1068 mount -t hugetlbfs nodev /mnt/huge
1069 echo 64 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages
1070 - To configure the DPDK build for the combined shared library modify
1071 config/common_base in your DPDK as follows
1072 #
1073 # Compile to share library
1074 #
1075 -CONFIG_RTE_BUILD_SHARED_LIB=n
1076 +CONFIG_RTE_BUILD_SHARED_LIB=y
1077 - Prepare the configuration for the appropriate target as specified at:
1078 http://dpdk.org/doc/guides/linux_gsg/build_dpdk.html.
1079 For example:
1080 $ make config T=x86_64-native-linuxapp-gcc
1081 - Build the target:
1082 $ make
1083 - Install DPDK to /usr
1084 $ sudo make install prefix=/usr
1085 NOTE 1: You must run make install as the configuration of collectd with
1086 DPDK expects DPDK to be installed somewhere.
1087 NOTE 2: If you don't specify a prefix then DPDK will be installed in /usr/local/
1088 NOTE 3: If you are not root then use sudo to make install DPDK to the appropriate
1089 location.
1090 - Check that the DPDK library has been installed in /usr/lib or /lib
1091 $ ls /usr/lib | grep dpdk
1092 - Bind the interfaces to use with dpdkstat to DPDK:
1093 DPDK devices can be setup with either the VFIO (for DPDK 1.7+) or UIO modules.
1094 Note: UIO requires inserting an out of tree driver igb_uio.ko that is available
1095 in DPDK.
1096 UIO Setup:
1097 - Insert uio.ko:
1098 $ sudo modprobe uio
1099 - Insert igb_uio.ko:
1100 $ sudo insmod $DPDK_BUILD/kmod/igb_uio.ko
1101 - Bind network device to igb_uio:
1102 $ sudo $DPDK_DIR/tools/dpdk_nic_bind.py --bind=igb_uio eth1
1103 VFIO Setup:
1104 - VFIO needs to be supported in the kernel and the BIOS. More information can be found
1105 @ http://dpdk.org/doc/guides/linux_gsg/build_dpdk.html.
1106 - Insert the `vfio-pci.ko' module: modprobe vfio-pci
1107 - Set the correct permissions for the vfio device:
1108 $ sudo /usr/bin/chmod a+x /dev/vfio
1109 $ sudo /usr/bin/chmod 0666 /dev/vfio/*
1110 - Bind the network device to vfio-pci:
1111 $ sudo $DPDK_DIR/tools/dpdk_nic_bind.py --bind=vfio-pci eth1
1112 NOTE: Please ensure to add appropriate commands to bind the network
1113 interfaces to DPDK in a system startup script if collectd is
1114 booted at system startup time.
1115 - Run ldconfig to update the shared library cache.
1117 Build static DPDK library for use with collectd:
1118 - To configure DPDK to build the combined static library libdpdk.a
1119 ensure that CONFIG_RTE_BUILD_SHARED_LIB is set to n in
1120 config/common_base in your DPDK as follows:
1121 #
1122 # Compile to share library
1123 #
1124 CONFIG_RTE_BUILD_SHARED_LIB=n
1125 - Prepare the configuration for the appropriate target as specified at:
1126 http://dpdk.org/doc/guides/linux_gsg/build_dpdk.html.
1127 For example:
1128 $ make config T=x86_64-native-linuxapp-gcc
1129 - Build the target using -fPIC:
1130 $ make EXTRA_CFLAGS=-fPIC -j
1131 - Install DPDK to /usr
1132 $ sudo make install prefix=/usr
1134 Configure collectd with DPDK:
1135 NOTE: The Address-Space Layout Randomization (ASLR) security feature in Linux should
1136 be disabled, in order for the same hugepage memory mappings to be present in all
1137 DPDK multi-process applications. Note that this has security implications.
1138 To disable ASLR:
1139 $ echo 0 > /proc/sys/kernel/randomize_va_space
1140 To fully enable ASLR:
1141 $ echo 2 > /proc/sys/kernel/randomize_va_space
1142 See http://dpdk.org/doc/guides/prog_guide/multi_proc_support.html
1144 - Generate the build script as specified below. (i.e. run `build.sh').
1145 - Configure collectd with the DPDK shared library:
1146 ./configure --with-libdpdk=/usr
1147 NOTE: To configure collectd with the DPDK static library:
1148 ./configure --with-libdpdk=/usr CFLAGS=" -lpthread -Wl,--whole-archive
1149 -Wl,-ldpdk -Wl,-lm -Wl,-lrt -Wl,-lpcap -Wl,-ldl -Wl,--no-whole-archive"
1151 Libraries:
1152 ...
1153 libdpdk . . . . . . . . yes
1155 Modules:
1156 ...
1157 dpdkstat . . . . . . .yes
1160 - Make sure that dpdk and dpdkstat are enabled in the configuration log:
1162 - Build collectd:
1163 $ make -j && make -j install.
1164 NOTE: As mentioned above, if you are building on Ubuntu 14.04 with GCC <= 4.8.X,
1165 you need to use:
1166 $ make -j CFLAGS+='-mssse3' && make -j install
1168 Usage of dpdkstat:
1169 - The same PCI device configuration should be passed to the primary process
1170 as the secondary process uses the same port indexes as the primary.
1171 NOTE: A blacklist/whitelist of NICs isn't supported yet.
1173 Contact
1174 -------
1176 For questions, bug reports, development information and basically all other
1177 concerns please send an email to collectd's mailing list at
1178 <list at collectd.org>.
1180 For live discussion and more personal contact visit us in IRC, we're in
1181 channel #collectd on freenode.
1184 Author
1185 ------
1187 Florian octo Forster <octo at collectd.org>,
1188 Sebastian tokkee Harl <sh at tokkee.org>,
1189 and many contributors (see `AUTHORS').
1191 Please use GitHub reporting bugs and submitting pull requests.
1192 See CONTRIBUTING.md for details.