Fluentd is an open source data collector, which lets you unify the data collection and consumption for a better use and understanding of data.
Fluentd is written in Ruby for flexibility, with performance sensitive parts written in C. However, some users may have difficulty installing and operating a Ruby daemon. That’s why Treasure Data, Inc is providing the stable distribution of Fluentd, called td-agent. The differences between Fluentd and td-agent can be found here. This installation guide is for td-agent v3, the new stable version. td-agent v3 uses fluentd v1.0 in the core. See this page for the comparison between v2 and v3.
apt-get install td-agent
List of Directives The configuration file consists of the following directives:
source directives determine the input sources.match directives determine the output destinations.filter directives determine the event processing pipelines.system directives set system wide configuration.label directives group the output and filter for internal
routing@include directives include other files.root@ip-10-7-104-167:/etc/td-agent/conf.d# tail -n50 *
==> 10-source-systemd.conf <==
<source>
@type systemd
matches [{ "_SYSTEMD_UNIT": "jps.service" }, { "_SYSTEMD_UNIT": "atoc-watcher.service" }]
path /var/log/journal
read_from_head true
<storage>
@type local
path /var/run/td-agent/fluentd-journald.json
</storage>
tag jps.systemd
</source>
==> 20-filter-jps-transformer.conf <==
<filter jps.*>
@type record_transformer
<record>
host ip-10-7-104-167
labels_app atoc-watcher
labels_cluster jps-production
namespace_name jps-production
</record>
</filter>
==> 30-match-gelf.conf <==
<match **>
@type gelf
host log-listener.infra.local
port 12201
protocol tcp
@log_level info
flush_interval 5s
max_retry_wait 30
disable_retry_limit true
</match>