Fluentd and Graylog2 can be used for End-to-End Log Analysis.
Fluentd is free and fully open-source log collector which works on technique ‘Log Everything’
Graylog2 is a powerful and currently most popular log analysis framework powered by Elasticsearch and MongoDB.
Fluentd can output data to Graylog2 in the GELF format to take advantage of Graylog2’s analytics and visualization features.
Fluentd is written in Ruby with performance sensitive parts written in C. However, casual users may have difficulty installing and operating a Ruby daemon.
That’s why Treasure Data, Inc is providing the stable community distribution of Fluentd, called td-agent
. The differences between Fluentd and td-agent can be found here.
You can follow the installation guide in fluentd site
But sometimes you might get an error(s) which will creates issues in installing td-agent for “Fluentd with Graylog”
So lets start installing td-agent through the rpm as per the instructions and fix the errors
Download the rpm and plugin and Install through rpm
Then try to start the td-agent service
If its throwing error like `initialize`: Permission denied @ rb_sysopen (Error: EACCES), then check the absolute directory permissions
And change it accordingly
Then try to start the service again
If throwing error for gelf – then Install the module locally after downloading from rubygems site (search for Download link and click on it)
Once done, check and change the permissions/ownership
Then, start the service again
That’s it – You have successfully resolved all the td-agent errors and its now up and running …
Happy Learning