Same here, fluentd is much better, performance wise.
But then I had to give ES more RAM because it couldn't take the hammering.
In fact, increasing the throughput to ES was causing some pretty spectacular crashes, with the /var/log partition at 100% because of the verbosity of the dumps.
Logstash sucks from both operational and developing perspective. I replaced it everywhere I could by sending
structured logs directly from the app or by using newer integrated beats features.
Comments
ELK stack user here - we actually found logstash to be our bottleneck. Changing it out for fluentd fixed our woes.
Same here, fluentd is much better, performance wise.
But then I had to give ES more RAM because it couldn't take the hammering.
In fact, increasing the throughput to ES was causing some pretty spectacular crashes, with the /var/log partition at 100% because of the verbosity of the dumps.
Logstash sucks from both operational and developing perspective. I replaced it everywhere I could by sending structured logs directly from the app or by using newer integrated beats features.