Over a million developers have joined DZone.

Amazon Recommends Fluentd as Best Practice for Data Collection over Flume and Scribe

· Cloud Zone

Download the Essential Cloud Buyer’s Guide to learn important factors to consider before selecting a provider as well as buying criteria to help you make the best decision for your infrastructure needs, brought to you in partnership with Internap.

This month, Parviz Deyham from Amazon Web Services promoted Fluentd as the best data collection tool for Amazon Elastic MapReduce (EMR), a hosted Hadoop framework running on Amazon Elastic Compute Cloud (EC2) and Amazon Simple Storage Service (S3).

In the best practices whitepaper, Parviz, an Enterpise Solution Architect at AWS, notes that "Fluentd is easier to install and maintain and has better documentation and support than Flume and Scribe." Collecting data in a scalable and reliable manner has an important place in big data architecture. Many big data analytics solutions fail to provide robust tools for data collection or require that developers write custom data collectors from the origin to the final collection point.

While such attempts to write custom data collectors are important, users can leverage open source frameworks that have already been written to provide scalable and efficient distributed data collection. Open source software is part of our DNA at Treasure Data and we are thrilled that Parviz sees the value in a versatile and lightweight data collection tool like Fluentd to stream data efficiently to the cloud.

We would like to thank the entire Fluentd community for their dedication and contributions to making Fluentd a first class data collection tool. This recommendation from Amazon validates the philosophy and benefits of Fluentd’s architecture, maintainability, and simplicity.

The Cloud Zone is brought to you in partnership with Internap. Read Bare-Metal Cloud 101 to learn about bare-metal cloud and how it has emerged as a way to complement virtualized services.


Published at DZone with permission of Sadayuki Furuhashi, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

The best of DZone straight to your inbox.

Please provide a valid email address.

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}