Using ElasticSearch for Big Data Analysis
Learn how leveraging ElasticSearch to build the data infrastructure makes it easier to linearly scale as new data nodes are added in the future.
Join the DZone community and get the full member experience.Join For Free
Collecting data is good and collecting Big Data is better, but analyzing Big Data is not easy. It requires knowledge of enterprise search engines for making content from different sources like enterprise databases, social media, sensor data, etc. searchable to a defined audience. ElasticSearch, Apache Solr, and Sphinx are some of the free and open-source enterprise search software.
While applications of ElasticSearch have advanced to Machine Learning, in this blog, I have tried to summarize a simple use case for Big Data analysis using ElasticSearch. ElasticSearch recently added Machine Learning algorithms to its enterprise stack for the purpose of finding anomalies in time-series log data.
Key Benefits of ElasticSearch Implementation
ElasticSearch is used for web search, log analysis, and Big Data analytics. ElasticSearch is more popular because it is easy to install, it scales out to hundreds of nodes with no additional software, and it is easy to work with due to its built-in REST API. Following is the summary of key benefits; for a detailed account, check out this article.
Ease of indexing.
How to Leverage ElasticSearch for Big Data Analysis
At Maruti Techlabs, we have a challenge of combining structured and unstructured data for big data analysis. We want to improve the user experience in searching data of used car parts for our client based in Austin, Texas. A potential customer can find used parts for his car on this portal. A huge amount of data (around 42 million) affects the usability of the system performance and query response time. We have designed a solution that brings Big Data stores and traditional RDBMS toolsets to a single, unified data analytics platform that enables analysts to access any amount of data of any type for any analysis at any time. Both
Published at DZone with permission of Rohit Akiwatkar. See the original article here.
Opinions expressed by DZone contributors are their own.