Over a million developers have joined DZone.

Bringing Big Data Operational Analytics Into the 21st Century

DZone's Guide to

Bringing Big Data Operational Analytics Into the 21st Century

Learn about the role of machine data in mining, analyzing, and filtering machine data for even the most sophisticated operational intelligence campaigns.

· Big Data Zone ·
Free Resource

Hortonworks Sandbox for HDP and HDF is your chance to get started on learning, developing, testing and trying out new features. Each download comes preconfigured with interactive tutorials, sample data and developments from the Apache community.

The term big data was invented nearly 20 years ago. Yet, brands have only recently begun exploring its possibilities. While big data has played a significant role in marketing, there are countless other applications that brands are starting to consider. Operational analytics is the new frontier that they plan to tackle.

Machine Data Is Invaluable to Operational Intelligence

Log management was introduced decades ago when storage capacity was much more finite. The purpose was to record user activities so that system administrators could analyze it at a future date.

Logs would be stored for a few weeks before being overwritten by newer ones. System administrators would manually review logs to see if people were using the resources appropriately and to determine if they required any assistance.

IT staff don’t work with machine data as frequently as they used to. Newer tools offer much more expedient and user-friendly services. However, machine data still plays an important role in many modern applications. In fact, a 2014 study showed that the fastest growing European organizations relied more extensively on machine data than their peers. Here are some organizational applications that it still serves to this day:

  • Monitoring cloud applications that need additional resources to operate (cloud bursting)
  • Identifying inactive or malicious virtual machines
  • Drawing correlations between customer service variables, such as call center volume and customer satisfaction
  • Identifying unknown traffic sources and matching it against their fraud algorithms
  • Ensuring transaction times fall within reasonable limits
  • Optimizing marketing campaigns by studying user devices, time on site and other behavioral data

These objectives wouldn’t be possible without access to machine data at the deepest level. This is where data analysis tools come in.

Finding New Purposes for Machine Data

One of the newest SaaS applications for operational intelligence is Splunk. This platform allows brands to capture, track, and filter operational data in real-time.

Splunk provides more granular insights than most other tools because it collects data at its root. It assimilates machine data via log and text files and these files are subsequently converted into operational intelligence.

This data is stored in repositories that can be easily searched and converted into visual reports. Users can also subscribe to real-time alerts.

Splunk Cloud and Splunk Enterprise offer a wide range of solutions for organizational intelligence, including open library and interactive visualizations. Newer versions of these applications expected to have even more sophisticated features in the future.

Shay Mowlem, Splunk’s VP of Product Marketing and Management, told Forbes:

"As more and more organizations collect, analyze, and retain data at an astounding rate, storage is increasingly becoming the most expensive aspect of data analytics. Long-term data retention is becoming a critical issue as companies grapple with regulatory compliance, security investigations, and the need to better understand long-term business trends. Splunk is passionate about making big data analytics more affordable for organizations of every size. Reducing the cost of historical data retention and analysis is a major part of delivering that value to our customers."

One of the biggest purposes is with social media management. Brands need to mine social media data to optimize their ads, targeting capabilities and customer landing pages. Without access to user data, their success would be built on pure guesswork. Marketing automation tool Campaign Monitor, recently acquired customer data platform Tagga to do just that. By integrating Tagga into the platform, Campaign Monitor can now combine behavioral data such as social engagement and browsing history into a single customer profile to give marketers the ability to take large data sets about customers and send highly personalized content on an individual 1:1 basis.

The Role of Machine Data in Organizational Models

Machine data did not become obsolete with the advance of cloud computing. It still serves a fundamental purpose that brands need to capitalize on. Today’s tools give them the capabilities they need to mine, analyze, and filter machine data for even the most sophisticated operational intelligence campaigns.

Hortonworks Community Connection (HCC) is an online collaboration destination for developers, DevOps, customers and partners to get answers to questions, collaborate on technical articles and share code examples from GitHub.  Join the discussion.

big data ,data analytics ,machine data ,saas ,operational data ,data storage ,data intelligence

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}