Over a million developers have joined DZone.

Big Data Must Focus on Operational Outcome

DZone's Guide to

Big Data Must Focus on Operational Outcome

· Big Data Zone ·
Free Resource

The open source HPCC Systems platform is a proven, easy to use solution for managing data at scale. Visit our Easy Guide to learn more about this completely free platform, test drive some code in the online Playground, and get started today.

Transform 2013 took place in London last week and included an interview between technology maven and raconteur Dennis Howlett and TIBCO CTO Matt Quinn on the topic of Big Data.

When asked his position on Big Data, Quinn started off with the following:

We see the big challenge with Big Data at the moment is that it really doesn’t focus on the operational outcome. And so, we believe that Big Data is a problem of two halves. We see data at rest and data in motion. Data at rest is all of your historical information, your transactional information, but not just in ERP’s. One of the challenges people have had to come to grips with is the fact that Big Data really forces you to think about data as…data…not just in the database. Machine generated data and log files can become just as important as the stuff that’s in the pretty little database.

Howlett interjected that some people say machine data is more important because there’s more of it and you can do more with it. To that point, Quinn added that it’s actually easy to get at. Not always easy to process and deal with it at speed and volume, but nearly every system or device creates log files that offer an opportunity to understand what’s happening at a very granular level.

“What are you going to do about it?”

Quinn gave the following scenario: After analytics have been used to find patterns in machine data, the key is to look at data as it happens through monitoring data in motion. He described a world not where we see reports weeks later about where fraud took place, but instead see data moving as events while fraud is occurring or even is most likely to occur. There are even indicators to the start of a fraud pattern that allow businesses to get a jump on what’s about to happen.

Ultimately, in Quinn’s words, “Challenge number one is very operational, but the second challenge is that Big Data tends to be very domain-specific.” If you look at financial services and capital markets, Big Data has been the reality for decades but their focus has been on data in motion and not so much at rest. At a retailer, on the other hand, the focus has been on data at rest, mostly as transactions. Retailers are now realizing there’s a richness of real-time data that needs to be addressed.

Summing up, Quinn said that when we know what the outcome needs to be, we can deploy a raft of technologies to make the solution happen. Starting with what data an organization has and what they want to do, they can work backward from there to figure out what has to be done. That’s the formula that leads to the modeling exercises that make Big Data operational and valuable.

Dennis Howlett

Dennis Howlett advises software developers about how they need to develop for customer needs and the requirement to understand the narrative in customer satisfaction. He helps developers shape strategy and product, write the occasional piece for Diginomica and other blogs. He also advises professional practitioners on their marketing strategies in a connected world.

Matt Quinn

Matt Quinn has been with TIBCO for 14 years. During this time he has had several worldwide roles leading up his current role of CTO. Quinn works with all product groups to create a common, corporate-wide vision for all of TIBCO’s products and technologies.

Managing data at scale doesn’t have to be hard. Find out how the completely free, open source HPCC Systems platform makes it easier to update, easier to program, easier to integrate data, and easier to manage clusters. Download and get started today.


Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}