Over a million developers have joined DZone.

Process is Data and Data is Process

DZone's Guide to

Process is Data and Data is Process

· Big Data Zone ·
Free Resource

The open source HPCC Systems platform is a proven, easy to use solution for managing data at scale. Visit our Easy Guide to learn more about this completely free platform, test drive some code in the online Playground, and get started today.

We’re a funny lot. For years we’ve been talking about data and process as though they were two different things. Guess what … they’re not. Process involves the activities of getting work done and data both describes process and is also the beginning and end points of work. We consume data in processes and we produce data in processes. Data kicks off a process and data ends a process. They are inseparable.

Wait a minute … does that mean digging a hole in the ground is somehow data? Does it consume data and produce data? Does data kick it off and end its process? Absolutely. The timing, location, size, depth, rationale and value of having that hole may not have been digitally captured in the past, but every factor that goes into producing and using that hole is absolutely in the realm of data. It always was.

Where Big Data Factors in

What’s happening, and this is where big data enters the story, is that we are now discovering the data that was uncaptured about hole digging and a myriad of other things. The explosion of data that we’re seeing isn’t from ‘new’ data, but from data that was previously unseen … uncaptured, untapped, unused. Invisible, but absolutely in existence.

The way to dig that hole may have been locked in the mind of that worker or given in a hole-digging class as unstructured data (from his supervisor as oral instruction). Even the hard work of digging  can also be considered data. As we move to automate and machines become a bigger part of our lives, that unstructured data quickly becomes very important. It becomes the instruction manual for how machines operate, optimize and help us dig better, smarter, more efficient holes.

Where Process is About to Change

And this is why we’re on the verge of something enormous. We call it big data, but in reality we’re at the front edge of the digitizing of our world. What tipped the scales in data’s favor were a few advances over the past two decades:

  • Fast drop in data storage and retrieval cost
  • Quick rise in computational power and techniques
  • Easy ways to share information, like the Internet
  • Rapid globalization and the need to compete against cheap labor with something smarter

Along the way, Cloud, Mobile and Social happened just to make it interesting, but that’s essentially the landscape we face today. Smart companies are embracing what’s coming next and becoming data and process powerhouses. Watch the progress of Nielsen and others and you’ll see exactly where the global economy is headed.

Managing data at scale doesn’t have to be hard. Find out how the completely free, open source HPCC Systems platform makes it easier to update, easier to program, easier to integrate data, and easier to manage clusters. Download and get started today.


Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}