Over a million developers have joined DZone.

On Organizations’ Real Requirements Around Big Data

DZone's Guide to

On Organizations’ Real Requirements Around Big Data

· Big Data Zone ·
Free Resource

Hortonworks Sandbox for HDP and HDF is your chance to get started on learning, developing, testing and trying out new features. Each download comes preconfigured with interactive tutorials, sample data and developments from the Apache community.

Every time another vendor takes the “big data” moniker and applies it to some legacy system a cat somewhere on the internet dies. Like “cloud” a couple of years ago, big data has become the term de jour – and vendors seem to think that simply using it will give them some magic power to up their revenues, their margins and their count of unicorns and rainbows. Yeah, in case you didn’t realize, the overuse of the term makes me cringe.

But that doesn’t mean that big data isn’t a valid concept per se. We live in a world where the amount of information is increasing rapidly. Where the number of individual devices connected to the net is exponentially rising. where organizations want to make some sense of the world around them – all reasons where big data can help.

So it was interesting to see the results of a survey that GigaSpaces commissioned into the use of big data in industry, and the pressing drivers for this use. The survey that GigaSpaces did was of 243 IT executives and developers spanning a bunch of different industries. Obviously 243 is a small number, and no doubt the survey pool was skewed somewhat by the focus, network and perspective of GigaSpaces, but as a general snapshot, it’s informative. So, some interesting statistics:

  • 80% say big data processing is “mission critical”
  • 70% have a need to process big data in real time
  • 80% are planning to move their big data processing to the cloud

So – what does this all mean?


Like I said, big data provides a real opportunity for organizations to make smarter decisions in a more rapid manner. But to get over the line in terms of selling projects within an enterprise – big data initiatives really can’t require a significant investment in kit. They also can’t create a drain on existing hardware. A move to cloud processing is therefore a natural step. At the same time, there is little point in assessing big data after the fact – the goal of big data is to be able to react to trends near instantaneously.

It’s interesting that the past couple of weeks have seen some interesting announcements from both AWS and HP about big data analysis initiatives that deliver on both these trends – likely a sign of things to come. Big data is going to be an important industry shift over the next few years – but users need to have products that allow it to harness big data insights rapidly and outside of their existing technology kit – this survey reiterates that fact.

Hortonworks Community Connection (HCC) is an online collaboration destination for developers, DevOps, customers and partners to get answers to questions, collaborate on technical articles and share code examples from GitHub.  Join the discussion.


Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}