Over a million developers have joined DZone.

Big Data Guide: Strategic Importance of Keeping Track of Big Data

DZone's Guide to

Big Data Guide: Strategic Importance of Keeping Track of Big Data

Although much of big data is about as useful as the text message you sent to your friend 15 minutes ago, the part of it that is useful can transform the world.

· Big Data Zone ·
Free Resource

Hortonworks Sandbox for HDP and HDF is your chance to get started on learning, developing, testing and trying out new features. Each download comes preconfigured with interactive tutorials, sample data and developments from the Apache community.

You are probably already feeling some of the impact of living in a data-driven world. You rarely come across a Google search that doesn’t answer your question, and often enough you find more than enough information to write your own tome on any topic you can imagine.

Moreover, your hard drive is probably filled with so much data accumulated over the years that you might wonder what would happen to all your data if it crashed and you hadn’t backed up everything. Fortunately, hard drive data recovery experts can quickly resolve this issue.

In one of his books, author Deepak Chopra narrates how his hard drive crashed when he was writing a book he had spent months researching. Since he had not yet backed up his data, he immediately experienced crushing despair. He had no idea how to reconstruct his pivotal ideas or retrace his in-depth research findings. Fortunately, he discovered that it was possible to completely recover a hard-drive and was amazed (and relieved) when he quickly got his restored hard-drive in the mail.

Keeping Track of Big Data

In four years from now, there will be 5,200 GB of data per person. International Data Corporation, a research group, believes that there will be 50 times more data in the next decade. In order to grasp how big that big data is becoming, let’s look at how measurements have increased over time:

  • When computers were invented, experts first talked about bytes, which is 8 bits.
  • Later, they talked about kilobytes (1000 bytes).
  • Yet this was remarkably modest when compared to the megabyte (1,000,000 bytes).
  • Then people were thrilled to have hard drives that stored their data in gigabytes (1,000,000,000 bytes).
  • Today, however, it’s possible to buy a personal computer with a terabyte (1 000 000 000 000 bytes).
  • When we get to supercomputers, we’re now talking in terms of a petabyte (1 000 000 000 000 000 bytes).
  • However, now we are measuring things in exabyte (1 000 000 000 000 000 000 bytes) too. For instance, 5 exabytes would be the total amount of words ever spoken by human beings.
  • Yet even this is small compared to the zettabyte (1 000 000 000 000 000 000 000 bytes). For instance, this year, Internet traffic is estimated to be about 1.3 zettabytes.

How Gaming Companies Are Leveraging Big Data

Gaming providers also use big data to optimize their business models. Companies that offer online slots monitor the amount of time and money players spend on their online games. This helps them determine which games are performing the best and how to improve on them to earn loyal, active users.

Why Big Data Is Growing So Fast

You might be wondering how data is growing so fast. After all, as a species, we’ve done fairly well for ourselves with a comparatively little amount of data and now we’re talking about living in a world where the sum of human knowledge will be fifty times more?

Basically, we now have more ways to capture data. Not only are our computer systems getting better at storing information, but we are also collecting data from embedded systems.

You can find embedded systems all over the place like sensors in clothes, bridges, buildings, and medical devices.

We are also getting better at compression technologies and data deduplication (a special data compression technique for eliminating duplicate copies). Storage utilization rates have increased through thin provisioning, which is the allocation of disk array capacity for storage and virtualization. In addition, we have also reduced how much data is transmitted across networks to be stored in data centers.

Can’t Keep Up?

Most of this data is arising from information sharing. It’s unstructured data that is being shared through video, email, and files. This accounts for 90% of all the data that will be created.

While there is still discovery of new information through science, as well as an exchange of ideas that creates new interpretations of how things work and what things mean, much of it is trivial information. We’re talking about cute cat videos, here. So, don’t panic, you will be fine and chances of waking up one day almost completely illiterate because the world was reinvented while you slept are remote. Yes, the world will change but it will be at a pace that you can manage. The job you will be doing in the next decade has probably not even been invented yet, but you will be up to speed by then.

Benefits of Big Data

Although much of big data is about as useful as the text message you sent to your friend 15 minutes ago, the part of it that is useful can transform the world.

While it's possible to appreciate the value of big data in practical things like improving marketing insights and delivering better customer satisfaction, big data is set to transform civilization.

According to Bernard Marr, who wrote the book, Big Data: Using smart big data analytics and metrics to make better decisions and improve performance:

“The advantages are so limitless because we now have data on everything and it can help us get new insights on everything. I see the whole spectrum of big data from NASA using it to analyze real-time data on Mars — and I find it particularly amazing how big data is used in healthcare to predict treatment plans and to predict diseases. It opens up completely new avenues in terms of combining data with other things such as robotics where you have smart, intelligent machines that can do a lot of the jobs that are currently done by people. Machines will be doing them much better.”

Hortonworks Community Connection (HCC) is an online collaboration destination for developers, DevOps, customers and partners to get answers to questions, collaborate on technical articles and share code examples from GitHub.  Join the discussion.

big data ,big data management ,disaster recovery

Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}