Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Big Data—Process, Success Stories, and Lessons Learnt

DZone's Guide to

Big Data—Process, Success Stories, and Lessons Learnt

· Big Data Zone
Free Resource

Learn how you can maximize big data in the cloud with Apache Hadoop. Download this eBook now. Brought to you in partnership with Hortonworks.


The buzz of “Big data” is spreading like an epidemic. Each and every industry is talking about it but the real picture seems to be bit hazy. In the upcoming lines, I would be talking about what has been done to get big data implementation across varied sectors, were there any success and what is the lesson learnt from those.

What is Big Data

Just as the word says, any huge information set which is under diagnosis to achieve some desired result is referred as big data. Now the “huge” part is the challenge here, as this huge can be parameterized by three v’s – volume, velocity and variety.

How Big Data evolved

Data management had always been a big issue and existed since 19th century, and therefore innovation and new invention to ease up those challenges have always been in place.

Some of the event listed below, will make things on more clear on how “Big data” evolved –

My First Encounter with Big Data

And rest is history, since then we are juggling and coping up with data sets and finding ways to handle it effectively.

It’s worth mentioning for me, how as an individual I was hit by it because like me many people wouldn’t ever realize that they are using big data analytics and in fact surrounded by it, I mostly do online shopping through many of the big e-commerce websites. Just another day, and I was checking my mail and it had a mail from flipkart , based on previous purchases it analyzed my data and send me a list of similar products which are currently running under offers. Another instance which catches my mind is when I logged into my bank account and again they showed me bar chart with which category I spent my money most, shopping, travel or food. These were the simplest data analysis done and it gave me a clear picture as where my hard core earned money is being pumped the most.

Is Big Data Industry Specific

Well, not at all it’s a generic term and could be used by any industry across globe from a drycleaner to a financial firm. Across different sectors the data may differ and so the desired result. For instance in automotive industry big data analytics could be used for vehicles warranty issues during production planning or sales prediction for a new launch, region wise could be very well handled.

Big Data Implementation

So you have the “Big Data” and now you are in need to use it wisely, to make it more productive, improve your decisions making capability and raise a bar against your competitors. What needs to be done?

·  Aim - What are you expecting out of your data? There could be many expectations [sample below] but it’s advisable to target one at a time and then include the next to simplify complexity.

o  Cost Reduction

o  Enhancing your productivity

o  Enhance your decision making capability

o  Faster execution

·  Strategy – Design a strategy to target that data

·  Start Small – Target a smaller chunk of data, rather than taking all parameters.

·  Achieving Milestone  – Lay the plan to achieve the aim with the strategy drafted

o  Tools Required – Going out in the market or build your own or simple MS excel will do

o  Time Required – Time is a constraint, you could only be the winner if you wrap it up first and smartly

o  Which data to use? – Structure out, good data and bad data

o  Budget – This would depend on company’s cost equation

o  Any Help needed – With evolving technology, it’s not always possible to get in house talent. So decide on it, contractors, outsource or new hiring’s?

o  Data Security – no compromise could be made with data security. 

·  Execution & Outcome – Things to get in action and match it with your expectation

·  Ongoing Data Analytics – It’s not a onetime activity, you need to manage and enhance it overtime and that’s biggest challenge.

Has any Firm Implemented Big Data yet?

There are many firms across globe who have successfully taken the advantage of Big Data analytics –

[a href="http://smartdatacollective.com/bigdatastartups/201286/why-ups-spends-over-1-billion-big-data-annually"]United Parcel Service (UPS) , one of the world’s biggest packages shipping company has beautifully blended big data analytics into their daily work, resulting in enhanced on time services globally.

In the entertainment industry, Caesars (formerly Harrah’s) , has exceptionally used the power of big data turning themselves into largest gaming company by revenue.

Bank of America having one of the largest customer base , exploring the big data analytics to provide a better experience to customers and also enhance their relationship with customers .

Also, it’s worth mentioning a report by Accenture where 92 % of customers where happy with the business outcomes after implementing big data analytics and 94 % are satisfied and glad that analytics are meeting their expectations.

Lesson Learnt

1.  Take One Step at a time

2.  As analytics is important and so is its storage

3.  Data Compression helps is effectively utilizing storage house

4.  Before Compressing make sure to sort it, it helps in optimizing data compression.

5.  Secure the data channel.

To Wrap Up

As we saw the accomplishments have been made, and some a making a steady progress towards it. The industry and its findings are relatively new, but the sure thing is that it would lead to better tomorrow. If we are clear on what we need, it would surely lead to the desired results we want to achieve. Big Data offers innovation, expansion, prolongevity and also probes us to take preventive measures and lay protocols to handle confidentiality, property rights, data reliability and individual liberty, so as individual we could be assured that our data is in safe hands and not out for exploitation. 

Hortonworks DataFlow is an integrated platform that makes data ingestion fast, easy, and secure. Download the white paper now.  Brought to you in partnership with Hortonworks

Topics:

Opinions expressed by DZone contributors are their own.

The best of DZone straight to your inbox.

SEE AN EXAMPLE
Please provide a valid email address.

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.
Subscribe

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}