Over a million developers have joined DZone.

Big Data and Market Research Have A Lot in Common

What are you trying to accomplish with big data?

· Big Data Zone

Hortonworks DataFlow is an integrated platform that makes data ingestion fast, easy, and secure. Download the white paper now.  Brought to you in partnership with Hortonworks

I was just reading an article, Insights From Early Data Lake Adopters, in the spring edition of Big Data Quarterly and it appears that people and companies pursuing big data initiatives would be well-served to remember some of the basics learned in market research 101.

Know what you're going to do with the data before you start collecting every byte of it. With the proliferation of sensor data, companies can become overwhelmed with data very quickly if they collect every piece of data rather than exceptions.

Imagine a city that's measuring flows in its water infrastructure. Does the city need to know how much water is flowing through a particular section of pipe every second or even every minute? Wouldn't the city be better served to determine a "normal" flow rate based of day and time and then look for exceptions to the data that may indicate a leak or a big event taking place?

By being more strategic in why you are collecting the data and what you are going to do with it you don't have to capture, store and analyze every data point. This reduces computing time and storage costs. While those costs might seem inconsequential today, determine how much data you're going to collect over the course of 365 days, and 3,650 days. Is the cost still inconsequential? What's the ROI on those costs?

I know a lot of companies want to collect all of the data they can and then figure out what to do with it after the fact. This reminds me of all of the surveys I've seen with 30+ questions, matrices, and open-ends that take more than 10 minutes to answer - wasting customers' and respondents' time. Rule number one in market research is don't ask the question if you don't know what you're going to do with the answer. It's rude and inconsiderate to the respondent and has led to a reduction in response rates and a loss of customers.

If you ask a customer a question and the customer answers it, it's incumbent upon you to thank the customer for their response and let them know how you'll address it. Granted there are very few companies following this "best practice" today; however, as companies become more customer centric, they'll be more conscientious about following up with customers who take the time to provide their invaluable input.

I've worked with clients that didn't even read the results of their customer surveys before creating a new website or developing a new product or service. Why did they waste their customers' time with the survey? Is it any wonder customers aren't engaged with these companies? The companies sure aren't engaged with their customers.

With more data being collected on prospect and customer actions, customer expectations of your company, and the water company, are going to increase. If the city knows my water main is leaking, isn't it incumbent upon them to let me know rather than wasting water and sending me a water bill I can't afford? Today they may get away with it. In five years, they'll have no excuse of taking advantage of their customers or wasting a precious natural resource.

While the customer experience (#CX) bar is very low today, customers will become more savvy about the data companies are collecting and how they are, or are not, using that data to improve the customer experience. If you can save a customer time, make their lives simpler and easier, you can earn a customer for life. What if your competitor beats you to the punch? You'll lose a customer for life.

With all of the data you're collecting on your physical plant, how much are you saving on energy, water, and operating expenses? How much space is being wasted and how will your next building use the data you've collected to be a smarter building? How much are you improving the employees' working environment so you attract the type of employees you need to be successful?

Big data is new and exciting for companies; however, few companies have figured out how to use the data to improve their business, reduce cost, improve revenue, and customer satisfaction.

Ultimately the companies that figure out how to use big data to improve their business, across many areas, will be successful while those who do not will fall by the wayside.

Are you using what you've learned from big data to inform your strategic planning or just collecting data?

Hortonworks Sandbox is a personal, portable Apache Hadoop® environment that comes with dozens of interactive Hadoop and it's ecosystem tutorials and the most exciting developments from the latest HDP distribution, brought to you in partnership with Hortonworks.

big data,iot,data lakes,databases,data anlysis

The best of DZone straight to your inbox.

Please provide a valid email address.

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}