Why the Impact of Big Data Extends Further Than You Think
Why the Impact of Big Data Extends Further Than You Think
See what separates Big Data projects from true, successful Big Data programs, the dangers of data silos, and tips for combining Big Data initiatives.
Join the DZone community and get the full member experience.Join For Free
Hortonworks Sandbox for HDP and HDF is your chance to get started on learning, developing, testing and trying out new features. Each download comes preconfigured with interactive tutorials, sample data and developments from the Apache community.
C-suite executives: What is the impact of big data in your organization? If you suggest I speak to the CIO, you might be surprised when you take a closer look.
Big data is at a tipping point across many corporate disciplines, from marketing to customer support. Corporate leaders across every part of their businesses are using it to innovate and gain new insights into their existing business.
The impact of big data is spreading rapidly throughout many companies and becoming a part of their DNA. The danger, however, is that some senior managers may be unaware of its effect.
Siloed Data Can Cause Problems
Big data deployments are commonly localized. When organizations first begin experimenting with big data, they often do so with small proof-of-concept projects. Sometimes the senior management may be aware of these, but in many cases, they may be run as “skunk works” projects conducted autonomously to test a team’s assumptions.
This creates several problems for senior executives. The first is that they don’t have a unified view of the organization’s big data initiatives, which tends to keep big data from becoming a strategic focus. The biggest value of data will come when connecting the dots across an entire organization. For instance, a single view of a customer does not live in marketing or in sales—it spans the entire organization. It’s hard to encourage joined-up thinking and pursue a bigger goal when different people in the organization are fostering a piecemeal approach to big data projects.
Another problem is a lack of governance. Data can be a powerful tool, but it can also be destructive to your business if misused. If ad hoc projects are started without adequate oversight, they can lead to the inappropriate use of personal information.
Without a strategic approach to big data, executives risk developing pockets of noncompliance as well-intentioned middle managers experiment with sensitive data. All it takes is one slip, and it could leave chief executives trying to explain themselves while the company makes negative headlines.
Time to Combine Big Data Projects
By pulling these projects together into a holistic strategy, business leaders can take control and help write the company’s big data narrative. This offers several advantages. First, it gives business leaders more control over data privacy and security. By imposing best practices in an enterprise-wide policy, executives reduce the risk of rogue managers creating security gaps in the corporate infrastructure. In a world where data provenance and protection is more important than ever, this point cannot be overstated.
Gaining control over data governance will make the compliance department happy. Instead of having to find and audit a universe of undocumented big data projects, the compliance team will already be aware of them, and will be assured that the project managers drew from a data governance playbook when creating them. This should make auditing far easier and reduce the risk of noncompliance.
The other benefit of creating a governing strategy for big data projects is that the whole can become more than the sum of its individual parts. While few companies expect to pile all their information into a single, huge data ocean, an end-to-end view of these projects can nevertheless introduce opportunities for efficiency and data aggregation. Multiple projects may share information in a single data lake.
Managing data in this way can increase the variety of information available to a single big data project. For example, an ad hoc project might only draw on the sales database, limiting its results. By making unstructured data from other sources available to that project, a company could generate more accurate, insightful results.
These partnerships require a strategic vision—which, in turn, increases the impact of big data at the corporate level. A comprehensive, top-down big data strategy creates space for the executive team to forge a vision of what it wants big data to achieve.
Making Big Data a Program, Not a Project
How can corporate leaders turn things around and get to this point? The first step lies in developing a mature big data strategy. Business leaders must treat big data as a program, not a project, with the highest level of executive buy-in.
Then you should give the program the attention it deserves by turning to experts to design big data architectures around open architectures: It’s unlikely that your big data program will support just one technology or put data in only one location.
Once you have a program with well-defined compliance practices and technical specifications, your organization can document its existing siloed projects, assessing each one to see if it can be incorporated into the new framework.
Even projects that cannot be integrated will still be valuable. Architects can learn from most deployments, understanding what worked and what didn’t. They can then factor those insights into a redesign, as they build the project from scratch with new, enterprise-approved parameters.
The time to do this is now. Data-driven companies such as Uber, Facebook, and Amazon are already disrupting markets by the dozen. They are ahead of the wave. By developing a strategy that embraces the impact of big data across your organization and beyond, you can surf this wave too.
Published at DZone with permission of Piet Loubser , DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.