Why Your Silos Are a No-Go
Why Your Silos Are a No-Go
One way to remove friction is by leaving the data at its source and let data owners iteratively share it with the organization via an API at their own pace.
Join the DZone community and get the full member experience.Join For Free
Hortonworks Sandbox for HDP and HDF is your chance to get started on learning, developing, testing and trying out new features. Each download comes preconfigured with interactive tutorials, sample data and developments from the Apache community.
Everyone loves their own data: collecting it, analyzing it, drawing conclusions from it. But often, when you allow departments or business units within your organization to gather their own data, that data isn’t shared. Then, much like the Middle Ages, when warlords ignored the crown, built their own castles, and collected their own taxes, fiefdoms emerge that erode the larger kingdom. Some data is duplicated, while other data is ignored. Conclusions only take a portion of the bigger picture into account. And, chaos eventually ensues. So, take a moment to think about your organization: How many data warlords do you have running around?
People or departments owning data is inevitable. But as a leader, it’s your responsibility to cultivate a frictionless, data-driven culture.
I know, I know — it’s easier said than done... especially if you’ve got decades of culture or existing processes working against you. And we understand that breaking your data out of silos is just one of many data analytics challenges you may face. But the point is, you don’t want to lose touch with your data. And make no mistake, no matter which department collected the data, it’s all yours. Unfortunately, the longer you let it sit in siloes, the harder your life will be down the road. Most importantly, that data is only valuable if it’s accessible to those who can make use of it and if it can be merged with other data, providing new insights.
However, as experience has shown us, when people are forced to share their data by dumping it all into a data warehouse, they tend to distance themselves from it (as the data moves outside of their department, ownership evaporates), leaving it without any management or curation. So, you want to build into your initiatives more incentive than simply avoiding punishment.
One way to remove friction is by leaving the data at its source and let data owners iteratively share it with the organization via an API at their own pace. As they realize the benefits of it, they will start sharing more and more data. Those who share their previously “proprietary” data will soon become your organization’s collaboration trendsetters — knights in shining armor, as it were. This approach also provides better transparency and efficiency since you’ll ultimately have a single, robust data catalog to draw from.
No API interface? Worry not. An API-in-a-box approach allows you to build an API layer on top of your data sources and start sharing data with other departments within weeks. API-in-a-box also allows data owners to retain ownership and manage their data, so others can access the data without any danger of changing or losing it.
When positioned as a cultural shift as much as an operational one with organic, positive incentives, getting people to share their data becomes less of a challenge.
Don’t think it works? A few years back, a little company called Amazon.com mandated that every legacy system that wanted to share or consume data had to share its data with other legacy systems through an API. In addition, all these APIs had to be potentially public. The result? The data behemoth we now know as Amazon Web Services (AWS). Even those who initially didn’t share eventually saw the benefit, and got on board.
All-in-all, sharing data is a much leaner, transparent method than the old “command and control” approach. It allows your organization to become nimble, agile and scrappy, getting the most out of every data point. Sharing works. And once you have all that data at your fingertips, you’ll hold the true keys to the kingdom.
Published at DZone with permission of Wolf Ruzicka . See the original article here.
Opinions expressed by DZone contributors are their own.