Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Why Your Silos Are a No-Go

DZone's Guide to

Why Your Silos Are a No-Go

One way to remove friction is by leaving the data at its source and let data owners iteratively share it with the organization via an API at their own pace.

· Big Data Zone ·
Free Resource

The open source HPCC Systems platform is a proven, easy to use solution for managing data at scale. Visit our Easy Guide to learn more about this completely free platform, test drive some code in the online Playground, and get started today.

Everyone loves their own data: collecting it, analyzing it, drawing conclusions from it. But often, when you allow departments or business units within your organization to gather their own data, that data isn’t shared. Then, much like the Middle Ages, when warlords ignored the crown, built their own castles, and collected their own taxes, fiefdoms emerge that erode the larger kingdom. Some data is duplicated, while other data is ignored. Conclusions only take a portion of the bigger picture into account. And, chaos eventually ensues. So, take a moment to think about your organization: How many data warlords do you have running around?

People or departments owning data is inevitable. But as a leader, it’s your responsibility to cultivate a frictionless, data-driven culture.

I know, I know — it’s easier said than done... especially if you’ve got decades of culture or existing processes working against you. And we understand that breaking your data out of silos is just one of many data analytics challenges you may face. But the point is, you don’t want to lose touch with your data. And make no mistake, no matter which department collected the data, it’s all yours. Unfortunately, the longer you let it sit in siloes, the harder your life will be down the road. Most importantly, that data is only valuable if it’s accessible to those who can make use of it and if it can be merged with other data, providing new insights.

However, as experience has shown us, when people are forced to share their data by dumping it all into a data warehouse, they tend to distance themselves from it (as the data moves outside of their department, ownership evaporates), leaving it without any management or curation. So, you want to build into your initiatives more incentive than simply avoiding punishment.

One way to remove friction is by leaving the data at its source and let data owners iteratively share it with the organization via an API at their own pace. As they realize the benefits of it, they will start sharing more and more data. Those who share their previously “proprietary” data will soon become your organization’s collaboration trendsetters — knights in shining armor, as it were. This approach also provides better transparency and efficiency since you’ll ultimately have a single, robust data catalog to draw from.

No API interface? Worry not. An API-in-a-box approach allows you to build an API layer on top of your data sources and start sharing data with other departments within weeks. API-in-a-box also allows data owners to retain ownership and manage their data, so others can access the data without any danger of changing or losing it.

When positioned as a cultural shift as much as an operational one with organic, positive incentives, getting people to share their data becomes less of a challenge.

Don’t think it works? A few years back, a little company called Amazon.com mandated that every legacy system that wanted to share or consume data had to share its data with other legacy systems through an API. In addition, all these APIs had to be potentially public. The result? The data behemoth we now know as Amazon Web Services (AWS). Even those who initially didn’t share eventually saw the benefit, and got on board.

All-in-all, sharing data is a much leaner, transparent method than the old “command and control” approach. It allows your organization to become nimble, agile and scrappy, getting the most out of every data point. Sharing works. And once you have all that data at your fingertips, you’ll hold the true keys to the kingdom.

Managing data at scale doesn’t have to be hard. Find out how the completely free, open source HPCC Systems platform makes it easier to update, easier to program, easier to integrate data, and easier to manage clusters. Download and get started today.

Topics:
data analytics ,api ,aws ,silos ,big data

Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}