Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

3 Data Integration, Management and Architecture Trends for 2019

DZone's Guide to

3 Data Integration, Management and Architecture Trends for 2019

What will the new year hold for these three key areas of big data? We take a look at this question and try to provide some clarity.

· Big Data Zone ·
Free Resource

The Architect’s Guide to Big Data Application Performance. Get the Guide.

The coming year almost certainly has some surprises in store for the technology space, but one factor IT professionals can bank on is that digital transformation will remain a top goal for enterprises. Digital transformation provides a solid advantage by facilitating operational excellence; it enables service or product innovation, and it increases customer value over time. That’s why enabling digital transformation is among CIOs’ top three objectives in 11 out of 15 industries, according to Gartner’s 2018 CIO Agenda Industry Insights report.

Both the agent and currency of every digital transformation initiative is data. IT professionals seeking to help their companies pursue digital business models in the coming year must be ready to quickly integrate an ever-increasing number of endpoints and applications required by business leaders. They must also be ready to manage and deploy digital assets and data from new data sources, a majority of which will be cloud-based, and do so at the speed of innovation. For IT leaders, the challenge will be to develop an IT strategy and architecture that can support the integration and data management needs of digital transformation.

As IT leaders make key architectural decisions, three data integration, management and architecture trends will bear watching. These trends will form crucial inflection points for companies pursuing digital transformation in 2019 and beyond. Here’s a brief look at each: 

DIY – time to call in the experts:

IT leaders have discovered that aging systems that require IT professionals to connect data endpoints can’t keep up with increasingly complex data needs and business requirements to quickly incorporate applications. The pressure will only grow as companies implement digital business models. Integration Platform-as-a-Service (iPaaS) is a newer approach that facilitates more agile integration, including cloud-based applications. As currently characterized by industry insiders, iPaaS provides a modern architecture for integration and data management, but it also equips business leaders and citizen integrators with DIY tools to complete own integrations.

But is DIY the right approach, given the increasing data integration and management complexity driven by digital transformation, which involves more sources and greater more data compliance and security considerations?

Many IT leaders are opting instead for an iPaaS model with an advanced architecture that also offers managed services to address the complexity of integration, security, compliance, etc. It’s a logical progression. Homeowners routinely tackle simple home repair and maintenance tasks themselves but call in the professionals when facing more complex jobs. The same is true of enterprises that initially took a DIY approach to data integration; as the compliance, security, infrastructure and operational challenges associated with the function got more complex (and consumed an increasing share of scarce IT resources), they called in the professionals, i.e., turned to managed services providers so they could refocus on their core competencies.     

Industry giants seek consolidation: Seasoned IT pros may remember the go-to-market plan followed by big companies like Oracle and SAP a generation ago. When those companies rolled out the solutions that eventually became the legacy systems that enterprises are currently shedding to conduct business in the cloud, they created closed ecosystems that kept users in the fold as new data integration and management needs arose. Something similar is happening now, with major industry players like Salesforce consolidating integration, data ownership, and data residency into their platforms. As IT leaders make decisions about how to support digital transformation, they’ll need to consider how industry consolidation — and the closed ecosystems that typically evolve from consolidation — could foreclose options downstream.

When it purchased MuleSoft earlier this year, Salesforce seemed to move away from its previous stance as an agnostic platform with open APIs and third-party integrators. The shift on the part of Salesforce and other industry giants appears to signal a desire for greater ecosystem control and ownership of the connectivity to authoritative data sources. This shift was possibly prompted by competition from leading cloud technology players like Amazon Web Services and Google.

The application-centric vs. data-centric divide grows: The industry consolidation described above highlights a related trend: industry leaders are going all-in on market strategies built around applications or data. Reminiscent of the country’s political divide, there’s little room for middle ground as this fault line emerges, and companies are picking sides throughout the industry. It’s important for IT leaders to think through the implications for digital transformation as they choose sides. There’s significant innovation in the application space, but it is data that drives digital business, and for that reason, a data-centric approach makes more sense for companies that are pursuing digital transformation.

Companies like Liaison come down firmly on the data side, serving as neutral brokers between data sources and applications so that business customers can select the applications that meet their unique needs. New applications can give businesses a significant competitive edge. But before they can take advantage of all the innovation in the application space, businesses have to make an architecture-related decision: do they want to keep their options open or commit to an application ecosystem?

The road to digital transformation is rocky for many companies. One major challenge some encounter is the scarcity of IT professionals, which has been amplified by historically low unemployment rates. Companies seeking highly skilled IT employees face fierce competition for talent, and the shortage may be exacerbated by new restrictions on H-1B visas. That’s a serious problem for companies that want to create an in-house team to handle data integration and management instead of choosing a managed services vendor.  

As the DIY integration era comes to an end, major industry players seek closed ecosystems, and the application-centric vs. data-centric chasm widens, enterprises will explore more options for facilitating digital transformation. One attractive alternative is to choose an advanced platform that allows the company to maximize the value of data instead of devoting considerable resources to the technical implementation and maintenance of integration solutions. Look for more companies to choose that option as these three trends accelerate in 2019 and the coming years.

Learn how taking a DataOps approach will help you speed up processes and increase data quality by providing streamlined analytics pipelines via automation and testing. Learn More.

Topics:
digital transformation ,data integration ,big data ,data management ,data architecture

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}