Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

On Modern Warehousing and Cloud Integration

DZone's Guide to

On Modern Warehousing and Cloud Integration

Read on to learn more about Modern Warehousing, Cloud Integration, and the challenges as well as benefits associated with them.

· Integration Zone ·
Free Resource

SnapLogic is the leading self-service enterprise-grade integration platform. Download the 2018 GartnerMagic Quadrant for Enterprise iPaaS or play around on the platform, risk free, for 30 days.

Data warehousing continues to be central in today’s organizations as data has become more of a corporate asset for companies. However, data warehousing is not without its challenges. We recently sponsored the Eckerson Group webcast, “The Step-by-Step Guide to Modern Data Warehousing,” and I’ve compiled some quick takeaways.

Challenges of conventional data warehousing vs. modern data warehousing

To solve the growing challenges associated with conventional data warehousing, many organizations are moving to a cloud data warehouse. Not only can a cloud data warehouse enable business scalability, elasticity, and managed infrastructure, it also lowers costs, increases process and deployment speed, and comes with security and governance.

The four major cloud data warehouse players — Amazon Redshift, Google BigQuery, Microsoft Azure, and SnowflakeDB — are different in terms of offerings, but in the end, all help accomplish the same goal. However, you can’t move your data warehouse to the cloud with just a cloud data warehouse. You may also need a subset of tools that help make the data migration easier and more manageable, including:

  • Data integration tools or an integration platform as a service (iPaaS)
  • Data warehouse automation tools (DWA)
  • Data Virtualization tools (DV)


iPaaS Solutions

Data integration tools are used to build or rebuild ETL processes and to populate the data warehouse when the warehouse is in the cloud. This makes it important to find an iPaaS solution that is specifically built for the cloud. There are multiple solutions out there, one of which is SnapLogic, a cloud-first iPaaS. Read more on how to select the best iPaaS tool for your organization.

Data Warehouse Automation Tools

This automation can be used to deconstruct your legacy ETL and re-engineer it to metadata, helping to recreate and regenerate the process automatically. Data warehouse automation tools include Attunity, WhereScape, and others.

Data Virtualization Tools

Moving large amounts of data at one time can be overwhelming and result in many issues, which is why it’s a best practice to move to the cloud incrementally. With data virtualization tools, you can create a layer between where the data is stored and where the data is consumed, providing data views that help hide the physical realities of the data being migrated — all while the business goes on as usual. Data virtualization tools include Denodo, Data Virtuality, and more.

The Big Picture

Moving data to the cloud is a more complicated process than it appears to be. Migrating the data is only one step of many.

The many steps it takes to migrate to a cloud data warehouse

Business Case

There are many benefits that make cloud data warehousing an initiative to pursue, including:

  • Agility
  • Performance
  • Growth
  • Cost savings
  • Labor savings


As you make your business case for cloud data warehousing, ask these questions:

  • What are the drivers to move to the cloud?
  • What are the business benefits and who cares about them?
  • What are the technical benefits?
  • What are the business disadvantages of not migrating and who feels the pain?
  • What are the technical disadvantages of not migrating and who feels the pain?


Once you’ve answered these questions, assess the architecture as demonstrated by the slide below.

Data warehouse architectural assessment for modern data warehousing

From there, you’re ready to map out your migration strategy, technology strategy, and then execute your migration to the cloud.

To learn more about cloud data warehousing, view our SlideShare presentation below and watch the webcast, “The Step-by-Step Guide to Modern Data Warehousing” on-demand.

With SnapLogic’s integration platform you can save millions of dollars, increase integrator productivity by 5X, and reduce integration time to value by 90%. Sign up for our risk-free 30-day trial!

Topics:
integration ,data warehouse ,cloud integration ,ipaas

Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}