DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
What's in store for DevOps in 2023? Hear from the experts in our "DZone 2023 Preview: DevOps Edition" on Fri, Jan 27!
Save your seat
  1. DZone
  2. Data Engineering
  3. Big Data
  4. Database Migration Tools

Database Migration Tools

In this post, we take a look at the three main kinds of database migration tools (on-premise, open source, and cloud-based) and the situation in which each is best.

Garrett Alley user avatar by
Garrett Alley
·
Apr. 16, 19 · Analysis
Like (3)
Save
Tweet
Share
31.28K Views

Join the DZone community and get the full member experience.

Join For Free

There's no denying that the world is driven by data. And that data usually lives in a database. As enterprises like yours increasingly look to extract maximum value and insights from data through big data analytics, they're finding that sometimes it's necessary to move their data from one database to another. This process is called, appropriately, database migration.

Database migration tools allow you to literally move data from one type of database to another or to another destination like a data warehouse or data lake. Migrating databases - say, from on-premise to the cloud - can help reduce costs, improve business agility with more flexible systems, and centralize enterprise data to create a single source of truth.

But migrating databases requires careful planning to ensure that all your data is properly accounted for, transported, and protected. And those plans can only be properly executed with the right database migration tools.

Available Database Migration Tools

Database migration tools are typically categorized as on-premise, open source, or cloud-based. Which type of tool you'll need depends heavily on the size of the database, how many schemas and tables you are going to migrate, and whether or not there are large objects (LOBs) that need migrating as well. Here's what that means for you:

On-Premise Database Migration Tools

As a general rule, on-premise solutions are designed to migrate databases within a large or medium enterprise without moving data to the cloud, which is why some companies with specific security restrictions prefer them. These tools are ideal for changing data warehouses or moving the location of your primary data store, or if you are simply bringing together data from disparate sources on-premise because they're capable of managing the broadest range of data integration initiatives as a single platform.

  • IBM InfoSphere
  • Informatica PowerCenter
  • Microsoft SQL
  • Oracle Data Service Integrator

Open Source Database Migration Tools

Likewise, open source tools are developed by communities of database pros and are made available for free or at a very low cost. These tools are designed to be universal across databases, allowing you to migrate data from and to any database using standard filters and can run in real-time or in batches.

However, because they're universal, they aren't at all specific. So, it may require some custom development of plugins or other features to align with your company's specific needs which means spending on skilled staff or bringing on consulting developers to fill those needs.

  • Apache NiFi
  • Flyway
  • Pentaho

Cloud-Based Database Migration Tools

Moving big, critical data sets - entire databases - requires the latest-generation cloud-based migration tools. Known for their power and flexibility, cloud-based migration tools can handle large amounts and various types of data and applications with ease, offering enhanced security capabilities and greater agility than other types of tools.

Cloud-based database migration tools can integrate well with both the source and the target systems, enabling you to migrate databases using various database engines with minimal effort or overhead. The best tools make it easy to set up and migrate data by eliminating time wasted restructuring database design and re-coding.

  • Alooma
  • Matillion
  • Snaplogic

How to Select the Right Database Migration Tool

Database migration comes with significant challenges around locating previously siloed data, protecting against data corruption, and securing data during transit. Each should play a pivotal role in selecting a database migration tool. Here are some other considerations to keep in mind:

  • Type. Determine if you'll need a framework/language-dependent tool or an independent tool. Framework and language-dependent tools are common if you use a popular language or framework such as Ruby, PHP, Python, or Django. If you don't want to be locked into a particular language or framework, look for an independent tool that acts as a source control for your database.
  • Cost. If long-view cost management is a chief concern, cloud-based tools are your best bet since they will save you money in the long run on both physical infrastructure and human resources. On the other hand, you can get the most bang for little to no money from open source tools, allowing you to focus unused resources elsewhere. But remember that you'll need the right expertise in-house to set up and configure them.
  • Security. If the databases you're migrating are mission-critical or otherwise highly sensitive (perhaps due to compliance requirements), cloud-based tools come out on top as they rank high for security and are usually compliance-certified. On-premise tools can also be very secure, but only to the extent that your on-premise infrastructure is secure.
  • Data model. The schema is a blueprint of how the database is structured, and it varies based on the rules of a given database. When you move data from one system to another, look for a tool that aligns with the new database.
  • Data transformation. One of the most critical elements of database migration is transforming the data. Look for a solution that can normalize your data, remove duplicate information, and detect errors while moving your data to the target database.
Database design Big data Open source

Published at DZone with permission of Garrett Alley, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Popular on DZone

  • ChatGPT vs. GPT3: The Ultimate Comparison
  • Essential Mobile App Security Tips for Coders in 2023: Make Your App Unhackable
  • Core Machine Learning Metrics
  • Required Knowledge To Pass AWS Certified Data Analytics Specialty Exam

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com
  • +1 (919) 678-0300

Let's be friends: