Top 3 Challenges That a CTO Will Face While Migrating a Database
In this article, take. look at three challenges that a CTO could face while migrating a database.
Join the DZone community and get the full member experience.Join For Free
The competitive world demands that applications be more performant, bringing pace to the overall functioning of the application. Well, adopting new technologies and moving away from the legacy system can be miraculous. Migrating to a new database is one such miracle that can help applications leverage the overall performance and functionality. But, the process of migrating databases is a challenging task. Let us explore the top 3 challenges that are faced by the CTO while performing a migration.
1. Devising a Migration Strategy
Database migration is a strategic process. It is fraught with a lot of risks. Devising a plan for safe, secure and efficient migration of databases is really important for the migration to be successful. Well, for many teams, deciding whether to pursue a “big bang” migration — a single step process or to leverage a more methodical, incremental approach called a “trickle” or “parallel run” migration.
There are various factors that can influence the migration strategy such as allocated budget, timeline or time constraints, and available human resources to execute the migration.
“According to research, enterprise database migration projects can cost up to $875K and experience more than $250K in cost overruns.”
Financial and time implications can add another layer of complexity to the planning and strategy process.
2. Lack of Integration Process and Proper Collaboration
A typical process of data migration involves different people using myriad technologies. For instance, the use of spreadsheets to document data, which are usually prone to human errors and are difficult to be translated doing data analysis or performing data transformations. The use of various technologies can sometimes lead to failure in the transfer of data and its design between the analysis, development, testing and implementation phases. Sometimes things get lost in translation which eventually increases the cost and wastes a lot of time. For this, organizations should look to utilize the platform at its full potential. It should be able to successfully link the critical inputs and outputs from each of the stages to help in reducing the error and save time and money.
As we have already mentioned, data migration involves different people who use different technologies. In some cases, a mix of your employees and some external engineers would work on database migration. It might happen that some of these people may not be in the same location. Working at different locations in silos might impact the overall efficiency, create more data silos and can lead to misinterpretations. While working together, it can be difficult to deal with situations when things go wrong. Most people try to blame rather than resolving issues. Making use of collaborative tools helps the parties involved in the migration in observing the same picture of data as it moves through various project stages, leaving little or no room for assumptions and misunderstandings.
3. Emphasis on Securing Data and Systems
Data migration isn’t just time-consuming and costly but also has the potential for increased risk without the right protocols and plans in place. In any migration, there is a treasure trove of high-value intellectual property that has the risk of being leaked, lost or otherwise accessed by unauthorized users (either inadvertently or with malicious intent). Any of the instances could mean significant damage to the company reputation, customer churn, or even potential lawsuits or punitive fines.
For instance, Twitter migrated from MySQL to its custom database Manhattan. The lesson that they learned,
“Over the years, as we’ve migrated data from MySQL to Manhattan to take advantage of better availability, lower latency, and easier development, we’ve also adopted additional storage engines (LSM, b+tree…) to better serve our traffic patterns. Additionally, we’ve learned from incidents and have started protecting our storage layers from abuse by sending a backpressure signal and enabling query filtering.”
The process of database migration can be smooth if you can devise a proper migration strategy with appropriate collaboration among various teams. It is also essential to lay emphasis on data security and systems so that data loss can be avoided, hence successful migration can be accomplished.
Published at DZone with permission of Hiren Dhaduk. See the original article here.
Opinions expressed by DZone contributors are their own.