Over a million developers have joined DZone.

Minding the Database: Devs vs. DBAs

DZone 's Guide to

Minding the Database: Devs vs. DBAs

Let's take a look at this article from the new Database Guide. Explore minding the database and look at Devs vs. DBAs.

· Database Zone ·
Free Resource

This article is featured in the new DZone Guide to Databases: Relational and Beyond. Get your free copy for more insightful articles, industry statistics, and more!

There has been a lot of attention and focus of late on cloud platforms, containers, and microservices. Fueled by the hype and promises, software teams of all sizes and shapes are looking to take advantage of these new methods of delivering software. However, many of these teams are conveniently omitting one of the most important elements of a functional end-user software experience: the underlying data and databases. At the end of the day, whether using containerized microservices or not, the application is meaningless without data. The dirty detail that nobody seems to talk about is how to manage and evolve the persistence layer so that it can support an application layer architected to leverage the most exciting serverless cloud platform or coolest new microservices solution. 

To add further intrigue to this glaring omission, there has been a major shift in the industry wherein application developers are responsible for initially authoring database changes. If the term "application developer" is confusing, the trend can be put another way: the responsibility of writing the first draft of database code has shifted away from database professionals such as DBAs and data architects. That immediately begets the question: what does this mean for the database?

There has always been some friction between application teams and infrastructure teams — or put in a more familiar way — between Dev and Ops. Application teams generate change and constantly need the infrastructure to keep up. Ops teams, on the other hand, want to keep systems stable and available — and change can be risky for these groups. As expected, the shifting trend — where application developers author the database code needed to support the application feature or enhancement — adds to the tension that has long existed between Dev and Ops teams.

To be clear, the changes that have fallen to generalist developers are simply the changes needed to directly support an update to the application. The broader data architecture efforts, query performance tuning, security patches, system upgrades, and other work outside of directly supporting a change in the application are still firmly in the purview of DBAs and data architects. Fundamentally though, the challenge lies in supporting the opposing needs of those looking to make small, incremental changes to the structure and logic of a database and those looking to maintain the uptime, performance, and availability of a database.

Historically, the tools for making changes to and maintaining the database were focused on meeting the needs of DBAs as they were primarily responsible for both functions. Given the glacial pace of change relative to today, these tools were optimized more for operations than for development. The primary focus was on keeping databases up, running, and performant. As software development and delivery cycles have greatly accelerated, these tools have struggled to keep up. At the heart of the matter is that these tools were not designed for rapid, iterative, incremental changes — much less parallel development with feature-based or trunk-based approaches.

To address this gap, a new class of tools, optimized for rapid, iterative development across parallel workflows are needed to support the growing legions for application developers that are now authoring database changes have emerged. A lexicon of terms has quickly formed to describe the two different approaches that now exist. Tools that are better suited for understanding the overall state of the database, primarily meant for operations teams, data architects, and DBAs are often described as "state-based" or "declarative" tools. Meanwhile, tools that are better suited to making small changes to a database, primarily meant for developers needing to modify the database, are described as"migration-based" or "imperative" tools.

Both out-of-box vendor tools — for example, Microsoft's DACPAC, SSMS, and SSDT tools — and old, established proprietary tools — such asQuest Software's TOAD — are state-based, focusing on the ideal state definition and using a comparison (or "diff") to generate a migration script for each target database that needs to be evolved. Meanwhile, a lot of relatively newer tools, which are focused on enabling Agile development teams to track and migrate databases — such as Datical andLiquibase — are migration-based.

Image title

Figure 1: An illustration of how state-based and migration-based approach evolving the current state of a database.

A natural follow-up at this point is: what is the best approach — or better, what's the best tool? This depends heavily on the needs of the team. If working with a data-heavy application in an Agile framework, with a steady stream of database changes that are a part of every application sprint, it makes sense to adopt a migration-based tool. Conversely, if database changes are rare, many of the free, vendor-provided state-based tools may suffice. Veterans like Martin Fowler have addressed this topic extensively and strongly promote migration-based approaches to Agile development teams that need to rapidly and incrementally make changes to databases.

Why is the migration-based approach better suited to modern Agile and DevOps focused teams? It's because migration-based approaches better map to the key principles of Agile and DevOps workflows. In the name of accelerating velocity, improving quality, and ensuring transparency, these modern software workflows espouse the following:

  • Small, incremental changes that are directly tied to business requirements

  • Automated validation, build, and deployment of code changes

  • Enabling self-service and fast feedback loops

  • Repeatable, consistent deployments that are done with immutable artifacts ("build once, deploy often" is the mantra)

  • Planning for and accommodating the flexibility to change requirements or implementation midway through a sprint

To begin, the state-based approach does not promote small, incremental changes. Developers work with and are consequently empowered to edit the entire state definition — which does not reinforce making small changes. Even if developers are disciplined and make small changes to the state definition, the generated migration script is often hundreds of lines long even for relatively simple changes. Worse, the generated script may also need to be edited in many cases to prevent data loss or other undesirable outcomes — especially as many changes cannot easily be derived from a comparison (for example: changing a relationship from one-to-many to many-to-many, or renaming a column). With all the manual intervention required, the state-based approach does not easily allow for developer self-service nor fast feedback loops.

Making matters worse, the state-based approach cannot adhere to the"build once, deploy often" best-practice as each deployment involves a comparison and script generation step that might yield different results. This adds risk to database deployments and prevents deployments from being automated — as each generated script requires manual inspection — and potentially manual modification — prior to deployment. Lastly, teams lack the flexibility with this approach to accommodate changes. If there is a change in requirements or an issue discovered in testing, a developer needs to make a change to the state model, get a new migration script generated from the tool, review and possibly hand-edit the migration, and only then proceed forward.

As an alternative, the migration-based approach puts the burden on developers to focus on the migration. Once the migration itself has been explicitly defined, it's possible for intelligent automation to validate the small, incremental change, build it into an immutable artifact, and enable consistent, automated downstream deployments. By treating the migration as the first-class object of interest, it becomes possible to enable developer self-service, fast feedback loops, and the flexibility to quickly rework an individual migration instead of contending with the entire state model and a script generation process as demanded by the state-based approach.

Commercial solutions like Datical build upon the migration-based approach. For example, Datical's simulation capability allows teams to take a validated artifact of database changes and forecast the impact on a given target database. This type of rich, context-aware feedback can be automated, allowing better developer self-service and improving database code quality.

In summary, the needs of operators and developers are different, and as developers increasingly take on database code changes, it's important for organizations to use the proper tools and processes to reduce friction while improving throughput and quality. This doesn't mean that teams need to discard the old state-based tools; instead, it means that teams looking to bring Agile and DevOps practices to their database code changes need to invest in a migration-based solution to better enable developers while allowing operators to continue relying on state-based tools to manage system uptime.

This article is featured in the new DZone Guide to Databases: Relational and Beyond. Get your free copy for more insightful articles, industry statistics, and more!

database ,database guide ,dzone guide ,devs vs dbas

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}