Evolving the Database for DevOps
Let's take a look at an article from the new DZone Database Guide, which explores how the role of the DBA is evolving and how DevOps should not be the central point.
Join the DZone community and get the full member experience.Join For Free
I’m writing this article with feelings of “conflicted passion.” As a database administrator with two decades of experience, devoted to investments in platform expertise, I’ve honed my skills. I was known for building database systems, focused on making the most with the technology of a given database platform. While working at Oracle, I utilized strong skills in partitioning, PL/SQL code and functions, and other Oracle-specific features. A similar scenario developed when I worked with Microsoft SQL Server, MySQL, Informix, and others. I used the features that were built by the database vendor to make the most of the database platform. This solution created a double-edged sword. The business received the most from the platform they chose, but the cost was a single-platform solution for coding, release, and skillset. The solution often had challenges bridging to environments on other platforms, often requiring ITdepartments to have policies on what they would and would not support. Due to this cost, the DevOps engineer in me knows that we must change for DevOps to encompass the database more successfully, resulting in the conflict I feel.
Automation is key to information technology meeting increasing customer demands. The amount of online content on DevOps and agile processes tells us the importance of this evolution, but a majority of the focus in at the application tier and not the database platform applications rely on to provide much of its functionality. As my career moved from DBA to DevOps engineer, my focus changed from database platform-centric solutions to locating ways to automate and enforcing vanilla approaches to problem-solving.
DBA Heal Thyself
One of the goals of automation is to remove human intervention, which, in turn, removes human error and human bias, the former being a large attraction for database administrators (DBAs) to move toward DevOps. Automation of everything already present shouldn’t be the finish line of DevOps, though. A continual enhancement and improvement to decrease the length of the development cycle and perfecting steps within the cycle should be an ongoing priority.
Many traditional DBAs are hesitant to embrace DevOps in for fear of their roles being made obsolete. For systems to evolve, human initiative, introspection, and inspiration are required. By removing tedious, repetitious tasks, we’re able to free up DBAsto focus on these valuable endeavors. When DBAs realize the latter goal and recognize how their role can evolve as part of it, they have less hesitation and can become an important and pivotal part of the DevOps initiative.
Let’s first remove the misconception of the single database. The role of the DBA is not going away, but evolving.02. DevOps, although essential for most businesses to get ahead, must be done with the data being a central point.03. Databases will serve the industry best by being less platform-specific and being better at serving up data quickly and transparently. Evolving the Database for DevOps platform environment. Consider the actual percentage of businesses with a single database platform. If we ignore policy and take the data from the environment as any indicator, the numbers will show that less than 10% are truly single platform.No matter the policies set by the IT department to drive a single platform decision, a unique vendor requirement or business critical system on a secondary platform is very likely to exist. If you speak with any DBA, it’s a common challenge of at least one or more outlier environments they’re left to support, even if their training and experience haven’t prepared them for it. With this myth dispersed, we can now assume that most technical environments have more than one database platform in a given technical environment.
Moving beyond this first myth, we might think we’re closer to taking on the overwhelming hurdle of culture and processes part of any DevOps project, but the fact that we commonly leave the review of database changes to the end of the workflow demonstrates we aren’t. It’s a sign of poor communication between teams. And when we talk about the communication break between DBAs and Development, it’s almost synonymous with IT. It’s also a clear indicator that the cultural divide hasn’t been addressed as part of the DevOps shift. Many may claim that it’s due to philosophical or control differences between the teams, but at some point, this divide must be met and addressed as the blocker it is.
There is a roadblock factor that we experience repeatedly around this challenge. DevOps teams tighten timelines and rarely have the window to wait for a DBA to review changes, yet they also may feel hesitation to bring the DBA in earlier when concerns could add time at any point in the schedule. It’s essential that a process be designed where developers, operations, and DBAs agree how changes to the database are handled and steps are defined to involve the DBA from the beginning. The earlier that DBAs are brought in, the fewer issues that will arise, creating a higher rate of success as time proceeds. Anxiety gives way to anticipated collaboration between each of the teams — a primary goal of all DevOps teams.
Change Is Bad — No, Change Is Good
DBAs aren’t fans of change, as consistency is often synonymous with stability. A DBA is often recognized as competent by environment stability, resulting in a high uptime for access to database environments. This being the case, DBAs can be viewed as a roadblock in their hesitation around change — and the development cycle is all about change. With development cycles part of continuous delivery, and with continuous integration putting significant pressure on stability, a DBA can begin to pressure toward slowdown of the development cycle. Even having all changes tracked in version control doesn’t mean that outages won’t occur, and outages should be expected at some point, even with mature DevOps practices. The most important goal is to have the DBA part of the process to eliminate as much potential of an outage or impact as possible. And if one does occur, lessons should be learned to remove the potential for any repeat.
One of the biggest ways to eliminate impacts from a development cycle is to ensure there are proper development and testing environments that are duplicates of production. Using virtualization to provide as many development and testing environments — as many as needed for multiple development cycles simultaneously in process — should be a priority of any DevOps group. Notice that I assigned responsibility for this to the DevOps group instead of the DBA group. The reason for this distinction is that we’ve held the DBA responsible for this for too long. As part of DevOps, and as teams are redistributed towards multi-role teams (developer, DBA, administrator, etc.)on each, it distributes the responsibility and the ownership for the resources across the business, too. It creates a buy-in for all involved to be part of the solution, using newer technical enhancements like virtualization, snapshots, and security measures to provide what’s needed to succeed.
Many traditionalDBAs are hesitant to embrace DevOps in for fear of their roles being made obsolete.
These teams may consider containerizing environments, as well. There are numerous types of container technology outside of just Docker. A popular solution, Kubernetes, can be used to create “pods” that will allow identification of what tiers belong to each other and the ability to package these separate tiers as one. It will conserve resources vs. using virtual machines and save money in the end. Containerizing can be very beneficial at the database level, too. There are options to snapshot and use virtualized databases as part of containers — even in pods of multiple database platforms, applications, and structured/unstructured flat files.
With this change to the housing of database sources together, a secondary data source for central analytics might not be required as often as it’s proposed. SQL may have similar“flavors” between database platforms, but statements can result in different outcomes depending on the database platform. By using your databases as only data stores and by eliminating much of the proprietary code, forcing it outside the database layer, there will be less impact of changes between database engines. This will leave tedious and simple management at this point in technology, such as index management, query optimization, and statistics collection at the database tier to be automated by the database engine. The database is no more than a data store at this point, and this is where the DBA in me has a conflict. I must move beyond my pride in my database platform skills or loyalty to any product. The DevOps engineer in me must embrace solutions that work at a higher level than any unique platform.
In the end, we will hopefully see interfaces(both command line and graphical) that possess the ability to interact with a database platform, no matter the vendor.
This evolution will make use of object-relational mapping(ORM) frameworks, along with other tools to simplify the demands. I will make a choice to standardize how I deal with our database tiers and move more outside of the database so that we can create a set of unified processes when automating database tier work.
I will also move away from tightly coupled architecture environments with a centric database. Tightly coupled architecture can create more complexity than most DevOpsendeavors are designed to manage. One way to resolve this is to build microservices on top of databases. By granting each microservice its own database — specifically, a virtualized copy of a database — DevOps is able to move as fast as any code and it’s siloed to the point of fewer collisions with other development cycles, only requiring a final merge as the last step in the development cycle before deployment.
As a DBA, I will now need to (again) move past my traditional role to be centric to a specific database platform and simply treat the database as a storage layer. The database layer is there to serve up data to fulfill and analyze; nothing more. Yes, most database technologies have the technologies and features to be used for more than just to store data, but by doing so, it creates a complexity layer and lock-in to the database platform that also creates challenges later in the development lifecycle. By using microservices, you decouple the database from this challenge. The microservice will serve as a conduit for other microservices to deliver data vs. direct query of the database, limiting the demands on platform-centric code or SQL.
In the end, we will hopefully see interfaces (both command line and graphical) that possess the ability to interact with a database platform, no matter the vendor. We already see this with Scala-based Spark. Developers can code in C#, Java, SQL, or Python, and yet, the output is the same at the analytical level. Database management must evolve in a similar way. With this need fulfilled, the conflict ends and the passion emerges for database management that is transparent to the database administrator, leaving cycles open to focus on DevOps automation, data quality, and code performance.
Opinions expressed by DZone contributors are their own.