How to Implement Data DevOps

DZone 's Guide to

How to Implement Data DevOps

Have an executive sponsor and make sure DevOps, developers, and DBAs collaborate.

· Big Data Zone ·
Free Resource

In this article, I continue my discussion with Derek Hutson, President and CEO at Datical following our initial conversation about, “Why do we need Data DevOps?” In this segment, I wanted to get his thoughts on implementing Data DevOps.

What should clients consider when beginning Data DevOps Initiatives?

Begin by looking at your application portfolio carefully and asking:

  • Where do I need speed?
  • Where do I need quality?
  • Where do I need appropriate levels of security and audit ability and governance?

Start there, and then look at how you are organized. Do I have a database and data representatives on my DevOps teams? If not, how do I change that?

When you have the pressure of delivering a new release, and, at the same time you're trying to innovate and implement Data DevOps, get executive sponsorship, ensure you have the right organizational structure in place, it's easy to quickly get overwhelmed. In this case, having a commitment to not try and do every application is important; instead, pick key applications and go from there. That's why you start from an implementation standpoint.

Should DataDevOpsbe integrated into the DevOps Toolchain

Data DevOps must be integrated into the DevOps toolchain; it should never be separated. We're experts in databases. We're experts in DevOps. We need to be experts on the DevOps side so that we can integrate the database into the toolchain.

We will keep the bar high and say data should be part of your DevOps toolchain in order to realize the maximum benefit. You don't want to continue with a separate process the way it is today. We've said to the database teams, you're a unique process, we still have a database because of the value that's there but it needs to be integrated into the DevOps toolchain.

Who leads the charge for Data DevOps? 

It's got to be an executive that really has the recognition of the need to do it. However, they're usually the sponsor. We've seen developers lead the charge. We've seen DevOps teams lead the charge. We've seen DBAs lead the charge. I think it depends on the organization, where the problem recognition is coming from, and who is assigned or who is asked to solve the problem.

Where we've seen this work best is when you've got separate teams representing the three different legs of the stool involved. When those three teams collaborate, and there's executive sponsorship, they're going to solve the problem. So, thinking about who leads the charge, it goes back to which application and who is involved, bought in, and committed to changing it.

What tools and technologies do you recommend to clients for adoption?

There are myriad. Begin by taking an inventory of what you have in terms of tooling and data, then identify what data they want to integrate into their DevOps toolchain. Thinks about the key classes of technology —database release automation, data automation.

I think about something like Delphic that’s able to provision environments very quickly and provide those test environments in an efficient and inexpensive way.

There's more on the data-side in terms of tooling that provides data normalization and ensures that you have appropriate context. Then there's test data automation and database release automation. I'm sure there are others. But I think about those three big buckets.

What are some of the challenges that you see companies needing to overcome as they try to integrate data into the DevOps process?

First and foremost is the resistance to change. If you're going to do Data DevOps, even if you're going to do DevOps period, you're going to have to be willing to commit to some kind of a change in the way that you do things today.

You can't just say we're going to do everything exactly the same. You may have to restructure the way that you organize your teams, so you have the right people aligned and committed to making the change.

DevOps cuts horizontally across silos. You have to organize horizontally to be able to touch dev, operations, test, quality, release, and production. Then, that loop comes right back. Align horizontally across those; otherwise, stovepipes are a key challenge to overcome.

Realize the need to incur some short-term costs. Anytime you begin to automate, the next time you do that task it’s probably faster to do it manually. It takes a while to automate, but the 15th, 20th, and 15,000th  time that you do it with automation is where your ROI is nearly infinite.

Unfortunately, we see many organizations say that automating processes is not the most critical thing to do today, so they choose to wait. In this case, those organizations are choosing short-term pain over long-term gain.

What do developers need to learn or know to facilitate the implementation of a Data DevOps methodology?

There are times when developers say, “my job is to check this code in and make sure I don't break the build. Somebody else has to deal with it from that point forward. I know my application changes are going to require changes to the database, etc.”

It's a measure of empathy with what happens next. If I'm checking in an application change that is going to require 50, 100, or 500 changes to the database, I need to empathize with who's got to make those changes and ask, “is there a better way to do this?”

The ROI and the payback may be very high for the developer if he or she will say, “I'm committed to understanding what happens next. Maybe, I need to do something slightly different on my side to be able to facilitate a much better experience throughout the entire lifecycle, or the DevOps toolchain.”

It's understanding what happens from an operation standpoint with data or database from a DevOps perspective. Developers can partner with the operations teams to say, “how are we going to solve this problem since we will both benefit from solving it.”

We've got to work together to do this because, in a silo, the database teams can't change by themselves; they need the developer. Additionally, the organization has to agree to some adjustments in the way they build software to realize the full benefits and ROI of Data DevOps.

So there's an empathy layer, there's understanding, and there's working together to empower operations teams to make necessary changes. When you do that, everybody in the toolchain, everybody in the release process will benefit — especially developers.

As we discussed on the previous question, be willing to trade a little short-term pain, discomfort, and change for everybody to realize long-term benefits.

big data, data devops, devops

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}