Last week, I had the opportunity to speak with Gene Kim, author of the Phoenix Project and co-founder of IT Revolution. Gene will be a keynote speaker at JUC West, so I asked him about his experience eariler this year at JUC East, the growth of DevOps, how large older companies are adapting, and meeting Kohsuke Kawaguchi.
MW: You’ve been a keynote speaker at the other JUC events this year, correct?
GK: Actually I just went to the one at DC. It was really great! I finally got to meet the legend, Kohsuke, the man behind it all.
MW: That was your first time meeting him?
GK: Yeah, I’ve been an admirer of his for years, obviously, and I’ve heard great stories about him, but it was the first time I met him in person.
MW: Wow. I kind of figured that you, Jez Humble, Kohsuke, Martin Fowler, all these other prominent members of the community just knew each other in general like a DevOps Illuminati.
GK: *laughs* Oh no, that’s funny because it was such a treat to meet him. And what better place than at the Jenkins User Conference?
MW: How was your experience at JUC East, and what are you excited about for the west coast event?
GK: I want to dwell on KK for a moment. It’s true what everyone’s been saying for years that he’s an incredibly soft-spoken, humble individual. That was so true! It’s a stark contrast that he’s at the Jenkins User Conference where there are a thousand people, avid fans of his work and have created things of incredible value around what he created. It was such a treat!
MW: Was there any news from that conference you were excited about, or anything coming up at JUC West?
GK: One of the big surprises to me was the huge cross section of industries that were represented there from a user perspective. You had people from CapitalOne, GSA, the government organizations…it was such a treat to see a broad diverse list of organizations that do great stuff around Jenkins. I guess it shouldn’t surprise me, but it surprised me nonetheless to see the vibrant vendor ecosystem being created around Jenkins. It’s not just CloudBees, you have organizations like Pivotal, Xebia Labs, IBM, and Red Had there. It’s an amazing thing when you build something that gets put to use so widely, but when you end up creating a whole ecosystem of consultants and other software vendors, it’s not something you see very often.
One of the reasons I’m so delighted to be at the conference, and why I’m excited to be there again, is that I think so often, if you rewind the clock back a couple years, the people using Jenkins and doing CD pipelines were primarily the unicorns like Google, Amazon, Netflix, and Etsy. My passion right now is not so much how the unicorns are transforming, they’re doing that just fine, but how the horses are transforming, the large complex organizations that are taking the same principles as the unicorns and getting the same great outcomes. That’s where my area of passion is right now. So any time that you can get horses together and have them talk about things that only unicorns used to care about is a very special thing.
MW: What do you think has driven that shift? Stories about performance improvements?
GK: Well that’s what it’s got to be. There are two key messages I’ve seen. One is that high performers are massively outperforming their non-high performing peers. They have faster deploy rates, shorter lead times, and higher change success rates. But the second find that keeps coming up over and over again is that it’s relevant to every industry category. The fact that the Jenkins community has incorporated all of those is additional firsthand validation that DevOps is for everybody.
MW: I know you worked on the latest State of DevOps report with Puppet Labs, and I was wondering if there were any findings that surprised you?
GK: Oh man, there were so many. One of them was that I mentioned agility metrics: lead time and deployment frequencies, and the level at which high performers outperformed low performers was the same as last year. However, the reliability measures were significantly better this year. If you look at Figure 1, the mean time to restore for high performers had a 168x higher rate than low performers, compared to 48x last year, and the change success rate among high performers was 60x over last year’s 3x. What in the world, right? I’d say high performers are not just getting better, but also more reliable. Even if it’s painful you need to do it more frequently. This year has shown us that the only way you can be reliable is by implementing these DevOps practices.
Another thing that just made my year was this notion that DevOps helps us break the mythical “man month.” Fredric Brooks taught us that if you double the number of developers in general, you doubled the testing effort, deployment effort, and the integration effort to actually get to customers. What figure 9 shows us is just like Jez Humble said in Continuous Delivery: if you have developers continually testing, writing, and deploying their code, and if you have the automated processes and architecture to support us, low performers’ number of deployments per day per developer goes down as the team, whereas for the high performers, the number of deployments per day per developer goes up. In my mind it really shows that we can increase developer productivity linearly with the number of developers, and we can break the myth of the “man month” under certain conditions. I don’t think there is any technology leader that doesn’t care about that.
As an aside, what’s happening at large organizations like CapitalOne is that just like Netflix and Etsy, they’re trying to create a very technical brand to show people what innovative stuff they’re doing as a hiring vehicle. It’s just breathtaking to see because I remember a few years ago, if you were from a bank you couldn’t go to conferences if you were a tech person. Now you have CapitalOne at OSCON, JUC, Agile2015, DevOps Enterprise Conference…it’s amazing that they’re as visible as someone from Netflix would be, and that’s a beautiful thing to me.
MW: Yeah, when I was at Agile, CapitalOne was there and so was Lockheed Martin, but they said “we’re not here to sell anything or showcase a tool, we want to try and help government how to do agile right, because we want to move in that direction, and government is our biggest customer.” It’s really cool how bigger companies are making that shift. And one thing that is becoming a more common refrain is that every company is a software company, and everyone wants their software to work well and to be delivered quickly. People are starting to understand that from a leadership role, and I think this is a huge factor in big companies like CapitalOne developing their own open source tools.
GK: Yeah yeah yeah. Did Mark Schwartz give a keynote there, from the US citizenship and immigration services at the department of homeland security?
MW: I don’t think so. I only got to hear one keynote, unfortunately.
GK: Yeah it’s another place where government agencies just keep seeing success stories, like the CIO of US CIS is putting pressure on the software and large system integrators that serve the federal government. It’s just breathtaking to see. You mentioned Lockheed Martin, and it’s not just the system integrators, it’s also coming from the agencies, saying: “This is how we’re going to work and we expect our partners to work with us.” We live in amazing times!
MW: It’s incredible. When I started at DZone 3 years ago DevOps was nowhere near the point it is today, and now it seems to be taken as fact from everyone who isn’t afraid about being able to implement these changes.
GK: As a side note, I think what’s different about DevOps, and I think Jenkins is a huge part of it. Before, changes would come from the top-down. This crazy agile people would come back from a conference and bring these radical ideas to the organization. What’s different about DevOps is that when you have tools like Jenkins, which radically improves the lives of developers, it spreads like wildfire. I mean, who wants to go back to the old way where you’re integrating only once at the end of the process? You have a competency inside an organization and you end up with this feeding frenzy where everyone’s in line to eat next. It increases productivity, it increases quality of life, it increases agility, it increases reliability, and it decreases burnout. You end up with these bottom up AND top down forces that accelerate change. It’s a key part of why DevOps is being adopted so quickly as opposed to agile 11 years ago. So more kudos to KK!
MW: Looking to the future a bit, what is the next step for the DevOps movement? What will some of the major challenges be?
GK: I think I know the top five problems facing the DevOps movement, and this comes from the DevOps Enterprise Conference I held in October. I asked every speaker to end with a slid with one of two titles. It was either “Here’s what I don’t know how to do” or “here’s what I’m looking for help with.” The five issues were:
“How do I create automated testing for legacy applications,”
“What are the cultural and leadership issues around transition,”
“Everyone wants to go faster in dev and ops, but those bastards in compliance and security keep saying no. How do we get them into the tribe as well,”
“What are the next generation of roles and responsibilities, especially for operations,” and
“What message should I use to drive my bootup programs?”
This is how we’re focusing the next conference. Technology is a big factor in all of those. It’s all around “how do I get there faster safely and reliably,” which ultimately leaves you with a more competitive organization.
In the old days, our job in operations is that we did our work in ticketing systems. We’d build servers, set them up, run them, and get a ticketing service to test protocols and so forth. So many stories told at JUC East were that humans don’t do these things; your tools do it for you. Now you create testing and environments on demand. You deploy on demand. It’s the wild world we’re going into.
MW: That’ll definitely feed into the role issues you mentioned earlier, especially with testers who won’t be doing manual testing anymore.
GK: Good and bad, right? No more error-prone manual testing, but who’s going to write all the automated tests?
MW: Is there anything else I should have asked or something our audience should know about.
GK: Well DevOps, and Agile 11 years ago, just accelerated this massive change in how we organize everything, especially in operations. I think developers always like small teams, but in operations, it’s by the book. They say there are two extremes of how we organize: market-oriented teams like developers who deliver directly to customers without worrying about other departments, while operations is functionally oriented which is organized for cost and other concerns. Agile has forced this switch to functional orientation to a market orientation. You’re talking about two separate ends of the spectrum and it’s breathtaking to see how DevOps and agile has transformed operations.