Key Takeaways From Continuous Discussions (#c9d9) Episode 42: Release Pipelines

last week, we hosted another episode of our continuous discussion ( #c9d9 ) podcast — this time discussing release pipelines .

our expert panel included: gioia ballin, software engineer at measurance; jamie ingilby, software development leader; juni mukherjee, author of “ continuous delivery pipeline – where does it choke? ”; j. paul reed, managing partner at release engineering approaches and host of the shipshow podcast ; and our very own anders wallgren and sam fell.

during this episode, we covered the hard truths and common hurdles along your pipeline — from code check-in to release — and some best practices to address them on your path to achieving continuous delivery. we discussed continuous integration, the importance of testing, and challenges of configuration, deployment and release, and more. continue reading for best practices and advice from our expert panelists.

continuous integration

• ballin on ci in small teams: “when your team is very small, time resources are scarce, and you have to give power to the developers. you don’t have the time to ask someone to deploy your service; you have to be able to deploy your service by yourself.”
• the way mukherjee sees ci? “the way i visualize it is you have the feature branches, where you are relatively isolated and can make mistakes without really putting the entire team up for sale. then, when you check into main line, mentally prepare yourself to see it in production. that way you go into main line with the amount of confidence that you had before.”
• ingilby on the challenges of ci in legacy and monolith applications: “in these applications where there isn’t maybe the branching strategy to support it, continuous integration is sometimes the first hurdle that people have to get over. and sometimes it’s a lot more difficult to get through that one first hurdle than we might otherwise think.”
• first, operationalize continuous integration, says reed : “often times, legacy applications or not, i find that the conversation really starts around operationalizing your continuous integration platform, and then having the organization react to when it fails – when they’re failing unit tests, leaving them red for months at a time, or having tests that might be flapper tests, that go red and green, actually spend time on fixing those.”

testing

• advice on testing in release pipelines, per ingilby : “you have to consider everything that is considered ‘testing’ for your product – we’re not just talking about functional testing, or acceptance tests, it could be everything that’s required in order for you to build confidence in your artifact as a releasable product.”
• testing is the hardest part of continuous delivery, according to reed : “testing is the hardest part, because we talk about unit tests and static tests and all these different types of tests. we say that we should have them – and the fact that we’re still saying that – the more interesting question to me is – it’s not that we don’t get it, it’s just that we don’t do it.”
• according to mukherjee , tests move you from “suicidal speed” to “responsible speed.” she goes on to say, “tests are first class citizens – a test pipeline is as meaningful as anything else.”
• testing can save you dollars, according to ballin : “software testing is so important because software always contains defects, defects leads to failures, and failures potentially lead to money losses.”

configuration, deployment, release

• reed stresses the importance of release engineering. he says successful organizations: “[are] caring about those none functional requirements of operational requirements, release engineering, tooling, asking how to do ab testing and feature flagging.  all these things are release engineering problems, and we’re starting to see its impact on security.”
• mukherjee : “don’t make this a tooling problem. it’s essentially a process problem, and if you say process drives architecture and architecture drives tools, then once we lay out a cool process for ourselves, i think it’s very reasonable to then follow it up with the right architecture and the right tool set – and the tools will evolve.”
• advice for when you have multiple environments before production, per ballin : “if you have several environments and you have to pass through all the environments, the best way in my opinion is to streamline the pipeline until the very last step before production.”
• everyone on the team should be akin to the release process, according to ingilby : “it’s important that the final step is part of a step that anyone on the team feels confident in doing. there are lots of organizations that have formal, or even informal, deployment heroes that do all of the deploying and have special access – this isn’t a scalable pattern and doesn’t encourage your software architecture to be easily deployable and resilient.”

watch the full episode here

want more continuous discussions ? we hold our #c9d9 ’s every other tuesday at 10 a.m. pst, which features expert panelists talking about devops, continuous delivery, agile and more. next tuesday on continuous discussions, we’ll be talking about devops and cd for non-web apps .