Dmitry Koltuno wrote a great article in Forbes about his experience with DevOps during a three year project to develop a hotel operations platform. As with any early-stage company, Koltuno’s budget was tight so teams had to be lean. As they gained traction, new clients came with new requests, and soon they had a team of six developers working on the same stack. They worked within a monolithic code base that needed to be manually built and deployed. There were no tools and they were too busy to implement any.
They started running into delays and consistency issues. Whenever problems came up, it would take the team away from the code. They also faced constant context-switching due to production issues, and keeping their infrastructure components up to date took a lot of unplanned time. A large amount of infrastructure maintenance had crept in.
They didn’t have much foundation to build on. Though they had basic scripts for repetitive deployment tasks, setting up new infrastructure was manual and any changes would take time to execute, tune and propagate to all environments. There were also subtle inconsistencies between development, staging and production that were difficult to spot. They didn’t have a solution for Configuration Management (CM), Continuous Integration (CI) or Continuous Delivery (CD).
There were many places for improvement, but they assumed that with a few months of work with their plan, they’d be on autopilot.
In retrospect, Koltuno acknowledges, they grossly underestimated the effort needed to clean up an existing infrastructure and didn’t appreciate the on-going commitment of maintaining automation. However, they still didn’t commit to a full-time DevOps hire, instead giving their Head of Enterprise Architecture, the task to oversee and execute the consultant’s plan.
The head of EA enlisted the help of their development lead and started implementing automated deployment, scripted configuration and Continuous Delivery. Although this seemed simple, timelines grew. DevOps does not provide standalone solutions: it introduces toolsets that facilitate new traditions and hence requires deep integration in the development process.
Their DevOps roadmap was big, and they also needed time for production support, infrastructure upgrades and research of new tools. Soon, two senior staff members were spending a significant part of their week on it. By not acknowledging that we needed full-time DevOps hires, they were converting two of their lead engineers into the role implicitly.
Koltuno learned that DevOps is an essential role to hire full-time early on. He says you should hire when your product has shipped, or when you have at least three developers on board. Live products require more monitoring than expected. If you haven’t hired this person yet, realize that DevOps work is happening in the background and your team would move faster if that function was explicitly owned.
Koltuno says that “If you’re not ready to make the leap, consider a seasoned consultant to help set a basic foundation. We found that with the right plan, you can hire freelancers in order to manage costs. However, this is only a temporary solution.
A great DevOps engineer is a company changer. They ensure the system is running smoothly and being monitored, and they can respond to issues as they arise. The DevOps engineer ensures that your developers are never doing repetitive tasks, and that the infrastructure is kept up to date as the stack evolves. As processes change and the company grows, the DevOps engineer automates as much as possible to accelerate work. Because of DevOps, developers can focus on their core work so you can deliver products earlier and more reliably.”