A Taxonomy for Devops Tools

during a recent discussion with a group of experienced devops practitioners, we came up with the following "taxonomy" for the service delivery part of the devops tooling landscape:

task execution

what do these tools do?

• allow you to run arbitrary commands or scripts on one or multiple target systems
• depending on the tool, not just ad-hoc, but also or on a regular basis

what are they good at?

• one-off or regular maintenance, "cleanup" or management activity (e.g. clearing caches, killing stale processes)
• think "cron++"

what are they not so good at?

• coordinating processes across multiple machines
• targeting hybrid environments (e.g. support for windows varies widely between tools)
• providing out-of-the-box logic – script it yourself

remarks

• most of the other tool categories include a "task execution framework", since they also need to run commands on many servers. just a question of how easy it is to leverage that framework.

cloud & container management

what do these tools do?

• cloud management: allow you to manage virtual machines, networks, storage, routing, load balancing etc…basically, manage virtual datacenters
• container management: allow you to define and manage "runtime containers" or "runtime environments", which may be virtual machines, networks etc., or a tool-specific container type

what are they good at?

• providing on-demand environments and resources for your delivery pipelines and runtime environment
• depending on the tool, providing a definition model for environments or containers that can be instantiated on a developer’s laptop as well as in the production environment

what are they not so good at?

• portable definition models are limited and too restrictive, flexible ones are not portable enough
• often require an application architecture geared to the container model – not easy to adopt for existing applications
• limited or no support for "role-based definition control", e.g. ensuring certain parts of the container or environment definition are fixed
• management of persistent data and linking to externally consumed services hardly ever covered
• container and environment definition still generally a manual process – script it yourself

remarks

• providing lightweight containers (typically, based on lxc) as a "development deliverable" is currently a fashionable idea
  whether this can work for you depends largely on whether you can (re-)architect your applications to work in this model
• load balancing, scaling, "a/b pool management" etc. are still things you need to solve (effectively, this is the difference between a paas and a "container" or "environment" pool)

system provisioning & configuration

what do these tools do?

• allow you to define the desired state of a system, often in a declarative manner
• ensure a target system is brought to the desired state, and stays there
• via their underlying task execution frameworks, allow for the execution of ad-hoc commands on a server

what are they good at?

• ensuring many machines are in a known state and kept in that state
• providing a lot of out-of-the-box content for system configuration and service installation tasks
• depending on the tool, reporting on differences between the intended and actual states of a machine

what are they not so good at?

• coordinating an action sequence across multiple machines

application integration

what do these tools do?

• take source code and other development artifacts and turn them into a versioned deliverable that is a candidate for release
• run extensive code-level and integration tests, code analysis tools etc. for validation
• depending on the tool, performing this validation on candidate code changes before integration into the main branch

what are they good at?

• providing a lot of out-of-the-box integrations with build and code-level testing and –analysis tools
• chaining and distributing build and testing tasks for efficiency
• maintaining the "releasability" of your main branch

what are they not so good at?

• coordinating an action sequence across multiple machines
• providing out-of-the-box logic for tasks beyond the construction and archiving of the deliverable – script it yourself
• depending on the tool, providing a domain model that represents the delivery process

application deployment

what do these tools do?

• take a new version of a deliverable (typically, an application) and get it running in a target environment (i.e. a set of target systems)
• depending on the tool, optionally create target environments for the new version on demand

why are they good at?

• coordinating actions across multiple target systems
• ensuring deliverables are environment-independent and handling any required environment-specific configuration
• providing a lot of out-of-the-box content for application-tier tasks

what are they not so good at?

• providing out-of-the-box logic for system configuration & service installation tasks
• supporting build, validation and packaging of the deliverable
• providing a domain model to support the entire delivery process, esp. including team-based tasks
• providing out-of-the-box logic for application-tier tasks – script it yourself
• providing a domain model that aligns with application delivery concepts

pipeline orchestration

what do these tools do?

• allow you to define the sequence of tasks that make up your delivery/release process, i.e. your "delivery pipeline"
• depending on the tool, support processes that combine manual and automated tasks
• depending on the tool, show features are currently at which stage of the process

why are they good at?

• providing visibility into the end-to-end delivery process
• depending on the tool, allow drill-down into not just which applications are where, but which features those applications implement, which code versions were used to build the applications etc.
• providing an end-to-end audit trail for each application running in production

what are they not so good at?

• providing out-of-the-box logic for tasks carried out in the pipeline – script it yourself or invoke other tools
• depending on the tool, supporting team-based tasks and variable processes
• depending on the tool, supporting process improvement

we then tried to place these tools in relation to

1. which part(s) of the system stack they address
2. where in the end-to-end/"code-to-cache" spectrum they fall

and came up with the following diagram:

what do you think? which of these categories of tools do you use? do you have your own categorization?