All the way back in 1997, when I worked in the land of consulting, I established a practice that I called “Software Engineering Process Improvement” (SEPI) to address what I saw was a growing trend in the industry – the need to help coach organizations in ways to make their software development processes and practices more efficient and to deliver more value. I’ve had the privilege of watching the trend of software development practices change over time addressing ever-evolving technologies and business realities.
In this post, I’m going to provide some predictions that revolve around the world of agile software development that I believe will manifest (or continue to manifest) in 2013. Here is a summary of those predictions:
· Agile will/must Evolve: Gartner has indicated that Agile is in the Trough of Disillusionment in their Hype cycle. Agile hasn’t lived up to its hype, but that doesn’t discredit it, it only means it must evolve further to address these issues.
· Significant DevOps Influence on Agile: Agile is about removal of waste and the optimization of the whole. Operations and maintenance has rarely been considered in Agile practices. In 2013, DevOps will influence Agile and help bring value across the entire lifecycle
· Agile for the CEO: Books such as “Lean Startup” help emphasize the fact that Agile practices aren’t just for development teams. The same basic principles that underpin Agile and Lean also means good business. Agile isn’t just for Development teams any longer.
· Strong Lean Intersection: We’ve already seen a lot of movement from Lean into the world of Agile. In 2013, this will solidify.
· Patterns of Agility: There is no one right way to implement Agile within a team or an organization, however, more discrete “adoption design patterns” will emerge that will help make Agile adoption more palatable and successful.
· Renewed focus On Specification: A response to the backlash against over specification, software requirements will come back into focus being led by the convergence of techniques such as BDD and wireframe validation.
· git Dominance: GIT make an even greater splash in 2013 as we see more and more organization move to GIT or GIT based tooling for their cloud AND enterprise needs.
Now, if you’re interested, here is a more comprehensive explanation of each point.
Agile Will/Must Evolve
Has Agile lived up to its Hype? Well, from my experience, backed by Gartner’s Hype Cycle results on Agile, it doesn’t seem that Agile is “complete” yet. There is no doubt that Agile (in all of its forms... from engineering practices to team communication practices) has drastically influenced how almost all software is being created today. For example, Agile has worked its way in to the Project Management Institute and is by far the fastest growing form of software development methodology used today. However, Agile still has its problems. First, you can’t simply flick a switch and be Agile. Organizations have begun to understand that adoption of Agile is very much about organizational change – it needs to be well positioned, teams need to be appropriate trained, and it should be rolled out incrementally. In addition, Enterprise Agile and Agile at Scale has yet to become mainstream, as we see the majority of Agile adoption happening at the team level, versus the organizational level. Agile will require some additional “tweaks” regarding end to end maturity, adoption and scale to bring it out of the Trough of Disillusion.
DevOps Influence on Agile
The latest buzzword in the software development community is DevOps. DevOps is a software development method that stresses communication, collaboration and integration between software developers and information technology (IT) professionals. DevOps is a response to the interdependence of software development and IT operations. What is key to understand is that the concept of development working closely with QA and IT to produce solutions is not a new concept. A more holistic approach has existing for decades stemming from a more holistic mindset of process and flow (more on that later when I talk about Lean). DevOps involves more than just software deployments: it’s a set of processes and methods for thinking about communication and collaboration between departments.
This is really one of the key missing ingredients in the traditional (simple) view of Agile today. Most organizations don’t want to deal with DevOps for a few reasons: the nature of organizational silos as well as the added complexity to Agile adoption. Most Agile methodologies promote tight integration between the business and developers, but it doesn’t stress how QA and IT should also be working as part of the entire software lifecycle. By all means, the traditional teachings of Agile really do focus on the Development teams. In today’s word, we’re seeing a collapse of departments – increasingly the line between development, QA, and operations is becoming very blurry and these silos will further collapse with the DevOps mindset.
This isn’t a surprising trend, and in some ways its saddens me that its only becoming a trend in 2013. So, why DevOps now? There are a few reasons for this. First, it’s the recognition that there is waste and risk between the boundaries of traditional development and operations. The “handoff” to operations is full of waste in both directions. Was the system been developed for the correct operational environment? Is the development team taking into consideration operations and maintenance cycles? Does the operational team understand the requirements of the system? What is required for operational and maintenance teams to properly deploy, maintain and troubleshoot the system? This recognition has been forced by an even greater need of efficiency and value production in software development, driven either by shrinking budgets or a “startup” mentality.
Second, DevOps can help us really understand how our system is being used. Many teams have begun using deep functional instrumentation (some call it application telemetry) to help understand usage patterns of the application. This telemetry, implemented in a production and operational setting, feed back into the product teams giving them insight into how users ACTUALLY use the software. This can not be done without an operational mindset.
Agile for the CEO – The Agile Enterprise
For many years, Agile has been seen as only appealing to the Development team. Agile was considered by senior management as “something the techies do.” In 2013, Agile will come to senior management, as long as it continues to mature and encompass a greater representation of DevOps while being finely tuned to the needs of the business. In fact, taking this even further, businesses will start to embrace the fact that they must work hand in hand with IT in order to truly push the boundaries of business and innovation – and ultimately survival – a trend some are already calling Business-Agile Enterprise. Ultimately, Agile is about producing the highest amount of business value with the smallest investment – to maximize return of investment by maximizing throughput, minimizing operation and maintenance costs, with the smallest investment possible. IT projects have become the lifeblood of new business operations – and for these reasons why wouldn’t the CEO want to have a deeper understanding of Agility.
With that said, Agile needs to be repositioned. Many years ago I was escorted out of a building after I suggested to the CEO of the company that I will be introducing Agile techniques to his IT teams and the CEO believed that Agile was just an excuse for his developers not to work hard and produce results when he needed them. Since then, I have never used the term Agile with a CEO, instead I would borrow from the land Lean and talk about removing waste, focusing on the production of business value, reducing risk by having constant review and accurate progress metrics: Ultimately getting more value out of the IT investment. As we see Agile move into the mainstream, coupled with influences such as Lean Thinking, this is a perfect opportunity to ensure that we are thinking holistically about the business from the top down as well as the bottom up.
I’ve been a proponent of the application of Lean Thinking to the Application Lifecycle Management spectrum for over 15 years. There are a lot of reasons Lean Thinking works in software development and I’ve often stated that “Lean describes why Agile works.” We’re going to see an increased velocity of Lean Thinking applied to software development in 2013, specifically in its relation and interaction with the business.
To some extent, some of the things I’ve already been talking about have been heavily influenced by Lean Thinking – specifically the greater focus of DevOps in Agile. In addition, books such as “Lean Startup” demonstrate that Lean Thinking combined with agile and incremental practices made really great business sense. This is just one example of how Lean will not only “sneak” its way into our way of developing software, but how it will be used to help unite some disparate worlds.
Beware, however, of the Lean hype trap. I’m seeing a lot of organizations switch their marketing from “Agile” to “Agile/Lean” very quickly and I suspect that this is due to the overall market trend. Years ago (and still to this day) many misunderstood Agile in the same way Lean is likely being misunderstood. I’d walk into an organization and ask them what methodologies they use to manage their teams and processes and would be met with “We use Agile.. we don’t write any specs, we don’t provide estimates, we can’t tell you when we’re going to be done.. so, we’re Agile” – I suspect many organizations will misuse the term “Lean” in the same way “.. we’re very Lean because we follow a few Agile practices… they are the same right?”
Patterns of Agility
Agile “patterns” have always provided value to those who want to adopt agile practices. Scrum (for example) is an example of a pattern of Agile, including many practices and rules that should be followed as a whole.
Scrum, however, may not always work in all situations. In addition, Scrum doesn’t address a great many set of Agile practices that fit well together (such as agile design and build practices, devops, etc). A few years ago, Stephen Forte and I started talking on something called the “Agile Buffet” where organizations would “pull” in agile practices based on their specific needs. We would always end our lectures by constructing a few “agile recipes” with the audience to demonstrate how this worked.
In 2013, I think we’re going to see more specific pre-baked “agile recipes” that will act as guidance to customers of different types, providing them with a list of ingredients along with the steps required to mix together agile practices that fit their specific needs and tastes. Heck, if it looks like the prediction isn’t coming true, I may just sit down and write these myself ;-)
Renewed focus on Specification
I remember the good old days where I worked on a team employing the Rational Unified Process to the extent of its abilities to build a large and complex insurance solution. The specification and elaboration processes were much larger than the implementation effort – and much of what we produced (from use cases, to use case diagrams, sequence diagrams, class diagrams, activity diagrams, and every other document and diagram you can think of) weren’t very consumable by anyone we provided them to. At the end of the day if we wanted to validate requirements with users we ended up using wireframes – developers who were meant to consume the rest of our piles of produced specifications never really used anything we produced, solving that problem by coming over and having a conversation about what to build.
For many reasons there has been a backlash against specifications. I know many “agilists” who spit in the face of the word “specification” claiming it’s not agile to create specifications – that it’s wasteful (this may come back to the misunderstanding of Agile in general that I keep on observing as well). In reality, this just isn’t true. There needs to be some form of specification created before you start writing code – some way to envision and communicate intent, to validate approach before effort is exerted. Today we see a great amount of effort being put into wireframing and designer mockups. These are fantastic since, for the most part, these types of specification are easier to produce than writing code.
Now, don’t get me wrong. I’m not saying that you should all go out and write exhaustive specifications now. What I’m saying will happen in 2013 will be a re-inflation of the world of specification used in different ways in Agile. Wireframes are great – but that may not all be what you need. I’m a big believer of BDD, and even BDD (a highly promoted Agile technique) leverages very discrete specifications in the form of Gherkin. Agile doesn’t tell you NOT to product specifications, but do so for a reason and produce them in a way that is directly consumable by someone or something.
Over the years there have been great strides in “specification” types and tooling and I think that in 2013 these techniques will become more mainstream helping organizations capture/produce/validate requirements even easier, as well as help to generate application components and associated tests even more efficiently. In fact, I believe the tie between “specification” and “test” is the strongest – so, look for some fantastic innovation in this area.
There is not a lot I’m going to say about git dominance. git has been growing in significance for years, especially in the realm of enthusiasts and startups. Git hasn’t made very strong crossroads into the enterprise – yet. In 2013 we’re going to see this change as git will continue to make large leaps and become an important fixture in both enterprise and lightweight development shops. In 2012, even Microsoft started to embrace this fact by providing basic git replication support to its Team Foundation Server product opening the door for merger of client-server and fully distributed source control approaches. We’re going to see much, much more investment around git in 2013 across the board.
Joel Semeniuk, EVP Agile Project Management Division, TelerikJoel Semeniuk is the Executive Vice President of Telerik’s Agile Project Management Division and is responsible for heading the TeamPulse family of products along with its integration with other Telerik products. Joel joined Telerik in 2010, coming from Imaginet – a company he helped establish in 1997. With 18 years of experience delivering world-class software, Joel is also a well published author and public speaker on Agile and Team Management topics, and has worked with companies around the world to improve their application lifecycle through the adoption of Agile and Lean techniques. You can follow Joel on Twitter @JoelSemeniuk.