Why Agile Fails Because of Corporate Culture
Large corporations are increasingly longing to be like startups, but the road to Agile success is lined with failed projects, frustrated employees, and more.
Join the DZone community and get the full member experience.Join For Free
Large corporations are increasingly longing to be like startups: Flat hierarchies and the absence of formal procedures result in unseen productivity; doing without restraining bureaucracy allows for remarkable speed, innovation, and creativity.
Speed, innovation, and creativity: It's exactly these things most corporations are desperately lacking. In turn, it's only logical for big companies to want to take a leaf out of the startups' book. Where the clumsy imitation of a start-up mentality fails or maybe isn't even an option, the magic word agile soon echoes through the aisles. Because you know, the competition is already successfully agile, every consultant can deliver ready-made, highly scaled agile wonders on demand in no time at all, and even the house and farm service provider has had experience with it for a long time, but simply wasn't asked about it until now. So "agile" gets written on the banner, swarms of self-proclaimed evangelists get unleashed onto the staff and off they go on a journey into a brighter future.
But the road to agile success is lined with failed projects, frustrated employees, resistance, trench warfare, burned money and wasted time. Why do large companies have such a hard time with the self-indoctrinated start-up mentality?
From painful personal experience, I believe there are two reasons for this, which always occur in the same context:
- They simply aren't made for it
- They want to be agile instead of becoming agile
Why Large Companies Aren't Made for Agile
Large companies are large because they have perfected the ability to streamline themselves for efficiency. Everything has to function with as little effort (cost) as possible and therefore has to be unified and standardized, deprived of any individuality and, of course, scalable.
Growth is the only premise, scaling is the mother of paradigms. In fact, however, efficiency as the foundation for growth nowadays is no longer an advantage:
It'n no longer just a matter of solving the same problem as efficiently as possible, but rather of identifying the most important problem out of emerging problems (effectiveness) and then solving it efficiently. What it means is that effectiveness has to be valued over efficiency. Moreover, efficiency is no longer achieved by standardizing stuff, but rather by thinking outside of the box when trying to leverage the existing knowledge in the company to solve a new problem.
(Ari Byland, translated from German)
Scaling - the second bedrock paradigm of large companies - has side effects that are barely beneficial to the agile mindset. Scaling (and growth) require roles and processes, regulations, hierarchies, salary levels, performance targets and annual reviews, growth planning and learning paths.
As a matter of fact any company will, beyond a certain size, will end up in an organization with several overlapping types of silos - regardless of how it earns its money:
Fig. 1: Classic silos: The matrix organization © Thomas Klein / redbots GmbH
The problem with these silos is that they kill even the most tiny trace of emerging agility. Silos are installed to establish responsibility and accountability, both in terms of personnel and expertise. They are limited by budgets and serve to map performance, regulate staff training and further education and are the cornerstone of everything that moves in the haze of the term "scaling". But even just the topic of budgeting alone is nothing but the death of agile:
Budgeting is built on two fundamental assumptions:
The future is predictable and, that people can't be trusted.
It doesn't get more un-agile.
Budgeting and its accomplice, the "estimates", are therefore based on the assumption that the future is predictable or at least estimable. Agile methodologies, on the other hand, were born out of the fact that the whole "estimating and planning" thing apparently didn't work out quite well frequently - and instead, people wanted to be able to quickly react to change.
Management Doesn't Mean Leadership
The trunkload of items a silo carries (or results from?) features a certain tough item: Management. In some way, agile organizations also sport a kind of "management" - however, there's a crucial difference: While a classic matrix organization has managers, each with their own competence and responsibility, the agile organization holds the term leadership. And leadership isn't tied to a spot in an organizational chart but anchored guts and minds of the people who are lead by someone because they trust them. Someone who, instead of managing a project, supports the people behind it to make them become their own best, who motivates and encourages peers instead of telling them what to do and taking decisions for them.
In total, the characteristics of silos ultimately lead to effects which are the exact opposite of agility, because they... :
- introduce/increase dependencies for solving problems individually
- impede collaborative, solution-oriented work
(especially beyond the scope of a team or project)
- restrict communication and knowledge sharing
- limit reasoning, creativity, and innovation
- prevent learning and further development beyond the daily business and
- foster a bureaucratic mindset, routine and dullness
It's hard to imagine a more hostile environment for agility.
Companies who try to be agile without sacrifying their beloved silos (and the processes and roles associated with them), will face a much bigger - yet often hidden - problem. One that cannot be solved neither technically or functionally, by processes or even by doing without processes, because it is cultural in nature. A change in the way of working in the company will not and cannot succeed on its own unless there is a corresponding change in the corporate culture. In fact, the cultural change would first have to be implemented, experienced and underpinned with new values emerging from that experience. Only then -but then out of genuine conviction- companies will be successful in changing to the new way of working. Yes, I'm talking about the often-cited agile mindset.
"Agile mindset" doesn't mean the usage of sticky notes for teamwork but living to the values of openness, transparency, courage and respect - to mention a few.
If You Can't Get Agility Right on a Small Scale, Scaling Agile Will Just Scale Your Dimension of Failure
With the traditional values of the old corporate culture (scaling, growth) being so long and deeply anchored in the company’s DNA, very little time, if any, passes before calls for scaling agile are heard after the first step towards an agile way of working was made. In here, scaling bears a new danger: All the remaining, rather small frictions in the processes also get scaled - up to a point where roadblocks appear in areas unknown in the past. In other words, if you can't get agility right on a small scale, you'll be doomed to fail when scaling.
The price to pay of successfully working agile is high: A perceived loss of control and a dsciplined focus on the essential stuff (with the team deciding upon what is essential). For many folks, especially those in mid-level project management (which itself is becoming almost redundant now) this price is too high. To cope with that, attempts are made to "adapt" the agile framework so that it fits the established silos and processes. The fragile newborn agile framework is thus torn apart, overhauled and reassembled. I call this "frankengile" - a creature that looks agile from the outside, but behaves like a typical, classic matrix delivery on the inside. And that's what the company feels comfortable with. That's what they know from the past. And hey, now that they know it how it worky, they immediately also know how to scale it in the blink of an eye.
Unfortunately, though, it is like cutting the wings on a new airplane to make it fit the existing hangar: The fancy new thing shows off to people passing by the hangar. Everyone on the ground is happily bragging about it. Yet - that aircraft will never take off.
All attempts to scale agile usually end up in a weird mixture of chaos, patronizing and arbitrariness that doesn't work any better than anything before, but - because it's scaled - now frustrates more people than ever. A monstrous mammoth framework that makes you wonder what could actually be agile about it.
Time to Destroy the Myth of the "Scaled Agile" Panacea
But ain't there no way out of the dilemma that agile doesn't fit large companies? Indeed, there is: But its logical consequence are simply not feasible for large companies:
Don’t scale agile... descale your organization.
(Henrik Kniberg, video)
This statement indeed comes from nobody less than Henrik Kniberg, a pioneer of agile methodologies and one of the leading participants in the design of what is known as the famous Spotify model.
Kniberg's retrospective assessments of his own publication have already been commented on and analyzed frequently - specifically in order to be able to deduce why scaling agile methodologies into the dimensions of large companies mostly fails. I particularly like the following quote from an article by Anthony Mersino, who comments on Kniberg's statement and gets to the core of the issue:
It isn’t about changing agile to fit your company, it is changing your organization to achieve business agility.
This nails it. If we now consider the deeper meaning behind the statement, it becomes clear that we have been going in circles and came back right to the point of corporate culture: Agile principles simply don't match the organic corporate culture of large companies.
Before we move on to the second point, I would like to make a comment on the Spotify model: It's finally time to destroy the myth of the "scaled agile" panacea. As Kniberg himself has reported several times, it was never intended as a model or framework for scaled agile work, but merely a description of the modifications and extensions to existing agile frameworks, used at the customer Spotify.
Unfortunately though, the topic developed such momentum that it was snatched out of his hands by starving disciples of scaling, who then promptly installed Squads, Tribes, and Guilds in their own and other companies, only to be left waiting for what they thought would be a certain success. Unfortunately, in almost no case did this work out particularly well, not even for Spotify itself, as many stakeholders have since reported.
And, to be honest, whenever I take a look at the visual representation of the Spotify model, I don't see nothing but yet another (probably multi-nested) matrix organization:
Fig. 2: Modern silos: The "Spotify model" Henrik Kniberg / Anders Ivarsson (pdf)
Why It Is a Problem to Be Agile Instead of Becoming Agile
Agility thrives on adapting to change caused by internal or external factors, observed by continuous self-assessment. The ability to do so is based on two fundamental skills:
- Trust in the capability of the involved actors to decide on the best-suited solution given the current situation on their own - and providing appropriate creative space (and time) to enable them to act accordingly
- Allow people to fail! Moreover, accept failure as a positive event that enables us to learn, improve and change for the better. In fact, failure is the one and only experience we make that actually sets the ground for improving
In most companies, however, providing creative space and encouraging to fail aren't really core values, are they? Therefore, it's the creation of this awareness which is is a fundamental obstacle and the only real reason why agility fails because of corporate culture.
The weirdest fact about all this: The failure of agility in the company is itself a failure from which companies learn nothing. Why? Because failure is not seen as an opportunity for improvement, but as failure. And so it goes on and on; a vicious circle from which companies can only break out through a culture change.
This change requires certain skills from all those involved. For some, these skills have yet to be awakened; for others, they're probably present and may only just need to be fostered. But all of those involved will need to be coached, on a long-term basis. Becoming agile is an evolutionary process and it can't just be avoided or saved-upon by taking advantage of the wisdom of others or applying arbitrary agile frameworks from a consultant's shelve to be agile.
And let me stress that: Evolution.
That means it's ever-ongoing. It's not a one-off big bang thing and whoohoo you're agile. As a C-level guy or gal, as a board member or shareholder it will take loads of guts to go that way. Think twice. Unless you're willing to make steps into the future without ever reaching a "done" state you should probably better stay un-agile. There's nothing wrong with running a traditional, classic company. It's at least far better than wasting money by doing agile wrong.
Okay, you get it. But how can such a change succeed? Or, by using Kniberg's words, how do you "de-scale" an organization? In short, why not by taking the disruptive character of agile working literally and thus decomposing silos that make up a scaled company. And ideally, you do that in an agile way: By starting small, trying things out, failing, and continuously improving.
In other words, it's not by following a plan in a centralized step-by-step manner, but rather by giving up a certain amount of control. By letting people try things out (but please make sure they are mentored) and let them openly exchange their views on the results.
Yes, this admittedly may sound like a total chaos, but it's nothing different than a small-scale greenfield strategy where people are allowed to try things out. The important part is that you don't just start an experiment at a single point, but let several teams takeoff in parallel. These teams should not be too identical in scope, in order to avoid the impression of competition, and there should not be too many teams, in order to be able to guarantee constant and dedicated support from agile coaches.
On a journey to an agile enterprise, I'd recommend the following points as a travel guide:
- Agility (creativity and innovation) needs freedom
- Allowing for freedom means giving up control
Both in terms of time and budget
- But: Experimental activities must be (generously) time-boxed
- Don't ask for knowledge transfer but encourage it
- Shift from JourFix / status meeting to learning lunch, brown bag
- Use individual dashboards on open data instead of siloed reports
- Do spikes, hackathons, pairing, "public" demos with colleagues
- Use wikis instead of Intranet, try InnerSource with your code
- Accept failure (...and learn from it)
- Celebrate change instead of fearing it
- "Do the wrong thing - and talk about it!"
- Promote leadership instead of management
- Recognize, inspire and promote potential
in teams and individuals
- Create opportunities instead of giving instructions
- Apply strong personal coaching instead of rigid learning paths
- A leader works for his team, not the other way around!
These are just some random thoughts. It's neither an exhaustive list nor one with items to check-off one by one. Think of it like a list of ingredients and develop your own recipe from it. And please always remember that this is a development process, not a milestone plan. Do enjoy successes - and celebrate failure as a moment of awareness that your recipe is not yet to everyone's liking.
Being a star chef does not mean serving other chefs' dishes. You may be successful in the short term, but to stay at the top in the long term, it takes creativity and innovation. And these two ingredients are expensive - because they require the courage to fail and the determination to try again and again.
Opinions expressed by DZone contributors are their own.