A Former Pilot's Perspective: How To Cripple Your Team Morale, Motivation, and Productivity

During my IT career, and especially during the times where I was supposed to work in an Agile environment, I heard, saw, and experienced many uses of Agile. One theme caught my eyes and while I like numbers and some spreadsheets from time to time, I decided to "break my relation" with that theme - velocity. A former colleague and friend shared a similar view on the matter and we decided to pass the message through the aviation prism, as we are both former private pilots.

This work resulted in two conferences, the first version for a small group during a monthly schedule at Communauté Agile de Québec, and a second version, presented at a much bigger event, Agile Tour de Québec in October 2019,  was enhanced with the help of another fellow friend and agile coach.

Why Indicators In Agile Development?

In the history of Project Management, people have been formatted to measure and control everything. You even have one of the 5 Project Management processes called "Control and monitor". And when agility showed up, it was natural for people transitioning from "traditional" project management to "agile" project management to use reference points they were used to. And when agile arrived pretty stripped of those notions, they just put some in place. There you got velocity and its infamous points (more on that later).

One book, in particular, its title namely, has had a stronghold in managers' mindset. However, most of them have missed the point. And everything is in the title.

Scrum is not about doing twice the work in half the time, but is about "The art of doing twice the work in half the time". As human nature is all about shortcuts, this one was deemed hurtful for many. Art takes time to learn and master. It's not just something you apply like a recipe. Taking this shortcut managers confirmed a need to monitor and control... by using velocity and its points. 

It's the same shortcut that pushed Waterfall under the light, whereas it was simply a draft of a much more complex process in the study it appeared into.

From this point forward, most of the agile teams started being output-oriented instead of outcome-oriented like the agile manifesto recommended.

Another element to take into consideration is the software context. In 2008, Jurgen Appelo described IT development as a complex system in its blog post.

In this mixture, you can also add our VUCA working/organization context (Vulnerable, Uncertain, Challenging, and Ambiguous) and you have a good recipe for failure.

"Simple and complicated systems are fully predictable. [...] Complex systems are not constructed, they are grown." - Jurgen Appelo (2008)

Starting from there with the aviation metaphor, when we present in front of you the following three cockpits, the motorcycle being a relatively simple system, the automobile a relatively complicated one, and the plane a relatively complex one, where do you think the velocity indicator is?

While it is obvious for the motorcycle and the automobile, it is more difficult to find the plane velocity indicator. Even in the car, we can see that speed - which is not exactly velocity, but I will make the amalgam for the purpose of simplifying the explanation- is not the main indicator. You already have a lot of other information presented.  In a plane, it is one of the smallest indicators. It is important, but not as much as you think.

So with the evolution of the complexity of a system, we can notice that the velocity (speed) is less and less important to follow. 

While flying, as a pilot, you have a routine to do with your eyes. You check multiple indicators routinely for the purpose of safety and to ensure that you are flying in the right direction. One of the routines which are learned by many is the following :

1. Artificial Horizon
2. Altimeter
3. Artificial Horizon
4. Heading indicator (directional gyro)
5. Artificial Horizon
6. SPEED

And, of course, simultaneously you need to follow what's happening outside. 

It is confirmed by Doc Norton's newly published book:

"Velocity is a simple indicator of a very complex system. To measure creative work by throughput alone is to not measure it at all; quality and impact are essential" - Doc Norton, Escape Velocity

Then we can ask ourselves, as team members, managers, directors, or even CEOs, what is the message we are transmitting to our colleagues or employees. Are we looking for output or for outcome? If you are a manager or a Director, what is the perception of your teams? Virpi Oinonen has greatly illustrated this situation with one of his cartoons.

 With that said, velocity is still an indicator necessary but as a component of other metrics, as we will see later. For the time being, we want to give you keys to detect and understand the misuses of the velocity heard, seen, and experienced so far. This is not an exhaustive list. Being able to detect them, is the first step into acting on them and finding better ways.

How Velocity Is and Has Been Misused

Being Focused on Velocity

This one is pretty easy to detect. Everyone is focused on following the velocity and during sprint review, you may hear or have heard; "Yay, we have delivered 26 pts, congrats everyone !". However, if you place yourselves as the customer, it is difficult for him to determine what was deliver with these 26 pts.

It is similar to a pilot announcing to its passengers that they are flying at 900km/h. Are we far from our destination? Is it safe to fly at this speed? Is it airspeed or groundspeed?

So detaching yourselves from following only this indicator is critical to your team morale and work quality. We have to say that this behavior is decreasing in the last years. Thank you all Scrum Masters and Agile Coaches!

Forcing the Same Velocity To All Your Teams

"In a matter of facilitating the synchronization of the delivery...", or whatever reason people can give you, "...it is necessary for all teams to have the same velocity". 

This one is very harmful to the teams. Such as illustrated in the picture, when two different planes are flying at the same speed, they do not have the same difficulties neither the same stress on their system.

Here we have a P-51 Mustang flying together with an F-22 Lightning. The P-51 needs to fly at a very high speed so the F-22 can follow him. But doing so, the stress on the engine, the structure, and even the pilot is too great to sustain it for a long time. 

At the same time, the F-22 needs to fly at a very low speed to be able to follow the P-51. The stress on the pilot is as much important as a too low speed would certainly be fatal to the pilot and the plane. Here again, this is far from sustainable. 

Watch for your people not burning out.

Comparing Velocity Between Teams

As much harmful than the previous misuse. Teams are not identical, their context - technical or business - is not the same, the competencies needed and brought by your players are not the same and their daily routines are not the same. It is then a waste of time to try to compare velocity between teams.

It is like comparing the two types of planes illustrated here. Why aren't they flying at the same speed? Why haven't they the same shape? 

While they are both great at training pilots, the left one has characteristics of its own that the right one blows out of the water. And it is normal. The Cessna is one of the most used planes for training pilots while the Canadair is to train jet pilots and accessorily supporting the Snowbirds (Canadian Acrobatic Air Patrol). I have yet to see Cessna patrolling like the Patrouille de France or the Red Arrows for that matter.

Asking For a Higher Velocity Each Sprint

In the following video, you will see an experienced pilot flying a performant aircraft. However, they will take-off from a high-altitude airport with a short runway.

A short runway is already a challenge as not all aircraft are agile to the extend of taking-off in short distances and being at high altitude means that the mass of the air is thinner than sea level. The consequences are that more speed, more distance, more power is needed to put the plane in the air.

While taking off, the plane is stalling very quickly (meaning the sustentation on the wing is working) and even if the trained pilot managed to keep the plane stable, we can see that top of the trees are higher than the plane. Which is very dangerous and luckily, nobody is hurt. You can even hear the pilot saying he will never do that again later on the video.

Pushing teams has to be done when ultimately necessary and never regularly. It can be a way of testing their limits but even though, I would not recommend doing that.

Keeping the Same Velocity, Sprint After Sprint

What if you find the same plane, with the same flying crew and pilots? Do you believe the latter should always aim for the same speed at landing? Do you think that flying conditions are always the same whatever the team's experience?

It's the same for your teams, while experienced and used to work in a complex system, you never know what can happen. A sudden crash, the classic "it was working yesterday", etc.

Watch the following videos considering its the same plane and crew. Then ask yourselves if everything will always be the same and if the velocity is as planned, as always.

Your Team Is Crippled: Watch For the Impacts Yourself 

Not much to say as a picture is a thousand words, but if we had to quickly pass through each point:

• False estimations: it has been demonstrated that teams will game the system whenever they can. If you push a velocity onto them, they will comply, but you will get a completely false predictability
• Team morale going down: when they lost the will because of numbers, team members tends to bail-out or be counter-productive
• Customer Dissatisfaction: yes, they are impacted. A team working for the velocity number at the expense of delivering value to the customer is bad. For your team and your organization
• Lower quality: if a team needs to deliver points, it will deliver points. But you need to find the trade-off. most of the cases, quality is on the top of the list. 
• Standardization of practices: meaning the lost envy to innovate and progress. Points  and fully busy over continuous improvement
• Technical debt: hitting the velocity objective sprint after sprint has never been done. There are obviously parts that are left behind.

If you want to avoid such avenues, I recommend you to continue the reading. I sincerely hope that, at this point, you have the keys to detect such misuses and now we will show you some keys to mitigate them and board into an uplifting journey.

No Velocity? Then What?

While we have passed the more common misuse, it is time for a little theory and fact checkpoint.

First of all, neither in the Scrum Guide nor the Agile Manifesto do we found a mention about "velocity". While in the Scrum guide they try to make reference to it, they have removed the burndown charts part early on. On the agile manifesto on the other hand, the minimalist principle state :

"Working software is the primary measure of progress" - Agile Manifesto

I give that it is stated "primary measure", and I confirm there are other ways. Including a sense of  velocity  speed. 

Speaking of which, here are some statistics across the world on the uses and practices observed. It is usually well resumed in the VersionOne State of Agile Report. In its 12th edition (edited in 2018 for the year 2017), the survey report that the way success is measured with Agile Projects is done mainly by the following three indicators :

• Customer/user satisfaction - 46%
• Business value delivered - 43 %
• Velocity - 42%

What is notable, is when comparing these numbers with the precedent survey, the first two saw a 20 percent points increase while the velocity lost 25 points (it was at 67%). As you have guessed, the precedent year, velocity was on top. In 2018, for the year 2017, velocity dropped another 5% points to 38%. In 2019, for 2018, velocity dropped another 1% point to 37%.

What is most important are the other indicators which took over Velocity in the rankings :

• Customer/user satisfaction
• Business value delivered

Another major upward trend observed was the metric "budget vs actual cost" in 2017, confirmed by the same rank (4th) a year later.

So, here we are. Indicators are other than Velocity. Noice! Let's dive into the options.

Other Indicators

Again, we do not want nor have the knowledge to dress an exhaustive list, however, to the best of our knowledge these are the most interesting and underestimated indicators that can help you, daily.

Team Health

There are many, many options to monitor your team's health. Jimmy Janlen in his book "Agile Toolbox, visualization examples" reference various options, from which "Team Mood Barometer" or its "Stress Level Meter" are good means for that.

On the Japanese side, Niko Niko is one other technique to follow the team mood, by simply using smileys.

Others will use the attendance metric as a means to detect any motivation drop. Feel free to experiment, empirical steps are the bests!

Flight Parameters

Following Klaus Leopold Kanban's Flight Levels, this section is dedicated to levels of discussions and monitoring.

Strategic Flight Level

High-level discussions don't need micro-informations. Therefore a quarterly meeting should be enough for what we can call "portfolio management". Here a set of indicators you can establish while at this level:

• Percentage of projects delivered on time
• Average Lead Time (also Time to market)
• Customer satisfaction (net promoter score, fit4purpose, etc.)

Our aviation thematic could relate to this level like reading a pamphlet with what a flying seeing tour could allow us to fly over. No details, a global time for the tour, and the main attractions to watch for. Simple as that.

Tactic Flight Level

We are at the level where business meets IT. At this coordination level, it is more important to focalize discussions around collaboration and planification. Usually, you can establish a monthly meeting, with  the following indicators to support your conversations :

• Deliverables per period (with the objective to use Monte-Carlo simulations for predictability)
• Bugs in production (following a delivery)
• Budget consumption (what value delivered & in progress vs budget used so far)

It is easy to link this level to any map you can use for your flight purposes. Like in this example, you need to determine the flight paths you are allowed (or not) to fly through, as you need to identify secondary airports in case of emergency or any radio frequencies you will have to use.

Operation Flight Level

This one is mostly for the Team to help itself to coordinate its efforts. Obviously, any routine meeting is recommended to adopt a continuous amelioration mindset. Daily scrums, revues, retros, etc...

With what we now, the team can establish these sets of indicators :

• Team Delivery Rate (DR =WIP/Av CT)

1. Team's WIP (daily average)
2. Team's CT (85th percentile)

• Flow efficiency (effective time/passthrough time)

It's like creating our flight plan, which will be adapted during the flight, comparing what we estimated in the briefing room and the reality, so we can adjust immediately our next estimations between checkpoints. Aviation pilots were doing agile sooner than any of us!

From experience, using lean-agile metrics is always a good idea. However, most of the time, this needs a mindset change for many people as they are not used to read and understand such metrics, and there may still have a need to correlate these metrics with the organizational accountability that may not be that "agile".

If it's the case, do not hesitate to tell the management that the team has its own metric and will find a way to interface them with corporate metrics. 

Lead Time and Cycle Time

We have talked about Lead Time and Cycle Time many times and they have the right to have their own section. These indicators, while still "lag indicators" are critical to ensure the pertinence of your business and your team efforts.

While the Lead Time is the time between the moment you receive a request and the moment it is delivered, Cycle Time is the time between any start and end of a task supporting the delivery of the request.

While traveling, let's say, from Montreal to Hawaii, you will have a Lead Time that starts from the moment you enter Montreal's Airport and will end when you will exit Honolulu's Airport.

However, Cycle Times during this trip can be time between starting and ending your:

• Process of registration;
• Departure border passthrough process;
• Security check process;
• The first leg of the trip (MTL-TOR);
• The second leg of the trip (TOR-LOA);
• The third leg of the trip (LOA -HON);
• Arrival border passthrough process;
• Luggage retrieval process;

If you have noticed, these are "valuable" Cycles Times. It means that the time spent for these steps is valuable to your trip because they let you move one step closer to your final destination.

However, there are also "non-valuable" Cycles Times that you should not forget, such as "time to find the right registration desk", "time while waiting in the duty-free area", ["time for onboarding", "time to offboarding"] x3, ["time to transit to another plane"]x2, etc.

Being aware of all these cycle times will allow your team to focus efforts on eliminating the time wasted between valuable cycle times and then optimized the valuable ones. Many airports have taken actions on this, like Heathrow (London) which diminished the number of lost passengers during a month from thousand to a dozen thanks to better signage from checking to plane boarding.

Cumulative Flow Diagrams

Very close to lead time and cycle time you have the Cumulative Flow Diagrams, a much better version of any burn down turned up charts that help you determine your Delivery Rate.

It is well explained in the following video (I do not have any interest in nor am I a member of the company presenting it):

The "Unknown" Business Value

Most of the team I worked with had difficulty to grasp what is business value. I believe this is mostly due to the fact that it was difficult for them to be the customer of themselves. Their knowledge of the market, the competition, and final user needs were scarce. 

One way to illustrate business value is to determine, like in our example, what the customer really wants.

Is the customer looking for a comfortable flight that will bring him from the town center to town center with luxury food and seats, and obviously paying the price for that, or is he looking for a cheap flight without all the cow and bells?

Well, having the ability to answer such questions is important for any team. And that implies being able to connect to the user quickly and at any time, as much as being plugged to the market and its trends. It is mostly a business indicator but the IT division should be at least able to understand how the context of the organization can impact its decisions and orientations.

I like using the following matrix to help the discussion when it focused on R.O.I.

Statistical Approaches

If you are fond of numbers, one set of indicators that can help you are scattered plots and percentiles.

They will help to establish predictability within your team and help you with accountability puzzles you may face in organizations. You will however need a minimal set of data to make any use of it. Usually, we say you need between 1 to 3 months of data depending on the type of elements of work your teams have to churn through.

Scattered plot charts will help you determine the percentiles for a type of card for instance. When you have sufficient data, you will be able to say that "80% of working elements of 1 day of cycle time during the last 8 months are done within 22 days". Therefore when a batch of 100 1 cycle time cards are coming your way, you can predict, using Monte-Carlo Simulations - courtesy of Daniel Vacanti -, that these 80 cards may be finished within the next 22 days.

According to statistic studies, when someone says "on average it takes 2 days", he is in fact saying "there is a 54% chance that it will take 2 days". This is not a good estimate to base our plans on, right? So we will usually use the percentiles 80 to 90th to have better predictability.

Investments

This one is more tricky of the gang. However, there are hybrids approaches that can help with the formal accountability of any organization.

In the excellent book "#NoProjects: A culture of Continous Value", Evan Leybourn and Shane Hastie are trying to establish solid financial indicators for agile teams. While I wasn't able to personally test them, the example given in the book is pretty straightforward. They describe "Efficiency" and "Productivity" indicators according to what usual accountants will look for in projects for the organization's accountability. Which is related to following CapEx and OpEx (Capitalization and Operation Expenses).

In the cases I encountered, we simply established, for the sake of having a financial indicator for middle managers and directors we were dealing with, what we called hybrid indicators. Using the backlog, we created three ratios :

• Stories in progress vs total
• Stories delivered vs total
• Budget consumed vs total

In doing so, we were able to identify the projects in difficulty and discussed why they were not progressing. And it is pretty easy to read and understand. For instance, if your first ratio is 70%, the second 0%, and the third is 10%, it seems that you are starting this project and you have engaged a relatively small investment for almost 3/4 of the backlog.

However, if the third ratio is 60%, you have a serious problem to resolve. But this is still a lagging indicator so you need to be careful in using it and not using it for predictability.

In Conclusion

From all these indicators, we could have influenced you to built a dashboard. But this is not our final recommendation. You still need to live the action with your team, the indicators only supporting your discussions and decisions.

However, if you still want to go the dashboard way, look for an adaptive one instead of a static one. Like, prefer the futuristic dashboard of an Airbus 350 or the SpaceX Dragon Crew one (on the right) over the Concord one (on the left).

On a final note, we would recommend you to not dive into planning too far in advance, because the indicators cannot help you predict a year-long of work. They are mostly indicators that give you readings on your team actual health and in some cases a very short-term preview. As Jeff Gothelf put it here in his blog :

Many thanks to Guy Larochelle to build with me the original version of the conference and Jean-Romain Cordier for enhancing it for the knowledgeable audience of Quebec's Agile Tour.