How To Introduce Data-Driven Culture To Your Dev Team
In this article we will discuss applying agile principles can help dev leaders roll out a data-driven culture in their teams and more!
Join the DZone community and get the full member experience.Join For Free
In this article we will discuss:
Applying agile principles can help dev leaders roll out a data-driven culture in their teams.
Tailoring the “why” for your metrics differently to different people helps with translation.
Starting small reduces risk and increases the likelihood of bottom-up adoption.
Adapting your metrics to company goals can increase business alignment.
Embedding your use of data into daily rituals can help with stickiness.
The first time I ever thought about using data to help run my team was in 2015. CloudLock was growing fast and overnight (it felt like) I went from leading a single team of 6 devs to leading 5 teams with 50+ devs combined (as VP of Engineering).
I had no experience dealing with an organization of that size. The processes that had worked for us when we were smaller weren’t scaling for our current size and situation. I was pretty sure I needed to be doing more to help my team but I wasn’t sure what.
I’m not afraid to ask for help so I hired an agile coach to come in and talk to our entire team — engineering, product management, and UX. She made a huge difference. I’m still using a lot of what I learned from her today.
One of the most common challenges I hear from software development leaders is…
Every time I share how we do things at LinearB I hear Kara Minotti Becker’s voice in my head. So I figured I better see what she’s doing now. It turns out she is still an agile coach in high demand. It also turns out I still have a lot to learn from her.
The ideas and videos below come from our Zoom conversation on Friday, April 10th.
After we caught up (she has a 7-year old girl, I’m expecting my first later this month), Kara asked me...
What prompted you to reach out to me in 2015?
The short answer is, I was in over my head.
I asked her what she remembered about our engagement.
She mentioned, “You guys were hungry for understanding the cultural piece of agile. How it helps people work together. Not just how we set the machinery up.”
I remember that too. I’ve always cared about culture. When it comes down to it, as a dev leader, I can’t do anything without great engineers who are driven to help the team.
If I don’t create an environment where talented people want to work, where they feel safe to be themselves, where they can make friends and explore their creativity and learn and have fun…I have no chance to deliver results for the business. I also believe in the power of data. And that’s why I’m passionate about this question.
What Steps Can Engineering Leaders Take to Get Their Entire Team to Buy-In to a Data-Driven Culture?
Kara and I landed in four key areas.
1. Start Small and Take an Agile Approach
“Engineering leaders need to be careful not to crush their teams— Kara
under the weight of too many metrics.”
When it comes to a new product or feature, we collect the most important requirements, ship an MVP, collect feedback and iterate over time to make it better. For some reason, we forget to take that approach with our internal processes.
If we try to measure too many things from the beginning, we introduce risk. Scope creep. Delays. Loss of momentum. Just like a new feature, the first version of your metrics probably won’t be perfect regardless of how long you take to roll them out. You’re better off shipping something fast because you’ll start learning sooner.
Starting small also gives you a better chance of getting bottom-up support. When you pick 1-2 north star metrics that represent the main goal for your team, it’s easy for everyone to get on board. Anything more than that increases the likelihood of confusion.
PRO TIP: At CloudLock, when I had a new indicator I wanted to measure, I would try it out with a single team to start. After they spent some time using it, I would incorporate their feedback. Not only did this help me work out the kinks before rolling it out broadly, but it also created champions with the team who helped me convince everyone else it was a good idea.
See Kara explains the importance of starting small and iterating.
2. Align Your Engineering Metrics to Business Objectives
The next big question is…
What are the most important things I should be measuring?
If only it were that easy.
Kara emphasized, “There are no standard metrics that all agile teams should measure. It’s all about knowing what your company’s goals are and picking metrics that help you figure out how you’re contributing to that.”
She also believes that what you measure needs to adapt over time as your company’s goals change. “If your metrics are not aligned to company goals they are pointless.”
I went through three iterations in 5 years at CloudLock.
Value mode: Early on the company’s mission was to deliver as much new product value as possible. So for engineering, it was all about speed to value for us. We had a lot of incredible ideas for our product and our job was to implement them as quickly as possible. I was an individual contributor developer for most of this phase and I got promoted to team lead towards the end.
I remember our leadership focusing a lot on velocity during this time – the number of story points each team delivered in each sprint.
|I think velocity is the most dangerous, most misused metric in software development. It can be useful to inform sprint planning within an individual team. But it should never be shared outside of that team and it should never be used to measure productivity.
Our leadership was trying to measure dimensions of our process like speed and efficiency. We probably would have benefited from measuring Cycle Time but back then the ideas and tools didn’t exist to get that data easily. So we settled for what we could measure – things like velocity.
More on Cycle Time later in this post.
Growth mode: At this point, we had a lot of customers but adding more was still the main goal for the company. We had a bunch of salespeople and they were making commitments to prospects. We had a lot of new marketing people and they had launches and PR announcements.
We still needed to ship fast but it was even more important to hit our deadlines. Changing from the mindset of “ship new value as fast as possible” to “ship new value more predictably” was not easy. We were still under pressure to deliver more value but we couldn’t change priorities on the fly anymore.
This phase became all about predictability. By now I was the VP of Engineering. I measured iteration churn (how many items came in and out of the sprint after it started) and the percentage of story points delivered versus committed. These metrics were the best proxies I had to determine our consistency. And, to be clear, we only looked at this data for each team and never published these to the whole org or compared teams against each other using these metrics.
Customer mode: Eventually we reached the point where 500+ large enterprise customers used our product every day and relied on it. Keeping them happy became our priority. We brought in a VP of Customer Success. We had a top-line company goal of 90% customer retention year over year. This phase was all about quality.
You might think changing our mindset from predictability to quality would be a little easier than changing from speed to predictability. It was harder – for three reasons. First, our org was much bigger at this point so there more people we needed to get buy-in from. Second, going fast feels good for everyone. Devs like it. And the business likes it.
Nobody likes being told we need to delay a release even if they know a potential bug can have big consequences. Finally, our most important internal customer changed. In predictability, your main internal customers are still sales and marketing. With quality, your main internal customer becomes customer success and technical support. There were new people we had to get to know.
We measured the number of issues, bugs, and outages per release which we called “change failure rate”, as well as mean time to restore and mean time to resolve support tickets.
PRO TIP: Looking back on it, we were using a lot of output metrics when we should have been using process metrics. When you focus on an output like how many story points you delivered or how many bugs you had, what do you do when you miss your mark? Measuring the process helps you figure out where and how you can improve. And many process metrics are leading indicators of your outputs so measuring them helps you predict when you have a problem coming and allows you to do something about it proactively.
For example, at LinearB right now, instead of using velocity to measure speed, we look at Cycle Time which is the time it takes us to deliver new work from the first commit to release. By measuring each phase of our cycle, when this metric goes up as it did for us recently after we transition to 100% work-from-home, we know exactly where to focus to get back on track.
3. Translate the “Why” Behind Your Metrics to Different Audiences Differently
“Engineers are scared of metrics. They’re afraid of how they’re going to be used… who they’re going to be exposed to.”— Kara
I think Kara is right. It’s not just developers.
When shared in the wrong way, engineering metrics can be confusing to the rest of the business too.
“It is true that you have to show executive staff numbers that have meaning at their level.”
But that doesn’t mean we shouldn’t make an effort to teach our business the key aspects of software development.
As dev leaders, we are somewhat caught in the middle. So what can we do?
Context is everything. If you tailor the “why” behind each of your metrics differently for different audiences, you’ll have a better chance to land your message.
Using LinearB as a case study…
We’re a 25-month-old start-up with strong initial traction and a big vision which means we’re in value mode right now. Our mission is to deliver as much value into our product as possible.
Since we care about speed to value, Cycle Time is a core metric we track. We look at it in every CEO staff meeting, sprint retro, and dev all-hands. We’re especially focused on it in Q2 because our Cycle Time went up a lot last month after our shift to 100% work-from-home.
Cycle Time has a different significance to different people in our company.
Our dev leaders care about maintaining delivery momentum. They like Cycle time because it shows bottlenecks in our process. For example, we were able to see that our coding time and pick-up time were the main contributors to our Cycle Time going up in March. So to get us back on track, our dev leaders adapted some of our processes in those areas to fit work from home.
Our business leaders care about acquiring more customers. They like Cycle Time because they see if our delivery speed is trending positively and they can correlate that to new feature releases.
In Q2 our execs are also looking closely at a phase of Cycle Time called Review Time – the amount of time it takes to merge a PR after the review process starts. Review Time is fairly in the weeds but we’re using it as a proxy to measure teamwork between developers. If Review Time goes up, it means we may need better processes or technology to help our remote devs work together better.
Our developers care about innovating, building cool stuff and trying new things. They like Cycle Time because a faster cycle is a signal that they get to take on more challenges more often. They also like it because bottlenecks in the cycle (like in Release Time, for example) can be an indication that more automation is needed. Devs hate non-creative, manual work. And of course, our developers like to see the connection between their work and the success of the company. They think it’s cool that our CEO looks at an engineering process metric every week.
PRO TIP: Kara says when it comes to your dev team, “make sure they know what good looks like.” This can actually be hard when you’re starting because you may not always have benchmarks for everything you want to measure. But you can’t get everyone to rally around a metric if they don’t know what they’re shooting for. Set yourself up for success by creating small achievable milestones to start. This will make it easy to celebrate small wins along the way which builds momentum for the bigger goal.
4. Include Metrics in All of Your Existing Rituals
The best way for your metrics to become part of your culture is to “operationalize” them into your existing activities.
Daily stand-up, 1:1s, scrum of scrum, retrospective, all-hands, coffee talk… every meeting you hold would be better if you showed data.
We used to broadcast our LinearB dashboard on the TVs around our office. Now we pull them up in every Zoom meeting.
PRO TIP: Early in my journey of using data to run my team, a mentor told me to give praise and constructive criticism in the context of the metrics we care about as a team. Not only will that reinforce data as a key figure in how the team works, but it will also make your feedback more concrete and actionable. For example, at LinearB this quarter since we’re focused on pick-up time, each week we acknowledge the dev who helps their teammates by picking up PRs the fastest.
Using Metrics in Your Executive Staff Meeting
“We need to keep them [the executive team] looking and pointing in the same direction and their metrics are the only way they can do that.”
“We need to keep them [the executive team] looking and pointing in the same direction and their metrics are the only way they can do that.”
So Where Exactly Do I Start Introducing a Data-Driven Culture?
Kara has some great advice for how to start the whole process.
If you don’t have time to watch the 3 minute video, here is a summary of what she recommends:
- Get your people in a room.
- Ask leading questions about what problems they want to solve.
- Shut up and listen
Here’s how to get in touch with Kara:
Published at DZone with permission of Dan Lines, DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.