The metrics series, vol 1: A personal framework
How to get your team's temperature in a way that's actually constructive
I’ve been wanting to write a series of posts around software development metrics for a while, but hesitating because I didn’t feel like pontificating about it. However, what’s the fun in learning if you cannot spread the word? So… here goes nothing!
I’ve been managing teams for 3-ish years now. During this time I’ve spanned between inspecting and fine-tuning a single team of 6 people, or trying to stimulate momentum, consistency and some degree of predictability across up to 6 different units and around 40 people. This was actually humbling and showed me that different contexts need different approaches. Read between the lines: there’s no secret recipe. But here you have a framework that works for me.
In this post I’ll set the basis of my framework thorugh the 3 steps I follow when I START to measure (i.e. I am assigned to a new team). The specific details on how to measure every one of those things will be featured in the next posts. Maybe this whole thing should be an ebook, who knows.
Step 1: Pick your key indicators.
There’s no point in measuring everything, but furthermore, very often you’ll find the data you can track does not necessarily map to what you are really measuring.
So I try to make it simple. What’s my job?
I am accountable for the realization of the company strategy, through the delivery and operation of a specific software product, in a timely manner, that behaves as expected. I am also responsible for hiring and growing the teams that develop said product.
My framework will try to assess at every moment if I am meeting that goal through 6 indicators:
Are we delivering that product?
Is the product performing as desired?
Are our timings consistent with expectations? (deliver)
Are we able to iterate with agility? (operate)
Am I hiring well?
Are the teams thriving?
Are the individuals having a good experience in the team and the company?
As you can see, I ask my framework 6 questions; 2 of them related to the product, 2 related to the *speed* (or better said, the pace, the flow) and the last 3, to the humans. Based on my experience, success is unlikely if one of those 3 aspects fail. Your framework might look like this, if you are in a similar position and company, or it might look like something totally different. The key idea here is that it should allow you to empirically assess that the things that matter to you are going fine.
I will dig in each of these questions, the data points to answer them and their possible meanings in the next posts of this series.
Step 2: Be humble. Set your baseline.
Once you gather all that data, you will feel tempted to start extracting conclusions. Just don’t. The only thing you get when you start measuring is a random snapshot in the life of a team, and rushing to build a story around it will only make you cheat yourself and your people. Be patient and build your time series.
Some tips:
Metrics on product performance can be heavily influenced by market trends (i.e. seasonal products, new niche market, new competitor…). Do some research and know your industry before considering yourself ready to jump into conclusions.
If you are going to track delivery against deadlines, those should not be arbitrarily set. They should either be committed by the team (and you’ll still won’t be measuring their speed, but actually how good are they at refining, removing blockers, how precise where the specs, how mature they are as a team).
Metrics around agility and development operations can be very sensitive to holiday periods, members on sick leave, new joiners, departures… the good news is, those events will let you measure the resiliency of the team.
Hiring metrics will take you at least six months to be relevant, because humans need time to start belonging into a team, the market is somewhat seasonal, and changes in the pipeline need time to be propagated.
When measuring individuals, look at least at full semesters. Don’t fall into the vanity of measuring optimism in new joiners or people who just had a raise. Be honest, let the honeymoons and raises settle and monitor bigger cycles to get a real grasp of how the people experience the company.
Collective mood fluctuates. If you track the typical manager satisfaction, mission alignment or personal satisfaction metrics, for example, you’ll see they tend to move up and down in a very hectic way, but on the long run, they rarely cross the line of their very own, predictable range. Those numbers change very eloquently after a bad CEO meeting, a project decomission, a winter party or the annual raise. It’s normal, keep on looking at the overall trend.
Step 3. Improve one thing at a time.
Once you have a meaningful data set, make a list of 3 things that need to improve, then pick one you can actually influence. Work on that one for a while, and inspect as you go.
Although it’s really tempting to pick that Medium post you read on turbo performing teams and just apply the whole set of ideas, changing several things at a time will intoxicate your results and increase the challenge of interiorising new habits.
Lastly, be transparent. Share with your people about what you think can be improved, what’s the benefit you expect, and what do you want to try. You should have valid reasons to alter the way they are working at the moment, and they should understand how the changes you are asking from them will have a positive impact.
Humans are not your puppets, but on top of that, keeping it for yourself will make you miss the chance to have them helping you. So please be a good human and approach your changes as a joint effort, not your mad scientist experiment.