The need for measuring performance
As engineering managers, you are no stranger to metrics. From lines of codes to uptime, the software development industry is almost too obsessed with numbers. However, when it comes to measuring the performance of an engineering team, engineering managers often get stuck.
There are two types of people: those that don’t believe in quantitative metrics and those who do. I belong to the second group. That is not to say I don’t look at qualitative feedback, simply that quantitative metrics can tell us multiple stories and paint a better picture, if you know what to measure in the first place.
In this article, I will look at how you might think about what metrics to measure, based on the unique situation of your team and what steps to take in not only measuring, but improving the performance of your engineering team. Because the ultimate reason why you want to measure it in the first place is so you can make it better. Remember:
“You can’t improve what you don’t measure.”
Types of metrics
There are two types of metrics you should look at when it comes to measuring the success of a software engineering team.
Qualitative metrics
The first type of metric is qualitative. Usually, this can be achieved by talking to people; employees and customers.
Many software companies use NPS (Net Promoter Score) to measure customer satisfaction. Looking at customer-driven metrics ensures your software engineering team is delivering value to business and customers.
Under qualitative metrics, you may also want to get 360° feedback on individual team members to evaluate their performance.
When qualitatively measuring the performance of a team, pay attention to the low performers within the team. As the saying goes, one rotten apple spoils the barrel. Although it may just be one single person in a team of ten, that one person can have a significant impact on the performance and morale of the entire team, if their performance is not up to scratch. Being able to identify low-performing engineers as early as possible comes from experience.
There are multiple reasons why an engineer may be performing poorly. Some of them are:
- Disengaged with work (bored, not interested in tech, not interested in company’s goals);
- Other personal matters affecting them;
- Not feeling belonging in a team; and/or
- Do not have the required competency.
After identifying the root cause, the software engineering manager can then come up with an appropriate plan. Possible actions are:
- Move to a different team or project;
- Give them time off to recharge;
- Provide training/resources;
- Pair them with other more senior engineers; and/or
- Provide them with a smaller scope work.
If none of the above works after a set period of time, it is time to consider letting them go.
Quantitative metrics
The second type of metric is quantitative. They can be represented numerically and obtained scientifically. Examples of quantitative metrics that can be used to measure/manage quality, delivery and productivity in software product development are:
Time to delivery
- Cycle Time
- Lead Time
Quality
- Defect rate
- Percentage of test coverage
Uptime (reliability)
- Latency
- Time to recovery
- Incident detection time (Mean Time To Detection)
- Incident recovery time (Mean Time To Repair)
Speed of delivery
- Average age of closed bugs
- Team velocity
- Number of releases
- Number of story points completed
- Rate of story completion
As I’ve written in my book, The Engineering Manager’s How-To Guide, it’s important to measure what matters and use metrics that relate back to the business goals. Otherwise, software engineering teams become too distant from the ‘Why’ of the business.
How to get started
The following are recommended steps in putting together metrics for your engineering team:
- Start with the goal — An example of your goal might be to deliver reliable features at speed.
- Think about what metrics you want to measure that are aligned with the goal and how you want to measure them. Once you have defined these metrics for your engineering team, it’s important to monitor and stay close to them. Without monitoring and keeping track of them, it’s as good as not having any metrics at all.
- Assign people who will be the owners of the metrics and work with them. These people could be engineering managers, staff engineers or software engineers who are passionate about particular metrics.
- Put together tools and mechanisms. Trust but verify – Automation and alerting, Slack, If This Then That (ITTT) tool to set up smart alerting.
- Set a cadence for reviewing metrics. Eg: fortnightly meeting for quantitative metrics and quarterly review of qualitative feedback.
- Keep monitoring, reviewing and improving!
Thanks for reading and if you want further career advice and resources for software engineers and engineering managers feel free to check out my website.
***
If you or your CTO / technology lead would benefit from any of the services offered by the CTO Craft community, use the Contact Us button at the top or email us here and we’ll be in touch!
Subscribe to Tech Manager Weekly for a free weekly dose of tech culture, hiring, development, process and more.