There are a variety of well-known software development metrics that engineering teams can use to inform their decision-making. DORA metrics help teams track how well they’re doing in terms of the speed and quality of software delivery. Retrospectives and surveys shed light on how software engineers feel and what problems they face in their daily work.
Once you get started, there’s a virtually endless list of things you could measure. However, the most important question for any software company is: what are we getting in exchange for our investment in product development? The answer shouldn’t be a list of features or lines of code. Instead, your investment in software development should positively impact the key metrics of the business.
The tricky part for many software organisations is that product management often owns impact metrics. Cross-disciplinary work is not always painless, and it is easy for engineering leaders to just “stay in their lane.” However, aligning engineering teams with the needs of the business is crucial for their success. Engineering leaders, and especially CTOs, should invest in understanding how they can do this in practice.
In this article, we’ll discuss three ways to ensure your software teams are making an impact, not just shipping lines of code.
1. Feature adoption
Feature adoption is a great place to start when you’re aiming to understand the business impact of engineering. If no one is using the features you ship, you’re not building the right things or failing to make them discoverable. In any case, you’re not creating value for your customers.
Feature adoption doesn’t automatically guarantee growing revenues and profits. People who get value from your product are more likely to pay for it, but this relationship also deserves a critical look. When analysing how well different features correlate with key business metrics, engineering can often provide unique insights on the possible technical aspects affecting feature adoption, such as performance and reliability.
Help from engineering is also crucial when instrumenting feature adoption tracking for your apps. This is true even when you use ready-made tools for tracking feature adoption, such as Amplitude, Pendo, and Heap. More so, if you want to use something like Segment to relay events to your data warehouse.
Comparing feature adoption between paid and non-paid users for different features
2. Investment balance
Another way to understand the impact of software development is to categorise the engineering investment you’re making into different types of work. For example, do you know how much engineering effort is going into driving your company’s top priorities, keeping the lights on, or dealing with technical debt?
Engineering leaders should actively promote a healthy view of engineering investment, where the organisation is not only focused on building new stuff but intentionally invests in improving productivity. This allows the organisation to continually ship their top priorities and not end up in a slump, where technical debt slows down everything and puts the business in an awkward position for a long period of time.
To get accurate data about your investment distribution, you should pull data from both your issue tracker (e.g., Jira) and version control (e.g., GitHub). Issue trackers usually capture non-coding activities better, whereas version control tools offer you a view into the coding activity.
Investment distribution between different categories of work
3. Business-specific leading indicators
Lagging indicators such as revenue and profits change too slowly to steer the work of software teams that are continuously shipping new code. To fix this, your organisation needs leading indicators that give you more immediate signs of the business impact of engineering.
Here are a few engineering-related examples your on can draw from:
- Slack famously found that teams that had exchanged 2,000+ messages had really tried the product and 93% ended up as long-term customers. A proxy like this could be used by developers responsible for optimising the onboarding experience
- A company might want to focus on enterprise customers to drive up their Average Revenue per Customer. Since revenue numbers are lagging, they could use conversion rates and churn of larger companies as leading indicators instead. This directs engineering to focus more on problems that are common with enterprise customers like user management, security and compliance, etc.
- A software team building a robot for picking up products in a fulfilment centre could optimise the error rates of picking the wrong product or dropping the product.
Some leading indicators will be useful for years, while others will help you measure and improve engineering impact in the short term. That’s why engineering leaders should also frequently review the usefulness of the leading indicators they’re using.
Wrapping up
It’s easy for CTOs and engineering leaders to focus on the technical side of things and lose sight of the business outcomes of their work. You should resist this and ensure you always know how you’re measuring the business impact of engineering. At the end of the day, it doesn’t matter how quickly you’re building software and how flawless the code is unless you’re providing value to your customers — and the business.
While identifying leading indicators that are meaningful for engineering may take a few rounds of experimentation, you can get started with tracking feature adoption and the balance of engineering investment quite quickly. The sooner you start, the sooner you’ll be able to help your engineering teams focus on the work that drives the key metrics of the business.
***
Our next CTO Craft Conference is confirmed! We’ll be live online on November 14th – 16th with three days packed full of expert speakers, fascinating panels and the ever favourite, fireside chats.
For more information, read our recent blog.
If you or your CTO / technology lead would benefit from any of the services offered by the CTO Craft community, use the Contact Us button at the top and we’ll be in touch!
Subscribe to Tech Manager Weekly for a free weekly dose of tech culture, hiring, development, process and more!