Measuring progress in software, can it be done?

Thorny topic this week, can you measure progress in a software business?

Done well, the measurement of progress through Key Performance Indicators (KPIs) is essential for any growing business to know how it is doing. When it comes to software though it often seems to break down; it’s challenging to know what to measure and why. Often things we’d like to measure turn out to be impossible, and things we can measure don’t help us.

For example, CEOs are often keen to measure a development teams ‘velocity’. Sadly though, any measurement that you can do around velocity is usually both highly subjective and context dependent. In practice, it means that it cannot be used to make comparisons between teams or predictions (no matter how hard you may try!) about future progress.

The result is that, in a lot of businesses, they give up. The CEO gnashes their teeth and wishes they had more insight into where all the money they spend on developers is going. But, not measuring isn’t an acceptable answer around here!

So, if we accept that that measuring can be difficult and needs to be done intelligently, what can we do? I’ve already discussed, for example, the need to pair behavioural and output metrics to ensure that what you think the numbers are telling you is, in fact, real. Are there measures specific to software and software progress?

For example, if you are “spending” 60 hours of development activity per week, how do you measure what output you are getting from all of this activity? How do you measure progress?

The first thing we should recognise is that there is a bit of a “you can’t get there from here” problem. What I mean is that if you do not have clear goals for the business and are not already measuring progress in these terms, then you can’t measure progress in development. You need to do that first. Then come back here. (Don’t worry, I’ll wait here for you).

It may seem obvious to you right now, but something that too easily gets lost when we get busy, is that creating software is a means to an end: we want to make a change in the world, and building software is how we intend to make that change happen. What this suggests is that our measurement of “software progress” is really measurement of “business progress” in a different guise.

This is where the second trap for the unwary lives, and one I’ve made myself too many times: software is not usually planned around business goals in any meaningful sense. If we’re doing it right our development priorities are driven by things like value proposition design and assumption mapping to ensure that we are customer focused and testing hypotheses around customer value. But there is a missing link. We assume that if we do the right thing for the customer, the right thing will equally happen for the business - too often this is simply not true!

It’s too easy for our software planning to be driven by features and customer journey. It may be directed around what we think the customer wants (at least if we’re value-proposition focused as we should be), but there isn’t a clear connection to what the business wants.

So, what is the answer?

In the first place the answer is that measurement of development progress is really about the Return on Investment (ROI) on development spend. Experience has taught me that measuring this means planning software development activities around changes to critical business measures, and only then in terms of value proposition/customer behaviours.

For example, if we were a SaaS business, we might target the goals for a sprint as:

Monthly recurring revenue +5%

Customer churn -2%

Employee satisfaction +2%

Operational spend -5%

We would then plan around what features we could work on that we think can best achieve these specific ends. Software progress is then around how quickly we get these changes in our business KPIs. If we’re spending a certain amount each month on development, we can measure ROI on that activity and decide: “Am I getting the progress I expect?” from a basis of real world measures that mean something to business.

How do you actually do this? I’m glad you asked. A process I’ve found helpful is called Impact Mapping. This may be a new term to you but it’s the best way I have found to close the loop so that development spend can be seen in terms of the real-world return to the business that any software CEO should care about.

By Impact Mapping you directly connect development activity to business goals, focus the organisation on delivering on business goals, avoid building software that doesn’t lead in the right direction, and give yourself a way of measuring progress against spend.

Have you tried Impact Mapping? I’d love to hear about your experiences. Does it sound interesting? I’d be happy to explain more over a coffee.