Posted by: communicationcloud | December 9, 2011

Measuring success. Part 2: Creating a measurement framework

As I mentioned in a recent article, it’s not uncommon for people to ask me what are good measures for successful software customer experience. My answer is probably not particularly helpful: it depends what you mean by “successful”.

In an attempt to give a more useful answer, in my last post I wrote about how to create a success map that defines success in a UX, technical communication or customer service team. Here’s that post, in case you missed it: Measuring success…part 1: Defining success

This article is the promised follow-up post, in which I attempt to explain how you can make use of the success map to define a set of measures that you can use to evaluate and communicate your progress towards success.

Here goes…

So, you’ve mapped out what your purpose is. To work out how successful you are at achieving it, you’re going to need to define some metrics. I’m going to use the success map I mentioned in my previous post as an example, so here it is again to refer to:

If any of your goals are defined outside of your team, then note down the measures for them first. In most cases, these will be the goals at the top of your success map – probably the ones that relate to the financial or customer perspective. For example, you may have been told already that the cost of support is to be measured as “the cost of the support team per product license sold”. Make sure you understand exactly how this is measured and what the target value is – without that you may find it difficult to deliver improvements that affect it in the way that you want to. This is probably also a good time to ensure that you’re able to keep updated about the current value.

Once you’ve added these external goals, it’s time to develop your own measures. I find it easiest to start at the bottom and work up to develop measures, and then work back down the map again to add in the specifics of target values that you want to achieve.

  • A couple of points on defining measures;
    When you’re defining measures, it might be useful to remember that not all measures need to be based on large quantities of formally gathered data. For example, if you have a goal for team members to feel valued, then you could measure this by asking members once a month to score how valued they currently feel on a scale of 1-10.
  • If you struggle to come up with measures then it may be that you need to add more sub-goals to your success map, and use an index measure to evaluate success. For example, in the “Customer” section on the example success map above, the goal “improve search engine performance” is very difficult to measure in itself (in this case, it’s because search engine performance is a complex interaction of different factors). Luckily, we’ve already defined two sub-goals for this which are easier to measure (I wrote a article on measuring findability, where I covered this: How do you know your site search is working?).
  • As well as the measure and a target value, it’s also useful to keep a note of other factors about each measure: where the data will come from, the frequency that you will update your measurement, who is responsible for calculating this, and so on.

Step-by-step example of developing a measurement framework

1. Define the measures, working up the success map from the bottom, as described above.

Learning:

  • Goal=Product training.  Measure = % of team members who have had full training on all products (alternatively: % of products each team member has had training in all products)
  • Goal=Customer feedback. Measure=frequency of customer survey (alternative: number of customer comments per product per month)

Internal:

  • Goal= Produce the right product-support topics. Measure=Number of support calls about issues for which there is no information topic available

Alternatively, this may be a case where you want to add in more sub-goals on the success map – e.g. two sub-goals: Don’t produce anything that won’t be used and Produce information that covers all the main issues that users have.

Which you choose here really depends on your specific context.

Customer:

  • Goal=Increase online self-service capability. Measure=Number of support calls dealt with by support team per license sold

Financial:

  • Goal=Reduce cost of supporting users. Measure (external)=cost of support team per product license sold

2. Define target values for each of your measures, working down the success map  (this is because the goals we want to achieve are defined from the top down).

Financial:

  • Goal=Reduce cost of supporting users. Measure (external)=cost of support team per product license sold

Target= £1

Customer:

  • Goal=Increase online self-service capability. Measure=Number of support calls dealt with by support team per license sold

Target: 2 calls per license

Internal:

  • Goal= Produce the right product-support topics. Measure=Number of support calls about issues for which there is no information topic available

Target: <5 calls per month about issues for which there is no information topic available

Learning:

  • Goal=Product training.  Measure = % of team members who have had full training on all products external goal.

Target: Each team member has full training in at least 75% of products

  • Goal=Customer feedback. Measure=frequency of customer survey (alternative: number of customer comments per product per month)

Target: monthly

The measurement framework

Put all this together, and you end up with a measurement framework that looks something like the following:

Incidentally, if this is beginning to look like a balanced scorecard to anyone, that’s not entirely coincidental; this measurement framework is a good way to build a balanced scorecard, which in turn is a useful tool for communicating about your progress.

What do other people do?

In this post and my previous one, I’ve described a way of defining, measuring, and communicating success. This is just one way of doing it, though. There’s an interesting additional perspective in this article by Rahel Baillie, plus some ideas about the associated issue of demonstrating value too: The ROI of content 

How do you do measure and communicate success?

If anyone has a go at using the framework I’ve described here I’d love to hear how it goes (and I’m happy to answer any questions you have while you’re doing it, too).

If there’s any interest, I may follow up at some point with some detail on types of measure and what they’re good for.

Postscript

Once you’ve worked up your success map defining measures, and back down again to put target values against each measure, you may have spotted a mismatch between what you need to achieve and what’s possible. Let’s say you need to reduce support costs by 10% in a year, but there’s no possibility of getting all the training you’ve identified as part of what’s needed to achieve this. As far as your success map and measurement framework go, this is okay! These tools have worked to identify a mismatch between goals and capabilities. To resolve the mismatch, you’ll need to do one of the following:

  • Change the target for the headline goal
  • Add in extra capability
  • Find alternative ways of achieving your goals

Some related articles

Here are links to other articles I reference in this post:

By me:

By other people:

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Categories

%d bloggers like this: