The Common Sense About Metrics

These days, you cannot do much without running across some discussion of metrics and measures. It’s not that I think that there’s anything wrong with it, but I do think that it’s a potential point of failure if not done at the right time and properly.

In any area where we are concerned about quality, cost, or performance, the first thing that many seem to think about are critical success factors (CSFs), key performance indicators (KPIs) or common “process-centric” metrics (e.g. “average hold time,” “problem closure rate,” etc.).

The Common Sense About Metrics

It might seem that these are the right things for a service provider organisation to pay attention to, but it may not get us the results that we really want. Why? There is an underlying assumption that, because these are the things that we “in the ITSM space” often talk about, these are the things we “should” be paying attention to. After all, this is “best practice,” right?

Well, not so fast there. While it may be common, I wouldn’t say that it’s the right thing to do. The main reason is that the context from which the question gets answered here is critical. If that isn’t properly established, we are almost guaranteed to miss the mark. The main reason here is the fact that there is always a skew towards a single perspective, which is most often that of the provider.

The Key Questions

The two key questions that we want to keep focused on in any discussion of metrics or measures are

1. Who is it important to?
2. What is important to them?

If we can’t establish the first question, the second question is automatically going to be a problem. The bottom line here is that, if we don’t know the answers to these two questions, we are just guessing at whether or not it is something that we should be tracking. From my perspective, getting to the point where we can actively discern what is/isn’t important is critical.

The underlying assumption with the key questions is that, in an economy that is based upon a service experience, customers will place value those things that directly impact their experience.

What Is Really Important?

Unfortunately, most practitioners are not used to this kind of thinking. They are most familiar and comfortable with looking at it from their (likely silo-based) perspective. It’s not that this is wrong, but it’s most definitely limited. Given our historical boundaries, it’s why we need to find ways to develop our ability to think in this way.

At a minimum, we need to examine the situation from multiple roles (both the customer and service provider perspective) and orientations. Each of these combinations will give us a different view into what metrics we can/should pay attention to. Additionally, each view (minimally) has an inward and outward facing component to it.

So, if a practitioner makes a change to a supporting process area (say change management, for example), more often than not, they will consider how the change affects the service provider, not necessarily how it impacts the customers experience of the service actually being provided! In some cases, it’s even worse than this, as some practitioners will limit their thinking to the scope of the supporting process area in isolation of all others.

There are elements of role play and scenario-based thinking here. To start, we need to think about things as if we were the customer and ask the key questions. Ask the questions and record what you believe the answers are. Once you’ve done this, if at all possible, engage your actual customer to get their feedback, and compare the responses from the actual customer’s perspective. If done over time, this will help improve your accuracy and reliability in taking a “foreign perspective.

None of this is difficult, but it does require skills that are normally not required.

Making an Example of Someone

Given we’ve talked about what we can be doing, let’s do a little thought exercise to reinforce the key points we’ve just reviewed. To do this, we’ll use our friendly service desk manager to examine how this might play out operationally.

As we noted earlier, there is a customer and a service provider side to any conversation about metrics and there is an internal and external component to it. To examine this, we’ll arrange things in a bulleted hierarchy/list:

• Customer/Service provider perspective
  • Internal/External
    • Central concern(s)
      • Candidate metric

For the purposes of this exercise, we’ll look at the things that our service desk manager might be regularly reporting to the customer organisation, as part of an effort to help build the case for the value that the I.T. organisation provides. In addition to this, we’d also expect that there is also regular provider-side reporting to assess how well the organisation is performing.

For any given organisation, the mix of these might look something like this:

• Customer perspective
  • Internal
    • Efficient and effective operations
      • Time to restore service [average, min, max; trend line]
      • # of service disruptions
    • Being a good steward of corporate resources
      • Cost of support
  • External
    • Reliable delivery against stated mission objectives
      • Customer satisfaction rating [average, min, max, trend line]

• Service provider perspective
  • Internal
    • Driving operational efficiency
      • Staffing levels [by period]
  • Building reserves for future investment and contingency
    • Cost of service delivery
    • Cost of service support
  • External
    • Demonstrating customer value
      • First call resolution rate
      • Support satisfaction rating [average, min, max, trend line]
      • Cost of service

The key thing in considering any candidate metrics is to consider that metrics can be combined in a way that helps tell a story. If our intent is to help demonstrate business/customer value, our story should emphasize those aspects of value that resonate with the customer. We should also be able to turn right around and craft another story that we can play specifically for the service provider organisation and emphasize those aspects that resonate.

In either case, we need to compare the different metrics to each other over time, visualize the relation-ships between decisions made on both the customer and service provider sides (e.g. staffing levels, funding, finite resource allocations, etc.) and their operational effects. In this way, we use “hard data” to enhance the credibility of our stories.

Summary

As you can see, there can be significant differences between what a customer values and what a service a provider values. This is both right and proper, as their needs are not created equal. In order to think about what metrics are appropriate to pay attention to, at any given point in time, we must consider both the customer and service provider perspective. This requires that practitioners develop specific critical thinking skills. The greater degree to which these skills are developed, the more easily you will be able to select a more balanced set of metrics that provide real insight into how well your organisation is actually delivering service.

We believe that service providers exist to provide business value for their customers. Indeed, it’s front and center when they make decisions about how they will organize themselves to service their customers. Given this, service providers need to pay attention to those metrics that will help reliably build the case for clear, compel-ling demonstrations of business value.

About Ken Gonzalez

Kenneth Gonzalez is an internationally recognized industry expert and leader in technology management, Outside-In Thinking, Lean and IT service management. He helps business, practitioners and customers alike better understand and apply these principles to deliver value and tangible results.