Managing the Business of IT (4/4)

By Mark Helme


Date 15 July 2010 Tags

Why “performance targets” don’t deliver performance

CIOs worry about what constitutes success – what they need to achieve, and how they should be judged. Apart from questions of applications and infrastructure and cost and change projects, they need to have a clear understanding of the levers they can pull and the effects this will have. They need to understand the practice of performance management, and of setting targets.

Performance is not an absolute but should be matched to need; but what should we measure, and how often? How should these statistics be used, and most importantly, how will people respond if they are turned into performance targets? Because performance target are often used as management incentives.

Even if you know what behaviours or outcomes you want, and even if you can measure what you think are reasonable proxies, there is still a real danger that target and outcome pass like ships in the night. People then try to hit the targets, not what the target was supposed to incentivise. Below we look at some examples of the unintended consequences of targets designed in good faith to motivate particular behaviour. We argue that IT may actually get in the way; because even if it facilitates measurement, it does not always do it in a good way, as we shall see..

Old economics text books rarely looked at how incentives actually work (preferring the myth of the “rational economic man”) but recent research recognise that people respond to incentives in many and unexpected ways, and consider the rest of economics to be mere commentary on this fact.1

Evidently there is a fair amount of good practice already out there. For example:

  • Measure things that can be measured easily (e.g., numbers of people on line, not their reasons for being on line) and which are likely to mean something to people. It is also often recognised that reporting on the availability of IT elements doesn’t necessarily translate into something people can experience, i.e., the health of the end to end service. (“I’m hitting all my SLAs but customer satisfaction is still bad!”) It probably means something’s wrong with the SLAs).
  • People understand “process success” (e.g., number of orders shipped or planes leaving on time, incidents resolved) rather than the events which contribute to the outcomes (orders input, boarding tickets printed, telephones answered).
  • On top of this we need to decide what a reasonable outcome is, and whether they need to vary for specific users, processes or times of the month (e.g., Finance at month end, Marketing at product launch time, the Chairman always). Evidently better performance typically comes at a cost, but why for everyone, all the time?

So we set desired levels of performance, and we start to measure inputs and outputs. Just measuring something can change behaviour – the Hawthorne effect. However, this is often exaggerated and fades quickly – what really makes a difference is turning our measuresinto management targets. The issue isn’t whether the target is achievable, but whether it drives the right behaviour. Too many managers think their work has to involve setting targets (because that’s just what managers do), and just hope that their staff understand what is wanted.

This isn’t physics, it’s psychology: – people respond to targets, especially if they have incentives to hit them (and penalties for failing to). That’s why we set them and put them in SLAs with outsourcers (to keep them honest and focussed). But they don’t always drive people to do the right thing.

People respond to incentives, but often in ways we don’t want and didn’t expect. The philosopher of science Jerry Ravetz once wrote, “wherever there’s a system there’s a racket to beat it”, and the evidence isn’t lacking. People can cheat (like closing tickets before the incident is resolved) or they can concentrate on hitting a target relating to an element of the IT chain at the expense of “doing the right thing”. Something similar appears to be endemic in the NHS and the police services, if some recent (anonymously written) books are to be believed.2 Getting patients out of the A&E wards (and into corridors) within 4 hours and inflating the detection rate by including trivial crimes which are simultaneously logged and solved without anyone going near the justice system would never happen without targets. In Hanoi, under French colonial rule, a programme paying people a bounty for each rat pelt handed in was intended to exterminate rats. Not surprisingly people farmed rats instead, and took the bounty. New houses are mandated to have a proportion of “green” light bulbs, but they take time to come on. So they are used in places hardly ever used, and are left on to the point of using more, not less, energy.

It is essential to understand the real “incentive structure” of a target/measurement regime, and to recognise how people will in fact be incentivised to behave, and to try to predict the unintended (and unwanted) consequences before instituting it. (But having said that, what should the French have done about the rats?)

“What gets measured gets managed” said Peter Drucker. And it’s true – managing without data is hard, and attention will be focussed on what is measured. The danger is obvious: throughout the entire re-engineering movement from the 1980’s onwards, companies have sought to ensure that key processes were designed to increase customer satisfaction at the lowest possible cost, often requiring extensive automation. The automation that has provided hitherto unimagined opportunities to measure things, has supported, if not directly caused, an explosion of performance and measurement targets, which in turn has led to unwanted consequences.

Reengineering was meant to design business process from an end to end perspective, and if possible through the lens of the customer. A fragmented target regime just subverts that.

In our experience there is a real danger that any target used to drive behaviour at a sub-process level runs a serious risk of being subverted. People will chase targets, not what they know to be in the organisation’s interest, because, perversely, the organisation has not made it in their interest to do that.

This does not mean that we don’t need data; we are not better off flying blind. We need to measure extremely carefully, and extensively, and we need to be sophisticated enough not to be deluded by an inappropriate and amateur use of statistics. We must not be bumped into extensive remediation programmes without understanding underlying drivers and root causes of our problems. But there is a huge difference between a measurement and a target. Having the data helps better decisions, but people need to understand the real goals of an organisation, and the culture needs to reinforce this. Otherwise people may chase (their own) inappropriate targets.

Gather and disseminate as much information as is needed for people to intelligently use it to support the super-ordinate goals of the organisation. Articulating what those goals are and communicating them to staff, emphasising that internal functions will be judged on how comprehensively they are achieved, is clearly at least as important as any measurement regime. A rich set of information enables better trade-offs, and helps people to take wider responsibility.

Some time ago (in If you’re not smart sourcing what are you doing? ) we suggested defining required outcomes and getting the suppliers to agree to deliver them, but this is hard to do in practice, especially if Procurement or Legal insists on a very tightly drawn up service credit regime with fragmented SLAs. It seems to us that one of the principal failures of outsourcing relationships is that the SLA structure, far from focussing the outsourcer sets up a parallel universe.

The number of times we have heard the wail “That’s not what I thought I was signing up for!” is surely an indictment of the industry. Next time we’ll look at this in more detail.

1. See for example the writings by: Steven E Landsburg and Tim Harford

2. In Stitches: The Highs and Lows of Life as an A&E Doctor by Dr Nick Edwards and Wasting Police Time: The Crazy World of the War on Crime by PC David Copperfield.

« Back to blog
Subscribe to our RSS feed.
©2018 Differentis   Sitemap Privacy Policy Terms of Use Website by zoomedia