Unintended consequences of poor KPIs

In his insightful eBook titled, Generating and Selecting KPI Sets, Peter Hill argues that characteristics of good KPIs include alignment to the organisation goal, measurement of quality and correlate with process performance.

My experience is in agreement with Peter's assertions. Too many organisations fail to use "good" KPIs and reap the reward of usually negative unintended consequences.

Many examples come to mind. I will use two personal examples to illustrate.

I will start with a personal example of setting a poor KPI and I will follow with a poor sales and marketing KPI I inherited which dislocated an entire market.

Twenty five years ago I was in a role that had responsibility for thinking about new ways of boosting productivity of an industrial lubricants blending and package filling plant. The scope of the role included plant planning and scheduling as well as capital planning and process improvements.

The plant was beset with two major problems. Its productivity at an aggregated level of litres produced per man-hour was one-fifth the national average of five plants.

Most of the projects implemented in the plant to improve this situation in the past were capital intensive. I wanted to show that we could change the behaviour of people by measurement. I wanted to show that capital investment was not the only lever we had to pull.

Given that our production management team viewed productivity as our biggest problem, it was "obvious" to me that productivity was the measure we should use. We chose to measure the productivity of filling at the level of each package size. Thus, we had a measure and a target for filling 205 litre drums and another one for 20 litre drums and so on. We also had differentiated targets for different filling lines with different levels of automation.

This was a big deal. It took eighteen months of negotiation with the union and payment of allowances to be able to collect data to measure the productivity at the level we wanted.

At first, the measurement of productivity showed great results. We improved productivity by twenty percent just by measuring it. Undoubtedly, there was a degree of Hawthorne effect, but we did not care.

After about twelve months, however, things were not quite as rosy as we had thought.

Our annual cost analysis to determine our filling costs by product and by package size, demonstrated our costs had hardly changed at all and certainly not by twenty percent. It was clear that our "improvement" in productivity was not showing through in reduced per package labour dollar costs.

Further than that, our customer service levels which were already poor had deteriorated. Our out-of-stock levels had increased, our levels of returned stock had got out of hand and morale in the distribution area had plummeted. Angry customers regularly vented their feelings to our drivers and clerks as did our marketers and distributors with me and my manager.

It was not until a few years later, when I was manager of the plant, that it became evident to me what unintended consequences had resulted from the selection of package filling rates as our main indicators of performance.

The unintended consequences were:

• The plant supervisor developed an informal prime KPI of the number of 205 litre drums (the largest package) filled per hour as he thought this was a sign of high plant turnover.
• Our leading hands became skilled at allocating hours in both rostering staff and compiling the data (e.g. falsifying records to put time into training hours) to make the informal prime key performance indicator always look good.
• The leading hands became skilful at manufacturing excuses to not fill difficult, short run small package orders in order to keep reported filling rates up.
• Niche products, which were difficult to fill and usually were in small packages with short runs, were thus out of stock constantly.
• Large customers ordered the niche products from time to time and put them on back order when they were not available.
• When the niche products were not available for long periods of time, customers complained bitterly and/or ordered stock from a competitor. Their complaints were understandable as niche products were usually required to keep machinery running or was necessary for scheduled maintenance.
• When customers ordered from competitors, backorders which were sent out to them were returned as unnecessary.
• When customers complained enough about their impending plant shut down, our schedules were hastily rearranged with consequential loss of real productivity.
• Our leading hands became more skilful in reallocating hours to hide the loss of productivity.

The lessons learnt from that episode, put into Peter's language were:

• The KPIs were not aligned to the corporate goal which was sales. The levels of out-of-stocks would have been a better measure.
• The KPI was not measured by the filling process. We had no means of signing in and out of a machine to count labour hours and packages filled.
• The KPI did not measure quality. It only measured packages filled, not packages filled against the schedule.

We did eventually implement two replacement measures:

• The items in stock as a percentage of the planned number (some stock items were blend-to-order only).
• The degree to which stock keeping units were being replenished within the planned lead time.

Within six months of implementing those measures, returns were negligible, complaints were minimal, and the actual warehouse stock levels reduced by thirty-five percent and costs per litre fell by ten percent.

The warehouse stock level was an additional unknown unintended consequence of the measures. Our leading hands had, in an attempt to increase filling rates of 205 litre drums also been in the habit of making extended runs of high volume grades above the recommended stock levels. Extended runs meant no change-over time for flushing lines etc. Higher unit filling rates resulted from filling the same product over a period of a few days when compared with several different runs of a few hours each.

Many years later, I was marketing manager of the same organisation. This time I inherited a poor KPI which had distorted the market for lubricants in Australia.

The organisation I worked for was the market leader by market share. It had, however, not increased profit for ten years. It never made a loss, but profit was steady at around $10M every year. Return on investment fell slightly each year. This was despite product launches, technical innovation, new market entries, cost reduction exercises and several attempts to increase prices.

The formal targets for the business were measured as contribution to fixed costs. The business shared many facilities with other business units, so this was chosen over profit for reporting management.

The informal KPI was sales "volume" in either mega litres or tonnes. Sales volume in dollars was important informally but not as important as physical quantity. This was easily observed by listening to the stories which were told. The champion sales person was the one who sold higher quantities, not the one who made more money.

In fact, money was hardly ever mentioned as an informal measure, except in terms of a derivative of the formal target. That derivative was known as the "marketing margin". The marketing margin was defined as [unit sales price] minus [the sum of the unit cost of goods sold and unit manufacturing and distribution costs]. Marketing margin was a strong formal and informal KPI.

All eighty or so souls in marketing and sales KNEW what an appropriate marketing margin was; fifteen cents per litre for large volumes, thirty cents per litre for second tier volumes and "who cares" for small volumes. The very fact that people KNEW what was a good margin and that sales volume was such a strong informal target dislocated the lubricants market in Australia.

The manyfold unintended consequences of this target and KPI combination were:

• High volume, "acceptable marketing contribution" business was encouraged over small volume, very high contribution margin business.
• The organisation had high market shares in mining, steel, processing oil and wholesale. Wholesale constituted selling the base oils which constituted 80-90% of a typical lubricant to competitors.
• It had very low market shares in high margin business of food processing, high end manufacturing, power generation and motor cycle lubricants.
• Large quantities of base oils were imported to supply the high volumes required for the processing industries and wholesale supply. This was even though the Australian market was over supplied and major competitors exported their surplus or closed down plant for extended maintenance periods.
• Full time staff were allocated to very large volume customers who made very little, if any, contribution to fixed costs once the sales and support costs were factored in.
• The booming wholesale trade spawned many small competitors with very low barriers to entry. The new competitors competed on price.
• Each time a new initiative in the production area yielded a reduction in costs the benefits of those cost reductions flowed to the customer within a year. As the margin crept up due to the reduction in costs, sales people became uneasy about "gouging" the customer and reduced the price to keep margins within bounds of what they KNEW were acceptable margins. They were fearful of losing the volume.
• Every time a price increase was implemented, it unwound within three months for similar reasons to that accompanying a cost reduction. However, price increases carried the additional burden of some customers lodging a protest.

Changing the informal target to contribution to fixed costs took a lot of effort, although it happened in a relatively short time. Multiple actions such as walking away from a national tender of 10% of the Australian market in favour of redeploying the people to penetrate into the low volume high margin food processing business concentrated everyone's mind. Combined with a nationwide person-to-person communication blitz to over one thousand direct and reseller sales staff the message got through.

Once it was established that money was the target and the price discount levels was a prime KPI, several consequences occurred:

• Wholesale and processing volumes declined as prices went up.
• Competitors raised their prices.
• Staff redeployed to penetrating new markets, including exporting finished products to China, were successful. They sold more additional volume than we lost by walking away from very low margin business. The higher volume driven through the plants reduced unit costs.
• Australia's base oil supply and demand rebalanced keeping a firmer footing under prices.
• The reduced unit costs were retained as increased contribution to fixed costs as the KPI was discount off list price, not margin.
• A price increase was successful, the first to be so in ten years.
• Contribution to fixed costs rose to over $22M in two years, Return on investment more than doubled.

The lessons learnt from this episode in Peter's language included:

• The original KPI did not correlate with process performance. The result of a sale should be a customer who believes they are getting value for money and a seller who thinks they are getting a just return on their effort.

Only price can be used in such calculations. Margin does not enter into the customer's calculation. A sales person usually does not have enough knowledge to know what margin is truly acceptable. They should know, however, what price the market will bear.

• The original KPI likewise did not measure the sales process performance.

KPIs undoubtedly drive performance. Aside from a goal, KPIs are the most important gift you can give your people to aid their performance. Be aware, however, that even poor KPIs drive performance giving rise to unintended, poor, consequences.