Not everything that counts can be counted, and not everything that can be counted counts.
In an earlier series of articles we discussed how to set performance levels for Performance Based Contracts (PBCs) (see Setting the Performance Level Part 1, Part 2 and Part 3). The general principles and approaches described in this series of articles work for all types of performance measures (both objective and subjective). However, only specific elements of these relate to subjective performance measures that have a qualitative (as opposed to a quantitative) result based on the buyer’s subjective measurement of the seller’s performance. Given this, what approach do you use when designing subjective performance measures and setting their performance levels?
Before we look at this, in a different previous article (see Objective vs. Subjective Measures), we described objective and subjective performance measures based on an Oxford dictionary definition as:
- subjective – “Based on or influenced by personal feelings, tastes, or opinions”; or
- objective – “(Of a person or their judgement) not influenced by personal feelings or opinions in considering and representing facts”.
Based on this definition, subjective performance measures seems better at measuring outcomes such as customer performance, behaviours and culture that rely on perceived performance. In measuring outcomes that are inherently subjective, it is easier to use qualitative descriptions of performance (e.g. good, fair and poor, etc.) as opposed to quantitative performance (e.g. 78%, 3 hours, 23 parts, etc.). Firstly, lets look at how to score subjective performance measures.
Over years of experience my colleagues and I have simplified to two types of scoring techniques subjective performance measures; scored and unscored.
Scored Subjective Performance Measures
This type of subjective performance measure uses multiple levels of scores or grades to define performance. While many descriptions are used, my colleagues and I have standardised on the following terminology:
- Superior – representing a level of performance level above the Required Performance Level and typically reflected in the colour ‘blue’ in traffic lights noting using superior performance is optional;
- Good – representing a level of performance that meets or it better than the Required Performance Level and typically reflected in the colour ‘green’ in traffic lights;
- Fair – representing a level of performance that meets or is better than the Minimum Level of Performance but less than the Required Performance Level and typically reflected in the colour ‘amber’ in traffic lights; and
- Poor – representing a level of performance that is worse than the Minimum Level of Performance and typically reflected in the colour ‘red’ in traffic lights.
Typical scored subjective performance measures score the seller’s ability to deliver the range of performances and can include a variety of areas such as product or service delivery performance, behaviours such as customer satisfaction and responsiveness, cost including transparency, etc.
Unscored Subjective Performance Measures
Where a performance measure does not need multiple levels of scores or grades to define performance, or where a performance measure only represents compliance (i.e. compliant or non-compliant), it is possible to use a simpler unscored subjective performance measure. Again, while many descriptions are used, my colleagues and I have standardised on the following terminology:
- Satisfactory– representing a level of performance that meets the Required Performance Level and typically reflected in the colour ‘green’ in traffic lights; or
- Unsatisfactory – representing a level of performance that is worse than the Required Level of Performance and typically reflected in the colour ‘red’ in traffic lights.
While unscored subjective performance measures are uncommon, they are used to score the seller’s compliance with legislative areas such as safety, financial reporting, etc. where there is typically an external performance standard set and is either met or not met.
Figure 1 illustrates these two scoring approaches to subjective performance measures.