Designing a Subjective Performance Measure (Part 1)

Posted on Sat 08 Jul 2017 by Andrew Jacopino

Not everything that counts can be counted, and not everything that can be counted counts.

Albert Einstein

In an earlier series of articles we discussed how to set performance levels for Performance Based Contracts (PBCs) (see Setting the Performance Level Part 1, Part 2 and Part 3). The general principles and approaches described in this series of articles work for all types of performance measures (both objective and subjective). However, only specific elements of these relate to subjective performance measures that have a qualitative (as opposed to a quantitative) result based on the buyer’s subjective measurement of the seller’s performance. Given this, what approach do you use when designing subjective performance measures and setting their performance levels?

Before we look at this, in a different previous article (see Objective vs. Subjective Measures), we described objective and subjective performance measures based on an Oxford dictionary definition as:

  • subjective – “Based on or influenced by personal feelings, tastes, or opinions”; or
  • objective – “(Of a person or their judgement) not influenced by personal feelings or opinions in considering and representing facts”.

Based on this definition, subjective performance measures seems better at measuring outcomes such as customer performance, behaviours and culture that rely on perceived performance. In measuring outcomes that are inherently subjective, it is easier to use qualitative descriptions of performance (e.g. good, fair and poor, etc.) as opposed to quantitative performance (e.g. 78%, 3 hours, 23 parts, etc.). Firstly, lets look at how to score subjective performance measures.

Over years of experience my colleagues and I have simplified to two types of scoring techniques subjective performance measures; scored and unscored.

Scored Subjective Performance Measures

This type of subjective performance measure uses multiple levels of scores or grades to define performance. While many descriptions are used, my colleagues and I have standardised on the following terminology:

  • Superior – representing a level of performance level above the Required Performance Level and typically reflected in the colour ‘blue’ in traffic lights noting using superior performance is optional;
  • Good – representing a level of performance that meets or it better than the Required Performance Level and typically reflected in the colour ‘green’ in traffic lights;
  • Fair – representing a level of performance that meets or is better than the Minimum Level of Performance but less than the Required Performance Level and typically reflected in the colour ‘amber’ in traffic lights; and
  • Poor – representing a level of performance that is worse than the Minimum Level of Performance and typically reflected in the colour ‘red’ in traffic lights.

Typical scored subjective performance measures score the seller’s ability to deliver the range of performances and can include a variety of areas such as product or service delivery performance, behaviours such as customer satisfaction and responsiveness, cost including transparency, etc.

Unscored Subjective Performance Measures

Where a performance measure does not need multiple levels of scores or grades to define performance, or where a performance measure only represents compliance (i.e. compliant or non-compliant), it is possible to use a simpler unscored subjective performance measure. Again, while many descriptions are used, my colleagues and I have standardised on the following terminology:

  • Satisfactory– representing a level of performance that meets the Required Performance Level and typically reflected in the colour ‘green’ in traffic lights; or
  • Unsatisfactory – representing a level of performance that is worse than the Required Level of Performance and typically reflected in the colour ‘red’ in traffic lights.

While unscored subjective performance measures are uncommon, they are used to score the seller’s compliance with legislative areas such as safety, financial reporting, etc. where there is typically an external performance standard set and is either met or not met.

Figure 1 illustrates these two scoring approaches to subjective performance measures.

Figure 1 – Subjective Performance Measures – Performance Levels

Posted in Performance Measure, the How | Tagged , , , , , , | 3 Comments

Measuring Training Performance

Posted on Sat 24 Jun 2017 by Andrew Jacopino

“Job training empowers people to realize their dreams and improve their lives.”

Sylvia Mathews Burwell

I recently came across an article in Harvard Business Review titled “A Better Metric for the Value of a Worker Training Program”. The article focuses on recommending a new worker training program measure, Cost Per Employed Day (CPED), designed to measure the cost-benefit of a specific worker training program by measuring both the cost and job placement, but also how the student is going 6 months later. While I don’t intend to discuss this specific measure at length it did highlight one recurring Performance Based Contracting (PBC) question; how to measure and drive positive training performance.

In many circumstances, training, while critical for business operations, is part of enabling services and as such is routinely outsourced. In acknowledging the critical nature of these services these outsourced arrangements are candidates for PBCs requiring the design of training specific performance measures and payment regimes.

The solution may seem simple; something like, the “number of students that successfully complete a course”. While at first glance this is OK, it has a number of unintended consequences, especially if linked to money.

For example, if payment is linked to “successful completion of the course” this may drive organisational behaviour to reduce the performance levels (e.g. exam marks or standard of the exam) to make sure success and therefore payment. However, if the tests are standardised and outside the control of the training organisation, it may increase the ‘prerequisites’ for students attending the course to make it highly likely that everyone succeeds (e.g. you need a masters degree before attempting the course). Alternatively, the organisation may focus on teaching to the test as opposed to giving the student the necessary knowledge and skills for competency in the workplace.

Many of you may feel this is an overly pessimistic perspective of our training institutions, however, there are numerous examples of training organisations including national universities that have been guilty of elements of the above to ensure continued student rankings and funding.

So what is the solution? Unfortunately, there is no simple answer. As described in the article it is about a holistic approach to measuring the whole training process including:

  • education process such as training facility and instructor availability, course materials, exam results, etc.;
  • long-term competency as measured by workplace at least 3 months (preferably 6 months) later;
  • student experience including instructor participation, course administration, etc.; and
  • (potentially) the cost.

This approach lends itself to a number of performance measures, many of which will be subjective informed by student, manager and workplace surveys.

In summary, in measuring training performance we need clarity about the outcome we want to drive, and align this with the performance measures used. Otherwise, we run the very real risk of driving behaviours and outcomes we neither intend nor want.

Posted in Performance Measure, the How | Tagged , , , , , , | Leave a comment

Process for Developing a Performance Based Contract (PBC)

Posted on Sat 10 Jun 2017 by Andrew Jacopino

Many years ago when preparing to teach a 1-day Performance Based Contracting (PBC) workshop it occurred to me that while we had an 8-step process for developing it wasn’t entirely accurate. In fact of the 100+ PBC that I have been fortunate enough to be involved in the overwhelming majority (80+%) of these revisited previous process steps numerous times, especially on the highly complex projects. As such, the intent of this article is to describe the general non-linear process for developing a PBC.

Firstly, based on the collective experience of my colleagues and I the process of developing a PBC is not linear. That is, completing Step 3 does not mean that you will not revisit it after completing Step 5. In fact, based on our experience a good PBC practitioner will continue to revisit and test the design and underlying assumptions of the previous steps.

Secondly, while many people will go directly to selecting performance measures as they have a good understanding of what is or could be measured, it is essential that we understand why we are using a PBC (the intent) and what behaviours we are trying to drive (the outcome). Without this, we may be driving the wrong behaviours resulting in perverse incentives.

Based on this we contend that the PBC development process is actually a spiral development process based on 3 spirals that are illustrated in Figure 1.

Figure 1 – Performance Based Contracting (PBC) Development Process

 

Spiral 1 – Defining the Requirements of the PBC (Requirements Definition)

The intent of spiral 1 is threefold as follows:

  1. define the scope of the PBC;
  2. define the overall environment that the PBC is operating in (e.g. how the scope of the PBC compares with the overall buyer outcome – we refer to this as the enterprise outcome which may require multiple contractors and internal buyer personnel to deliver); and
  3. what the performance (and typically cost) objectives are (e.g. defining PBC success).

The key output of spiral 1 is a deep understanding and an agreed description of what the desired outcome is, including the enterprise outcome. In our experience, highly successful PBCs are very clear about their definition of success (requirements), which is then linked to various rewards and remedies (consequences). Importantly, if spiral 1 is not completed, we fall into a common pitfall; measuring what we have always measured with little to no consideration of what behaviour we are trying to drive and why.

Spiral 2 – Initial Design of the PBC (Initial Design)

Following on and using the results from spiral 1, the intent of spiral 2 is twofold as follows:

  1. develop, capture, and refine a list of candidate performance measures to ensure that the draft set of performance measures are necessary (not too many) and sufficient (not too few) given the contractual scope of work; and
  2. define the initial list of rewards and remedies (consequences) that the candidate performance measures are linked to.

The key output of spiral 2 is a draft performance management hierarchy that can be defined as a system of performance measures that represent both the PBC and overall enterprise outcome through a number of performance measures representing each Key Result Area (KRA) at one of three performance measure levels (e.g. Strategic Performance Measures (SPMs), Key Performance Indicators (SHIs) and System Health Indicators (SHIs)). See a previous article on XX for more information on the types of performance measures that are available to PBC practitioners.

It is strongly recommended that detailed drafting of the candidate performance measures, including business rules, performance levels, etc., not be undertaken until the draft performance management hierarchy is agreed to. In our experience completing detailed drafting of performance measures until the whole draft performance management hierarchy is agreed to can often result in significant re-work if alternate performance measures are desired.

Spiral 3 – Detailed Design of the PBC (Detailed Design)

The intent of spiral 3 is, based on the draft performance management hierarchy from spiral 2, to develop a completed project specific Performance Management Framework (PMF) including all relevant performance clauses within conditions of contract, statement of work, payment, glossary, etc. Importantly, as part of finalising the performance measures the PBC practitioner will need to develop the following for each performance measure:

  1. performance levels including, if necessary, define an averaging methodology (see previous articles Setting the Performance Part 1,  Part 2 and Part 3);
  2. specific business rules (sometimes referred to as ground rules and assumption (GR&A) to include/exclude specific contract events (e.g. third party interference));
  3. if there a multiple performance measures, determine the individual performance measure weighting (see previous article Performance Measure Weighting);
  4. whether specific remedies will apply (e.g. Stop Payment and Termination events), and if so, how many occurrences before they apply;
  5. whether an Award Term/Rolling Wave incentive will apply;
  6. consequence on price including when to apply to the At-Risk Amount (ARA) (e.g. per event, per performance period, per year); and
  7. implication of performance reporting on general contract management reporting.

The key output of spiral 3 is a first draft of the suite of documents that collectively make up the PMF.

In summary, using the spiral development process illustrated in Figure 1 will result in an better PBC that more accurately represents the required outcome, which is developed in quicker time with less resources and re-work. And who doesn’t want that result!

Posted in Behaviours, Outcome, Performance Measure, the How | Tagged , , , , , , , | Leave a comment

Stretch Goals

Posted on Sat 27 May 2017 by Andrew Jacopino

“The greater danger for most of us isn’t that our aim is too high and miss it, but that it is too low and we reach it.”

Michelangelo

In writing the articles on setting performance levels I came across a Harvard Business Review article on “The Stretch Goal Paradox”. This article describes Stretch Goals as different from ordinary challenging goals in two aspects:

  1. Extreme Difficultly – levels that go beyond current capability and performance; and
  2. Extreme Novelty – to bring a stretch goal within reach brand-new paths and approaches must be found (that is revolutionary ideas as opposed evolutionary ideas).

Importantly the authors found that the behavioural “consequences of setting and then missing stretch goals can be profound with failures fostering employee fear and helplessness, killing motivation and ultimately damaging performance.” Accordingly, the authors of the article suggest careful consideration whether to use stretch goals vs. more achievable goals.

The reason I am writing about this is many people in developing a Performance Based Contracts (PBCs) want to include incentives for superior performance. The question is whether the superior performance represents a stretch goal based on the definition above or whether it is slightly better performance than the required level of performance (see the earlier article on Setting the Performance Level (Part 1) that describes the fours levels of performance used in PBCs).

In my career I have seen all types of PBCs from those with no incentives for superior performance, those with incentives for superior performance, and even those with incentives for stretch goals. However, as the article suggested , we need to carefully consider the reason for offering an incentive. Is the buyer expecting that superior performance or a stretch goal be delivered by the seller? And does the amount of incentive enough reward the seller for the extra effort (e.g. is the Return On Investment (ROI) enough to motivate the seller)?

In setting PBCs performance levels the use of incentives, and especially those that are stretch goals, should be carefully considered to avoid the same behavioural issues identified by the authors in either the buyer (e.g. an expectation of superior performance) or seller (e.g. seller management will expect the delivery team to always get the extra rewards).  The key to avoiding these behaviours is balancing achievability and stretch levels.

Posted in Behaviours, payment curve, Performance Measure, the How | Tagged , , , , , , | 3 Comments

Setting the Performance Levels (Part 3)

Posted on Sat 13 May 2017 by Andrew Jacopino

In the previous two articles (see Setting a Performance Level (Part 1) and Setting a Performance Level (Part 2) we discussed the general requirements for setting performance levels in a Performance Based Contract (PBC). In this final part we are going to look at how to set the performance level.

In most PBCs setting the performance levels will ultimately be the role of the buyer regardless of whether the performance levels are collaboratively developed. They will typically define the expected level of performance that a seller will have to deliver to meet the buyer’s outcome (the Required Performance Level); that is, what performance level defines success for the buyer. The buyer will also define the level of performance where they no longer get any benefit from the product / service delivered by the seller (the Minimum Performance Level).

In establishing both these performance levels the buyer can take either of the two following approaches (although sometimes you may use both):

  1. Top-Down Approach, or
  2. Bottom-Up Approach.

Top Down Approach

In a top down approach, the performance levels can be taken from a range of ‘strategic’ documents that reflect the buyer’s need in terms of what performance levels would define overall success. Documents may include white papers, business cases, etc. typically used to describe the need of a buyer outlining performance expectations in order to respond to a new or changed business need. For example, the business case to outsource a particular product / service may need the seller to deliver a specific performance level to make sure the buyer can continue their internal business.

A top down approach is typically used in establishing a new product or service.

Bottom Up Approach

Unlike the top down approach, the bottom up approach uses historical performance data to set a realistic performance levels noting that it is relies on existing and relevant performance data for assessment. However, at times this may not be possible and other sources of data will need considering. In order of preference these are:

  1. historical data – using historical performance data for the same (or similar) products / services) is the most accurate method of performance predictions. However, before using historical data PBC practitioners must verify whether there are any changes in the following since they can lead to significant changes in performance estimates from the historical basis:
    1. Configuration. Is the complexity of the product / services similar? Are there more or less products being supported under this PBC? Is there a variation in the model of the product being assessed and the new contract?
    2. Role. Is the future usage the same between the current fleet and the new fleet? For example, the current fleet is only used 8 hours a day Monday to Friday and the new fleet is required 24 hours per day 7 days a week.
    3. Environment. Is the future operating environment the same as the historical fleet? For example , the current environment is used inside climate controlled buildings and the new fleet is required to be used outside from -10 degrees Celsius to +45 degrees Celsius.
  2. mathematical predictions / simulations – this method uses a range of techniques to find the likely performance of the products / services including the likely variation or spread of performance.
  3. professional judgement – least preferred way of predicting future performance since it is subject to the person undertaking the judgement (e.g. ‘gut feel’) and is subject to agendas and biases (e.g. risk aversion, loss aversion, etc.).

As discussed at the start of this article it is very common for both the top down and bottom up approaches to be used together. When used together the bottom up approach is used to define the performance of the current product / service (i.e. what performance the buyer is getting), while the top down approach is then used to define the new performance levels. The used of both approaches allows a comparison the current performance and the future desired performance and can quickly highlight the business need for change (e.g. reduce the cost per item delivered, reduce in the cost per operating hours, etc.).

Regardless of which approach you use, the key is understanding the source of the performance level and if using a bottom up approach, ensuring it takes into account any of the variations listed above.

Posted in payment curve, the How | Tagged , , , , | 4 Comments

2017 International Performance Based Contracting (PBC) Conference

Posted on Sun 07 May 2017 by Andrew Jacopino

For those you are not aware, the Australian Department of Defence is holding it’s biannual International Performance Based Contracting (PBC) Conference 2017 from 15–16 May 2017 here in Canberra, Australia. Attendance is free of charge, however, registration is essential and space is limited.  The final date for nominations has been extended until the Close of Business on Monday 8th May 2017.

The intent of the International PBC Conference 2017 is to give a frank and robust forum where government agencies, defence industry members and international partners can discuss the evolution of the current best practice PBCs and how these have shaped positive outcomes for the Australian government and the defence industry. Moreover, attendees will benefit from a comprehensive introduction to the roles that incentive-based contracting, supplier knowledge management or behavioural economics may play in the future of PBC.

While the focus of this conference is the defence sector, many of the presentations and case studies give an excellent introduction and insight into a sector that has been actively using PBCs for over 14 years, spending over AUD$2B per year through PBCs, and has seen the approach evolve over this time noting buyer, seller and user perspectives are represented. Additionally, the international speaker will help highlight how other organisations are approaching this contracting beneficial contracting technique.

If you are interested, for more information, including a copy of the agenda, and the method for registering please visit the Australian Department of Defence PBC Centre of Excellence website.

Posted in Conference | Tagged , , , , | Leave a comment

Setting the Performance Levels (Part 2)

Posted on Sat 29 Apr 2017 by Andrew Jacopino

In the earlier article (see Setting the Performance Levels (Part 1)) we defined the four performance levels used in Performance Based Contracts (PBCs). They were the Required Performance Level, Minimum Performance Level, Inflection / Elbow Performance Level and the Incentive Performance Level. In this article we will look at the general principles that should be followed in setting these performance levels.

General Principles in Setting PBC Performance Levels

In setting performance levels the following general considerations should be taken into account:

  • Do the performance levels change during the year? For example, some performance levels have seasonal variation (e.g. to align with the weather such as summer or monsoon seasons). That is, during specific months of the year (say peak periods) the buyer’s performance requirements may increase, or have less variation in the performance delivered. At other times, the buyer’s performance requirements may have more flexibility.
  • Do the performance requirements change during the term of the contract? Is it different at the start of the contract (e.g. do the performance levels change from when establishing a new product / service as opposed to maintaining the product / service)? Can a transition program allow for a scaled increase in Performance Levels over time?
  • Is the performance level achievable or is it a ‘stretch goal’?
  • Do the performance measures need an extensive list of business rules (e.g. automatic exclusions) to make the performance level achievable? In setting the performance levels does it use ‘adjusted’ performance (i.e. with the business rules applied) or ‘raw / actual’ performance (i.e. without any business rules)?
  • Does the buyer need a very high performance level? At very high performance levels each additional level of performance will typically leads to escalating prices. As such, the buyer needs to understand the cost drivers associated with very high performance (i.e. S-curve).

In addition to these general questions we need to understand the mathematical basis of the performance level. Many practitioners may ignore this part since they may feel uncomfortable with maths, and specifically statistics. However, it is essential to that practitioners address this in setting the performance levels based on the following three elements.

The first element is averaging. When a performance measure uses averaging (e.g. the daily number of ‘on-time’ deliveries measured daily but then averaged over the month) it results in a smaller spread in the performance levels between highest and lowest possible performance results. Importantly, as the period of which the averaging occurs gets longer (e.g. from weekly to monthly to quarterly) the spread between highest and lowest possible performance results continues to get smaller and smaller. Therefore, practitioners need to check that spread between their Required Performance Level and Minimum Performance Level takes into account the amount of averaging applied (e.g. daily performance levels are not used in setting the performance levels when the performance is averaged monthly).

The second element is the difference between the mean (the average of the performance) and the variance (the spread of the performance). Mathematically the average will typically (based on a normal or Gaussian distribution) represent the point where 50% of the performance is below this point and 50% is above this point. However, from a PBC perspective, do you want to set your Required Performance Level as the average that has a 50/50 chance of success? Most buyers want more confidence that their requirements will be met. Most sellers want more confidence of 100% payment. Therefore, when setting the performance levels, especially when comparing with historical performance, we don’t usually set it using the mean. However, this only applies when setting the performance levels using pervious historical performance (see Setting the Performance Levels (Part 3) for further details on the Bottom Up approach).

The third element is statistical significance. This is the concept that ensures that enough data exists to make meaningful deductions of performance from. For example, if a performance measure uses a 12 month average then it requires at least 12 months of data before the first assessment can be made. If this same measure is then updated quarterly, it will need to continue to use the previous 12 months data updated with the new quarter and discarding the previous quarter. These are typically referred to as “rolling averages”. Reliability performance measures, such as Mean Time Between Failure (MTBF) are high dependent on statistical significance.

In the next article we will look at how to bring these general principles together to set our PBC performance levels.

Posted in payment curve, Performance Measure, the How, Uncategorized | Tagged , , , , | 4 Comments

Setting the Performance Level (Part 1)

Posted on Sat 21 Jan 2017 by Andrew Jacopino

“You get what you measure. Measure the wrong thing and you get the wrong behaviours.“

John H. Lingle

In our experience one of the most challenging steps in establishing a Performance Based Contract (PBC) is setting the performance levels for each performance measure.

In an earlier post (The Role of a Performance Measure) we established that the primary role of a performance measure was to communicate.  Either communicate the buyers need (requirement) (e.g. what performance to I need out of this contract) or to communicate the sellers performance (e.g. did the required performance get delivered). The setting of performance levels simply continues this communication between buyer and seller.

Specifically, from a buyer’s perspective a performance level defines their performance requirements to the seller. However, for most PBCs it is not about setting a single performance requirement for each performance measure, but rather how rewards and remedies may change with the seller’s actual performance. For example, if the seller’s actual performance is worse than the buyer’s performance requirement, this would result in a reduced payment to the seller. Alternatively, and where there is benefit to the buyer, if the seller’s actual performance is better than the buyer’s performance requirement, this would result in an increased payment to the seller.

From a seller’s perspective, given the relationship between performance and rewards and remedies in a PBC, the performance levels represent a large part of the commercial risk and will be carefully considered by the seller. This is true of not only the performance level that results is 100% payment, but also for the performance level that results in 0% payment, and potentially the performance level that results in an incentive (e.g. 110% payment).

Before setting our performance levels we need to first define the 4 performance levels of a PBC as illustrated in Figure 1 below noting these levels equally apply to both quantitative performance and qualitative performance. These are as follows:

  • Required Performance Level – this is the performance level the buyer expects delivered by the seller and should be the design point for the sellers solution (e.g. to make sure consistent ensure delivery of this level of performance to the buyer the seller needs an amount of staff, spare parts, vehicles, etc.). This typically represents the level where the buyer would pay the seller all (100%) of the performance fee against this performance measure.
  • Minimum Performance Level – this is the performance level where the buyer receives no value from the service being delivered by the seller noting this point may not be ‘0’ performance (e.g. the buyer’s requirement may specify a minimum number (level) of emergency vehicles required each day as opposed to the minimum number being 0 vehicles). This typically represents the level where the buyer would not pay the seller any of the performance fee (0%) against this performance measure.
  • Inflection / Elbow Performance Level – this is a performance level between the required and the minimum performance level and “shapes” transition from the minimum performance level to required performance levels. This can be a simple as a straight line between the two levels (e.g. linear), or as in the case for the Australian Department of Defence, this typically sits half-way between the minimum and required performance levels and represents the point where the buyer would pay the seller 80% of the performance fee against this performance measure. However, there are many ways of doing this.
  • Incentive Performance Level – this is a performance level above the required performance level where the buyer may pay the seller more than the performance fee (greater than 100%) against this performance measure. However, paying an incentive usually only occurs where the delivery of superior performance delivers added value to the buyer and will usually be limited to a specific amount.
PBC Performance Levels

Figure 1 : PBC Performance Levels

Having defined the 4 performance levels in a PBC, in the next post, we’ll discuss how to we set each.

Posted in payment curve, Performance Measure, the How | Tagged , , , , , | 10 Comments

Happy Holidays to all the Performance Based Contracting (PBC) Blog Readers

Posted on Tue 27 Dec 2016 by Andrew Jacopino

At this joyous time of year, we are grateful for the time we can spend with our own friends and family, and we wish you all abundance, happiness, and peace in a New Year filled with hope. Happy holidays!

Posted in Admin | Leave a comment

Changing Performance Measures

Posted on Sun 23 Oct 2016 by Andrew Jacopino

I recently read an article by Robert Wolcott in the Harvard Business Review (HBR) titled “Don’t be Tyrannized by Old Metrics”. The article discussed the virtue of companies continuously reviewing, and where necessary, changing performance measures especially where there was a corresponding change in strategy (or in the case of Performance Based Contracting (PBC) a change in the contract scope of work).

While Robert’s article is mainly focused on those metrics that you may find in a company Balanced Scorecard, the 4 points he raised make good sense for any PBC practitioner to consider:

  1. Know your metrics and behaviours they drive. Those familiar with this blog may remember that our definition of a highly successful PBC is one that firstly drives the right behaviour. Since if this is not being achieved, or worse still, the PBC drives perverse incentives, the PBC will fail.
  1. Track your metrics at your peripheries. Research shows that a good source of innovation is located in industries that are outside your industry (i.e. at your periphery). Therefore, we should be looking and learning from these peripheral industries to see what performance measure they are using and whether they would be applicable to your area.
  1. Prioritise metrics that reflects value to customers, rather than simply volume or efficiency. This is great advice. Linked to knowing the performance measures and the behaviour they drive is what the focus of these performance measures are. While Peter Druker famously said “You can’t manage what you can’t measure”, given modern data systems ability to record and report I am not sure reporting 200+ performance measures every day helps focus our attention. Instead, we should focus our attention on those critical performance measures that measures success (insight) as opposed to the approach of “if we can measure it we should measure it” (oversight).
  1. Experiment with emerging, alternative – and iterate. This is the most controversial point, especially when considering from a commercial perspective. While we should explore new and innovative performance measures, a PBC starts within a known commercial relationship of risk and reward. For example, if the seller does A the buyer will pay $X. However, if the seller does (A – 10%) then the buyer will pay $Y. However, by the changing performance measure, including the commensurate change in performance level, we also change the basis of payment and the associated commercial risk. While changing is OK, we simply need to acknowledge the impact. One option to help deliver this is to run these new performance measures in parallel with the old ones to make sure that they work as expected and not jump directly to them. We sometimes refer to them as “ghost metrics”. If they work well, we can then look to change as part of a formal contract change proposal.

Robert’s parting advice is to review your metrics at least annual to check if relevant but also to check whether you could do better. My colleagues and I agree that we should all be reviewing any Performance Management Framework (PMF) at least annually as part of a contract annual strategic review. Indeed, those familiar with the Australian Standard for Defence Contracts (ASDEFCON) templates, especially the Support version, would be aware that this annual PMF review is a core part of the contract.

So in summary, and as with most things in life it is about balance; balancing the need to review and change contract performance measures with the need to explore innovative new and alternate performance measures, especially where this changes the underlying commercial basis.

Posted in Contract Management, Performance Measure, the How, the What | Tagged , , , , , , , , , | 3 Comments