Unintended and Perverse Outcomes

A colleague from the Australian Department of Defence (thanks Hayley!) recently sent me a link to an article from the Canberra Times about “Losing our way: How the cult of the KPI has damaged our moral compass”.  This article is partly in response to recent revelations in Australia from the Banking Royal Commission and the Victorian State Police (one of the 7 Australian state / province police departments).

Evidence presented to the Banking Royal Commission showed that Commonwealth Bank staff had inappropriately set up thousands of children’s bank accounts (often called Dollarmites accounts) either using their own money or the bank’s money to meet aggressive targets tied to an incentive scheme.  Commonwealth Bank staff would do so when parents had signed up their kids for school banking, but had not deposited money into the account within 30 days.  If no deposit was made, the sign-up would not count towards sales targets and financial rewards.  For disclosure, my children have Commonwealth Bank Dolomites accounts, however, all deposits have come from their family (see Dollarmites bites: the scandal behind the Commonwealth Bank’s junior savings program for more information).

For the Victorian State Police, The Age newspaper revealed that Victoria Police officers falsified 258,000 roadside alcohol breath tests over 5½ years (about 1.5% of all tests carried out during that time) by inflating breath test bags themselves to meet quotas noting there were no financial incentive for officers to fake tests (see How the Victorian police faked breath tests for more information).

The reason I am bringing it up in a Performance Based Contracting (PBC) blog is that these outcomes are familiar traps for new PBC practitioners resulting in unintended or perverse outcomes.  Specifically, the pitfalls identified in the article mirror our PBC pitfalls being:

  1. Avoid using a single quantitative performance measure – instead, use a variety of objective / quantitative and subjective / qualitative performance measures including measures for enterprise performance and behaviours (see The Allure of a Single Measure and Objective vs. Subjective Performance Measures).
  2. Avoid using only monetary performance measures – instead, use a variety of rewards (positive – ‘carrot’ – incentives) and remedies (negative – ‘stick’ – disincentives) as part of your overall Performance Management Framework (PMF) including measuring at a variety of levels and rewarding enterprise performance and behaviours (see When is a KPI not a KPI).
  3. Avoid using stretch targets – instead, make sure the performance is achievable regardless of whether a reward or remedy (see Setting the Performance Levels (Part 1, Part 2 and Part 3) and Stretch Goals).
  4. Avoid ‘setting and forgetting’ your KPIs – instead, use the performance measures and PMF as part of an ongoing discussion on past, present and future performance requirements and achievements.

So as a reader of this blog I hope you concluded that these perverse outcomes were highly likely since, from a behavioural economics and performance management design perspective, they were all unfortunately setup to fail.

The key for each of you is to make sure that you learn the lessons described here and avoid the same pitfalls in the design of your PBC.

Posted in Behavioural Economics, Case Study, Incentives, Outcome | Tagged , , , , , , , , , | Leave a comment

Performance Based Contracting Book Update

Firstly, sorry for being offline recently.  I have been madly finishing the “Mastering Performance Based Contracts” book which I am pleased to say is undergoing the final touches before release, which is now scheduled for 1stSeptember 2018.

As part of the release of the book I will potentially be in Europe in September 2018 and in the US / Canada in November 2018 to run 1-day and 3-day Performance Based Contracting courses based on the book through the International Association of Contract and Commercial Management (IACCM).  The courses are being offered on both a public and private basis and can be tailored to suit specific audiences.

If you are interested please let me know and I will send you the IACCM point of contact or you can visit the IACCM website.

I will also be back to providing articles on the blog as of next Saturday.

Again, sorry for the absence but hope you enjoy the forthcoming book and courses.

Posted in Book | Tagged , , , , | Leave a comment

Supply Support Performance Measures – Part 3

In the first article we looked at the various performance measures we can use for Supply Support in Performance Based Contracts including grouping them into 3 main types; demand satisfaction performance measures, inventory holding performance measures and customer wait time performance measures.  In the last article we looked at demand satisfaction performance measures in more detail.  In this third and final article we’ll look at the remaining 2 groups of Supply Support performance measures (inventory holding performance measures and customer wait time performance measures) and give a simply diagram summarising which ones you should use and when.

Inventory Holding Performance Measures

Inventory holding performance measures need seller’s to maintain agreed stock levels at specified locations.  While this limits the seller’s freedom in how they meet a buyer’s demand, it does also minimise the seller’s exposure to inaccurate and uncertain buyer demand profiles.

Given this, inventory holding performance measures are typically used where buyers wants confidence in their demands by specifying both the required stock levels and locations.  One such example we have used is a performance measure called Minimum Asset Level (MAL), which is a measure of the number of stock items as a percentage of al stock items that are above the minimum stock level for each location.  The inventory check can be done on a continuous basis (i.e. 24 hours a day / 7 days a week) or periodic basis (e.g. at 9am each day).

In certain circumstances using inventory holding performance measures is the best approach.  For example, in some circumstances the holding of a minimum level of spares, especially those with low usage rates, is not economically feasible.  However, what if these low use spares cause a critical failure and long delay?  In asset management language, these low probability – high consequence items are called ‘insurance spares’.  For example, many Air Forces hold extra, very expensive spare aircraft engines if they are needed due to random failures such as Foreign Object Damage (FOD) including the accidental injection of a bird into an engine during take-off or landing.  While these events have a very low probability, when they occur they result in a very expensive asset being unavailable until a replacement engine is available.  Accordingly, many Air Forces will specify extra ‘spare’ engines to be held with the maintenance organisations for these events.  However, in the commercial aviation sector to solve this problem, an engine provider may combine (pool) spare engines for a number of airlines in the same region or airport to cut the costs to all airlines involved.

Successful use of inventory holding performance measures is dependent on accurately setting the stock levels and locations.  Setting the levels too high will result in higher stock holding cost, but will meet all demands.  Setting the levels too low will result in delays and increased costs based on the asset being unavailable for use.

Inventory holding performance measures are best used where failed delivery results in severe consequences where the buyer and seller have agreed stock levels and locations including the holding of low rate “insurance spares”.

Customer Wait Time Performance Measures

Customer wait time performance measures need seller’s to simply give an item within an agreed time period.  Unlike the previous two type of supply support performance measures this approach makes the seller reactive to demands significantly reducing the seller’s exposure to inaccurate and uncertain buyer demand profiles.  This focuses the seller on the timeliness between order and satisfaction, depending only on their processes for satisfying the demand which could be through procurement, manufacturing or repair.  One customer wait time performance measure example is Turn Around Time (TAT) which is measure of time from placement to satisfaction of a demand (e.g. a 30 day TAT means that it takes 30 days from the buyer placing an order to the seller satisfying the demand).  Successful use of customer wait time performance measures is dependent on seller’s processes for satisfying demands.

Customer wait time performance measures are typically used where there is a higher cost of holding additional (spare) items than the cost due to a delay in delivery of the item.  Similarly, customer wait time performance measures are also used where there are low number of demands, again with low consequence due to a delay.

However, customer wait time performance measures should not be used where there are severe consequences for failed/ delayed delivery and where there are stable and high numbers of demands.  In this case, demand satisfaction performance measures should be used.


These articles should guide the PBC practitioner to the best type of Supply Support performance measure for the specific PBC.  A summary of these descriptions, including when it is best to use / not to use, is provided in the figure below.  Given there is some overlap between the three types of Supply Support performance measures a performance test case should be developed and tested to make sure the ideal Supply Support performance measures are used.

Types of Supply Support Performance Measures

Posted in Performance Measure, the How | Tagged , , , , , , , , , , | Leave a comment

Supply Support Performance Measures – Part 2

In the last article we looked at the various performance measures we can use for Supply Support in Performance Based Contracts including grouping them into 3 main types; demand satisfaction performance measures, inventory holding performance measures and customer wait time performance measures.  In this article we are going to look at demand satisfaction performance measures in more detail.

Demand Satisfaction Performance Measures

At the highest level of integration with the buyer’s business demand satisfaction performance measures focus on contracting for satisfaction of the buyer’s demand.  These represent by performance measures such as Demand Satisfaction Rate (DSR) or DIFOT which is simply the number of successful demands as a percentage of total demands (e.g. 98% DSR means the seller satisfied 98% of buyer demands based on the right number, type and delivery location of all items within the agreed timeframe).

A demand satisfaction performance measure gives the seller freedom of how to meet the demand in terms of quantity and location of items being demanded whether through:

  • just in time procurement, manufacturing or repairs;
  • holding stock in single or multiple warehouses; and
  • whether to allow sharing of stock between buyers using the same items (e.g. pooling spares).

However, to do this, the seller must have a good understanding of buyer demand profile, and importantly, the demand profile must be stable and accurate. The seller then uses this information, through a process called “spares determination” which typically involves dedicated software tools such as OPUS10, to balance the number and location of items vs. total cost of the inventory (including storage).

So while demand satisfaction performance measures seem a good outcome for both buyer and seller, the same aspects that makes it attractive also causes limitations that need  consideration.

Specifically, what if we only have limited knowledge on the buyer demand profile resulting in high level of uncertainty and inaccuracy?  For example, what if the seller bases their support on 50 demands per month, however, they are getting 100 demands per month?  Does the buyer expect that the seller can support this and if not, can the buyer apply commercial consequences (e.g. withholding profit) for a failure to satisfy these demands?  What about if it is the other way around with the seller expecting 100 demands but only getting 50 demands.  Does the buyer expect the seller to give back this reduction in cost?  At first glance you may think this is OK since the seller is easily meeting the demand satisfaction requirements.  However, what if those extra (spare) items have a shelf-life, like food, or become obsolete like ICT items (e.g. laptops and mobile phones)?  The reduced consumption now leads to waste and was this considered in the seller’s pricing or is this a cost for the buyer?

Additionally, unless the overall performance measure hierarchy includes lower level supply support performance measures, by only using demand satisfaction performance measures the buyer has limited insight and confidence into future performance.

Finally, while the buyer may agree to a very high level of satisfaction (e.g. 98%) this allows 2% failed deliveries.  But what if those few deliveries result in a severe or catastrophic consequence such as the only reason a large expensive container ship in port cannot leave or a passenger airline cannot make it’s scheduled departure? So what happens now?

The use of demand satisfaction performance measures sometimes requires that either the seller is given certain protections (e.g. if buyer demands go above a certain level they are not held accountable) or the seller places a higher price on the delivery of items reflecting this dependency, which in some circumstances, may make it unaffordable to the buyer.  In these situations, it is important that both buyer and seller understand the conditions placed on the performance measure.

Success in using demand satisfaction performance measures is highly dependent on knowledge, accuracy and certainty of the buyer’s demand profile. Given this dependency is best to use demand satisfaction performance measures where there is certainty and stability of demands.  Additionally, demand satisfaction performance measures are best used for higher numbers of demands where there are no severe consequences for small numbers of failed delivery.  Otherwise, the PBC practitioners should consider the next two types of Supply Support performance measures.

In the next and final article in this series (Part 3), we’ll look at the remaining 2 groups of Supply Support performance measures (inventory holding performance measures and customer wait time performance measures) and give a simply diagram summarising which ones you should use and when.

Posted in Performance Measure, the How | Tagged , , , , , , , , , | Leave a comment

International Association of Contract and Commercial Management (IACCM) “Ask the Expert” Presentation

For those of you who were unable to make it to my “Ask the Expert” webinar presentation on Performance Based Contracting on Wednesday 9thMay 2018 kindly hosted by the International Association of Contract and Commercial Management (IACCM), you can see the recording and slides via the following link .

A special thanks to Jennifer Goddard, who hosted me and ran the IT perfectly, and Mark Heminway, for putting all the advertising and notifications together.  Thanks to both of you for helping me.  It ran perfectly.

If you have a further interest in the topic of Performance Based Contracting please check out my blog at www.performancebasedcontracting.com which has more information and case studies, or feel free to contact me.

Posted in Presentation | Tagged , , , , , | Leave a comment

International Association of Contract and Commercial Management (IACCM) “Ask the Expert” Presentation

I will be delivering a webinar presentation on Performance Based Contracting (PBC) as part of the International Association of Contract and Commercial Management (IACCM) “Ask the Expert” series this Wednesday 9th May 2018 from 1200 – 1300 Australian Eastern Standard Time.

Based on my 12+ years of domestic and international PBC experience for buyers and sellers alike, and my regular articles on this site, the presentation will talk about Performance Based Contracts including what they are, what benefits they provide, how they work and some key success factors to consider when setting up and managing a PBC.

You can reach the webinar via the IACCM website via the following link.  For those unable to be at the presentation I will make it available in the near future on this site.

I hope to see you there.

Posted in Presentation, the How, the What, the Why | Tagged , , , , , | Leave a comment

Case Study – Performance Based Contracting Offer Definition and Improvement Activity (PBC ODIA)

One technique becoming more common during tendering for high value and / or high complexity Defence equipment acquisition and sustainment is a multi-stage tendering process.  The later stage of this process takes one or more tenderer (potentially down selected from a larger group of tenderers) through interactive discussions between buyer and seller(s) to explore and refine both the needs and the solution(s) being offered.  This is critical where there is uncertainty in the exact buyer’s needs since a conventional procurement process would traditionally limit this interaction and exploration of need(s) and solution(s).  So why am I bring this up on a PBC Blog?

Many years ago, as part of a buyer procurement team acquiring a new capability, we specifically included a PBC element in the ODIA for two reasons.  Firstly, to allow the buyer better insight and therefore confidence of the seller’s proposed solution.  To do this, the buyer set the same 3 performance scenarios (one good, one poor and one bad performance scenario) and then had each of the sellers describe how they how manage these events and what consequences would occur be based on the PBC offered.  Secondly, to allow the seller a mechanism to discuss areas of ambiguity or uncertainty in the buyer’s need(s) since this can drive cost in seller’s solutions.

In this case one of the tenderers, well-known to the buyer and having a lot of experience in Defence PBCs, indicated that they thought the buyer’s PBC was not fair resulting in many ‘non-compliances’ with the buyer’s PBC clauses (i.e. in their tender response they stated, if selected as the preferred tenderer, they would not agree to specific PBC clauses).  Given the seller’s experience and no other tenderer had this concern the ODIA process asked this tenderer to describe their concerns based on calculation of the 3 performance scenarios.

The tenderer working through the performance scenarios highlighted a number of events that would result in the overall performance score being a negative number (i.e. be below 0%).  In this circumstance, regardless of whether the seller fixed the performance issue, the overall performance score was a negative number.  While I have seen performance management frameworks that are very sensitive to performance events, in this circumstance this was not one of those.

With both buyer and seller trying to understand how this was occurring we discovered that inside one of seller’s calculations of performance there was a simple mathematical error; a ‘-‘ instead of a ‘+’.  Once this error was fixed the seller’s concern disappeared.  Given this discovery the tenderer asked whether they could come back the next as part of their larger scheduled OIDA sessions for another PBC discussion.  The next day the tenderer came in with a simple response.  They now were happy with the PBC arrangement in the contract, had no non-compliances and thanked the buyer’s team.

While I have highlighted this as a seller’s misunderstanding, I have equally seen errors in buyer’s PBCs being identified by sellers; no one is immune to this.

In summary this case study highlights 2 lessons.  Firstly, it is important that no matter how experienced both buyer and seller are, they need to fully understand how the PBC will work, including what consequences apply, by testing it against at least 3 scenarios; one based on good performance (all ‘green’), one based on poor performance (mix of ‘green’ and ‘amber’) and one based on bad performance (lots of ‘red’).  Secondly, it provides the buyer and seller a mechanism for discussing aspects of the PBC that drive costs such as performance levels, business rules (e.g. performance exclusions), etc. including any misunderstandings.  By having an interactive dialogue we can only end up with a more successful PBC for both buyer and seller, who doesn’t want that outcome!

Posted in Case Study | Tagged , , , , , , | Leave a comment