How to Nudge a Performance Based Contract (PBC) – Part 1

Commercial instruments, especially sophisticated ones such as Performance Based Contracts (PBCs), have the ability to shape the behaviour of both buyer and seller.  While many commercial practitioners see contracts in their simplest form, an explicit and directive agreement between buyer and seller, some acknowledge that contracts can be used for much more.  At their best, contracts can define, shape and nudge positive outcomes, including behaviours, in both buyer and seller.

In 2008 Richard Thaler and Cass Sunstein published, Nudge: Improving Decisions about Health, Wealth, and Happiness, which used behavioural economics as a method for helping people improve their decisions.  This book has had a profound impact on the field of economics introducing the notion of designing choice architectures to help people make better decisions by predicting how humans behave in certain situations.  Moreover, they gave us a blue print of how we could use various economic instruments to “nudge” human behaviour.  This is achieved by designing a “choice architecture” allowing individuals to choose their own path while highlighting the advantages and disadvantages of each choice; termed liberal paternalism.

Since this early description there have been many uptakes of the “nudge” approach including the creation of Behavioural Insight Teams (BIT)in a number of countries helping various local, state and federal governments design, modify and apply policies. However, applications are typically not focused on activities involving commercial contracts.

Despite the success of the book and the concept there are a number of proponents that disagree with the approach.  Some feel that providing a nudge towards a particular choice is itself a form of bias, and more extreme commentary claiming it is removing choice from the individual, even if offered.

So why am I bringing this up?

In the early 2000’s a number of organisations, especially in the Defence sector, began using Performance Based Contracts (PBC), or sometimes referred to as Performance Based Logistics (PBL) contracts or Contracting for Availability, with the intent of improving performance at a decreased price.  Indeed, the ability for a PBC to simultaneously deliver these outcomes[1][2][3][4]          has seen them more popular and their use more widespread.

Performance Based Contracting (PBC) Definition

Performance Based Contracting is an outcomes-oriented contracting method that ties a range of monetary and non-monetary consequences to the contractor based on their accomplishment of measurable and achievable performance requirements.

But for many people designing and applying PBCs over the past decade will see similarities between the concept of nudge and the intent of a PBC.  That is a highly successful PBC will:

  1. drive the right behaviour in the seller by addressing the Seller Needs;
  2. provide adequate commercial protections for the buyer by addressing the Buyer Needs; and
  3. balances both these needs within a usable Commercial Construct.

Accordingly, a good PBC aligns the Seller Needs against delivery of the Buyer Needs that is described in the Commercial Construct. Figure 1 highlights the interaction between these three areas based on, an asset management scope of work where:

  • Buyer Needs represent the traditional specification of requirements. In sustaining Complex Materiel[5] these requirements can be grouped into three broad areas of Asset Usage, Asset Optimisation and Asset Preservation underpinned by Safety Culture, Cost Consciousness and Positive Behaviours.
  • Seller Needs represents both the financial and non-financial outcomes required by the seller ranging from contract price and profit margin to contract duration and recognition schemes.
  • Commercial Construct represents the agreement, typically a contract, between the parties that fairly motivates and delivers both Buyer Needs and Seller Needs. The optimal Commercial Construct is a balance of conventional (written) contracting protections and a collaborative (relational) contracting approach.

Figure 1 : Interaction between Seller Need, Buyer Need and Commercial Construct

In order to achieve this each PBC requires a Performance Management Framework (PMF) which:

 “. . . ensures that the delivery of the enterprise outcome by creating a self-regulating agreement which uses a range of incentives to guide and disable choice.”

However, it is more than simply the Key Performance Indicators (KPI), but rather how all facets of the contract fit together to drive behaviour.  Indeed, key to the successful operation of the PBC is the notion of self-regulation where the seller’s performance determines the commercial consequences; positive or negative.  This idea of self-regulation aligns with the Behavioural Economics notion of “nudging” where, rather than the buyer directing the seller exactly how to deliver the outcome, the PMF should “nudge” the seller’s decisions allowing choice in full knowledge of the buyer’s requirements and consequences of the seller’s actions.

[1]            BOYCE, J. and BANGHART, A., “Performance Based Logistics and Project Proof Point – A Study of PBL Effectiveness”, Defense AT&L: Product Support Issue, March-April 2012

[2]            GUAJARDO, J.A.; COHEN, M.A.; NETESSINE, S and KIM S-H “Impact of Performance-Based Contracting on Product Reliability: An Empirical Analysis”, July 2009, revised February 2010, INSEAD Working Paper No. 2011/49/TOM

[3]            “Performance & Outcome Based Contracts 2015”, International Association of Contract and Commercial Management (IACCM), 2015

[4]       DOOGAN, C., LINGER, H., HOLMES, D. and BJERKNES, G., “Enquiry into Performance Based Contracting Practice – Phase 2 Case Studies”, June 2018

[5]       Complex materiel can be defined as those assets that support military operations through flying (e.g. aircraft and helicopters), sailing (e.g. ships, boats and submarines), driving (e.g. wheeled and tracked vehicles) and transmitting (e.g. satellite ground stations).

Posted in Behavioural Economics, Nudging, the How, the Why | Tagged , , , , , , , | Leave a comment

Perverse Incentive Follow-up

In earlier articles (see Perverse Incentives – Part 1 and Part 2,  and Unintended and Perverse Outcomes) I discussed the importance of checking the Performance Management Framework (PMF) within your Performance Based Contract (PBC) to watch for perverse incentives.  Those incentives, either positive or negative, that have an unintended and undesirable result which are contrary to the original intent.  In the case of a PBC, the interest of the buyer.

One of the case studies I used was the widespread practice of Victorian Police officers falsifying over 258,000 roadside alcohol breath tests over 5½ years (about 1.5% of all tests carried out during that time) by inflating breath test bags themselves to meet quotas noting there were no financial incentive for officers to fake tests.  For those not familiar with the Victorian Police force, they are a state level police force for 1 of the 7 state / territories in Australia noting there are only state and federal police forces in Australia with the state of Victoria having a population of 6.5 million in 2018 and an area of 227,436 km², which is sightly smaller than the area of the United Kingdom.

The Victoria Police conducted an independent investigation into the falsification of preliminary breath tests (PBTs) by their officers, called Taskforce Deliver, led by former Chief Commissioner Neil Comrie AO APM.  In the Taskforce Deliver report the taskforce found that the falsification of breath tests was widespread but fortunately may have occurred at much lower rates than first estimated by Victoria Police.  From a PBC perspective there are some interesting observations that equally apply to designing a PMF.  These are as follows:

Screen Shot 2019-04-20 at 8.02.30 pm

Importantly, many of the aspects highlighted in the report are common pitfalls when establishing a PBC.  These are:

  • Using quantitative performance measures due to their apparent simplicity and objectiveness in application (i.e. gathering, reporting and analysing) when qualitative performance measures should have been considered.
  • Applying performance measures without the enabling ‘infrastructure’ such as training, governance and IT to support operating the measures.
  • Not understanding / linking the Required Performance Level to a high-level outcome, including testing whether the performance level was achievable with current resources.
  • Using only a single performance measure to measure success (or otherwise) of a complex multi-dimensional activity (see The Allure of a Single Measure).

In summary, some of the issues raised in the Taskforce Deliver report could have been avoided, or at least minimised, through a better practice approach to applying performance measures to nudge behaviour.  While much of this better practice is covered in this blog and my book, Mastering Performance Based Contracting, over the next couple of months we will look at better practice performance measure selection and design.

Posted in Behavioural Economics, Book, Case Study, Incentives, Nudging, perverse incentives, the How, Uncategorized | Tagged , , , , , , , , , | Leave a comment

Award Term – Part 3

Having described the both the key elements and operation of an Award Term in the previous 2 posts (see Award Term – Part 1 and Award Term – Part 2), to aid in our understanding I will now look at 3 scenarios highlighting how an Award Term contract will work.

Scenario 1 – Good Performance

In scenario 1, consider the original example from the first post (see Award Term – Part 1) where the seller is delivering good performance, behaviours and value.  This scenario is shown in the figure below.  In this scenario in year 2 of the contract the buyer undertakes the Award Term assessment process resulting in a satisfactory rating.  In this case (i.e. the first Award Term) the buyer would only grant 1 additional year (i.e. year 7).

Performance Based Contracting Award Term Scenario 1

If the seller continues to deliver good performance, behaviours and value in through the Award Term assessment process the seller is again given a satisfactory ratingand is granted 1 additional year (i.e. year 8). In year 4 of the contract, given the seller has continuously delivered good performance, behaviours and value the buyer grants an additional two years (i.e. years 9 and 10) to the contract term.

Scenario 1 highlights the ability of the buyer to vary the amount of contract duration awarded through the Award Term process.  Moreover, while it is unlikely that an initial Award Term would be longer than the minimum duration (e.g. 1 year in our example), it does highlight continued, long-term good / superior performance, behaviours and value can reward the seller with longer Award Term durations.

Scenario 2 – Poor Performance

Unlike scenario 1, in scenario 2 the seller is delivering poor performance, behaviours and value. This scenario is shown in the figure below.  In year 2 of the contract the buyer undertakes the Award Term assessment process resulting in an unsatisfactory rating.  In this case the buyer would not grant any additional contract term beyond the original contract term (i.e. 6 years).  During year 3 the seller continues to deliver poor performance, behaviours and value and again is assessed as unsatisfactoryand not grants an additional year of contract term.  The original contract term of 6 years remains. Importantly, should the seller be assessed as unsatisfactory in year 4 of the contract the Off-Ramp Date will be reached resulting in an automatic re-tendering activity for some or all the contract scope.

Performance Based Contracting Award Term Scenario 2

Scenario 2 highlights the buyer’s ability not to grant additional contract tenure through the Award Term assessment process for poor performance, behaviours and value. Moreover, it highlights that for continued poor performance, behaviours and value the contract will “naturally expire (end)” since no contract extensions are granted.

Importantly, where any of the poor performance, behaviours and value result in other rewards and remedies such as a reduction in performance fee, stop payment and termination can still apply.

Scenario 3 – Mixed Performance

In scenario 3 the seller is delivering poor performance, behaviours and value during the first two years of operation.  This scenario is shown in the figure below.  In year 2 of the contract the buyer undertakes the Award Term assessment process resulting in an unsatisfactory rating.  In this case the buyer would not grant an additional contract term beyond the original contract term (i.e. 6 years).

In this scenario, given the buyer’s feedback the seller makes changes to their contract delivery approach and in year 3 the seller delivers good performance, behaviours and value resulting in a satisfactory ratingfrom the Award Term assessment process.  In this situation  (i.e. a previous unsatisfactory rating) the buyer would only likely grant 1 additional year (i.e. year 7) with a new total contract term of 7 years.

The seller then continues to deliver good performance, behaviours and value and in year 4 again receives a satisfactory rating from the Award Term assessment process with the buyer granting 1 additional year (i.e. year 8) with a new total contract term of 8 years.

Scenario 3 highlights the strength of the Award Term process to drive seller behaviours by illustrating how poor performance, behaviours and value are not rewarded, but how through remediation, the seller can recover any lost contract term by focusing on the delivery of long-term performance, behaviours and value.  A fair outcome for both buyer and seller.

Posted in Contract Consequences, Incentives, the How | Tagged , , , , , , , | Leave a comment

Award Term – Part 2

In the earlier post (see Award Term – Part 1) I looked at the key elements of an Award Term.  In this post let’s look at how the buyer determines whether to grant an Award Term to the seller.

Award Term Assessment Method

Before considering the assessment criteria there are 3 methods that can be used to decide whether an Award Term should be granted.   While each of these 3 methods are valid, the need to seek a fair position for both buyer and seller has led to an increased used of the ‘Balanced’ method described below as it offers a good mix of flexibility and transparency.

Award Term Assessment Method Advantages Disadvantages
Balanced – where the Award Term is granted based on a combination of specified general rules and buyer discretion
  • Provides seller clarity on the requirements and process for an extension
  • Allows buyer discretion to grant an extension
  • Discretion based on human judgement
  • More complex contract drafting than Qualitative approach

Award Term Assessment Method

Award Term Assessment Criteria

In addition to the assessment methods the buyer will need to develop the Award Term assessment criteria. A couple of years ago a senior executive once joked that from the buyer’s perspective that the Award Term assessment criteria were two simple questions; (1) do I still love you? and (2) can I still afford you?  Humour aside, the intent of these two questions is sound.  Firstly, is the seller’s both performance and relationship / behaviours still delivering the buyer’s outcome (i.e. am I still in love)? Secondly, has the seller improved the value of the contract by reducing the price of the services or improving / increasing the scope of the services (i.e. can I still afford you)?  The message, while silly, is simple; for a contract extension to be granted the seller needs to deliver good performance supported by positive, collaborative behaviours and improved value for money over the life of the contract.

The table below provides further Award Term assessment criteria based on the Support variant of the Australian Standard for Defence Contracts (ASDEFCON) contract templates.

Core Award Term Assessment Criteria
(Seller . . .)
Optional Award Term Assessment Criteria
  • performs obligations in a way that satisfies the Contract objectives;
  • behaviours have positively contributed to seller performance;
  • performance against each financial performance measure for the review period is assessed as meeting or exceeding the contracted level of performance; and
  • performance against all non-financial performance measures is assessed as meeting or exceeding the contracted level of performance.
  • outcome of any cost review is assessed as acceptable.
  • no Remediation Plan required during review period, or if one has been raised, the seller has completed all steps to the satisfaction of the buyer.

Core and Optional Award Term Assessment Criteria

Importantly, as these contract extensions reflect additional price to the buyer, it is recommended that any Award Term determination occur before the buyer’s annual budget cycle to allow this price, and any variation, to be included in the buyer budget for the next Financial Year.  This may need the Award Term determination to occur many months before the end of a financial period since in many organisations, especially large organisations, the budget cycle can be up to 4 months earlier.

In the last post of the series I will look at 3 examples to hopefully aid the understanding and practical application of this Performance Based Contracting approach.

Posted in Contract Consequences, Incentives, the How | Tagged , , , , , , , | 1 Comment

Award Term – Part 1

Central to the use of Performance Based Contracts (PBC) is the balancing of buyer needs with seller (contract) consequences, referring to the variety of commercial rewards and remedies that apply as part of a PBC and can include contract extension, payment, remediation plans, stop payment, contract termination, etc.

One of the main incentives used in PBCs is the awarding of additional contract duration (or term) for good or superior performance.  This additional tenure, called an Award Term by many PBC practitioners, is the process that determines whether additional contract duration will be granted.  While Award Term is the label generally used in Australia, other labels include Rolling Wave or Rolling Window, both highlighting the reoccurring nature of granting of contract extensions.

Over the next few blog entries I will discuss what an Award Term is, including key features, and how to use them.  At the end I will give some scenarios to illustrate how they work.  So how does an Award Term work and what are the key features?

The key elements of an Award Term contract are as follows:

  1. Initial Term– this is the base duration of the first contract and for many long-term contracts (greater than 10 years) the initial term will typically be between 5 – 7 years. For shorter-term contracts (less than 10 years) this will typically be 3 years.
  2. Maximum Term– if all Award Terms were granted, this is the total contract duration.
  3. Award Term– this the duration of the extension that, when granted, will cumulatively add onto the initial term. The size of the duration balances the cost associated with assessing and granting the Award Term with the need to keep the seller motivated by having Award Term assessments.  While many Performance Based Contracts will use the simplicity of a fixed extension duration of 1 or 2 years, many advanced Performance Based Contract will use a variable Award Term.  For example, using a variable Award Term the buyer could initially grant only a 1-year extension.  Once the seller has proven their performance, the buyer could grant a 2 or 3 year extension instead.  Should the seller’s performance reduce, the buyer can either go back to shorter extensions (e.g. 1 year) as a form of performance probation or not grant an extension.
  4. Off-Ramp Date– is the date where the buyer will automatically begin a retendering activity for either all or some of the contract scope. This date is set by allowing the buyer sufficient time to undertake a retendering activity before expiry of the current contract. For many large contracts, this can be between 1 and 2 years.  For many government agencies, this will be 2 years.
  5. Initial Award Term Assessment Date– is the date where the buyer is first assessing the seller for a contract extension. Typically, it is at least 2 years before the Off Ramp Date to allow the seller at least one failed Award Term Assessment before the buyer commences a retendering activity.

To help illustrate these elements and their sequence consider the example shown below.  In this example, consider a large contract that provides a range of support services to a buyer including maintenance, logistics and training.

award-term-generic.png

Example Award Term Performance Based Contract

Firstly, given the large scope of the contract, it is likely that the initial term of the contract would be at least 6 years and has a maximum term of 15 years.  Additionally, the buyer has sought internal funding for $150 million based on the assumption that the price per year was $10 million up until the end of the Maximum Term.  However, it should be remembered that the seller is only entitled to $60 million as that is the price of the initial term of the contract.  Any additional contract term, and therefore price, will need to be earned through the Award Term process.

Secondly, given the complexity and size of the contract it typically takes the buyer 2 years to undertake a complete tendering process from writing the tender through waiting for a response, evaluation, negotiation and signature, the buyer has set the Off-Ramp Date at 2 years from the end of the Initial Term.

Finally, to allow at least two assessment periods between the initial Award Term Assessment date and the Off-Ramp Date, the Initial Off Ramp Assessment Date has been set 2 years earlier than the Off-Ramp Assessment Date.

In the next entry we’ll look at the assessment methodology for whether an Award Term is granted.

Posted in Contract Consequences, the How | Tagged , , , , , , , | 4 Comments

Comparing Performance Across Cultural Divides

As part of teaching in the United Kingdom (UK) one of the attendees mentioned a common issue found in Performance Based Contracting (PBC); the wish to compare performance across cultural boundaries.  In this case the attendee mentioned their organisation was using a Net Promoter Score performance measure to compare performance of a UK and US part of the same organisation.  A Net Promoter Score performance measure is one that uses feedback to a single question: How likely are you to recommend our company/product/service to a friend or colleague?  The scoring for this answer is most often based on a 0 to 10 scale.

In this case the US part of the organisation was getting 9.8 out of 10; unquestionably, a great good score.  However, the UK part of the same organisation was only getting 8.8 out of 10.  So why the difference?  Initially, most people will believe that the UK part of the organisation is simply not performing as well as the US part.  However, is this true?

Culturally, people from the US are more likely to give full scores (i.e. 10 out of 10) for excellent performance.  Therefore, the 9.8 out of 10 is possible.  However, regardless of the same excellent performance, people from the UK will not give 10 out of 10 since, culturally, they believe even excellent service can be improved.  Therefore, they must leave room in the score to improve, intending to drive future improvement.  Hence, 9 out of 10, and the average of 8.8 instead of 9.8.

When the UK attendee asked their clients whether there was anything else they could do to improve the performance to a 10, the answer was ‘no’. When the attendee questioned why they didn’t score 10 out of 10, each client simply stated they could not give a 10 out of 10 just in case the performance got better.  However, they were completely satisfied with the services delivered with attendee not being able to do anything better.  A score of 9 was simply the best they could give.

Many of you reading this may believe this is a story; a parable to help explain the cultural issues. However, it is true, and importantly, not an isolated instance.  Accordingly, PBC practitioners need to be careful when directly comparing performance scores between cultures.

Posted in Behaviours, Case Study | Tagged , , , , , , | Leave a comment

Perverse Incentives – Part 2

In the previous article (see Perverse Incentives – Part 1) I discussed why PBC practitioners need to be aware of the potential for perverse incentives in our PBC arrangements.  In this article let’s look at how PBC practitioners, both buyers and sellers, can avoid accidentally including perverse incentive. In a previous article (see Unintended and Perverse Outcomes) I looked at a couple of recent cases that highlighted some possible actions.

From the buyer’s perspective, this requires “testing” of the PBC to see whether the Performance Management Framework (PMF) works as intended based on the three tests described above. You can undertake this testing by:

  • testing the arrangement for all areas of performance, from 100% (or above if incentivised) to 0% performance, observing what conditions result in this, and potential response from the seller;
  • including non-monetary performance measures such as Strategic Performance Measures (SPMs), potentially reflecting enterprise performance and enterprise behaviour  aligning buyer and seller outcomes and behaviours (see When is a KPI not a KPI? for SPM description).
  • monitoring of the performance and behaviours of the seller to watch whether unintended outcomes are occurring, their response as part of  the routine contract management activities.

However, “testing” it isn’t solely the buyer’s responsibility.  I believe sellers have a reciprocal responsibility to make sure, regardless of whether the PMF of the arrangement allows it, to behave fairly.  For example, during both the tendering process and operation of the contract, I believe sellers should have the courage to tell the buyer of any issues they find in the arrangement, especially one that may lead to a perverse incentive.  Equally, buyers should reward constructive feedback from sellers.

For example, in building the first transcontinental railroad in the 1860s, the United States Congress agreed to pay the builders per mile of track laid; a form of ‘time and materials’ basis of payment.  As a result, Thomas C. Durant of Union Pacific Railroad lengthened part of the route forming a bow shape unnecessarily adding miles of track to maximise payment.  In this case, perhaps both buyer and seller needed another performance measure rewarding the most direct route and speed of construction.

While many will argue that this perverse outcome was a result of dishonesty on behalf of the seller, I believe we as PBC practitioners should acknowledge the potential for perverse incentives in our arrangements and take action to remove, minimise or mitigate putting buyers and sellers in this position.

But regardless of the approach, we all need to be aware of perverse incentives to make sure we don’t end up in our very own cobra effect.

Posted in Behavioural Economics, Behaviours, the What | Tagged , , , , , , , , | Leave a comment