The Mathematics of a Performance Based Contract (PBC) – Part 1

Over the 15 years of applying and teaching Performance Based Contracting (PBC) I have always had trouble trying to describe the mathematics of PBCs, especially to those who

Reminded by my family’s current viewing of the US TV Series, Numbers, I am inspired to try and make PBC mathematics more accessible in order for non-mathy people to better understand, and hopefully use, these techniques.  So where is maths used in a PBC?  Interestingly, it is used throughout a PBC, from the performance levels to the percentage weighting of each of the say three Key Performance Indicators (KPIs) (e.g. 50% for KPI-1, 30% for KPI-2 and 20% for KPI-3) through to adjustment of the performance fee / At-Risk Amount.  However, for the majority of circumstances, the math in these instances is fairly simple and doesn’t tend to worry most people.  But there are two areas where this isn’t the case:

  1. working out the Adjusted Performance Score (APS) from an Achieved Performance score for a non-linear payment curve; and
  2. using historical data to set a performance level.

In these instances while the maths is fairly simple, they tend to confuse people leading to either avoidance by using an alternate approach which doesn’t require the same level of mathematical ability or removal altogether.  So let’s look at each in turn starting with the Adjusted Performance Score calculation.

Calculating the Adjusted Performance Score (APS) in a Performance Based Contract

In a series of previous articles (see Payment Curves Part 1, Part 2 and Part 3) I highlight the various methods for describing how to turn an Achieved Performance score (i.e. the raw score of the performance measure such as number of days late from a milestone, percentage of satisfied deliveries, number of outages per 1,000 operating hours, etc.) into an APS, which is always a percentage.  While there are a number of ways of doing this, a common method is to use a straight line between the minimum performance level (at which APS = 0%) and the required performance level (at which APS = 100%).  This can be a simple straight line between these 2 points, or as in the Australian Department of Defence approach, this line is broken into 2 segments with an inflection / elbow point making it two lines. However, the issue remains the same, how do I calculate the APS from the Achieved Performance score based on the straight line?

Consider the generic non-linear Payment Curve described in Figure 1 which has four “bands” of performance.

Generic Non-Linear PBC Payment CurveFigure 1 – Generic Non-Linear PBC Payment Curve

Here, all but Band D has a requirement to do some maths in order to determine the APS from the Achieved Performance level using the equation for a straight line:

y = mx + b

where:

  • y = Adjusted Performance Score (APS), or the vertical axis of the payment curve in percentage
  • m = gradient of slope of the line – the lower the value of m the less steep the line, while a higher value of m results in a steeper line
  • x = Achieved Performance score, or the horizontal axis, in whatever scale (e.g. percentage of satisfied deliveries)
  • b = starting or ‘offset’ point which is the value for y when x = 0 (e.g. for Band B the minimum APS is 80%, therefore z = 80%)

If you are a bit rusty you can get more information from the following website (see https://www.mathsisfun.com/equation_of_line.html).

In the next article I will look at how we apply this to a PBC using an example from the standard Australian Department of Defence Non-Linear PBC Payment Curve.

Posted in Consequence Analysis, payment curve, Performance Measure, the How | Tagged , , , , , , , | 2 Comments

How to Nudge a Performance Based Contract (PBC) – Part 2

In the previous article (see How to Nudge a Performance Based Contract (PBC) – Part 1) I described how successful contracts deliver the desired outcomes of both buyers and sellers; the classic “win-win” outcome that we all strive for.  We achieve this outcome by balancing the requirements of both buyer and seller with the consequences (rewards and remedies) that provides both positive (carrot) and negative (stick) incentives.

Examining the relationship between the three areas described in Figure 1 of Part 1 (Buyer Need, Seller Need and Commercial Construct), as illustrated in Figure 1, it is possible to observe the following characteristics:

  • moving horizontally (left / right) changes the nature of the agreement; the left-hand edge is a conventional commercial construct transferring risk from buyer to seller through a prescriptive, written contract whereas the right-hand edge is a highly collaborative (relational) approach that shares commercial risk between the parties with limited formal detail; and
  • moving vertically (up / down) changes the outcome for both the buyer and seller; the top edge meets the Buyer Needs but may be seen by the seller as punitive, “unfair” and a “bad deal” whereas the bottom edge meets the Seller Needs but may be seen by the buyer as risk free providing no assurance of performance; “money for nothing”.

Figure 1 : Relationship between Seller Need, Buyer Need and Commercial Construct

Acknowledging these extreme edges, the ideal PBC is one that balances each of these three areas represented by the overlapping region in the centre of the diagram where the Buyers Needs are delivered by linking these to the Sellers Needs through a fair collaborative Commercial Construct.

So how do we achieve this balance?

A PBC is fundamentally a behavioural economics approach where the buyer is trying to “nudge” seller behaviour to deliver their outcome by applying range of complementary rewards and sanctions.

Those familiar with Thaler and Sunstein’s concept of nudging may disagree with my analogy since they typically refer to nudging in a social engineering context (e.g. influencing the uptake of health insurance) and typically applied with a lighter touch.  However, I believe the intent is same.  Using a range of complementary rewards and remedies within a PBC creates a choice architecture that guides and disables seller choice rather than direct legal consequences of a conventional contract such as contractual breach.

While this type of nudge is more direct than Thaler and Sunstein probably envisioned, perhaps we can call it a “heavy nudge”, I believe the intent is same.  Moreover, I believe those of us involved in developing PBCs should strive to deliver a PMF that allows a seller to self-regulate their own behaviour by defining the consequences of their actions or omissions, including positive rewards (incentives), for delivery of the required outcomes.  One of the ways of achieving this is by designing our Commercial Construct with a range of consequences that scale (escalate) based on significance of performance variation from our requirements, in terms of duration, impact of the variation on the buyer, and whether this represents a continuing / repeating performance issue. This approach can be seen in Table 1.

Commitment
(Disable Choice)
Incentive
(Guide Choice)
Positive Incentive (Carrot)
  • Contract Duration and Extension
  • Incentives
  • Recognition schemes
Negative Incentive (Stick)
  • Stop Payment
  • Termination
  • At-Risk Amount
  • Liquidated Damages

Table 1 : Typology of Incentives and Commitments[1]

Here, commitments represent consequences that are either too good to refuse or too bad to accept. In these cases, the consequences, either positive (carrot) or negative (stick), must be of such significance to disable future choice.  On the other hand, incentives represent consequences that guide rather than disable choice. The difference between commitment and incentive is simply choice; does the consequence disable or merely guide choice.

For organisations that want to minimise the variation between PBC, one approach used with good results is to simplify contract drafting by codifying a default Commercial Construct that includes a range of consequences that scale (escalate) based on significance of performance variation from the buyer’s requirements.  The Australian Standard for Defence Contracting (ASDEFCON) suite of contract templates[2]are one example of this approach.

In summary, when commercial practitioners are developing their contracts they should be mindful of the field of behavioural economics ensuring their “Choice Architecture” delivers a fair and balanced commercial construct that delivers both the buyer’s and seller’s needs.  If commercial practitioners can do this, it should lead not only to positive outcomes, but also to positive and supportive behaviours.  And who wouldn’t want this!

[1]                Adapted from AYRES, I. “Carrots and Sticks”, Bantam, 2010

[2]               http://www.defence.gov.au/casg/DoingBusiness/ProcurementDefence/ContractingWithDefence/PoliciesGuidelinesTemplates/ContractingTemplates/asdefcon.aspx

Posted in Behavioural Economics, Nudging, the What, the Why | Tagged , , , , , , , | Leave a comment

How to Nudge a Performance Based Contract (PBC) – Part 1

Commercial instruments, especially sophisticated ones such as Performance Based Contracts (PBCs), have the ability to shape the behaviour of both buyer and seller.  While many commercial practitioners see contracts in their simplest form, an explicit and directive agreement between buyer and seller, some acknowledge that contracts can be used for much more.  At their best, contracts can define, shape and nudge positive outcomes, including behaviours, in both buyer and seller.

In 2008 Richard Thaler and Cass Sunstein published, Nudge: Improving Decisions about Health, Wealth, and Happiness, which used behavioural economics as a method for helping people improve their decisions.  This book has had a profound impact on the field of economics introducing the notion of designing choice architectures to help people make better decisions by predicting how humans behave in certain situations.  Moreover, they gave us a blue print of how we could use various economic instruments to “nudge” human behaviour.  This is achieved by designing a “choice architecture” allowing individuals to choose their own path while highlighting the advantages and disadvantages of each choice; termed liberal paternalism.

Since this early description there have been many uptakes of the “nudge” approach including the creation of Behavioural Insight Teams (BIT)in a number of countries helping various local, state and federal governments design, modify and apply policies. However, applications are typically not focused on activities involving commercial contracts.

Despite the success of the book and the concept there are a number of proponents that disagree with the approach.  Some feel that providing a nudge towards a particular choice is itself a form of bias, and more extreme commentary claiming it is removing choice from the individual, even if offered.

So why am I bringing this up?

In the early 2000’s a number of organisations, especially in the Defence sector, began using Performance Based Contracts (PBC), or sometimes referred to as Performance Based Logistics (PBL) contracts or Contracting for Availability, with the intent of improving performance at a decreased price.  Indeed, the ability for a PBC to simultaneously deliver these outcomes[1][2][3][4]          has seen them more popular and their use more widespread.

Performance Based Contracting (PBC) Definition

Performance Based Contracting is an outcomes-oriented contracting method that ties a range of monetary and non-monetary consequences to the contractor based on their accomplishment of measurable and achievable performance requirements.

But for many people designing and applying PBCs over the past decade will see similarities between the concept of nudge and the intent of a PBC.  That is a highly successful PBC will:

  1. drive the right behaviour in the seller by addressing the Seller Needs;
  2. provide adequate commercial protections for the buyer by addressing the Buyer Needs; and
  3. balances both these needs within a usable Commercial Construct.

Accordingly, a good PBC aligns the Seller Needs against delivery of the Buyer Needs that is described in the Commercial Construct. Figure 1 highlights the interaction between these three areas based on, an asset management scope of work where:

  • Buyer Needs represent the traditional specification of requirements. In sustaining Complex Materiel[5] these requirements can be grouped into three broad areas of Asset Usage, Asset Optimisation and Asset Preservation underpinned by Safety Culture, Cost Consciousness and Positive Behaviours.
  • Seller Needs represents both the financial and non-financial outcomes required by the seller ranging from contract price and profit margin to contract duration and recognition schemes.
  • Commercial Construct represents the agreement, typically a contract, between the parties that fairly motivates and delivers both Buyer Needs and Seller Needs. The optimal Commercial Construct is a balance of conventional (written) contracting protections and a collaborative (relational) contracting approach.

Figure 1 : Interaction between Seller Need, Buyer Need and Commercial Construct

In order to achieve this each PBC requires a Performance Management Framework (PMF) which:

 “. . . ensures that the delivery of the enterprise outcome by creating a self-regulating agreement which uses a range of incentives to guide and disable choice.”

However, it is more than simply the Key Performance Indicators (KPI), but rather how all facets of the contract fit together to drive behaviour.  Indeed, key to the successful operation of the PBC is the notion of self-regulation where the seller’s performance determines the commercial consequences; positive or negative.  This idea of self-regulation aligns with the Behavioural Economics notion of “nudging” where, rather than the buyer directing the seller exactly how to deliver the outcome, the PMF should “nudge” the seller’s decisions allowing choice in full knowledge of the buyer’s requirements and consequences of the seller’s actions.

[1]            BOYCE, J. and BANGHART, A., “Performance Based Logistics and Project Proof Point – A Study of PBL Effectiveness”, Defense AT&L: Product Support Issue, March-April 2012

[2]            GUAJARDO, J.A.; COHEN, M.A.; NETESSINE, S and KIM S-H “Impact of Performance-Based Contracting on Product Reliability: An Empirical Analysis”, July 2009, revised February 2010, INSEAD Working Paper No. 2011/49/TOM

[3]            “Performance & Outcome Based Contracts 2015”, International Association of Contract and Commercial Management (IACCM), 2015

[4]       DOOGAN, C., LINGER, H., HOLMES, D. and BJERKNES, G., “Enquiry into Performance Based Contracting Practice – Phase 2 Case Studies”, June 2018

[5]       Complex materiel can be defined as those assets that support military operations through flying (e.g. aircraft and helicopters), sailing (e.g. ships, boats and submarines), driving (e.g. wheeled and tracked vehicles) and transmitting (e.g. satellite ground stations).

Posted in Behavioural Economics, Nudging, the How, the Why | Tagged , , , , , , , | 1 Comment

Perverse Incentive Follow-up

In earlier articles (see Perverse Incentives – Part 1 and Part 2,  and Unintended and Perverse Outcomes) I discussed the importance of checking the Performance Management Framework (PMF) within your Performance Based Contract (PBC) to watch for perverse incentives.  Those incentives, either positive or negative, that have an unintended and undesirable result which are contrary to the original intent.  In the case of a PBC, the interest of the buyer.

One of the case studies I used was the widespread practice of Victorian Police officers falsifying over 258,000 roadside alcohol breath tests over 5½ years (about 1.5% of all tests carried out during that time) by inflating breath test bags themselves to meet quotas noting there were no financial incentive for officers to fake tests.  For those not familiar with the Victorian Police force, they are a state level police force for 1 of the 7 state / territories in Australia noting there are only state and federal police forces in Australia with the state of Victoria having a population of 6.5 million in 2018 and an area of 227,436 km², which is sightly smaller than the area of the United Kingdom.

The Victoria Police conducted an independent investigation into the falsification of preliminary breath tests (PBTs) by their officers, called Taskforce Deliver, led by former Chief Commissioner Neil Comrie AO APM.  In the Taskforce Deliver report the taskforce found that the falsification of breath tests was widespread but fortunately may have occurred at much lower rates than first estimated by Victoria Police.  From a PBC perspective there are some interesting observations that equally apply to designing a PMF.  These are as follows:

Screen Shot 2019-04-20 at 8.02.30 pm

Importantly, many of the aspects highlighted in the report are common pitfalls when establishing a PBC.  These are:

  • Using quantitative performance measures due to their apparent simplicity and objectiveness in application (i.e. gathering, reporting and analysing) when qualitative performance measures should have been considered.
  • Applying performance measures without the enabling ‘infrastructure’ such as training, governance and IT to support operating the measures.
  • Not understanding / linking the Required Performance Level to a high-level outcome, including testing whether the performance level was achievable with current resources.
  • Using only a single performance measure to measure success (or otherwise) of a complex multi-dimensional activity (see The Allure of a Single Measure).

In summary, some of the issues raised in the Taskforce Deliver report could have been avoided, or at least minimised, through a better practice approach to applying performance measures to nudge behaviour.  While much of this better practice is covered in this blog and my book, Mastering Performance Based Contracting, over the next couple of months we will look at better practice performance measure selection and design.

Posted in Behavioural Economics, Book, Case Study, Incentives, Nudging, perverse incentives, the How, Uncategorized | Tagged , , , , , , , , , | Leave a comment

Award Term – Part 3

Having described the both the key elements and operation of an Award Term in the previous 2 posts (see Award Term – Part 1 and Award Term – Part 2), to aid in our understanding I will now look at 3 scenarios highlighting how an Award Term contract will work.

Scenario 1 – Good Performance

In scenario 1, consider the original example from the first post (see Award Term – Part 1) where the seller is delivering good performance, behaviours and value.  This scenario is shown in the figure below.  In this scenario in year 2 of the contract the buyer undertakes the Award Term assessment process resulting in a satisfactory rating.  In this case (i.e. the first Award Term) the buyer would only grant 1 additional year (i.e. year 7).

Performance Based Contracting Award Term Scenario 1

If the seller continues to deliver good performance, behaviours and value in through the Award Term assessment process the seller is again given a satisfactory ratingand is granted 1 additional year (i.e. year 8). In year 4 of the contract, given the seller has continuously delivered good performance, behaviours and value the buyer grants an additional two years (i.e. years 9 and 10) to the contract term.

Scenario 1 highlights the ability of the buyer to vary the amount of contract duration awarded through the Award Term process.  Moreover, while it is unlikely that an initial Award Term would be longer than the minimum duration (e.g. 1 year in our example), it does highlight continued, long-term good / superior performance, behaviours and value can reward the seller with longer Award Term durations.

Scenario 2 – Poor Performance

Unlike scenario 1, in scenario 2 the seller is delivering poor performance, behaviours and value. This scenario is shown in the figure below.  In year 2 of the contract the buyer undertakes the Award Term assessment process resulting in an unsatisfactory rating.  In this case the buyer would not grant any additional contract term beyond the original contract term (i.e. 6 years).  During year 3 the seller continues to deliver poor performance, behaviours and value and again is assessed as unsatisfactoryand not grants an additional year of contract term.  The original contract term of 6 years remains. Importantly, should the seller be assessed as unsatisfactory in year 4 of the contract the Off-Ramp Date will be reached resulting in an automatic re-tendering activity for some or all the contract scope.

Performance Based Contracting Award Term Scenario 2

Scenario 2 highlights the buyer’s ability not to grant additional contract tenure through the Award Term assessment process for poor performance, behaviours and value. Moreover, it highlights that for continued poor performance, behaviours and value the contract will “naturally expire (end)” since no contract extensions are granted.

Importantly, where any of the poor performance, behaviours and value result in other rewards and remedies such as a reduction in performance fee, stop payment and termination can still apply.

Scenario 3 – Mixed Performance

In scenario 3 the seller is delivering poor performance, behaviours and value during the first two years of operation.  This scenario is shown in the figure below.  In year 2 of the contract the buyer undertakes the Award Term assessment process resulting in an unsatisfactory rating.  In this case the buyer would not grant an additional contract term beyond the original contract term (i.e. 6 years).

In this scenario, given the buyer’s feedback the seller makes changes to their contract delivery approach and in year 3 the seller delivers good performance, behaviours and value resulting in a satisfactory ratingfrom the Award Term assessment process.  In this situation  (i.e. a previous unsatisfactory rating) the buyer would only likely grant 1 additional year (i.e. year 7) with a new total contract term of 7 years.

The seller then continues to deliver good performance, behaviours and value and in year 4 again receives a satisfactory rating from the Award Term assessment process with the buyer granting 1 additional year (i.e. year 8) with a new total contract term of 8 years.

Scenario 3 highlights the strength of the Award Term process to drive seller behaviours by illustrating how poor performance, behaviours and value are not rewarded, but how through remediation, the seller can recover any lost contract term by focusing on the delivery of long-term performance, behaviours and value.  A fair outcome for both buyer and seller.

Posted in Contract Consequences, Incentives, the How | Tagged , , , , , , , | Leave a comment

Award Term – Part 2

In the earlier post (see Award Term – Part 1) I looked at the key elements of an Award Term.  In this post let’s look at how the buyer determines whether to grant an Award Term to the seller.

Award Term Assessment Method

Before considering the assessment criteria there are 3 methods that can be used to decide whether an Award Term should be granted.   While each of these 3 methods are valid, the need to seek a fair position for both buyer and seller has led to an increased used of the ‘Balanced’ method described below as it offers a good mix of flexibility and transparency.

Award Term Assessment Method Advantages Disadvantages
Balanced – where the Award Term is granted based on a combination of specified general rules and buyer discretion
  • Provides seller clarity on the requirements and process for an extension
  • Allows buyer discretion to grant an extension
  • Discretion based on human judgement
  • More complex contract drafting than Qualitative approach

Award Term Assessment Method

Award Term Assessment Criteria

In addition to the assessment methods the buyer will need to develop the Award Term assessment criteria. A couple of years ago a senior executive once joked that from the buyer’s perspective that the Award Term assessment criteria were two simple questions; (1) do I still love you? and (2) can I still afford you?  Humour aside, the intent of these two questions is sound.  Firstly, is the seller’s both performance and relationship / behaviours still delivering the buyer’s outcome (i.e. am I still in love)? Secondly, has the seller improved the value of the contract by reducing the price of the services or improving / increasing the scope of the services (i.e. can I still afford you)?  The message, while silly, is simple; for a contract extension to be granted the seller needs to deliver good performance supported by positive, collaborative behaviours and improved value for money over the life of the contract.

The table below provides further Award Term assessment criteria based on the Support variant of the Australian Standard for Defence Contracts (ASDEFCON) contract templates.

Core Award Term Assessment Criteria
(Seller . . .)
Optional Award Term Assessment Criteria
  • performs obligations in a way that satisfies the Contract objectives;
  • behaviours have positively contributed to seller performance;
  • performance against each financial performance measure for the review period is assessed as meeting or exceeding the contracted level of performance; and
  • performance against all non-financial performance measures is assessed as meeting or exceeding the contracted level of performance.
  • outcome of any cost review is assessed as acceptable.
  • no Remediation Plan required during review period, or if one has been raised, the seller has completed all steps to the satisfaction of the buyer.

Core and Optional Award Term Assessment Criteria

Importantly, as these contract extensions reflect additional price to the buyer, it is recommended that any Award Term determination occur before the buyer’s annual budget cycle to allow this price, and any variation, to be included in the buyer budget for the next Financial Year.  This may need the Award Term determination to occur many months before the end of a financial period since in many organisations, especially large organisations, the budget cycle can be up to 4 months earlier.

In the last post of the series I will look at 3 examples to hopefully aid the understanding and practical application of this Performance Based Contracting approach.

Posted in Contract Consequences, Incentives, the How | Tagged , , , , , , , | 1 Comment

Award Term – Part 1

Central to the use of Performance Based Contracts (PBC) is the balancing of buyer needs with seller (contract) consequences, referring to the variety of commercial rewards and remedies that apply as part of a PBC and can include contract extension, payment, remediation plans, stop payment, contract termination, etc.

One of the main incentives used in PBCs is the awarding of additional contract duration (or term) for good or superior performance.  This additional tenure, called an Award Term by many PBC practitioners, is the process that determines whether additional contract duration will be granted.  While Award Term is the label generally used in Australia, other labels include Rolling Wave or Rolling Window, both highlighting the reoccurring nature of granting of contract extensions.

Over the next few blog entries I will discuss what an Award Term is, including key features, and how to use them.  At the end I will give some scenarios to illustrate how they work.  So how does an Award Term work and what are the key features?

The key elements of an Award Term contract are as follows:

  1. Initial Term– this is the base duration of the first contract and for many long-term contracts (greater than 10 years) the initial term will typically be between 5 – 7 years. For shorter-term contracts (less than 10 years) this will typically be 3 years.
  2. Maximum Term– if all Award Terms were granted, this is the total contract duration.
  3. Award Term– this the duration of the extension that, when granted, will cumulatively add onto the initial term. The size of the duration balances the cost associated with assessing and granting the Award Term with the need to keep the seller motivated by having Award Term assessments.  While many Performance Based Contracts will use the simplicity of a fixed extension duration of 1 or 2 years, many advanced Performance Based Contract will use a variable Award Term.  For example, using a variable Award Term the buyer could initially grant only a 1-year extension.  Once the seller has proven their performance, the buyer could grant a 2 or 3 year extension instead.  Should the seller’s performance reduce, the buyer can either go back to shorter extensions (e.g. 1 year) as a form of performance probation or not grant an extension.
  4. Off-Ramp Date– is the date where the buyer will automatically begin a retendering activity for either all or some of the contract scope. This date is set by allowing the buyer sufficient time to undertake a retendering activity before expiry of the current contract. For many large contracts, this can be between 1 and 2 years.  For many government agencies, this will be 2 years.
  5. Initial Award Term Assessment Date– is the date where the buyer is first assessing the seller for a contract extension. Typically, it is at least 2 years before the Off Ramp Date to allow the seller at least one failed Award Term Assessment before the buyer commences a retendering activity.

To help illustrate these elements and their sequence consider the example shown below.  In this example, consider a large contract that provides a range of support services to a buyer including maintenance, logistics and training.

award-term-generic.png

Example Award Term Performance Based Contract

Firstly, given the large scope of the contract, it is likely that the initial term of the contract would be at least 6 years and has a maximum term of 15 years.  Additionally, the buyer has sought internal funding for $150 million based on the assumption that the price per year was $10 million up until the end of the Maximum Term.  However, it should be remembered that the seller is only entitled to $60 million as that is the price of the initial term of the contract.  Any additional contract term, and therefore price, will need to be earned through the Award Term process.

Secondly, given the complexity and size of the contract it typically takes the buyer 2 years to undertake a complete tendering process from writing the tender through waiting for a response, evaluation, negotiation and signature, the buyer has set the Off-Ramp Date at 2 years from the end of the Initial Term.

Finally, to allow at least two assessment periods between the initial Award Term Assessment date and the Off-Ramp Date, the Initial Off Ramp Assessment Date has been set 2 years earlier than the Off-Ramp Assessment Date.

In the next entry we’ll look at the assessment methodology for whether an Award Term is granted.

Posted in Contract Consequences, the How | Tagged , , , , , , , | 4 Comments