Performance Based Contracting (PBC) Book

Posted on Sat 30 Dec 2017 by Andrew Jacopino

With the imminent start of 2018 many of us are thinking about our New Years resolutions, including me. One of my resolutions is to have finished a Performance Based Contracting (PBC) book by 1st July 2018. Having already worked for over 1 year on this project I hope you will help me focus and make sure that I keep my promise!

Posted in Book, the How, the What, the Why | Tagged , , , , , | Leave a comment

Happy 2017 Holidays to all the Performance Based Contracting (PBC) Blog Readers

Posted on Sat 16 Dec 2017 by Andrew Jacopino

At this joyous time of year, we are grateful for the time we can spend with our own friends and family, and we wish you all abundance, happiness, and peace in a New Year filled with hope. Happy holidays!

Posted in Admin, Holiday Message | Tagged , , , | 2 Comments

Reliability in Performance Based Contracts (PBCs) – Part 2

Posted on Sat 09 Dec 2017 by Andrew Jacopino

In the previous article (see Reliability in PBCs – Part 1) we defined how reliability of a system or service could be used in a Performance Based Contract (PBC) including defining whether we were measuring success or failure, and whether the failure resulted in a mission [critical] failure or not (defined as a logistics failure). In this article we are going to look at the detail needed to measure reliability from a contractual perspective.

Previously we defined those elements that would result in either a mission or logistics failure. However, from a contractual perspective and linked to performance management, who is attributed to (owns) this failure, Buyer or Seller? Typically, this refers to either a chargeable (to the seller) or non-chargeable (to the buyer) failure. Unfortunately, determining whether a failure is chargeable or non-chargeable is not simple, especially when dealing with complex systems and commercial arrangements (e.g. multi-party contracts). As illustration of this consider the following scenario.

The buyer receives serviceable (working) vehicles from the seller that are then driven by the buyer’s employees to deliver buyer’s goods. The seller is responsible for ensuring that there are enough vehicles available to the buyer with all maintenance completed and the vehicles are “roadworthy” (i.e. no safety or mechanical issues such as working lights).

Now consider the buyer has failed to make a delivery due to the vehicle not working. Is this a chargeable failure? If the failure was due to a mechanical issue (e.g. engine not starting) then this is simple; chargeable failure to the seller. However, what if the vehicle was in an accident that was the fault of the buyer’s employee? Or the buyer’s driver did not put fuel in the vehicle?  What then? What if a third-party caused the accident (e.g. neither the buyer nor seller)? Alternatively, what if the vehicle was operating outside it’s normal role (e.g. carrying twice the maximum design weight making the brakes less effective) or in a different environment (e.g. extreme cold resulting in an icy road). What now? Is the failure chargeable or non-chargeable?

In a previous article (see Setting the Performance Levels (Part 3)) I discussed criticality of Configuration, Role and Environment (CRE) in setting performance levels. In failure sentencing (the name given in the Reliability Engineering community for this process), we should also consider the CRE.  While it is possible to transfer all risks from buyer to seller, in many circumstances this risk transfer is unaffordable for the buyer.

The alternative is to specify all these exclusions in the PBC. Indeed many years ago my colleagues and I would try, sometime resulting in up to 3 pages of possible exclusions like the ones above. However, explicitly considering and documenting is neither effective or efficient since it is impossible to capture all possible event.  Additionally, the drafting and management cost to both buyer and seller becomes too high.

To avoid this in recent years my colleagues and I have started either:

  • defined general sentencing principles (as opposed to specific rules); or
  • used “all in” performance measures that are not adjudicated (e.g. no failure sentencing is undertaken). See When a KPI is not a KPI for further details.

The key to which of these 2 types you use is highly dependent on whether the commercial relationship can include “all in” performance measures as part of the performance measures hierarchy. In my experience many contracts are more “master-servant” in nature (i.e. the buyer will tell the seller what to do and how to do it) as opposed to a more partnering style that emphasises a “shared destiny” approach.  However, where the commercial relationship is more partnering and collaborative in nature, the “all in” performance measures, especially when linked to incentive (see PBC Incentive Regime Part 1 and Part 2), are very powerful drivers of positive outcomes for both buyer and seller.

In summary, like availability performance measures, reliability performance measures should be included in many PBC performance measure hierarchies as they represent the goal of many PBC buyers and sellers. However, care must be taken when designing these reliability performance measures to make sure that they accurately reflect the need and take into account the specific CRE of the commercial arrangement. Moreover, unless the commercial relationship reflects a more partnering approach, the use of “all in” performance measures will be difficult to include.

Posted in Performance Measure, the How | Tagged , , , , , , | 1 Comment

Reliability in Performance Based Contracts (PBCs) – Part 1

Posted on Sat 02 Dec 2017 by Andrew Jacopino

“I can predict things. I can improve the uptime and the reliability. I can intervene and cause a better outcome before there’s a problem. ”

Michael Dell

In previous articles I described how to consider availability in Performance Based Contracts (PBCs). In this companion article, we are now going to look at reliability and how we describe it within our PBCs.

Similar to availability, considering reliability of a system or service is one of the highest priorities for both buyer and seller. But what is reliability and how do we describe it within our PBC?

In general terms reliability is defined as “The probability that an item can perform its intended function for a specified interval under stated conditions.”  Using this definition reliability requirements typically reflects either success or failure as a:

  • as a whole number (e.g. the number of successes or failures; 1, 10, 100);
  • as a percentage (e.g. 95% success or fail); or
  • as a rate (e.g. 1 failure per million operations).

Moreover, these measures of reliability represent either:

  • end-to-end, all-encompassing measures such as “mission success rate” defined as percentage of missions that successfully completed when considering all missions undertaken; or
  • technical specifications, such as Mean Time Between Failure (MTBF) or Mean Time Between Critical Failure (MTBCF), defined as the average time (measured in units such as operating hours, distance, number of operations, etc.) between failures.

While the term “mission” is being used here, this simply refers to the intended function or outcome. The mission could be the processing of a financial transaction, delivery of a meal, or completion of a flight from one city to another. The key to defining a successful mission is the seller has met the buyer’s (or end customer’s) need.

In terms of defining whether the measure is success or failure, people vary on which is better to use. I always prefer to use success, regardless of whether representing the buyer or seller, since it creates a positive discussion focused on success (as opposed to negative discussions on failures. The only exception to this is for very high reliability systems such as telecommunication / computer equipment that have ultra high percentages such as 99.995%. Here the numbers are so high both buyer and seller need to focus on the failures / exceptions as opposed to success.

Essential to defining reliability is criticality of the failure to the essential operation of the product or service; sometimes referred either:

  • Mission Critical Failure – which makes the product / services unusable (i.e. the ‘mission’ cannot continue); or
  • Logistics Failure – which may degrade the product / services, however, they can still be used / delivered, albeit at a lower quality or in a slower timeframe.

As you can see from these descriptions, the type of failure becomes central to how we deal with a reliability performance measure. For example, consider a vehicle, whether motorbike, car, bus or truck. All these vehicles have lights that enable them to drive safely at night or in low visibility conditions. So what if the lights on your car did not work and you had to drive to the shops for food on a beautiful clear, sunny day? How would you see this failure? A Mission Critical Failure or Logistics Failure? Since you could still drive safely to the shops then possible this is only a Logistics Failure. However, what if it was night-time (i.e. dark outside) or there was a long tunnel between your house and the shops? How would you consider this now? Mission Critical Failure or Logistics Failure?

Alternatively, what if your engine did not start or the radio / CD / MP3 player did not work? Typically, any failure to an engine would automatically be considered a Mission Critical Failure since the primary role of a vehicle is movement. However, the failure of the radio / CD / MP3 player would only be considered a Logistics Failure since it is not required to achieve the mission (although those of us with small children on long road trips may dispute this!).

To make it simpler to understand and to avoid disputes, especially for large, highly complex systems, many organisations define the systems that, if are not working, would automatically be considered a mission failure. For example, in the airline industry this list is known as the Master Minimum Equipment List (MMEL) that defines all the equipment that needs to be in working order for the aircraft to be considered in a working (sometimes called an operational or serviceable) state. In the Australian Department of Defence, more generically, this list is known as the Mission Critical Item List (MCIL).

Now we have defined what reliability is, in the next article, we will consider the commercial aspects of reliability by looking at who is responsible for failure.

Posted in Incentives, Performance Measure, the How | Tagged , , , , , , | 2 Comments

Treatment of Safety in a Performance Based Contract

Posted on Sat 07 Oct 2017 by Andrew Jacopino

One topic that has constantly come up over the years is how to treat safety in a Performance Based Contract (PBC).

In a previous article (see Designing Successful Performance Based Contracts) we described a PBC as having two main parts; (1) the need (or requirement) and (2) the consequence. In terms of the need part, it is widely acknowledged that safety is a critical element of our workplaces, regardless of whether part of the buyer organisation, seller organisation or a third party. Indeed many of our countries have specific legislation / laws that reflect this need (e.g. Workplace Health and Safety in Australia, The Health and Safety at Work etc Act 1974 in the United Kingdom, Canada Occupational Health and Safety Regulations (SOR/86-304) in Canada, etc.). Therefore, the need to have safety included in the Performance Management Framework (PMF) is without doubt. However, there is no agreement on either the specific need or consequence.

For example, some contracts have included easily measured lag performance measures such as Lost Time Injury Frequency Rate (LTIFR), which is a measure of the amount of time that seller’s staff are away from work due to a work related injury, as a ratio to the total work time (see the Safe Work Australia entry on LTIFR). Moreover, these contracts may link this measure to payment, either positive or negative, as a form of, consequence. Unfortunately, by linking safety to payment, especially a reduction in payment, drives some unintended consequences in sellers, especially those in financial distress. Specifically, this approach may encourage the misreporting of either injury severity or recovery duration. While this may seem overly pessimistic, especially considering the importance placed on safety, I am aware of a situation where an injured person was placed on training as opposed to convalescent leave. Fortunately, this was a rare example. However, I would argue why we need to create situations that reward misreporting?

Given this, my colleagues and I think that while it is essential that we measure safety, it is better to do it as a non-financial and qualitative performance measure. However, this does not mean there are no consequences for poor safety performance. Indeed, regardless of a signed contract, many countries have very strict obligations on organisations to protect their employees and third parties. Accordingly, we tend to use performance measures that such as Strategic Performance Indicator (SPMs) or a System Health Indicator (SHIs) linked to non-financial rewards and remedies such as additional contract tenure or future work noting this may include LTIFR (When is a KPI not a KPI describes the different tiers of performance measures) .

One such performance measure is the Safety Culture SPM which represents, as either Satisfactory or Unsatisfactory, a measure of the seller’s individual and group values, attitudes, competencies and patterns of behaviour that determines the commitment to, and the style and proficiency of, an organisation’s safety management system. Assessing this SPM is against 5 keys areas; (1) risk management, (2) management of work processes, (3) participation, communication and skills, (4) planning, design and procurement and (5) monitoring and review. In my experience, by using a Safety Culture SPM the performance conversation is now on overall safety performance as opposed to a single, quantitatively measurable sub-element such as LTIFR.

So next time you consider using a payment related safety metric in a PBC, pause and consider the behaviour you trying to drive. Then decide whether this approach is achieving this or whether there are other ways. Since in the end, regardless of whether buyer or seller, we all want the same outcome; a safe workplace.

Posted in Behaviours, Consequence Analysis, safety, the Why | Tagged , , , , , , , , | Leave a comment

Performance Based Contracting (PBC) Incentive Regime (Part 2)

Posted on Sat 23 Sep 2017 by Andrew Jacopino

In the previous article (see PBC Incentive Regime (Part 1)) I described the various benefits to both buyer and seller of having an incentive regime in a Performance Based Contract (PBC). In this article, let’s look at how to successfully make a PBC incentive regime.

Principles of a PBC Incentive Regime

To ensure a PBC incentive regime successfully delivers benefits I believe that we need to apply a number of principles as follows:

  • Incentive Gate – payment of the full incentive will need the seller to be both eligible for payment, and assessed as having delivered ‘Superior’ performance against the incentive Key Result Areas (KRAs) noting these can be different to other KRAs. In some cases, eligibility may be linked to seller performance against other contractual aspects such as contractual / behavioural requirements such as Health, Safety Environment (HSE), anti-slavery, bribery and corruption (ABC), etc. sometimes called to a ‘hygiene’ factors referring to their minimal performance requirements.
  • Minimum Performance – eligibility for the incentive is subject to general performance. That is, an incentive will be ineligible for payment where any of the KRAs are assessed as poor or not meeting minimum performance levels (i.e. there is no incentive available where one or more of the KRAs are assessed poor since under delivering in one area and over delivering in another area is not overall superior performance).
  • Assessment Method – assessment of the amount of incentive payable will be based on metrics noting of course they could use qualitative / subjective performance measures (see Designing a Subjective Performance Measure (Part 1) and Designing a Subjective Performance Measure (Part2)).
  • Maximum Incentive – there will be a cap (maximum) on the total amount of incentive available taking into account all incentives. For example, consider the effective profit rate achievable taking into (1) account base profit, (2) incentive and where applicable, (3) cost gain share, as a percentage of the actual costs. Alternatively, incentive funds may be reduced in specific circumstances such as the need to “top up” technical / project risk amounts, insurance claims, etc.
  • Timing of the Incentive – in many cases incentives are tied to buyer financial years, however, sometimes a buyer (at their discretion) may move unearned incentives into later years.
  • Multi-party Incentives – for multi-party (enterprise) incentives, the buyer will assess the enterprise performance and this performance score will apply to all organisations equally (e.g. the score applies equally to multiple sellers within the enterprise without adjudication).

PBC Incentive Key Success Factors

In addition to these principles, there are 3 key success factors required to deliver a successful PBC incentive regime; (1) aligning (targeting) incentives to buyer need, (2) balancing the amount of incentive on offer and (3) managing (calibrating) the expectation of getting the incentive. Let’s look at each in more detail.

1 – Aligning (Targeting) Incentives to Buyer Need

The first key success factor is aligning the incentive to the buyer’s need. This can be as simple as more performance against the current performance measures (e.g. faster delivery, less maintenance, etc.) or more complex (e.g. delivery of enterprise goals where the seller does not have total control of this outcome. Recent experience with large, long-term, strategic partnership PBCs highlight the need to think about enterprise performance and behaviours by using higher-level performance measures such as Strategic Performance Measure (SPMs) (see When is a KPI not a KPI?). This becomes critical where the incentive reflects the need for positive collaboration between multiple sellers in delivering the buyer enterprise goal.

Additionally, it is critical that the buyer can use (realise) the superior performance. For example, if the contract was for delivery of a spare part within 3 calendar days with incentives for early delivery. In this case, should an incentive be paid for early delivery of a part to the buyer’s site on a Sunday when no one is there to accept it? What benefit has been realised? Indeed, it may cost the buyer the have people on-site to accept the part early.

Finally, it may be possible to change the incentives on a semi-regular basis to make sure that they target the changing need of the buyer. While this makes the contract management more complicated, for long-term strategic partnerships this re-setting of incentive offers benefits to both buyer and seller (see Changing Performance Measures).

2 – Balancing the Amount of Incentive on Offer

The second Key Successful Factor is the size of the incentive. So how much should be offered? Rather than this being a simple formula, the key is balancing the risk / reward structure for the entire contract. Many, especially sellers, like the idea of symmetry between the At-Risk Amount (the amount put at risk for performance typically reflecting the profit margin) and the incentive. For example, a 10% At-Risk Amount (ARA) (and therefore 10% profit) would then result in a 10% incentive. While the seller may like this, for the buyer, and especially those in the public sector organisations (e.g. state and federal governments), a potential 20% profit margin is hard to defend when subject public scrutiny. That is, unlike the private sector, many state and federal governments must contend with audit offices, routine governmental hearings and extensive media coverage. Accordingly, paying an incentive should pass the simple question of “what benefit is the government getting for the extra money given that they have a contract?”.

3 – Managing (calibrating) the Expectation of Getting the Incentive

Finally, in my experience it is critical for a good relationship that both buyer and seller are aware of the expectations of either paying or receiving an incentive, beyond the principles described above. Some of these points are described below from each perspective.

Buyer perspective

  • Incentive Reflects Additional Effort, not Luck – that is, incentives should reward the seller’s extra effort to deliver superior performance as opposed to simple luck. While luck may still occur, it shouldn’t be recognised as effort needed to deliver superior performance.
  • Chance of Getting the Incentive is related to the Size of the Incentive – that is, if superior performance is hard to deliver, and is unlikely the result of luck, the size of the incentive should be relatively larger than superior performance that is simple (easy) to deliver.
  • Total Incentive not Guaranteed – that is, the incentive is not an “all or nothing” proposition, but rather the amount of the incentive payable will vary based on how much of the superior performance is delivered. Moreover, that the amount payable is still subject to any eligibility criteria (see PBC Incentive Regime (Part 1)) and delivery of minimum performance.

Seller perspective

  • Is it achievable (Achievable vs. Stretch Goal) – In a previous article (see Stretch Goals) we examined research showing for an incentive to work the seller must have a chance of getting it. It also showed that where there is no chance, regardless of effort, it has a negative effort. Moreover, where the superior performance is hard to deliver, the amount of the incentive should be larger.
  • Return on Investment (RoI) – that is, the amount of effort has a positive RoI. For example, many years ago when working with a large client in the United States we determined that the amount of the incentive on offer was large (e.g. $1 million dollars). Unfortunately, based on our estimates, it would have taken $6 million dollars to earn the $1 million dollars resulting in a net $5 million dollar loss. Clearly not a good financial outcome for the seller.

In summary, the deliberate and careful use of incentives in PBCs can deliver benefits to both buyer and seller. They can encourage and reward the delivery of positive collaborative behaviours and long-term / enterprise outcomes. The key to the successful application of PBC incentives is aligning (targeting) incentives to buyer outcomes, balancing the amount of incentive on offer and managing (calibrating) the expectation of getting the incentive. If you can achieve all this, the outcome of your PBC may exceed your expectations. And that is a great outcome!

Posted in Incentives, the How, the What | Tagged , , , , , | 1 Comment

Performance Based Contracting (PBC) Incentive Regime (Part 1)

Posted on Sun 10 Sep 2017 by Andrew Jacopino

“Incentive – a thing that motivates or encourages someone to do something.”

Oxford Dictionary

There is a lot of debate on the use of incentives in Performance Based Contracts (PBC). Some argue, especially buyers from public sector organisations (e.g. state and federal governments), “why should I pay more for something that I have already contracted for?”. Alternatively, some argue that incentives should exist to reward sellers who deliver better value for money than originally tended through early delivery, reduced cost, increased capability, etc. So who is right?

For many who know me may believe that I support one of these perspectives more strongly. In fact I believe a well-considered and constructed use of incentives in PBCs offers benefits to both buyer and seller as illustrated in Table 1.

Benefits of Incentives to Buyer Benefits of Incentives to Seller
1.     Encourages better performance

2.     Encourages reduction in price/ better value for money

3.     Encourages positive behaviours

4.     Encourages delivery of long-term / indirect enterprise outcomes through collaboration

 1.     Rewards superior performance

2.     Rewards reductions in price / better value for money

3.     Rewards positive behaviours

4.     Rewards collaboration

5.     Allows lost revenue to be earned back

But, before we look at how to implement an incentive regime in a PBC we need to understand the range of possible incentives that are available to PBC practitioners. These fall into 2 categories; (1) performance measure incentives or (2) contract incentives. Let’s now look at each in turn.

Performance Measure Incentive.  This is the simplest and best understood of the PBC incentives. In this case instead of performance measure score stopping at 100%, it continues above 100%. Therefore, in this instance the Weighted Performance Score (see the previous article Performance Measure Weighting) would also be above 100% potentially limited (capped). This is illustrated in Figure 1.

Figure 1 – Performance Based Contracting (PBC) Payment Curve

Contract Incentive – the other type of PBC incentive is related to specific contract terms. Some common ones include:

  • Earn back – the ability to earn back any withheld / lost fees, typically applied at the end of a financial (or calendar) year to allow the seller to keep up revenue despite earlier poor performance.
  • Contract Extension – the ability to earn extra contract terms without competition through the delivery of superior performance (which may include price and behaviours), sometimes referred to an Award Term / Rolling Wave / Rolling Window contracts.
  • Contract Scope / Workshare – the ability to earn extra contract scope from a single buyer / seller arrangement or contract workshare from a single buyer / multi seller arrangement.

For clarity, in the case of contract incentives there are no extra payments for the current scope of work. Instead, they rely on future (or lost) revenue as a reward.

In the next article we’ll look at how to successfully apply an incentive regime to a PBC.

Posted in Benefits, Incentives, the Why | Tagged , , , , , , | 2 Comments

Achieving Balance

Posted on Sat 19 Aug 2017 by Andrew Jacopino

“Everything should be made as simple as possible, but not simpler.”

Albert Einstein

I recently saw a video by Eric Barker author of “Barking Up The Wrong Tree.” In this video Barker reveals, based on two researchers at Harvard (Nash and Stevenson), the four areas you need to fulfil to be happy and have a work-life balance. While I generally wouldn’t reflect on this type of article in this blog, I wanted to give an analogy based on his discussion.

Specifically, Barker highlights that many people look at the “work-life balance” through a single measure such as happiness or money. He points out that approaching simplification in this way misses the overall picture of such a complex need. For example, there are many instances of where simply making more money doesn’t automatically mean relationships do well and people are happier. What Nash and Stevenson discovered was that executives who are most successful in terms of work-life balance used four measures, and by adding a bit in each one of those areas on a routine basis, they ended up finding a good approximation of an overall work-life balance.

In Performance Based Contracting (PBC) many practitioners, and especially their time poor executives, want the same simplified outcome; a single performance measure that encompasses the outcome. By following this want and using a single performance measure many practitioners lose the insight required to manage complex commercial arrangements.

To address this I previously discussed the need to have a number and range of performance measures linked to different consequences (see When is a KPI not a KPI and The Allure of a Single Measure). This approach not only focuses sellers on the overall ‘strategic’ outcomes required by the buyer, but also gives both organisations insight into short and long term performance.  By balancing both the number (quantity) and type (quantitative vs. qualitative) of performance measures. This is critical for highly successful PBCs. It is essential that PBC practitioners consider this as part of their performance measure design.

For the record, the 4 areas identified in the article in no particular order are:

  1. Happiness – “Are you enjoying what you’re doing?”
  2. Achievement – “Are you doing well, getting a head in your career?”
  3. Significance – “Is what you’re doing having positive effects in the people you love?”
  4. Legacy – “In some small way, is what you’re doing making the world a better place?”

In the end I hope that reader can take heed from Barker’s article and achieve balance whether it be in your PBC or your personal work life balance. If not, it is always good to have a stretch target (see Stretch Goals)!

Posted in Performance Measure, the What | Tagged , , , , , , , , | Leave a comment

Cost Performance Measures

Posted on Sat 05 Aug 2017 by Andrew Jacopino

Business, that’s easily defined – it’s other people’s money.

Peter Drucker

In addition to using the Basis of Payment as a method for shaping the cost behaviour of the seller it is also possible to use a “cost” performance measure as part of a Performance Based Contract (PBC). By using a “cost” performance measure it is possible to force the routine discussion of cost performance between buyer and seller as part of the overall performance management strategy.

In this discussion it is important to understand that while we may refer to “cost” this typically reflects the overall cost to the buyer (e.g. the total amount of money paid to the seller, usually called “price”) as opposed to the seller’s cost that does not typically include General and Administrative (G&A) overhead costs (e.g. those indirect overhead costs such as human resources, security and IT functions, etc.), profit associated with this activity or contingency / management reserve (e.g. an amount of money set aside by the seller to cover those unexpected costs). Figure 1 illustrates the difference between contract price and cost, and the various elements that make up contract price.

Figure 1 : Cost vs. Price

Typically these “cost” performance measures represent either the cost of the services / products delivered under the contract (e.g. $1,000,000 for the delivery of 1,000 flying hours per year for an aircraft) or that same services / products delivery cost per unit (e.g. $1,000 per flying hour).

Alternatively, rather than measuring the cost of the services / products delivered, the cost can reflect the “enterprise” Total Cost of Ownership (TCO) which can be defined as the total cost to deliver the capability regardless of organisation undertaking the activity. This includes costs associated with both operations and support including engineering, maintenance and logistics, training, etc. Importantly, if a TCO performance measure is going to be used, it is essential that it captures all costs regardless of organisation including any resources applied by the buyer (e.g. the cost associated with the buyer’s operations team). Similar to the “cost” performance measure, the “enterprise” TCO cost performance measure it can take two forms; TCO or TCO per unit.

Regardless of which type of cost performance measure used there are three key points that need to be considered as follows when designing a “cost” performance measure. Specifically, the performance measure must:

  1. not distract the seller from meeting its performance obligations under the contract (e.g. while cost may be important, so is the delivery of the contract performance);
  2. produce real and effective improvements to the support of the capability without detriment due to aggressive cost saving (e.g. at worst case, contain future contract cost without changing the contract performance levels); and
  3. provides sufficient incentive to the Contractor to achieve the cost outcome (e.g. link to a reward or remedy in the PBC such as increasing (or decreasing) contract duration).

Where “cost” performance measures form part of the performance measure hierarchy they are typically not related to any form of financial consequences, either positive or negative (e.g. a Key Performance Indicator (KPI) using our terminology). Instead, they form part of the Strategic Performance Measures (SPM) reflecting an “enterprise” focus on overall TCO for the capability.

In our experience, where a cost performance measure, especially an “enterprise” TCO performance measure, has been included as part of the overall performance management strategy we have observed better cost outcomes as the “enterprise” is forced to routinely discuss the topic of cost. The question really should be, can you afford not to include a cost performance measure?

Posted in Performance Measure, the How | Tagged , , , , , , , , , | Leave a comment

Designing a Subjective Performance Measure (Part 2)

Posted on Sat 22 Jul 2017 by Andrew Jacopino

In the previous article (Designing Subjective Performance Measures (Part 1)) we defined the typical performance levels used in subjective performance measures. In this article we will look at how to set these performance levels by defining the scoring process.

Defining the Scoring Process for a Subjective Performance Measure

Before starting it is important to use the general principles and approaches for setting the performance levels that have been previously provided (see Setting the Performance level (Part 1)). These apply equally for both objective and subjective performance measures.

However, in our experience there are two key aspects for describing subjective performance as follows:

  1. attribute(s) that describes the performance; and
  2. how often the attribute is observed (e.g. the frequency of occurrence).

For ease of use, we further highlight the relationship between these two aspects using a table with an example provided below in Table 1.

Subjective Performance Measure Attributes

The performance attributes for subjective performance measures are simply those characteristics that make up or describe the overall measure. For example, while many PBC practitioners want to measure single humanistic characteristics such as relationship and trust, the question is how to measure these in a repeatable way that assists both buyer and seller in the scoring process?

In this case, the performance measure attributes give a method where, used together with frequency (see below), we can define the attributes that make the overall qualitative characteristics. For example, in “The Trusted Advisor” by David Maister, Charles Green, Robert Galford, they define trust as a relationship between Credibility, Reliability, Intimacy and Self-orientation which each of these attributes having further definitions.

We have found that by using introductory words “The contractor . . .” together with a word picture for the attribute helps both buyer and seller understand the requirement. For example, “The contractor . . . Always . . . resolves disputes as the lowest possible level.”

Subjective Performance Measure Attribute Frequency

Linked to the performance measure attributes is how often the attribute is observed; that is the frequency of occurrence. While it is possible to use a quantitative approach to frequency (e.g. 80% of the time) we have found that by using a number it creates a pseudo quantitative performance measure requiring a recording and reporting system to collect quantitative performance data. This is not the intent of a qualitative performance measure. Accordingly, we have established the following qualitative description of attribute frequency:

  • Always – every time, without exception;
  • Often – generally, most times;
  • Sometimes – occasionally, now and again; and
  • Not Always – whenever an exception has been observed.

It is essential that in defining the qualitative performance levels we don’t inadvertently make them quantitative. For example, if the qualitative performance measure is Responsiveness, it is critical that Good is not specified as “80% of enquiries answered within 48 hours”. This is a quantitative performance measure. Instead, by using the qualitative frequency descriptors above this can be changed to “Good = the contractor OFTEN answers enquiries within 48 hours”.

While the difference may seem small, the intent of a qualitative performance measure is to make sure both buyer and seller have a performance discussion as opposed to accurately and definitively measuring performance. Accordingly, qualitative performance measures should not be related to financial rewards and remedies, but rather non-financial rewards and remedies. Given this, subjective performance measures are typically non-payment performance measures; that is either Strategic Performance Measure (SPMs) or System Health Indicators (SHIs).

Table 1 below provides an example of this approach to defining both attributes and frequency to define the scoring process and performance levels of a subjective performance measure.

Table 1 : Example of Scoring Attributes of a Qualitative Performance Measure

Aggregating Attributes

Finally, in setting our qualitative performance levels, where the qualitative performance measure has more than one attribute (as shown in Table 1 above), we need to decide how to aggregate the attributes to produce an overall performance score.

The main method used is deciding how many attributes at what performance score would lead to an overall superior, good, fair or poor, and how this changes as the number of attributes change.

For example, assuming superior in included in the scoring method, we would usually set this as “at least X of the Y performance attributes ranked as “Superior” and no single performance attribute ranked lower than Good.” In this case, by including both frequency requirements it ensures a high average performance against the attributes and no single ‘poor’ performers. Additionally, we may also set poor performance as two or more attributes assessed as poor. Table 2 below provides an example of this approach to scoring noting there were four performance attributes for this example.

Table 2 : Example of Aggregating Attributes of a
Qualitative Performance Measure

Summary

Qualitative performance measures can add significant value to the performance exchange between buyer and seller if the performance measures are designed properly using clear descriptions for the attributes and frequency. Given this, subjective performance measures are typically non-payment performance measures; that is either SPMs or SHIs.

Posted in Performance Measure, the How | Tagged , , , , , , | 2 Comments