When to use a Performance Based Contract?

Posted on Sun 19 Apr 2015 by Andrew Jacopino

While advice has been written (see the United States Department of Defence Product Support Business Case Analysis (BCA) Guidebook) on when to use a Performance Based Contract (PBC) (or Performance Based Logistics (PBL) contract) my colleagues and I have recently discussed a method for simplifying this process. The process we are leaning towards utilises 3 sequential questions as follows:

Step 1 – Should We? (Assess Net Benefits of Approach)

Step 1 simply poses the threshold question of whether a PBC approach will deliver net benefits to the buyer; that is, the benefits will outweigh the costs.

Areas that should be considered in this step are:

  • Extent of seller’s freedom to deliver the contract outcome including the application of novel or innovative solutions
  • Known accountability of the buyer, seller and any third parties in delivery of the overall outcome
  • Impact of performance on seller’s overall outcome
  • Cost of collecting, cleaning (including adjudication based on specific business rules), calculating, analysing, reporting and storing performance data
  • Access to accurate and meaningful data to measure performance
  • Future variation in performance through maturity of the item being supported

Step 2 – Could We? (Assess Key Enablers)

If step 1 identifies the net benefits that could be realised, we then need to decide whether we have, or could put it place, all the enablers to support the successful application of a PBC including cultural aspects.

Areas that should be considered in this step are:

  • Stakeholder support of a PBC approach including executive, users, contract managers, procurement and contracting, legal, etc
  • A governance structure that supports the regular reporting and discussion of performance management
  • A contracting architecture that has PBC built-in, or has the ability to be readily tailored
  • Potential suppliers are accepting of a PBC approach as a part of the Approach To Market

Step 3 – How (Select Approach)

If Step 2 confirms the enablers are in place, or strategies are agreed to develop/source them, we then need to decide which PBC approach to use.

For example in the past 18 months my colleagues and I have focused on refining ‘lite’ variations of our standard approach to a PBC. These ‘lite’ variations keep the intent and features of a PBC while reducing the management overhead for smaller support contracts. But more on this in a future post.

In summary, I believe that by having the discipline to ask 3 simple questions (Should I? Could I? and How?) as part of the development process for a PBC will result in a more highly successful PBCs. Something that I believe all of us want.

Posted in the Why | Tagged , , , , | Leave a comment

When is KPI not a KPI?

Posted on Wed 25 Mar 2015 by Andrew Jacopino

A Key Performance indicator is a quantifiable measure used to evaluate the success of an organization, employee, etc. in meeting objectives for performance.”

Oxford Dictionary

In the previous post I compared objective and subjective performance measures. However, we didn’t discuss what a performance measure is. So whether you call it a metric, a Key Performance Indicator (KPI), a performance measure or something else, all these terms refer to something that measures performance.

While I think this is a pretty good definition I want to expand our definitions to also reflect the consequences (the monetary or non-monetary rewards and sanctions) that the KPIs are linked to as I believe this approach provides clarity around the use and intent of the KPIs. So what does this framework look like?

Using the framework that my colleagues and I have created, firstly, rather than all measures being labelled KPIs, we refer to a more generic term, performance measure. Secondly, we introduced the notion of Key Result Areas (KRAs) to represent themes of performance measures such as Safety, Cost, Behaviours, Availability, etc. which may have 1 or more performance measures within that theme. Finally, we created 3 tiers of performance measures using labels that reflect their role (requirement) and consequence (reward or sanction). They are as follows:

  1. Strategic Performance Measure (SPM). A SPM is a recent term (24 months) used to reflect a high level focus (i.e. at the strategic level) assessing performance against areas such as safety, cost and behaviours. Additionally, SPMs can be used to reflect “enterprise” outcomes; that is, the highest level outcome (e.g. an aircraft completing a mission) acknowledging that the contractor is not solely responsible for failure or success of this outcome. However, by explicitly including “enterprise” outcomes in the performance framework highlights the buyers desired overall outcome. Given the indirect nature of SPMs, they are typically not linked to payment, but rather to other non-monetary rewards and sanctions such as contract extension.
  1. Key Performance Indicator (KPI). A KPI in our context is a performance measure that is directly related to contractor performance against the scope of work. Importantly, in our framework a KPI is linked to payment and as such, is a quantitative, lag performance measure. But more on this notion of lead and lag in a later post.
  1. System Health Indicators (SHIs). SHIs are performance measures that give us insight into past and future contractor performance by highlighting drivers and constraints to this performance. Typically, the role of an SHI is to provide confidence in the delivery of the KPIs. Again, given the nature of SHIs, they are typically not linked to payment, but rather to other non-monetary rewards and sanctions such as contract extension.

The relationship between SPMs, KPIs and SHIs is provided in the diagram below.

 

Performance Measure Tiers

Figure 1 – Tiers of Performance Measures

 To assist in understanding the framework we developed a diagram called the performance measure hierarchy, which show the relationship between all these performance measures and the KRAs. As you can see in the diagram below, the vertical direction (up/down) shows the KRAs highlighting which performance measures are being used to measure the theme (say safety), while also highlighting the various consequences they are linked. Alternatively, the horizontal direction (left/right) highlights the 3 tiers of performance measures (i.e. SPM, KPI or SHI) allowing a better understanding of the risk (e.g. what performance measures are linked to monetary sanctions).

Performance Measure Hierarchy
 Figure 2 –Performance Measures Hierarchy

 So while this framework may seem a little contrived, we would encourage it’s use to aid both the buyer and seller in understanding the intent of the performance measure since sometimes a KPI is not really a KPI.

Posted in Performance Measure, the How | Tagged , , , , , , , , , , , , , , , | 13 Comments

Objective vs Subjective Performance Measures

Posted on Wed 18 Mar 2015 by Andrew Jacopino

No sane person should believe that something is subjective merely because it cannot be settled beyond controversy .
Hilary Putnam, American Philosopher

In choosing ideal performance measures to be used in Performance Based Contracts (PBCs) there is much debate on whether they all need to be objective, or whether some subjective performance measures can be used?

For clarity, the Oxford dictionary defines objective as “Based on or influenced by personal feelings, tastes, or opinions” and subjective as “(Of a person or their judgement) not influenced by personal feelings or opinions in considering and representing facts”.

Objective measures are usually those related to physical activities or deliverables. For example, how many parts were received by the required delivery date or whether the maintenance was completed by the scheduled completion date. One reason for the desire for all performance measures to be objective is when linked to contractor payment, there is a clear and repeatable process between contractor performance and payment, regardless of who is undertaking the assessment.

While these type of performance measures are easily understood and therefore accurately measured (e.g. the part either did or did not arrive on Monday) sometimes performance measures include business rules to account for factors outside the contractor’s control. For example, what if the contractor tried to deliver the part on Monday but there was no customer to accept the part? What if the inventory management system was down at the time the part was delivered? So the question becomes how to account for these factors in the performance measure, and importantly, who decides whether these adjustment events apply and does the application of business rules make it subjective?

In addition to the business rules, many agree there are a range contractor attributes that should be discussed on a regular basis, and potentially taken into account when considering performance incentives and/or contract extensions. For example, what is the health of the relationship between the buyer, seller and other third parties? What about the safety culture of the organisation? What about the customer satisfaction?

So how do we measure things that are inherently subjective and how do we use these to drive contractor performance?  And how do we make a subjective performance measure repeatable?  One important factor in their use is that they are not typically linked to monetary rewards and sanctions, but rather to non-monetary rewards and sanctions such as contract extensions.  By doing this, the subjective performance measures acts as a point of conversation in the performance exchange between the buyer and seller.  But to do this, the design of an effective performance measure must define the attribute(s) for each performance level (e.g. say for good, fair and poor performance) which guides and informs the performance assessment.  But more on this in the next post.

So in summary, highly successful PBCs use both objective and subjective performance measures given the benefits to both buyer and seller. But the key to their effectiveness is in the detail of their design and what monetary or non-monetary rewards and/or sanctions (consequences) they are linked to.

Posted in Definition, Performance Measure | Tagged , , , , , , , | 5 Comments

Nudging Performance

Posted on Tue 24 Feb 2015 by Andrew Jacopino

For those that found my previous post that compared PBCs with the notion of nudging interesting I would highly recommend you read this insightful question and answer (Q&A) session with Richard Thaler, co-author of the book Nudge: Improving Decisions About Health, Wealth, and Happiness.  Specifically, this Q&A specifically discusses what makes up a good nudge.

So let me know your thoughts on whether you think PBCs nudge performance.

Posted in Behaviours, Nudging | Tagged , , , , , , , , | Leave a comment

Is Your Contract Fun?

Posted on Sat 21 Feb 2015 by Andrew Jacopino

Katherine Milkman, an associate professor at Wharton, University of Pennsylvania, starts her behavioural economics class with the following video.

.

Having watched this you may be wondering what this has to do with PBC? Let me explain.

If you agree with my definition of a PBC you will notice that fundamentally a PBC is simply a behavioural economics situation where we try to nudge contractor behaviour to deliver the buyer’s outcome through applying range of complementary rewards and sanctions.

Those familiar with Thaler and Sunstein’s concept of nudging may disagree with my analogy since it is typically used in a social engineering context (e.g. influencing the uptake of health insurance) and typically applied with a lighter touch. However, I would argue that the intent is same.  By using a range of complementary rewards and sanctions within a PBC creates a Choice Architecture that guides and disables contractor choice rather than direct legal consequences of a conventional contract such as contractual breach.

I accept that this type of nudge is more direct than Thaler and Sunstein probably envisioned, perhaps we can call it a “heavy nudge”, but again I think the intent is same.  Moreover, I believe those of us involved in developing PBCs should strive to deliver a Performance Management Framework that allows a contractor to self-regulate their behaviour by defining the consequences of their actions or omissions.

So while some may not consider your PBC fun in the traditional sense, I believe that highly successful PBCs should skilfully use a contract’s Choice Architecture to nudge the delivery of the required outcomes.

Posted in Behaviours, Definition, the Why | Tagged , , , , , , , , , , | 2 Comments

Defining a Performance Based Contract (PBC)

Posted on Mon 09 Feb 2015 by Andrew Jacopino

Over the past 12 months articles[1] have discussed the virtues of Performance Based Contracting (PBC), but may have led the reader to believe that PBCs are neither well understood nor used. This is not entirely true since forms of PBCs have been extensively used by both the Defence and civil sectors since the late 1990’s in a range of acquisition and support contracts. So why the confusion?

I believe that one of the reasons may be the various definitions and labels given to this type of contract. For example, in the US and Canadian defence sector they are called Performance Based Logistics (PBL) contracts. In the Australian defence sector we call them PBCs while the UK defence sector use both these terms but also use Contracting for Availability and Contracting for Capability. Alternatively, large elements of the civil sector use another term, Vested Outsourcing[2]. So what is a PBC?

While there are a variety of PBC definitions[3] available in looking at all these I believe they can all be combined as follows:

Performance Based Contracting is an outcomes-oriented contracting method that ties a range of monetary and non-monetary consequences to the contractor based on their accomplishment of measurable and achievable performance requirements.

In looking at this definition some may argue there is no difference to conventional contracts that apply  rewards or sanctions to the contractor based on their performance. While this may be true from an overview, the contract types differ in their application. To highlight these differences I offer the following five key characteristics of a PBC:

  1. Requirements focused on the contractual outcomes, and not how the work is performed
  2. Set of indicators tied to the outcome
  3. Achievable performance standard for each indicator
  4. Defined process to collect, analyse and report data for the selected indicator
  5. Range of monetary and non-monetary consequences, either rewards or sanctions for the contractor, based on performance

If we can settle on a definition and key characteristics for a PBC that are industry agnostic and without branding or proprietary marking, I believe this will go a long way on the road to making PBCs more accessible and hopefully better understood and used.

 

[1]                 http://www.corrs.com.au/publications/corrs-in-brief/outcome-based-contracting-is-on-the-up-who-s-doing-it-why-and-what-you-need-to-know-about-it/

[2]                 http://www.vestedway.com

[3]                 http://www.defence.gov.au/dmo/Multimedia/Next_GPBSC-9-5978.pdf, http://www.defence.gov.au/dmo/Multimedia/asd_pbc_v2-9-5979.pdf, http://principlesandpractices.org/wp-content/uploads/2013/04/PerformanceBased.pdf, http://www.acquisition.gov/far/01-07/html/Subpart_37_6.html, http://www.who.int/management/resources/finances/Section2-3.pdf, http://www.whitehouse.gov/omb/procurement_guide_pbsc, http://www.des.wa.gov/services/ContractingPurchasing/PoliciesTraining/Resources/Pages/performancedBasedContracting.aspx, https://www.uspsoig.gov/blog/maximizing-performance-based-contracting, http://www.acquisition.gov/sevensteps/library/DOEpb-contracting.pdf

Posted in Definition, the What, Uncategorized | Tagged , , , , , | 5 Comments

Designing Successful Performance Based Contracts

Posted on Sun 25 Jan 2015 by Andrew Jacopino

Successful contracts deliver the desired outcomes of both the buyers and seller through a form of commercial construct. This is the classic “win-win” outcome that we all strive for.

In Performance Based Contracting (PBC) our goal is the same; delivering this “win-win” outcome by examining whether the arrangement:

  1. drives the right behaviour in the seller by addressing the Seller’s Needs;
  2. provides adequate commercial protections for the buyer by addressing the Buyer’s Needs; and
  3. balances the needs within a usable Commercial Construct.

This is achieved by aligning the Seller’s Needs against delivery of the Buyer Needs as reflected in the Commercial Construct. The included diagram highlights the interaction between these three areas.

Relationship between Sellers Needs, Buyers Needs and the Commercial Construct

Relationship between Sellers Needs, Buyers Needs and the Commercial Construct

Highly successful PBCs deliver this by balancing the requirements of each party with the consequences (rewards and remedies) that provides both positive (carrot) and negative (stick) incentives.

Buyer Needs

The Buyer Needs represents the traditional specification of requirements. In sustaining ment of complex materiel[1] we group these requirements into three broad areas of Asset Usage, Asset Optimisation and Asset Preservation underpinned by Safety Culture, Cost Conscious and Positive Behaviours.

Seller Needs

The Seller Needs represents both the financial and non-financial outcomes required by the seller. These outcomes range from contract price and profit margin to contract duration and recognition schemes.

Commercial Construct

The Commercial Construct represents the agreement between the parties that fairly motivates and delivers both Buyer Needs and Seller Needs. The optimal Commercial Construct is a balance of conventional (written) contracting protections and a relational contracting approach.

Observations of the Interactions

Examining the relationship between these three areas it is possible to observe the following characteristics:

  • moving horizontally (left / right) changes the nature of the agreement; the left hand edge is a conventional Commercial Construct transferring risk from buyer to seller through a prescriptive, written contract whereas the right hand edge is a highly relational approach that shares risk with limited formal requirements; and
  • moving vertically (up / down) changes the outcome for both the buyer and seller; the top edge meets the Buyer Needs but may be seen by the seller as punitive and a “bad deal” whereas the bottom edge meets the Seller Needs but may be seen by the buyer as risk free providing no assurance of performance.

Acknowledging these extreme edges, the ideal PBC is one that balances each of these three areas represented by the overlapping region in the centre of the diagram where the Buyers Needs are delivered by linking these to the Sellers Needs through a fair Commercial Construct.

Designing Balance

So how do we achieve this balance?

The conventional approach to a Commercial Construct is through a series of commercial remedies such as the application of damages (e.g. Liquidated Damages), termination, etc,

An alternate option is to design our Commercial Construct with a range of consequences that scale (escalate) based on significance of performance variation from our requirements, both in terms of duration and impact of the variation on the buyer. This approach can be seen in Table 1[2].

Commitment
(Disable Choice)

Incentive
(Guide Choice)
Carrot
  • Contract Duration and Extension
  • Incentives
  • Recognition schemes
Stick
  • Stop Payment
  • Termination
  • At-Risk Amount
  • Liquidated Damages

Table 1 – Typology of Incentives and Commitments

Here commitments represents consequences that are either too good to refuse or too bad to accept. In these cases the consequences, either positive (carrot) or negative (stick), must be of such significance to disable future choice.  On the other hand, incentives represent consequences that guide rather than disable choice. Here the significance of the consequence allows choice.

One approach I have seen used with good results is to offer a default Commercial Construct to contract authors that includes a range of consequences that scale (escalate) based on significance of performance variation from our requirements.

Summary

Our goal should be to ensure that we design and maintain PBCs that drives the right behaviour in the seller, provides adequate commercial protections for the buyer, and balances these needs within a usable Commercial Construct. PBCs that appropriately balance these areas will result in an effective, efficient and affordable solution.

Please feel free to leave a comment on your thoughts on this framework.

[1]        Complex materiel is defined as those assets that support military operations through flying (e.g. aircraft and helicopters), sailing (e.g. ships, boats and submarines), driving (e.g. wheeled and tracked vehicles) and transmitting (e.g. satellite ground stations).

[2]     Adapted from AYRES, I. “Carrots and Sticks”, Bantam, 2010

Posted in Definition, the How | Tagged , , , , | 7 Comments

When the Fair Comes to Town

Posted on Sun 18 Jan 2015 by Andrew Jacopino

One of my favourite quotes when talking about fairness of commercial arrangements comes from a US friend of mine which goes:

“. . . the only fair I have seen is the one that comes to town twice a year.”

imageWhile I do enjoy the humour of this statement, unfortunately, it is an all too common feeling towards Performance Based Contracts (PBCs); that they are riskier than conventional contracts due to their very nature (i.e. performance based). However, the focus should be on whether this risk is real, or whether it perceived due to an emotional response from being performance managed.

In my observation most practitioners focus what consequences can be applied due to a failure to perform (e.g. reduction in profit, no additional contract extensions, etc.) rather than what consequences are likely to be applied. In many cases practitioners over-estimate the risk (noting that risk is defined as a combination of likelihood and consequence) by focusing on the consequences deterministically (that is, that they will occur during the term of the contract) not probabilistically (that is, what is the chance of this event occurring during the term of the contract). This risk averse behaviour can be a significant cost driver and can damage commercial relationships before they have even began.

So what do we do? Firstly, we need to acknowledge that the aim of a highly successful PBC is to pay the seller 100% of contract price since this would mean that the buyer has received 100% of their required outcomes; a win-win for both parties. However, this also means that for most PBCs the contract outcomes need to be achievable by the seller and not a stretch goal or target. That said, there are forms of incentivised contracts that can be used in this case which I will talk about in a future post. Secondly, that a failure by the seller to deliver 100% of the buyers required outcomes has consequences, and that these consequences may escalate as less of the buyers outcomes are delivered. And finally, we need to understand what is the likelihood of the specific consequence occurring to determine the real risk to both buyer and seller. Some of my colleagues and I call this last step consequence analysis and will part of a future post.

So in summary, to avoid “performance anxiety” both the buyer and seller needs to understand the probabilistic risk of non-performance and not simply assume that all consequences are going to occur equally, or even at all.  However, if either the buyer or seller truly believes that all consequences are likely, then I’d suggest it is a bad deal for both parties.

I would be interested in others observations and opinions.

Posted in Consequence Analysis, Risk Management, the How | Tagged , , , , , , | 1 Comment

Measuring and Incentivising Sustainment in the Age of Asset Management

Posted on Mon 12 Jan 2015 by Andrew Jacopino

In the early 1990’s when various western defence organisations started outsourcing the sustainment of complex materiel[1] they focused heavily on measuring and incentivising the four related “ilities“; that is availability, reliability, maintainability and supportability. Through the “ilities” we make sure that materiel was available to the user when they needed it, reliable when they used it, and maintainable and supportable when they eventually broke it, thereby allowing them to repeat the cycle again and again.

International and domestic evidence continues to prove that this approach to measuring and incentivising the sustainment of complex materiel has proven successful[2]. Indeed, this approach, enacted through Performance Based Contracting (PBC) or Performance Based Logistics (PBL), is the mainstay of the Defence approach to sustainment for almost two decades.

As we move toward longer term, strategic relationships in sustainment, however, we need to revisit whether our focus on the four “ilities” still proves the best approach to measurement and incentives. One area that we believe need addressing is asset preservation as this ensures the longevity of our complex materiel not just today or tomorrow, but also next month, next year and next decade. So the question is, how do we preserve our asset over its Life of Type (LOT)?

One possible approach is to utilise the recently released Asset Management methodology (ISO55000) which focuses on performance, cost and risk in both the short and long timeframes. Using this methodology as the foundation my colleagues propose that we consider three areas of measurement and incentives; (1) Asset Usage, (2) Asset Optimisation and (3) Asset Preservation. Before we look at each of these in more depth we need to recognise that this discussion does not intend to address specific performance measures that would be used to represent these areas. Rather, this selection would vary depending on the scope and the domain (e.g. Maintenance Repair and Overhaul (MRO) vs. supply chain or aerospace vs. maritime vs. land) they reflect. So let’s look at each in turn.

  • Asset Usage – the focus of this area is on user effectiveness and is equal to the earlier approach using the “ilities“. Simply put, Asset Usage is focused on whether an asset is available to be used, reliable once used, and able to be fixed and reused, again and again.
  • Asset Optimisation – unlike Asset Usage, the focus of this area is on the efficiency of the sustainment solution. Here activities such as Reliability Centred Maintenance (RCM), Condition Based Maintenance (CBM) and inventory optimisation all seek to minimise the both Contractor and Commonwealth resources consumed in sustaining the asset over both the short and long-term. Of note, there are those[3] that highlight that this constant optimisation through techniques such as Lean may result in highly optimal solutions but which have very little resilience to a single failure in the support system. Accordingly, the buyer need to be very careful that delivering efficiencies does not result in our sustainment solution being so lean that it becomes fragile. Not a desired quality, especially when supporting materiel that may be required at a moments notice.
  • Asset Preservation – the last area, and one that most buyers are especially interested in, is the focus on ensuring longevity of the asset. Indeed, given the trend towards longer-term, more strategic sustainment contracts, this need becomes even more critical.

Underpinning these three areas are the constants of Safety Culture, Cost Consciousness and Positive Behaviours. The reason for these being constant is that an asset that is available and reliable, both today and in the future, does not guarantee safety for the user or maintainer, affordability or a positive, collaborative relationship. This paper will not discuss or cover these common factors, but rather focus on the three areas above. However, we will revisit this topic in greater detail in a future post.

The illustration below shows the relationship between these three areas (i.e. Asset Usage, Asset Optimisation and Asset Preservation) and the constants of Safety Culture, Cost Consciousness and Positive Behaviours.

Relationship between Asset Usage, Asset Optimisation and Asset Preservation

Relationship between Asset Usage, Asset Optimisation and Asset Preservation

As illustrated, the three areas are in tension, and necessarily so. However, by ensuring these areas are adequately covered within the contractual measurement and incentive framework it explicitly defines the trade space that the asset manager (the seller or Contractor), as opposed to the Asset Owner (the buyer), can move around.

For example, there are times when we need to conduct preventative maintenance needs to preserve the asset regardless of whether the user needs it. That is not to mean that the asset cannot be used if there is an overriding, critical need such as Humanitarian Aid / Disaster Relief. However, what it does do is forces the understanding that the deferral of preventative maintenance, while appropriate, may reduce the longevity of the asset which needs to be eventually addressed; that is, it is a conscious choice. It is this conscious choice, or trade, that is at the centre of this approach. While we would never advocate that the sustainment organisation stop an asset being used, especially where the use can save lives, the asset users, asset managers and asset owners need to take a considered approach to both the short and long-term effects of any decision made. So where does this leave us?

If we accept this framework then the next step would be to look at the individual performance measures that we would assign to each of these areas, noting that while some may be constant, others will invariably change based on the scope of work and domain.

Please feel free to leave a comment on your thoughts on this framework.

[1]     Complex materiel is defined as those assets that support military operations through flying (e.g. aircraft and helicopters), sailing (e.g. ships, boats and submarines), driving (e.g. wheeled and tracked vehicles) and transmitting (e.g. satellite ground stations).

[2]        See BOYCE, J. and BANGHART, A., “Performance Based Logistics and Project Point Proof: A study of PBL Effectiveness”, Defense AT&L: Product Support Issue March-April 2012; GUAJARDO, COHEN, NETESSINE, KIM, 2010, ‘Impact of performance-based contracting on Product reliability: An empirical analysis’ INSEAD Working Paper No. 2011/49/TOM.

[3]        See TALEB, N.N., ‘The Forth Quadrant: A Map of the Limits of Statistics’, in “Thinking”, BROCKMAN, J., (Ed)

Posted in Asset Management, Behaviours, Definition, Performance Measure, the How | Tagged , , , , , , | 1 Comment

The Cost of Measuring Performance

Posted on Wed 07 Jan 2015 by Andrew Jacopino

Peter Drucker famously said,

“What gets measured gets managed.”

TIn this light, many practitioners in the field of Performance Based Contracting (PBC) attempt to apply large numbers of performance measures in trying to cover the entirety of the contract scope of work.

While generally this approach works it doesn’t consider 2 factors; (1) the ability of an individual or organisation to actively manage a large number of performance measures and (2) the cost of measurement. This particular post reflects the later cost of measurement.

The reason I bring it up is that I recently read a 2014 working paper by Morten Jerven on behalf of a European think tank, the Copenhagen Consensus. This paper provides an estimate of the cost of measuring the United Nations (UN) Millennium Development Goals (MDGs). The MDGs, while laudable in nature, comprised 8 targets measured using 60 performance indicators over the period of 1990 – 2015. His conservative estimate the cost of measuring success against these MDGs was a staggering USD$28B. The main cost driver was a lack of data to support the success (or failure) against the goal . While significant, the estimated cost of measuring the draft post-2015 goals is USD$254B based on 17 goals and 167 performance indicators. Morten’s discussion is not that we shouldn’t undertake these activities, but rather that we should consider the cost of measuring success (or failure) against each of these when considering how many goals and measures the UN should consider. Those interested in this thought-provoking article can find it at http://www.copenhagenconsensus.com/post-2015-consensus/datafordevelopment.

In our field of PBC I have observed this same problem. With the best of intent people choose performance measures that don’t have existing data sources or processes, or have processes requiring significant (and costly!) manual intervention. While it is perfectly OK to choose these performance measures, I would like people to consider the cost of measuring the “perfect” performance measure. Indeed a friend of mine from the seller side of PBCs has said that they could save between USD$200,000 to USD$300,000 per year per contract if the buyer had chosen a different performance measure. This isn’t the seller trying to avoid being accountable, but rather being cost conscious in delivering an affordable solution.

So in summary, as Morten Jerven points out, in selecting performance measures we need to carefully consider the cost of measurement to make sure that we can affordably manage success or failure of our contracts.

Posted in Performance Measure, the How | Tagged , , , , , , , , | Leave a comment