Supply Support Performance Measures – Part 1

“When we talk about ‘smart transportation,’ it is more than moving cargo from A to B. Digitization within transport and logistics means seamless service to our customers, visibility in the supply chain, and driving a more efficient business.”

Soren Skou

In this final article on complex materiel performance measures for Performance Based Contracts (PBCs) I am going to look specifically at Supply Support performance measures.  While many of you reading this may ask why cover this since these performance measures are relatively simple and well-known, such as Delivered In Full On Time (DIFOT), there are a number tricks and traps to catch those new to this field which I’d like to cover.

Firstly, we need to define Supply Support performance measures since many people include performance measures that capture other aspects such a maintenance and staffing.  However, I have defined these more generally as Supportability performance measures (see Supportability Performance Measures).  My preference when referring to Supply Support performance measures is the Australian Standard in Defence Contracts (ASDEFCON) definition:

Supply means the comprehensive function of providing Products and Services needed by users at the time and place required, and includes identification, requirement determination, procurement, receipt, inspection, storage, distribution, stock recording and accounting, reclamation and disposal.

Using this definition Supply Support performance measures look at the timeliness, completeness and accuracy of orders (demands) for items, whether they be repairable items (e.g. a vehicle engine or large office printer) or consumable items (e.g. vehicle tyres or printer paper).  However, given the comprehensive scope of work in this definition it is unlikely that all sellers will be involved to the same degree.  For example, some sellers will simply deliver an item on demand based on agreed lead-times.  However, other sellers may be in deeply embedded in the buyer’s business undertaking inventory management using sophisticated spares modelling to find the right balance of quantity and location of items vs. cost of the inventory vs. downtime waiting for an item.  So what are Supply Support performance measures?

While there are many Supply Support performance measures available to the PBC practitioner these can be grouped into 3 main types:

  1. demand satisfaction performance measures;
  2. inventory holding performance measures; or
  3. customer wait time performance measures.

Given each of these groups of performance measures have different focus and application, in the next article we will explore each in turn including when to use (and not to use) and some common performance measures for each.

Posted in Performance Measure, the How | Tagged , , , , , , , , , | Leave a comment

Supportability Performance Measures

“If you don’t understand how to run an efficient operation, new machinery will just give you new problems of operation and maintenance. The sure way to increase productivity is to better administrate man and machine.”

Edwards Deming

In earlier articles I described how to consider availability (see Availability in PBCs – Part 1 and Part 2) and reliability (see Reliability in PBCs – Part 1 and Part 2) in Performance Based Contracts (PBCs). In this companion article, we are now going to look at supportability and how we describe it within our PBCs.

Similar to availability and reliability, considering supportability of a system or service is one of the highest priorities for both buyer and seller and therefore typically a Key Result Area (KRA) in our PBC performance measure hierarchy.  But what is supportability and how do we describe it within our PBC?

One possible definition is the United Kingdom Defence Standard 00-44 that defines supportability as “A measure of the degree to which all resources required to operate and maintain the system/equipment can be provided in sufficient quantity and time.”  Importantly, this definition highlights a difference focus between availability / reliability and supportability; that of resources and timing.

Availability and reliability performance measures are about the ‘current mission’; the now.  Firstly, is the piece of equipment ready to use now and secondly, is the equipment reliable when I start using it so it won’t break before I can complete the ‘current mission’.

Different to this, supportability is about the ‘next mission’; the future.  So after using equipment to deliver mission success (the outcome), supportability is focusing on whether the buyer can reuse the equipment again for the next mission, the mission after that, and the mission after that, etc.  This reuse could be the result of maintenance (e.g. repairing broken elements such broken parts), replenishment (e.g. refilling used elements such as fuel and oil) or replacement (e.g. getting a completely new or refurbished piece of equipment).

Based on this definition many supportability performance measures reflect non-operational areas such as timeliness, quantity and resourcing of engineering, maintenance and supply (logistics) support.  Focusing on these elements gives the buyer confidence that the seller can deliver long-term mission success (i.e. availability and reliability).

Some examples of supportability performance measures include:

  • Engineering Support performance measures such as Engineering Backlog and Outstanding Corrective Action Reports that measures the engineering support systems ability to complete engineering actions in a timely and quality manner including closing out audit reports;
  • Maintenance Support performance measures such as Mean Time To Repair (MTTR), Mean Maintenance Time and Maintenance Mix that measures the repair times and percentage mix between preventative (scheduled), corrective (unscheduled) and emergent (appears when undertaking preventative or corrective maintenance) maintenance;
  • Supply Support performance measures such as Demand Satisfaction Rate (DSR) and Delivered In Full On Time (DIFOT) that measures the Supply Support system’s ability to deliver equipment and parts as requested;
  • General Support performance measures including areas such as training course satisfaction, number and length of staff vacancies, turn-over rate of staff including key staff, and competency / training of staff that measures the seller’s ability to deliver training outcomes and attract and keep quality staff, which is critical for specialised fields and / or remote localities.

In summary, while supportability performance measures in isolation give very little insight into the seller’s ability to deliver the buyer’s outcome, like availability and reliability, including supportability performance measures in PBC performance measure hierarchies give value insight and therefore confidence into the seller’s ability to consistently deliver outcomes over the long-term.

Posted in Definition, Outcome, Performance Measure, the How | Tagged , , , , , , , , , | 1 Comment

Revisiting Performance Based Contracts vs. Outcome Based Contracts vs. Relational Contracts

When a man’s suit fits, when the construction is beautiful, when the sewing and fabrics are there… in the end, you’ll look the best in it.

John Varvatos, American Designer

In a previous article (see Performance Based Contracting vs Outcomes Based Contract vs Relational Contract) I said that from my perspective there is no difference between Performance Based Contracts and Outcomes Based Contracts.  Moreover, in the same article I said for “Generation 3 PBCs”, that these also included relational contracting features.

Since this article there has is continuing emphasis by some organisations and advisors towards “Outcome Based Contracts”, including saying these were different to PBCs.  As such, I wanted to revisit this in light of the continued discussion on the similarities and differences.

Firstly, we need to understand what we mean by outcome.  Typically, supporters of Outcomes Based Contracts say it is about simply having buyers define the desired outcome and letting sellers tender their innovative solutions independent of the buyer telling them how.  Using this definition, an outcomes based contract can have PBC elements (i.e. a well-considered and defined risk-reward structure that drives the seller to deliver the buyer’s outcome) in addition to relational elements (i.e. a relational / behavioural charter, no fault dispute resolution clauses, etc.).  Alternatively, an outcomes based contract could simply be a conventional contract based on well-known and standard commercial principles using deliverables and milestones for payment.

In some of the commercial arrangement I have dealt with we have very deliberately separated the buyer’s outcome from the contract outcome.  In this case, the buyer’s intended outcome, say putting a ship to sea, may include aspects outside of the direct control or even influence of the seller including the buyer’s own processes and staff, and third-party sellers.  We call this level the ‘enterprise’ to distinguish it from the contract level.  In this case so while both buyer and seller are clear about the enterprise outcome, which may include collaborative behaviours, it is not feasible to ask sellers to offer a contract solution, innovative or otherwise, for how to do this that doesn’t involve a significant exclusions / business rules or financial risk premiums making any solution unaffordable.

That said, we have developed many PBCs that, using different tiers of performance measures that link to different risk-reward structures (consequences).  You can find out more about how to do this from the following article (see When is a KPI not a KPI?).

So as with many things in life, it is not simply one or the other.  Outcomes based contracting focuses on describing the buyers need and how to solve it; in this case by not constraining the solution.  However, an outcomes based contract can include both performance and relational elements.  Alternatively, PBCs can focus on the delivery of outcomes, including enterprise outcomes, which include collaborative behaviours.

So in the future, when you are considering what type of commercial arrangement you want to use, whether outcomes based, performance based or relational, I urge you to think about what features you need as opposed to the label put on it.  Since like a well-tailored suit, it is skill of the tailor and quality of the material as opposed to the label that makes it a perfect fit.

Posted in Behaviours, Definition, Outcome, the How | Tagged , , , | 1 Comment

Choosing a Good KPI? (Part 1)

Not everything that can be counted counts, and not everything that counts can be counted.

Albert Einstein

One question we often get is what is a good performance measure? While you could simply Google “performance measure” (or more typically “KPI”) and this will give you a surprising number of responses.  But a word of warning before you click those links.

While performance measures are central to a Performance Based Contract (PBC), and the associated Performance Management Framework (PMF), instead, we would ask you to think about the intended purpose of the performance measure. What do you want it to do?

So before we choose our performance measures we need to look at the role of performance measures in a PBC? Kenneth G. Charron in his 2006 article on “Why KPIs Belong in Supply Chain Contracts” suggests that:

  1. Performance Measures establish set up the buyer’s expectations in terms of performance
  2. Performance Measures can provide an incentive for the seller
  3. Performance Measures can offer an objective way to assess performance
  4. Performance Measures result in more consistent performance
  5. Performance reports will enable the buyer to compare the performance (Benchmark) of one seller to other sellers of the same services—past, present, and future
  6. Performance reports can help identify the cause of on-going service failures
  7. Performance reports can increase buyer communication and improve service over time
  8. Performance Measures can help organisations avoid or reduce the number of disputes and the resulting costs

Alternatively, having run many PBC courses and workshops we ask this same question with answers generally focused on:

  • Reducing contract price
  • Improving contract performance
  • Adjusting contract payment based on seller performance
  • Understanding commercial issues

But is this true?  Do simply having performance measures lead to these outcomes (e.g. improved performance)?  You have probably seen contracts with performance measures in them.  Did the existence of performance measures lead to better outcomes and was it simply having performance measures, or were there other aspects as well?

The intent of performance management is to ensure that requirements are consistently being met in an effective and efficient manner regardless of whether this is related to the performance of an organisation, process, employee, etc.  Performance management includes the process of setting performance expectations, monitoring performance, measuring results, and appraising and rewarding satisfactory performance.  A best practice performance measurement system will, at a minimum, be composed of the following elements:

  • formal, organised structure for performance measurement and reporting;
  • clearly defined roles, responsibilities and accountabilities for performance measurement and reporting;
  • well documented data quality standards and expectations for performance information, including monitoring and quality assurance procedures, which are clearly communicated across an organisation; and
  • assurance arrangements which may, for example, be in the form of approved data dictionaries that include adequate documentation of data sources, collection methods, standards and procedures with clearly spelt our calculation/costing methods, assumptions, etc.

So while a critical element are the performance measures, they are not the only element needed to drive behaviours.  Indeed, our approach to PBC describes 2 distinct parts to a PBC; the requirements and the consequences. As you can see from the above list most people believe that a performance measure delivers both. But do performance measures do both?  For example, if a seller exceeds the buyer’s performance requirement and is eligible for an incentive payment, is it the performance measure that pays the incentive?

In summary, before you simply take that list of ‘KPIs’ from google it is critical you understand what you are using the performance measure for and how it links to the other parts of the PBC.

In the next part of this series, we’ll look at the process for choosing and checking you have the right performance measures.

Posted in the How | Tagged , , , , , | Leave a comment

Case Study – Setting Performance Levels

Early in my career at the start of Performance Based Contracting (PBC) as a Reliability Engineer I was asked to check a services contract that was delivering availability for a particular piece of equipment.

The buyer was concerned that the equipment was not meeting the required performance levels, in this case an availability level, specified in the contract and that the seller had agreed to deliver through competition.

On the other hand the seller, who had agreed to deliver to the performance level in the contract, was delivering the best possible performance.  Having completed an independent review, it was clear that the seller was delivering the best performance in the world (compared to other users) and was better than the design specification. However, it was still lower that the contracted performance level. So what do you do?

Discarding the do nothing option, the first option is for the buyer to enforce the contract by forcing the seller to delivery the performance. Unfortunately, in this case it was physically impossible for the seller to do better than they had done (world best practice) and the only outcome here would be contract termination. Interestingly, if the buyer looked for another seller through an open market Request For Tender (RFT) I questioned whether they could get better performance and price than they were getting.

The second option is for the buyer and seller to agree to ‘reset’ the performance levels in the contract through a formal Contract Change Proposal (CCP) to those being delivered. This allows the buyer to set the current performance level making sure of continued  delivery while assuring the seller of continued tenure by removing the threat of termination for the delivery of world-class performance; a win-win for both buyer and seller. Fortunately, the buyer and seller settled on option 2.

The lesson for this example is that in setting performance levels (see Setting the Performance Levels (Part 1, Part 2 and Part 3) for further information on how to sett performance levels in a PBC):

  1. The buyer – must make sure that the required performance level is possible for a seller to deliver to – there is no point contracting for something that can’t be done. This will simply lead to failed outcomes and a broken relationship.
  2. The seller – must make sure that they do not sign up to a required performance level that cannot be delivered. This may need the seller to either not bid for the contract or offer an alternative performance level with a rationale for why. Some readers may think that perhaps the contract was too good to lose. However, in my experience, some contracts are too bad to win and potentially threaten both the financial outcome and reputation of the seller.

Simply put, while I am not blind to either buyer or seller trying to maximise their commercial outcome, there is no point starting a commercial relationship that cannot succeed.

Posted in Case Study | Tagged , , , , | 2 Comments

Starting a Performance Based Contract (PBC) – Part 2

In the previous article (see Starting a PBC – Part 1) I looked at the need and benefits of using a transition period when starting a Performance Based Contract (PBC). In this article I will look at key considerations to using a transition period when starting a PBC including examining the effect of the transition period on the PBCs Risk-Reward balance.

During a transition period the performance measures can either be implemented (i.e. turned on):

  • all at once; or
  • phased in over differing times either individually or as groups (e.g. start all supply support performance measures once all the spares part have been bought and delivered).

Additionally, more complex PBCs can also include:

  • specific ‘transitional’ performance measures that only apply during the transition period, or
  • setting performance levels at a lower performance level during the transition period recognising the need to balance the buyer’s need to hold the seller accountable for some level of performance (even if lower than the final performance level) and the seller’s need to minimise commercial risk during the implementation.

As part of designing the transition period it is also important to balance risk and reward. If the seller is ‘protected’ against non-performance (i.e. paid 100% of the At-Risk Amount regardless of performance) the commercial risk is different and the buyer may want to vary the commercial rewards during the same period. One option many PBCs now use is to ‘shape’ the profit margin during the transition period to reflect the change in commercial risk. This approach is illustrated in Figure 1. Please note that the values used in Figure 1 are for illustrative purposely only and are not recommendations or represent the values in real PBCs.

Figure 1 – PBC Transition Period and Shaped Profit

Important factors to consider when using a shaped profit margin approach:

  • While profit shaping commences with the start of the contract the seller can choose an early exit from the transition period. For example, instead of having a 6 month transition period the seller could choose to only have a 3 month, or indeed no, transition period and access the full profit margin from day 1.
  • The variation in profit (fee) must be enough to motivate seller’s acceptance of the additional risk (if any). That is, if the difference is too small or the original base profit margin is too high, there may be insufficient reward for the seller to take on the additional commercial risk and exit the transition period.
  • Ensure that variation in shaped profit is linked to other Risk-Reward structures within the contract (e.g. contract extension, stop payment, termination, etc.).
  • Consider whether the profit (fee) varies with individual performance measures, groups of performance measures or all performance measures.

In summary, the use of a transition period when starting a PBC is a critical step in establishing the right recording, scoring, reporting and reviewing cycles. However, applying a transition period is not a simple as having 12 months without consequences. Instead, it requires both buyer and seller to understand what outcomes they are seeking in the use of a transition period and to tailor it to specifically meet this need.

Posted in Contract Management, the How, Transition Period | Tagged , , , , | Leave a comment

Starting a Performance Based Contract (PBC) – Part 1

One routine question when starting a Performance Based Contract (PBC) is whether to implement all aspects of a PBC on day 1 of the contract. Some people, especially buyers, advocate starting all aspects of the PBC immediately, especially the consequences such as changes to performance payment. Others feel that starting the PBC with a small ‘phase in’ period is preferable. So who is right?

With a couple of exceptions I believe there are benefits from starting a PBC with a small transition period. Benefits include:

  • getting buyer and seller staff used to recording, scoring, reporting and reviewing performance (e.g. my observation has been that using this process for the first time will uncover issues no matter how clear everyone (buyer and seller alike) believes the contract is); and
  • avoiding initial and unrepresentative performance due to:
    • human error; and / or
    • small sample sizes / small durations.

The only exception to the need for a transition period is where the seller has already been delivering the same product or service and there is only minor change in scope or performance levels. In this situation, there is no need for the seller to have a transition period. Indeed, this may be part of the buyer’s justification for remaining with the seller as opposed to re-competing the contract.

Importantly, a transition period should not be viewed as a research phase for:

  • selection of new, either extra or different, or deleting current performance measures regardless of whether Strategic Performance Measures (SPMs), Key Performance Indicators (KPIs) or System Health Indicators (SHIs);
  • changing the weighting of the current performance measures; or
  • defining or changing the contracted performance levels for any of the performance measures regardless of whether SPMs, KPIs or SHIs.

However, where the product or service is so new there is no established performance baseline, the buyer may include a specific transition period to determine this performance level under actual contracted conditions (i.e. in the hands of the user in the field). This is very rare and the transition period would not extend past 12 months in duration. That said, to protect both parties and to prioritise the relationship at the start of the contract, I would always recommend including a minimum of 1 reporting period transition period regardless.

Typically the transition period will last for the first 1 or 2 reporting periods. For example, if performance is reported by the seller and reviewed by the buyer on a 3 monthly basis (e.g. quarterly), it would be common to include a transition period for the first 2 reporting period (i.e. 6 months). During the transition period the buyer and seller record and report performance as they would normally in the contract, however, there are no or “limited” consequences for both positive (e.g. incentives) and negative (e.g. abatements) variation in performance. This is illustrated in Figure 1.  Please note that the values used in Figure 1 are for illustrative purposely only and are not recommendations or represent the values in real PBCs.

PBC Transition

Figure 1 – Effect of a PBC Transition Period on Payment

The reason I have included “limited” consequences in the above description is that most PBCs will keep limited commercial consequences such as the right to terminate for other circumstances during the transition period . Indeed, it would be rare that during a transition period that a PBC did away with all commercial consequences for either buyer or seller.

In the next article, I will look at key considerations when using a transition period when starting a PBC including examining the effect of the transition period on the PBCs Risk-Reward balance.

Posted in Contract Management, the How, Transition Period | Tagged , , , , | 2 Comments