Swearing By Or Swearing At Benchmarking

I thought that I would post something about a perspective that I have on benchmarking that might be somewhat of a minority position as compared to other management consultants. My thoughts were triggered in the context of an operations project where I am working with an ex-Booz Allen Hamilton person.

Benchmarking client company operations against comparable company operations helps to place the client company in the context of competition, quantify areas of differences, and provide a fact-based foundation upon which management decisions can be made.

I totally agree with this in theory. In reality, one can run into a ton of issues, some of which include:

  • Existing metrics and measurements were collected for a service or product which is not comparable to the service or product of current concern – This is somewhat ill-defined, but as an example, suppose that a operations process related to wireless data service is different from a support process related to broadcast video service.
  • Client or management pokes a hole in the validity of benchmark and undermines the credibility of your case – May be tied to a variation of the first one, but in any case, think about how strongly one wants to hinge the consultative study on a single benchmark source or data point.
  • Benchmark information cannot be readily found – Ok, one can go recommend starting a benchmarking study before taking any action as these need not be cost-prohibitive studies to conduct. Consulting firms can also leverage their knowledge bases and networks to bring light to the table. I just caution taking actions that delay progress when other methods for fact-building can be taken (perhaps in parallel with the benchmarking).
  • Client metrics don’t exist, are questionable, or are not comparable – After jumping through hoops, the client measurement systems may not match the requirements and measurement guidelines used by participants in the benchmarking study …

So given the issues above (and I have to say in my experience, every operations consulting engagement I’ve been in has run into at least one of these issues), I would suggest that the prospects of using other methods for operations analysis not be discounted too much (and these methods do not require benchmarking as a necessary condition):

  • Structural analysis – e.g., whether an operations process runs in parallel or serial and whether the processes can be run in parallel versus serial is something that can be factually analyzed and need not necessarily be compared to other companies.
  • Bottleneck analysis – e.g., when resources are waiting downstream for another resource that is bottlenecked and when this happens with a high statistical certainty, this can indicate that the work structure is not optimal (perhaps flattening of the work structure, cross-training, or better planning is desirable).
  • Trend analysis – e.g., if costs are going up, cycle-times to respond are getting longer, and customers are getting more upset, these are facts that one can respond to without having to have benchmark data to the nth degree up-front. Many companies may do an OK job at getting static data, but longitudinal data over time can also be very informative.
  • Consistency analysis – although not as cut and dry, the argument is along the following lines: if a company choses a particular strategy (e.g., being a feature leader versus low-cost provider), then there are some tactics that are inconsistent or less consistent with the strategy (e.g., being a feature leader and running below average R&D investment rates).

So while I will say that I am all for benchmarks, if one finds oneself swearing at benchmarks, then there may also be an opportunity to step back and apply some other principles that do not require as much benchmark information to be readily available. A tremendous amount of value can also be added using analysis methods like the ones I mentioned above. There are many other methods – please feel free to share some of your favorites.

Bonus link: good website covering balanced scorecard information as well as quality functional deployment (QFD) principles (the latter of which I have seen applied more in product management scenarios).