🕵🏻‍♀️ Deep dive: On benchmarks

Or why you should be a tad careful with those shortcuts

People who have worked with me know how my face turns tomato red when most people start throwing benchmarks around.

I may or may not have been heard saying “these benchmarks are useless” a time or two or three. It is not because there is something wrong with comparing yourself to others (even though some of the best companies I worked for gave zero Effs about benchmarks). It is not because I believe 100% of benchmarks are useless. It is because some lack understanding of what they are comparing themselves to. 🍅

So, I want to make benchmarks great again!

Photo by Noah Buscher on Unsplash

Why bother with benchmarks: the Pros.

1. Setting Goals: When used well, benchmarks can be used to set high standards of achievement for company, products, motions, efficiency etc. They show how high you should aim.

2. Identify Performance Gaps: When used well, benchmarking helps identify performance gaps and areas for improvement. They show there might be a problem.

3. Resource Investment: When used well, benchmarking aids in identifying the most valuable investment choices. They show how much potentially it costs to solve a problem.

4. Market Opportunities and Trends: When used well, competitive benchmarking helps identify market opportunities and anticipate industry trends. They show an average of the big picture (hint: you get to decide what “big” means though).

This is it. This is what benchmarks are really good for. Every other use case is a derivative of one of those 4. Prove me wrong:)

Why you shouldn’t bother: the Cons.

Now here is my beef with people mindlessly throwing around benchmarks only to sound smart. They forget the following:

1. Lack of Context: Benchmark data often lacks context, making it difficult to understand how competitors achieved their results, how long it took them, what they tested previously, what unfair advantage they had etc. What’s worse is, most benchmark data lacks enough depth to allow you to slice and dice and cohort to really understand what’s going on. It is a snapshot at best and a manipulated image at worst.

2. Measurement Issues: Different companies may use varying metrics, complicating like-for-like comparisons and potentially skewing results. One man’s Win Rate will be another woman’s Close Rate. It shouldn’t be, but it happens. I cannot count on my hands any more amount of times I will dig into benchmark data and I will come out 1) knowing they measured things differently or 2) convinced they have not enough data to serve as a benchmark. Which leads me to,

3. Data Reliability: Finding reliable data can be challenging, as companies may not share data, and available data might not be trustworthy. This is especially true of private companies, who self report data. Ask yourself, who is filling out the survey, what are their motivations, and knowing the state of your data, do you believe their state of data is more reliable?

4. Backward-Looking: Benchmarks reflect past performance and do not necessarily indicate future success (or failure), as they do not account for real-time changes or future conditions. They are a lagging indicator. Nuff said.

5. Potential Distraction: Focusing too much on competitors' benchmarks can distract from internal growth and customer-focused strategies. This is why Uber never cared, WeWork never cared, Hopin never cared about the broader competition, or what exactly they are doing every minute of the day, etc etc.

---

Ok. Rant over.

Now, what do we do with this? Let’s make benchmarks great again. Here are my tips for making benchmarks work for you in meaningful ways:

Make Benchmarks Great Again

1. Define Benchmarking Goals: Clearly define what you want to achieve with benchmarking. Hint, it is one of the 4 Pros.

2. Limit Comparisons to only the Relevant: Choose a manageable number of comparables, that fit your situation. i.e. If you are B2C PLG Healthcare company in the US market, look at only those who are the same. Ideally choose the same cohort. Remove the Enterprise ones or the ones in New Zealand.

3. Use Reliable Data Sources: Ensure that data sources are trustworthy. If need be pick up the phone and validate. A validated range is better than an unvalidated single data point guesstimate.

4. Document Internal Processes: Thoroughly document your own processes to ensure apples to apples comparisons with the benchmark data. This should be obvious but again, common sense is not that common.

Remember there is more to life than just benchmarks: Use benchmarks alongside other performance measurement tools, SWOT analysis, PR reviews, GlasDoor reviews, and whatever else you need to paint a full picture and not just focus on a small corner.

---

Comments and questions are welcome, through the usual channel: LinkedIn message.