You Get What You Measure (So Be Careful What You Measure)

Remember the story about the genie who was freed from the magic lamp? “You get three wishes – any three wishes! – but choose carefully!” The greedy soul who rubs the lamp always wastes a wish or two. “I want to be ruler of the universe!” And poof! he’s transformed into a giant yardstick. He gets what he asked for…but it isn’t really want he wanted.

So too it is with measurements and metrics. I’m a big believer in the value of key metrics to drive focus, accountability, and results. When we measure something – like employee engagement, revenue growth, response time, or customer satisfaction – we learn how we’re doing: progress against plan, changes over time, and comparisons to industry best. With clarity around objectives, our teams can align around joint goals and ensure we are collectively prioritizing attention on those things that matter most to achieving desired results.

The good news is that when goals are clear, defining and monitoring key performance indicators (known as KPIs) is a very effective way to drive results. Good KPIs are SMART: specific, measurable, attainable, relevant, and time bound. As humans, we are excellent at adjusting our behavior based on what we’re measured against (sometimes consciously, sometimes unconsciously). And by measuring something on a regular basis, we keep track of how much progress we’ve made against our objectives so that we can take additional actions if necessary to achieve them.

Let’s use a few examples to explore how KPIs can be useful in providing clarity, focus, and accountability. First, from a personal point of view, what are your top priorities? Perhaps you’re committed to getting regular exercise, or you’re trying to lose weight, or you want to practice meditation, or you know you should get more sleep. Plenty of us have said these things – but just saying them doesn’t make them happen, right?

Say you want to turn “get more sleep” into a SMART KPI. First off, consider: what are you really trying to achieve, why does it matter, and how can you influence the outcome? Here perhaps your motivation to increase hours slept is to enhance daytime alertness, concentration, and overall health. A specific KPI could be “number of hours slept per night.” You might set a goal of achieving an average of seven hours of sleep each night within the next three months. Maybe you could use a wearable device to track how much you sleep, collect this data daily, and average over each seven-day period. You might review the data once a week (perhaps every Sunday), and then plan what actions you’ll take as a result. If you’ve already achieved or are on track toward your goal: stay the course. If not, consider how you might alter the outcome: perhaps you should try changing your bedtime routine, relocating electronic devices, getting black-out shades, sleeping in a different room from a snoring spouse, or seeking expert help (a friend or life coach or medical professional) if you’re way off your target and out of ideas. Then you take an action and check in on your KPI again the next Sunday to see if you’re closer to your goal. Celebrate small victories to reinforce behaviors and build momentum.

In business it can work in a similar manner. Top-level goals might be to increase sales revenue, decrease costs, reduce inventory, enhance customer satisfaction, or increase employee retention. Take “increase sales revenue.” It’s a pretty big topic – so let’s get specific. Do we mean overall dollars spent by all customers across the full portfolio? Or dollars per customer? In particular geographies? On certain product lines? What’s the goal: +5% relative to last year or relative to competitors or the market? How do we measure it? How often? What actions affect change? Who is accountable? There is both an art and a science to establishing KPIs and setting clear targets. Then there needs to be rigor in the review of progress so that others can intervene as necessary and provide help, diagnose issues, identify new actions, or pivot as needed.

But…here’s where the cautionary tale begins. Given that metrics and measurements are so powerful and drive behaviors, if we’re not careful in how we set and monitor them, results can be disastrous. Consider the perspectives of these esteemed leaders:

• Management guru Peter Drucker: “What gets measured gets managed – even when it’s pointless to measure and manage it, and even if it harms the purpose of the organization to do so.”
• Accounting historian H. Thomas Johnson: “Perhaps what you measure is what you get. More likely, what you measure is all you’ll get. What you don’t (or can’t) measure is lost.”
• Business authority Eliyahu Moshe Goldratt: “Tell me how you measure me and I will tell you how I will behave. If you measure me in an illogical way…do not complain about illogical behavior.”

The dark side of metrics is when you achieve the target but not the ultimate goal. Sometimes a measurement has unintended consequences like majoring on the minors (focusing on things that are not the most important) or measurement myopia (focusing only on what is measured) or analysis paralysis (an inability to make decisions due to the overwhelming amount of data). It may also drive bad behaviors like people trying to game the system. Let’s use a few examples to make this real.

Teaching: Standardized test scores are often used as a proxy for teaching quality, which has resulted in “teaching to the test” behaviors where teachers focus much of their classroom time on helping students prepare to excel on an end-of-year exam. Unfortunately, there are a variety of unintended consequences of a single-minded emphasis on student test scores. Students and teachers alike focus on a narrow set of educational topics covered on the test, taking time away from broader learning, customized instruction, critical thinking, and creativity. When the test is the goal, the learning can be lost once the test is over. While test scores improve, there is little evidence that this has led to higher educational quality or better educated students.

Hospitals: In the UK during the 1990s, hospitals were measured on and challenged to improve patient waiting times. While the focus and overt management of this metric was successful in reducing the amount of time patients waited to be seen, during the same period the death rate following emergency heart-attack admissions substantially increased. This unfortunate trade-off – shorter waiting times but more deaths – was an unintended outcome produced by choice of measure.

Innovation: Many businesses want to drive innovation – because innovative teams are more likely to produce differentiated products, come up with creative approaches to solve problems, and identify market disruptions – all of which are key enablers of business growth. But how do you measure and spur innovation? If we use “number of patents filed” as a proxy for innovation, the outcome may be unsatisfactory. It may drive bad behavior: everyone feels they should file patents, which drives up patent numbers but doesn’t necessarily create innovative teams. The patents may or may not be relevant, the time spent writing disclosures may or may not align to business priorities, and patents themselves are an incomplete measure of innovation given the importance today of open source, open ecosystems, and open innovation.

So, this is where it becomes critical to focus on over-arching goals and ensure we measure what matters most. Objectives and key results (known as OKRs) are designed to define success and monitor progress toward that success. Objectives are descriptions of bold, ambitious goals. Key results are quantitative measures of progress that can be answered definitively with a yes or no (did I achieve it or not?).

The OKR goal-setting framework is attributed to Andrew Grove at Intel back in the 1970s. More recently, many other businesses have adopted OKRs, including Google, Twitter, and Microsoft. My teams have implemented OKRs at IBM Research and in the IBM CIO organization. Former Intel salesperson and venture capitalist John Doerr published the book Measure What Matters in 2017 with guidance on implementing OKRs to drive business growth.

Back to our “increase sales revenue” goal from above. In OKR language, we might write an objective as: “Increase sales by 5% over last quarter.” And then we’d establish three to five key results that are critical enablers of this overall objective. For instance: “Generate $5 million in sales” + “Achieve 90% customer satisfaction” + “Acquire 40 new customers.” At the end of the quarter, we would know, with clarity, if we achieved each of these key results (yes we acquired 40 new customers or no we did not). The sales revenue, customer satisfaction, and new customer metrics are KPIs that help us monitor progress toward the OKRs and diagnose issues. This is how KPIs and OKRs work together and complement each other. As in this example, it is often a family of metrics that provide the most value, aligned to a clear objective.

Metrics and measurements play an important role in driving outcomes. What we choose to measure truly matters as it influences incentives, behaviors, and results. We must take care that the measurement doesn’t become an end in and of itself. As leaders, contributors, and individuals, we can derive great value from metrics, but we shouldn’t go overboard – as it is also true that there are many things that are important but not easily measured. In a deliberate bungling of the classic Rolling Stones lyric: You’ll often get what you measure, but you might find you don’t get what you want.