So I like measurements and KPIs as much as the next business manager but I am always concerned about being too overly-reliant on metrics, especially along the lines of whether the right things are being measured and optimized.

It’s not even whether or not you are measuring the right things, it’s also understanding that when you change one variable, in pursuit of maximizing some metric, then there are probably impacts to other variables. And if you’re not measuring those other variables as well then you’re impacting your business without even realizing it. For example, I once talked with someone who worked in a call center a while ago and they mentioned that the singular key metric they were measured on was “average call time”. You can imagine how that went. Call center agents pushed to just end the call as fast as possible, sometimes even just hanging up on people. How do you think that impacted the quality of experience for their customers?

There’s also frequently a lack of understanding of these impacts. Let’s say you are driving from Miami to Orlando and your sole measured KPI is duration. The distance is about 230 miles so theoretically the trip would take 4-5 hours, but you, in all your business wisdom, are pushing to optimize that KPI to, let’s say, 3 hours. Which results in an average speed of 78 mph. Is that reasonable, safe, and even achievable?

The reality is you cannot just optimize for a small number of metrics without having a broader understanding of the actual problem space, and thus understanding what downstream impacts will occur when you start to overly-adjust those metrics.