Get Started »
Metrics: The Good, The Bad and The Ugly
I fly a lot. Not that I want to. In fact, my driving radius is constantly increasing because airline service levels are so poor. Just in my last three consecutive flights on two different airlines (names withheld to protect the guilty) I have experienced the following:
An originating flight delayed for an hour, but the connecting flight not delayed so that I could fly to the hub city, but not to my final destination
A two hour delay for unplanned maintenance
A two hour delay for planned maintenance (obviously not very well planned)
Those three examples wasted five hours I can never get back. If I were to calculate how much of my life has been spent in the Delay Vortex that is our air travel system I’d probably be depressed. Is it measured in hours, days, weeks, months, or years?
Consider: The primary metric tracked in the Airline industry (and the US Department of Transportation) is on-time arrival of aircraft. This drives interesting behavior on the part of the airlines. I remember being on a business trip with 4 colleagues. We had a connecting flight at a major airport and we knew it was tight. Two of us arrived at the gate just as the agent was about to close the door. We explained to her that two colleagues were literally less than a minute behind us. With bureaucratic ruthlessness, she explained to us that the door had to be closed for the flight to leave on time and, subsequently, arrive on time.
Forget, for the moment, the absurdity of the statement from a factual basis (there is plenty of slop built into each flight time for excess taxiing and holding patterns), and think about what that lady was “measured” into doing. As far as I know, the aircraft doesn’t care when it arrives, but the passengers sure do. She made the decision that resulted in three people driving home that night in order to prevent the aircraft from leaving one minute later. It was a decision that did not consider the needs of the customer. Did she make this decision because she didn’t care? I doubt it. Are the executives who picked a poor metric stupid? I doubt that also. Is the government stupid for using a poor metric to measure the performance of an industry? I’ll keep my answer to that to myself . Does any of this make sense? Smart people will sure do things that don’t make sense when they are evaluated by ill-chosen metrics.
Toyota, long the benchmark for achieving brilliant results with average people using brilliant processes has even fallen prey to the lure of looking good instead of being good. Somewhere in the past decade, Toyota made a departure from their 60 year history of striving to be the best. They took a left turn in the direction of trying to be the biggest. They are paying the price. For the first time since the 1950’s, Toyota is losing money. They are also experiencing a rash of quality problems. So even the benchmark for low cost, high quality products can fall into this trap! But I digress… back to the airlines.
It gets worse. When there is inclement weather at a hub airport, the small commuter jets get grounded, but not the larger jets. Again, the focus is on on-time aircraft when the aircraft could care less whether or not it makes it to Houston on time.
Page 1/4
The result of this policy is that all the people who live in small cities that are served by commuter aircraft get delayed, and then they miss their connecting flight to large cities that are primarily served by large aircraft which leave as scheduled.
On one such occasion, I arrived at a gate to see the aircraft I was supposed to be on parked at the gate with the pilot and co-pilot doing their paperwork, clearly nowhere near ready to leave. The jet bridge was still extended, two feet short of the aircraft. I asked the gate agent if I could get on the plane since they obviously are not ready to leave yet. I was told that if I ran I could catch the last flight out that is leaving from two terminals away in 10 minutes. I asked if she would call ahead and let them know I was coming. She said, “No.” I told her if she would just get the crew to open the door, I’d be glad to jump the two feet from the jet bridge to the aircraft. She said, “You now have 9 minutes.” I ran. I missed that flight too, but at least I got 20 minutes of aerobic exercise out of the deal.
One final gripe: TSA. Each time I go through security I have to remove my coat, belt and shoes. I have to take my CPAP machine out of my luggage and it goes through testing 100% of the time. And, of course, my laptop and my one quart plastic bag with my toiletries in it has to come out. Once I get through the security check, I can get dressed and get all my stuff packed again. About 10% of the time, they rifle through my laptop case (generally when I’m running late) and about 50% of the time I find a notice in my checked luggage that informs me that my checked bag has also been rummaged through (they go ahead and wrinkle up my clothes for me while they are in there). The worst crime I’ve ever committed is speeding. Yet I get to experience this and there are still guys boarding planes with explosives in their shoes and underwear. It is all done for the appearance of doing something to keep us safe, but we all know through empirical evidence that it is ineffective. I’d be willing to submit to an FBI, Homeland Security, INS and Interpol background checks and be photographed, DNA sampled, fingerprinted, and retina scanned in exchange for the right to bypass this process.
What if the airlines and the government chose a metric that drove the right behaviors for the satisfaction of the customer? A good metric might be on-time arrival of passengers. Another would be total passenger lead time (how long does it take to check in, get through security, walk to your gate, learn of the gate change, walk to your new gate, wait through the delay because the crew got stuck in a snow storm in Indianapolis, board the aircraft, wait on the tarmac due to ground stop issued by DFW, fly to Dallas, wait on the tarmac for a gate to come available, taxi to the gate, wait on a crew to extend the jet bridge, wait for someone to unload your gate-checked bag, walk to baggage claim, wait for your baggage, get your rental car, drive away). If I could see improvement in those metrics, I’d be happier. Not satisfied, but happier. However, over the past 15 years, my experience is that those metrics are steadily declining…to the point that I’ll do most anything to avoid booking air travel.
But we can’t know because no one cares to measure them. If the metrics suggested above were used, would a gate agent hold up a jet for 60 seconds to allow three people to make it home to their families on time? Would they be willing to extend the jet bridge and re-open the door to get one more passenger to their destination on time? Would TSA find a way to reduce the inconvenience and delays inherent in their processes? Page 2/4
I use the airlines as an example to avoid offending my friends in manufacturing. It also affords me the opportunity to vent. However, how well do we do in Manufacturing? Fairly similar, I’m afraid. What do you measure that drives counter-productive behavior? Below are some examples of metrics that can drive the wrong behavior, with suggested alternatives.
Poor Metric
Behavior Driven
Better Metric
Behavior Driven
Monthly inventory snapshot
Looking good. Draw down inventory ahead of snapshot, Ramp back up afterwards. This increases the “bullwhip” effect in the supply base.
Daily inventory snapshot with target amount with upper and lower control limits. Any day outside control limits triggers problem response.
Being good. Fix the problems that lead to excess inventory
Monthly shipping revenue
Looking good. The shop floor is emptied out at the end of the month in order to expedite all possible shipments. At the beginning of the month, the shop floor has to be “primed” again at the beginning of the next month. Thus, there are three operating systems: End, Start, and Middle of month
Daily shipping revenue, with upper and lower control limits. Any day outside control limits triggers problem response.
Being good. Fix the problems that lead to not shipping the target amount
Machine Utilization
(% absorption)
Looking good. Run the machine even if the subsequent process doesn’t need the parts. Possibly overburdening equipment. Avoiding changeovers. Optimization of processes at the expense of the total system. This creates excess inventory and labor while tying up floor space.
Equipment productivity (unit capacity/$ capital)
Labor productivity
(man-hours/unit)
Floor space productivity (unit capacity/sq.ft.)
Being good. Drive toward productivity (effective use of resources to generate revenue). Focus on the total system, not single process effectiveness.
See more metrics examples continued on the next page.
Page 3/4
Below are more examples of metrics that can drive the wrong behavior, with suggested alternatives.
Poor Metric
Behavior Driven
Better Metric
Behavior Driven
Machine Cycle Time
Looking good. Operator is slave to the machine and is usually under-utilized
Takt Time
Being good. Work is operator-paced resulting in greater labor productivity. Equipment less likely to be overburdened
Daily production quantity
Looking good. Schedule can be juggled to make product that isn’t needed while product that is needed is not made.
Daily sequence attainment. Measure performance to planned sequence and mix, not just volume
Being good. Measuring ability to execute a schedule that meets the needs of customers. Fixing the problems that inhibit schedule attainment.
Cost (% of revenue)
Looking good. Meeting the budget, eliminating waste. Sounds “lean,” but it is “fake lean.”
Profit ($)
Being good. This is what we are in business for. What good are low cost and high margins if I don’t have volume? Zero cost but revenue of $1 I’m out of business!
Management expectations drive employee behaviors. Employee behaviors drive the culture of a manufacturing enterprise. Management expectations are driven by what we measure. Lean is about culture change. If you want to change your culture, a worthwhile activity is to assess how well the things you measure guide that culture. Some questions you might ask as you do this assessment:
Do we have a culture that attracts customers?
Do the metrics we have chosen drive the behaviors that we want to encourage? Will those behaviors result in the culture we are trying to achieve?
Does the metric for one process or function cause behavior that drives sub-optimization of another process or function?
Do individual and functional metrics align with the best interests of the organization?
Are metrics focused on improving our ability to meet the needs of the customer?
Do we drive metrics that are diametrically opposed to each other?
Do we have redundant metrics?
Do we have metrics that we track without knowing why?
To learn more about LMSPI and view our online archive, visit: www.LMSPI.com.