Good metric-making aims for the concrete and sensory

If you want to come up with effective metrics, forget brainstorming. Drop the creativity. Done well, this is an analytical exercise, says Stacey Barr, and it should aim at deriving concrete, sensory effects to measure.

She’s “the performance measure specialist,” and she lives in Brisbane, Australia. Stephen Few, the BI industry’s leading critic of performance dashboards, refers to her when he’s asked about performance metrics.

I talked to her yesterday as part of my research for my next BI This Week story.

People in search of metrics jump too soon into measuring, Stacey says. One of the first things she does for clients is to insert one critical step: defining what effect they are looking for—and then describing that effect in “sensory specific language.”

She finds that people’s first attempts use “fluffy, vague language to describe those results.” Prime examples: “quality,” “efficient,” “effective,” “sustainable,” “enhanced.” You’ve heard them all before.

“It’s this habit we’ve gotten into of using words that mean seven different things to three different people,” she says. “We’ve got to use words that are more concrete.”

When people are all sitting in a room talking about goals and results, they have the same images in their heads.

“It makes it much easier to measure,” she says. Concreteness is the key.

I suppose that making metrics is like making a movie. No image or sound can tell what a character thinks, feels or intends to do. He has to show it somehow. This concreteness is also what Jon Franklin calls for in his book Writing for Story: Craft Secrets of Dramatic Nonfiction.

Stacey once worked with a local council (to Californians, a county board of supervisors) to raise public participation in meetings. They had been measuring participation with the number of meetings held. That is, more meetings automatically counted as more participation.

More vague words: what does engagement or participation look like?

She asked them, “If the community were more engaged, what would people be doing that they’re not doing now?” At first they said things like more people showing up, a high proportion speaking up, more ideas proposed, and so on.

Eventually they came up with two measures that worked together.

• The number of “fresh faces,” those who’d never attended before. They had decided to give it a try.
• The number of familiar faces, those who’d come to at least half of all recent meetings. They had decided that showing up was worthwhile.

It works.

Come to think of it, doesn’t the need for concrete, sensory effects sound like storytelling?