Everybody knows what “retention” means until they have to design a metric. Ken Rudin, once of LucidEra and now general manager of analytics at the games site Zynga, thought that he and his team could “put something together” quickly — but it actually took “four solid weeks of discussion and debate.”

About 50 million people play Zynga games every day. It’s the leading online social gaming platform, according to Ken, and it’s grown from zero in 2007 to revenues of “a few” hundred million dollars annual revenue. Every day, the company captures 20 to 30 billion records of data, and Ken and his team use that data to improve revenue, viral marketing — and customer retention.

Zynga players play free. The revenue comes in a few dollars at a time for “virtual goods.” In the popular game FarmVille, for example, a player might get tired of the old-fashioned plow. The tractor upgrade costs $2.

“There are tons of different ways you can think about retention,” he laughs, “and which one should we use?”

How do you know when a customer has left? “Unless we don’t get a note saying, ‘Hi, we’re no longer playing,’ how do we know?”

Of course, no player’s going to make it that easy, so how long should Zynga wait before considering the player gone? A week? A man could have dropped his virtual pitchfork for a real vacation — or he could have plowed the last row.

Ken dealt with analytics all the time at LucidEra, but games were new to him. He’s learned a few things.

“It turns out, as you might imagine, that it depends on the game,” he says. The average simulation-game player tends to visit frequently, for example. Poker players, though, are much more likely to come back after, say, a three-month gap.

The retention curve also varies by the length of each player’s tenure. A new player who stays away 30 days is much less likely to return than a player who’s been at Zynga for years. Ken now puts users in three basic tenure buckets: “new,” “mature,” and “elder.”

Whatever question you try to answer, it has to be actionable. “There are metrics, and there are metrics that matter,” he says. If volume plunges, were the missing players mostly new ones? If so, it could indicate frustration; perhaps the games need better tutorials or less functionality at the beginning. Or were most of the missing the long-term customers? If so, perhaps the games haven’t offered enough challenge.

Ken expects growth when the economy improves. “When we look at what happens over holidays, such as July Fourth and Thanksgiving, usage really drops. Then it picks up as people go back to work,” he says. “[The games] are part of their routine. On vacation, players break their routines. They sleep late and spend more time with family. They don’t play the game.”

“It’s fascinating,” says Ken. “In analytics, so much of the problem is figuring out what the question really is.”

I think he means that it’s a great game.

Some loan officers used to go by rules of thumb. There were “The Three B’s: never lend to beauticians, bartenders or barbers” and “The Three P’s: never lend to preachers, plumbers or prostitutes.” Now we have an automated system, but it can’t tell an upstanding banker from a down-on-his-luck bartender.

Imagine a high-level banker who leaves his job for a promotion in another state. He’s trusted and respected for the job he did as senior risk manager, reporting to the board of directors, at a 19-branch bank in Atlanta. But for moving and taking that new job, his credit score declines. He’s forced to pay more for his new mortgage.

“That makes no sense,” he says, “It’s completely out of context,” says Clark Abrahams, SAS’s chief financial architect. He’s happily resettled, but he’s out to overhaul the U.S. credit scoring system.

The context the system missed is his ample capacity to repay the loan. He’s automatically put in the same basket as some other applicant who may live paycheck-to-paycheck.

Context is just what he would inject into the U.S. credit-scoring system. He calls the new system he’s promoting model CCAF (SEE-caff), for Comprehensive Credit Assessment Framework.

Today’s distorted scoring began decades ago, he explains. Before we had credit scoring, we had loan officers. They approved or denied loans based on their own experience and judgment. But that was unreliable and often unfair.

So when computers became available, banks developed scoring. Now we’ve swung the other way: proxy metrics, not common sense, rate credit applicants.

It’s a story in progress for TDWI’s BI This Week.

Several hundred practioners of the many aspects of business intelligence are gathering here in San Diego for this week’s TDWI conference. They know how to clean data, enable fast searches, design insight-accelerating tools and other wonders—and yet no one yet has a reliable metric to score restaurants. We still have to go out and sniff.

You’re now screaming at me: ask your concierge! I did. She sent me to Oceanaire, the Blue Point and several others. But all looked too slick and none smelled good.

Other readers are screaming different advice: look at Zagat, at TripAdvisor, at Google Earth! I did. The online reviews are all mixed—and which ones do I believe? What does the average rating really mean? The written reviews reflect mostly pretension and middle-class angst. Phrases like “they treated us like royalty” too often lead to evaluations like “cooked to perfection.”

This part of Web 2.0 doesn’t work. Nor did it work as Publishing 2.0, or whatever we might have called it back when Zagat ran on paper ballots and hardcopy. Hey, you clever people, create a reliable indicator for restaurant chemistry that I can compare with my own quantified preferences so I can predict my reaction.

I ended up at Candelas. No wait, no drunks, and no obsequious waiter. Just a nice place with delicious food.

Rules of Thumb is a fine website for those of use who enjoy proxy metrics, the things you use to judge when you can’t judge the real thing.

Picking a restaurant is an obsession on the site. One rule of its many rules is attributed to CBS’s Andy Rooney, who suggests you avoid cute names because owners aren’t serious about food. Other users believe that newspaper reviews in the window signal a restaurant that’s neither too bad nor too snooty. Still others believe a “high class” joint is good. The smartest one, though, says you should look first at the restroom.

There is a better way. My ex-wife managed restuarants for 22 years and knows the game. Put her anywhere in the world and the odds are 10 to 1 she’ll pick a good restaurant.

She says you have to keep in mind one thing: the biggest single factor in a restaurant’s quality is the attention to detail. Two restaurants with the same payroll, the same dining room, and the same ingredients can have much different quality. The difference is whether someone on staff is really paying attention.

Attention means keeping an eye on everything, even the small things. The quality-assurance route runs through the kitchen, the dining room and outside. A clean door, well-tended plants, and clean windows show that someone cared enough to keep them that way.

Next, see if the menu matches the season. Then go inside and sniff. Does it smell good? Do people seem happy?

The attention to what seems trivial indicates the level of attention paid to what you don’t see, such as ingredient selection, food storage and cooking.

I’ve almost got too much good stuff for my story in BI This Week about offbeat metrics. Stacey Barr, “the performance measure specialist” in Australia and Zach Gemignani at Juice Analytics in North Carolina both came through with insight-provoking cases.

Zach calls metrics for those hard-to-reach places where bookkeepers don’t go “franken-measures.” Stacey calls them “proxy measures.” By whatever name we call them, Zach and Stacey came up with good mini-cases.

• How a local government governing body figured out how to gauge community involvement. (I wrote about it a few days ago here.)
• How an IT team won the respect they craved. At first, they tried to count activities, but success came only when they counted results.
• How a semi-retired firefighter who’d pulled a few too many drunken kids out of wrecked cars proved to administrators that his prevention program really worked.
• How an online-education organization now decides when a customer should no longer be considered a customer.

The question now is how to weave together their stories and other material.

A story of my own might give the right structure. I used to wonder how to measure success of the rambling and eccentric old ski lodge I once helped lead. With this structure, the article would start with that question and end with an answer.

Not the real answer, though, which offers no hope. The answer will have to be personal: I now know how I could have done it.

If you want to come up with effective metrics, forget brainstorming. Drop the creativity. Done well, this is an analytical exercise, says Stacey Barr, and it should aim at deriving concrete, sensory effects to measure.

She’s “the performance measure specialist,” and she lives in Brisbane, Australia. Stephen Few, the BI industry’s leading critic of performance dashboards, refers to her when he’s asked about performance metrics.

I talked to her yesterday as part of my research for my next BI This Week story.

People in search of metrics jump too soon into measuring, Stacey says. One of the first things she does for clients is to insert one critical step: defining what effect they are looking for—and then describing that effect in “sensory specific language.”

She finds that people’s first attempts use “fluffy, vague language to describe those results.” Prime examples: “quality,” “efficient,” “effective,” “sustainable,” “enhanced.” You’ve heard them all before.

“It’s this habit we’ve gotten into of using words that mean seven different things to three different people,” she says. “We’ve got to use words that are more concrete.”

When people are all sitting in a room talking about goals and results, they have the same images in their heads.

“It makes it much easier to measure,” she says. Concreteness is the key.

I suppose that making metrics is like making a movie. No image or sound can tell what a character thinks, feels or intends to do. He has to show it somehow. This concreteness is also what Jon Franklin calls for in his book Writing for Story: Craft Secrets of Dramatic Nonfiction.

Stacey once worked with a local council (to Californians, a county board of supervisors) to raise public participation in meetings. They had been measuring participation with the number of meetings held. That is, more meetings automatically counted as more participation.

More vague words: what does engagement or participation look like?

She asked them, “If the community were more engaged, what would people be doing that they’re not doing now?” At first they said things like more people showing up, a high proportion speaking up, more ideas proposed, and so on.

Eventually they came up with two measures that worked together.

• The number of “fresh faces,” those who’d never attended before. They had decided to give it a try.
• The number of familiar faces, those who’d come to at least half of all recent meetings. They had decided that showing up was worthwhile.

It works.

Come to think of it, doesn’t the need for concrete, sensory effects sound like storytelling?

Three years ago, I spent four months in my Sicilian grandmother’s home town editing a book I had begun to hate. Time cues were sparse: I church bells four times an hour, a nearby friend for coffee once a day, and cannoli once a week. To ensure I made progress, I clung to my homemade FileMaker Pro-based timekeeper.

At first, the dismal results came in every day: When I felt that I had put in a good five or six hours of steady work, the end-of-day tally—with all the breaks for email, meals, snacks, and quick walks—usually amounted to about two hours of actual work.

That’s what got me thinking. Who says one-person operations can’t use business intelligence?

I don’t want MicroStrategy to outfit my tiny office, now near San Francisco, with its latest and greatest. No, but I do want a company like Intuit, ever more interested in the one-person market, to understand that money isn’t the only data individuals should track.

Time isn’t money, it’s more important than that. Productive time is the most consistent leading indicator there is. Waste your time and you’ll have no money to track.

This is business intelligence for the lone wolf, even the herded wolf.

Sure, you can use your own FileMaker setup or any of those client-billing applications. You click this button to start a billing period and that one to end it. But that takes discipline. I don’t know about you, but I exert every ounce of mine on staying on the program. I forget to click the little button and, damn!, at what point did that consultant’s column send me into a stupor?

If there’s one thing I’d like to find under the Christmas tree, it’s an application that does personal performance analytics.

It automatically tracks and categorizes work done on the computer. Out of the office, it works on an iPhone.

It detects what time I start in the morning and what time I drift away into surfing. It knows the difference between work, goofing off or eating at my desk while I scan email.

It considers all the clues to take a guess. The little brain inside says, “Let’s see, he was clicking away at Word until 9:21…” After that point, its log shows no phone calls, no Web pages downloading, no email opened or sent. The little microphone detected less noise than usual at 9:22, so no visitors had come by. “Hmm, let’s mark it at 9:21 and see what he says.”

A more forgiving “preferences” setting might guess a few minutes later. Like a loyal assistant, it would offer its guesses for review and approval.

“Sir, I would say you had a less than productive day yesterday. Am I correct?”

“No, you fool! I was thinking!”

“How much of it was productive, sir?”

Time tracking is just the beginning. It would also let me define my own, weird key performance indicators. Did I hit my daily hurdle of contact attempts? Did I run? Did I spend enough time this week actually working?

It would scan every scrap of evidence it could get its hands on to detect patterns.

Once in a while, I issue a grand invitation to myself. Come and see the trends. Buffet served.

There, after “brief introductory remarks,” the application offers rich what-if visual analysis, Tableau-like. I see what times of day I actually produce the most words, talk the most, listen the most, field the most email. Correlate production of words and phone calls with arrival of checks. Correlate this or that individual with checks or mood.

It’s like a smart, statistics-savvy aide. The difference is that when a human aide finds you passed out from exhaustion, it brings you a glass of water. But I don’t want that. I’m a lone wolf, and I get it myself.

I could easily find a parking place in Berkeley on Saturday! What could it mean? On a normal afternoon in Berkeley’s Gourmet Ghetto, I usually find just one space open, and often I have to drive around the block once…. Ah, it’s the end of Easter week and a bunch of the university people have gone away. So much for bigger meanings like recession, breaking news on TV or sunspots.

Sometimes such offbeat indicators do mean something. How would we know if the U.S. economy has begun sliding into a severe recession? By a surge in the number of eBay items for sale, according to one article in the March 23 New York Times. That would indicate, I assume, that people had become unusually motivated to liquidate whatever belongings they could.

These metrics measure hard-to-reach places, internal or external. They may be eccentric, creative and even strange.

That’s what I’m writing about next for TDWI’s BI This Week.

So far, my sources include these people:

• Stacey Barr, a woman in Australia who comes recommended by performance-dashboard expert Stephen Few.
• The ever-inventive Zach Gemignani of Juice Analytics, whose recent blog post on “Franken-measures” helped inspire the idea.

I listened to the yammering in Sierra Club committee meetings 20 years ago, before I got into technology, and thought that the “real” world knew better.

One little incident convinced me. At one meeting of the group overseeing the Sierra Club’s old ski lodge, Clair Tappaan Lodge, someone wanted to change the name of a room, the Puce Room. A woman who’d just redecorated in there wanted to rename it after the just-passed folk singer Kate Wolf.

I really liked the old name, especially apt for the room’s cold concrete walls and rusty dripping pipes. The committee, though, was about to approve the change. Then at the last minute, the mischievous manager piped up. “Ted has been in touch with a group that’s all upset with changing the name.” I had joked to him earlier that morning that I wish there were such a group.

I played along. “Yes,” I said deadpan, “they call themselves Traditionalists for Puce, and they’re really pissed off.”

I heard throats clear and papers ruffle. Something had changed. A few minutes later we voted: 7 to 1 for keeping the old name.

Legacy

In the years since, that committee continued to fiddle with the dining hall and micromanage staff. But they seem to have one significant legacy: the Club’s national board of directors has adopted their methods.

Now under consideration, about to be passed: Project Renewal. It’s the fourth or so in a years-long string of plans, reports, frameworks, strategies and schemes cooked up by consultants.

It is admirable in some ways. It institutes metrics for communications, by which I hope some of the nearly unreadable newsletters will improve.

Unfortunately, it does some things just as well as the worst of the business world. It scrubs out the best in favor of the top-down business world’s worst. Like so many poorly led businesses, this organization’s leadership doesn’t know how the place really works.

The deflowering

The flower of the Club, the force, the perfume, the motivation that kept people up all night at kitchen tables to complete work by deadline is about to become as corrupt and dysfunctional as the worst Bush-era federal agency. Something called “coordinating pairs” puts handcuffs on volunteer committee leaders—arrested, in effect, by paid staff. It’s just one example.

I assume the hope—the covert hope—of those who know what they’re doing is that one by one the arresting staff members will see their volunteer counterparts succumb to authority.

Did anyone cry out? Did someone mention the Obama phenomenon, with its surging forces eagerly signing on? Yes, wise leaders did. The Sierra Club, after all, helped show the world that structure’s force. But that was a long time ago before consultants knew that forces driven by emotion wouldn’t work.

Bravo, Sierra Club. You’ve learned a lot from the real world.

If there’s one reliable sign that a recession is coming, it’s when the experts say they see none coming. I’ve survived four. “Oh, maybe a slowdown, yes…” they say. Now, in today’s New York Times, Charles Duhigg argues that what’s unlikely is a “full blown depression.” Quoth Duhigg:

Why? Because so many of them have spent so much time studying the Great Depression and trying to figure out how to react more effectively if things turn really bad again.

Would that be the same kind of study France and Britain did after World War I to avoid World War II? The kind General Motors did to sustain its dominance over the pre-hybrid Toyota?

Quick, buy gold!

Really, to infer from this that a depression is likely could be just as silly as the people who take official denials about Martian spaceships as proof that they really exist.

I don’t infer anything. I just don’t remember hearing the D word during past recessions.