Mapping the many faces of “retention”

Everybody knows what “retention” means until they have to design a metric. Ken Rudin, once of LucidEra and now general manager of analytics at the games site Zynga, thought that he and his team could “put something together” quickly — but it actually took “four solid weeks of discussion and debate.”

About 50 million people play Zynga games every day. It’s the leading online social gaming platform, according to Ken, and it’s grown from zero in 2007 to revenues of “a few” hundred million dollars annual revenue. Every day, the company captures 20 to 30 billion records of data, and Ken and his team use that data to improve revenue, viral marketing — and customer retention.

Zynga players play free. The revenue comes in a few dollars at a time for “virtual goods.” In the popular game FarmVille, for example, a player might get tired of the old-fashioned plow. The tractor upgrade costs $2.

“There are tons of different ways you can think about retention,” he laughs, “and which one should we use?”

How do you know when a customer has left? “Unless we don’t get a note saying, ‘Hi, we’re no longer playing,’ how do we know?”

Of course, no player’s going to make it that easy, so how long should Zynga wait before considering the player gone? A week? A man could have dropped his virtual pitchfork for a real vacation — or he could have plowed the last row.

Ken dealt with analytics all the time at LucidEra, but games were new to him. He’s learned a few things.

“It turns out, as you might imagine, that it depends on the game,” he says. The average simulation-game player tends to visit frequently, for example. Poker players, though, are much more likely to come back after, say, a three-month gap.

The retention curve also varies by the length of each player’s tenure. A new player who stays away 30 days is much less likely to return than a player who’s been at Zynga for years. Ken now puts users in three basic tenure buckets: “new,” “mature,” and “elder.”

Whatever question you try to answer, it has to be actionable. “There are metrics, and there are metrics that matter,” he says. If volume plunges, were the missing players mostly new ones? If so, it could indicate frustration; perhaps the games need better tutorials or less functionality at the beginning. Or were most of the missing the long-term customers? If so, perhaps the games haven’t offered enough challenge.

Ken expects growth when the economy improves. “When we look at what happens over holidays, such as July Fourth and Thanksgiving, usage really drops. Then it picks up as people go back to work,” he says. “[The games] are part of their routine. On vacation, players break their routines. They sleep late and spend more time with family. They don’t play the game.”

“It’s fascinating,” says Ken. “In analytics, so much of the problem is figuring out what the question really is.”

I think he means that it’s a great game.