Tools and those who enable their misuse
Posted on | February 1, 2010 | 1 Comment
To get a data architect I know worked up, just ask him about how customers end up buying the wrong tools.
How about sales people who push federation tools on those who actually need data warehouses?
“It all sounds extremely sexy,” says my source, who works for a major business intelligence vendor and whom I can’t identify. “You have a lot of people who exaggerate their ability to combine data to provide business solutions. … They don’t prototype, they don’t profile, they don’t actually think about the problem or do testing or even send some high school data analyst out with Excel to put something together that [the customer] might want. They don’t do that.”
Many sales people tout EII because that’s what they have to sell, he says. “The EII tools give you your data, warts and all,” he says. It’ll work fine as a data warehouse substitute “if the data’s pretty clean to start with, if it has a somewhat similar structure, if you can define the data you need, if the data’s relatively common across all the sources, and if there’s not much duplication.”
Even if the salesperson has a more appropriate tool than what the customer asks for, the customer may never hear about it. “‘Fine!,’” thinks the salesperson. “‘If you want to buy a hammer, that’s fine. If you want to buy a wrench, that’s fine. It’s not like I care. It’s just sales to me.’”
Just once, says my source, he’d like to hear one of these questions: “How long does it take for a novice to become OK at this task?” Or, “How long would it take for an expert to become proficient at these two things?” Or, “If I have a failure, what is your tool’s usual process for recovery, and what gives your tool more integrity than others?”
Mark Madsen, meanwhile, has been been thinking about similar problems but from a different perspective. He’s research director at the Third Nature consultancy and a keynote speaker at this month’s TDWI conference in Las Vegas.
One source of problems he sees is vendor marketing. “It’s all about ‘our tool does this’ or ‘has these features,’” he writes in email. “A lot of people don’t think about them that way. They think about them as ‘what this tool is for.’” People end up using an ETL tool for real-time synchronization, for example, or a federation tool in place of a data warehouse.
Even product documentation can lead users down dark paths. “All those docs that say what the features are help when you know what feature you want,” he writes. “When you’re trying to accomplish a task, you’re thinking in a different way.” A common result: convoluted solutions.
“I once did something in an ETL tool,” he writes, “and the product developer said, ‘That’s not how you do that.’ They had built around an improper conception of how users apply it.”
Design schools tell you that every user has a theory of how anything works, he writes, which determines their approach to it. Wrong theories explain why people push on doors that need to be pulled, for example. He says that this insight has made him change his approach to teaching his courses or showing clients.
“I’ve realized that I need to start with the ‘what this thing is for’ and move into what you do with it, and how it works.”
Mark may go into this more in his keynote at this month’s TDWI World Conference in Las Vegas. His long-running “Clues to the Future of Business Intelligence” — perhaps the “Cats” of tech presentations — has been one of the most interesting I’ve seen in any tech industry. I expect “Stop Paving the Cowpath” to be worthwhile.
Tags: customers > data warehouses > las vegas > Mark Madsen > marketing > tdwi > tools
Marco looks to BI for help
Posted on | January 29, 2010 | 1 Comment
My friend Marco’s spam-bait operation was down last year, and he’s been asking me what business intelligence can do for him. He had just read one of TDWI’s promo emails last night when he called me again.
“I like Vegas. Should I go?” he asked from somewhere that sounded far away. I said it all depended on what he wanted to learn. Is making sense of his data important? If yes, go. But there seemed to be more to his question.
He’s gone through one shady business after another since the early ’60s, when as a teenager he sold drugs on the street. Now he sells fake email addresses in huge blocks to Eastern European spammers. All his customers have had a good education, he tells me, yet most retain some of their families’ traditional ways. He describes them picking over his blocks of email addresses as if over oranges in a bin, rejecting one, taking another. They seem to rely entirely on feel, and Marco makes sure each new batch feels “fresh” and authentic year after year.
“Cool. My data’s real, real important to me,” said Marco. “So’s my know-how, my experiments, my research. Those experts in Vegas dish on how to manage all that, man?”
Definitely the data, I said, but not much on the qualitative end of his research. He was disappointed.
“You know, you got me going on this insight thing, man,” he said. “And then you change the story. This business intelligence takes care of only some of my insight? Only some of it? What do they think, data’s the only way you get insight?”
He had a point. I thought fast. I said he should think of his operation like a speeding car. He liked that. I said he needed a “dashboard” to let him know how he was doing. He liked that, too. There was a course on Tuesday, I said, all about that.
“Cool, man. But what about my research? I got these journals I keep with my results and theories and shit like that. What about all that? I keep losing track of it all.”
I said I thought he was talking about knowledge management or something.
“Yeah, that sounds like what I want. Knowledge management. They don’t do that there?”
I explained that data was this event’s main focus. Other events … but he cut me off.
“No, man. Here’s what it’s about,” he said. “It’s all about marketing. I don’t know much about business intelligence, but I bet that every benefit, feature, whatever comes from a different tool. Each comes from a different vendor,” he said in a tired sing-song, “and the producers of this event have a line on a certain kind of vendor. To protect their game, they make up a category. Get hip, man. It’s always like that.”
He quickly added, as if he had already bored himself, “How’s the food there? Can a guy score somethin’ to eat?”
The best Caesar’s can offer, I said. Then he had to go answer the door. I heard urgent knocking.
Bring in the shrinks for decision analysis
Posted on | January 19, 2010 | 1 Comment
Now comes the hard part in business intelligence: figuring out how the humans can make better use of all our data and tools for decision making, writes Wayne Eckerson, director of TDWI Research. Let’s bring in the shrinks.
When Wayne points to a trend, it’s news even if others might have already foreseen it. He’s one of the industry’s most thoughtful observers, and one of the most deliberate.
In Tuesday morning’s blog post, he suggests improving BI by enlisting those who study how people make decisions.
To take BI to the next level, we need better insights into human behavior and perception. In other words, it’s time to recruit psychologists onto our BI teams.
He gave an example of one place that could have benefited from visits to the shrink’s couch.
A recent article in the Boston Globe called “Think Different, CIA” provides some instructive lessons for companies using BI tools to make decisions. The article describes a phenomenon that psychologists call “premature cognitive closure” to explain how humans in general, and intelligence analysts in particular, can get trapped by false assumptions, which can lead to massive intelligence failures. It turns out that humans over the course of eons have become great at filtering lots of data quickly to make sense of a situation. Unfortunately, those filters often blind us to additional evidence — or its absence — that would disprove our initial judgment or “theory.” In other words, humans rush to judgment and are blinded by biases. Of course, we all know this, but rarely do organizations implement policies and procedures to safeguard against such behaviors and prevent people from making poor decisions.
See his full post here.
Be sure to see the comments, too. He writes in reply to questions, “Like data governance, we need some principles for approaching and managing decisions. Maybe we should start a decision governance institute!?”
I can’t help notice: an institute.
See “CIA’s insights on the psychology of analysis” on Datadoodle.
Tags: decision analysis > decisions > prediction > tdwi > Wayne Eckerson
“Streetlights and Shadows”
Posted on | January 15, 2010 | 2 Comments
Some of the books Stephen Few reviews may at first glance to have little to do with data analysis. On second glance, though, they have everything to do with it. He often goes into the essence of thinking, insight, and decision making — core knowledge for BI practitioners.
See his latest, posted yesterday afternoon, on Gary Klein’s Streetlights and Shadows.
Mapping the many faces of “retention”
Posted on | January 15, 2010 | 2 Comments
Everybody knows what “retention” means until they have to design a metric. Ken Rudin, once of LucidEra and now general manager of analytics at the games site Zynga, thought that he and his team could “put something together” quickly — but it actually took “four solid weeks of discussion and debate.”
About 50 million people play Zynga games every day. It’s the leading online social gaming platform, according to Ken, and it’s grown from zero in 2007 to revenues of “a few” hundred million dollars annual revenue. Every day, the company captures 20 to 30 billion records of data, and Ken and his team use that data to improve revenue, viral marketing — and customer retention.
Zynga players play free. The revenue comes in a few dollars at a time for “virtual goods.” In the popular game FarmVille, for example, a player might get tired of the old-fashioned plow. The tractor upgrade costs $2.
“There are tons of different ways you can think about retention,” he laughs, “and which one should we use?”
How do you know when a customer has left? “Unless we don’t get a note saying, ‘Hi, we’re no longer playing,’ how do we know?”
Of course, no player’s going to make it that easy, so how long should Zynga wait before considering the player gone? A week? A man could have dropped his virtual pitchfork for a real vacation — or he could have plowed the last row.
Ken dealt with analytics all the time at LucidEra, but games were new to him. He’s learned a few things.
“It turns out, as you might imagine, that it depends on the game,” he says. The average simulation-game player tends to visit frequently, for example. Poker players, though, are much more likely to come back after, say, a three-month gap.
The retention curve also varies by the length of each player’s tenure. A new player who stays away 30 days is much less likely to return than a player who’s been at Zynga for years. Ken now puts users in three basic tenure buckets: “new,” “mature,” and “elder.”
Whatever question you try to answer, it has to be actionable. “There are metrics, and there are metrics that matter,” he says. If volume plunges, were the missing players mostly new ones? If so, it could indicate frustration; perhaps the games need better tutorials or less functionality at the beginning. Or were most of the missing the long-term customers? If so, perhaps the games haven’t offered enough challenge.
Ken expects growth when the economy improves. “When we look at what happens over holidays, such as July Fourth and Thanksgiving, usage really drops. Then it picks up as people go back to work,” he says. “[The games] are part of their routine. On vacation, players break their routines. They sleep late and spend more time with family. They don’t play the game.”
“It’s fascinating,” says Ken. “In analytics, so much of the problem is figuring out what the question really is.”
I think he means that it’s a great game.
Tags: analysis > customers > economy > games > games site > Ken Rudin > LucidEra > metric > metrics > retention curve > Zynga
Rolling heads can’t think
Posted on | January 12, 2010 | 2 Comments
Wolf Blitzer calls for heads to roll after the Christmas Day attack. But Jill Dychè is a data pro, and she’d rather let the heads think.
“Who should get fired?” is the same conversation as after screwups in corporations, writes Dychè, principal at Baseline Consulting.
Instead, the government should be addressing process issues. Indeed, the real conversation should be how to move forward. These questions should be asked now: “How should we bring identifying data together? What are the key sources? How should integration, access, and usage policies be formulated? What would a sustainable process look like?” Those questions aren’t “who” questions, they’re “how” questions, and they should be front-and-center in the national security conversation.
Read the full blog post.
Culture failure!
Posted on | January 7, 2010 | No Comments
See Oscar Berg’s post “Did You Ever Hear anyone Shout ‘Culture Failure’?” on his weblog, The Content Economy.
A culture failure is much more alarming and also much more uncomfortable than a simple process or technology failure. It signals that something is fundamentally wrong, something which is very complex and hard to change. It means that you not only have to change your own attitudes and behaviors, but also those of your colleagues, including management. You might need to change the entire incentive model, which in the end determines the bonus of your CEO. What is worse, you most likely also need to change the attitudes and behaviors of your CEO (”Impossible!”).
Originally referred by Henrik Mårtensson on Twitter.
Hoping for Citizen 2.0
Posted on | January 6, 2010 | 1 Comment
I like the sound of Government 2.0: Collaborate with citizens online and you can change government from a sewer-dwelling raccoon into a purring housecat.
Social media lets us try for a kind of politics that was impossible until now. I hope for great results. For many, Government 2.0, or “collaborative government,” will mean just “friending” a local cop. But in full flower, Government 2.0 can mean far better service, and far more government-and-citizen collaboration than ever before.
Even before we had social media, the glare of public attention was a proven antidote for bad politics. Citizens getting up their elbows in policymaking has always been another strong medicine.
Trouble is, that “sewer-dwelling raccoon” is always smarter than people think. When he’s hungry, he purrs like a housecat and covers stinky laws with high-minded names. Advertising fools just enough voters — so often complacent and impatient — to throw a new law onto the books. On and on it goes.
Such a stinky new law is just what Californians got in 2000. Proposition 34 was sold to voters as campaign finance reform. It was a ruse. (A few days ago, a report confirmed suspicions, and a major drafter of the proposition insisted he was “outraged.” Yeah, and round up the usual suspects.)
One other fix, more honest, came 100 years ago: California amended its constitution to give citizens the ballot proposition. It was the only way for voters to bypass the paralyzed Legislature and loosen the Southern Pacific Railroad’s grip. It worked. But more recently, ballot propositions have helped tie the state’s budget in knots.
In the long run, who knows how social media, visual analysis, and other tools may be used in government? What will matter most of all is who uses them. If it’s “the people,” which people?
I hope this new, pervasive politics mobilizes a new wave of smart activists — the way desktop publishing and, later, weblogs enabled editors and writers. Or the way tools like Tableau and Lyza are enabling independent-minded, creative analysts today.
As these activists learn about politics, I also hope that more citizens than ever before step up to watch, push, and verify. Such a voter would be Citizen 2.0, the real hope.
Otherwise, it’s going to be that raccoon again — this time on Twitter.
“Efficiency” can cost too much
Posted on | January 6, 2010 | No Comments
See Henrik Mårtensson’s “The Cost of Queues” on how an extreme focus on “cost effectiveness” can damage an organization. (Thanks to Jack Vinson for the referral on his blog “Knowledge Jolt with Jack.”)
If you try to become more cost effective by reducing capacity, and thereby capacity cost, all will be well at first, from an economic point of view. (The people who are let go are usually of a different opinion.) The catch is that this will increase the queues in the system. This increases lead times. Consequently, cost associated with lead time will also increase. (In manufacturing there is also a considerable storage cost due to increased inventory, but we will ignore that for the purposes of this blog post.)
Be a strategist, not a “geek”
Posted on | December 22, 2009 | 2 Comments
Dear Datadoodle: My title is “strategist analyst,” but I’ve become just “the data geek.” As soon as I get into the fine points of my data, they roll their eyes. In meetings, they make little jokes to each other, or they just stare out the window. Please help. I’ve got loads of great data but managers have no time for me anymore. The Data Geek
The Data Geek has lots of company, says Christine Muser, longtime data analyst, founder of CyCom Solutions, and writer of CyCom’s weblog Pharma-BI. She’s seen data analysts stumble over this problem. She had to deal with it herself.
“I love data,” she says, “so in my younger days I used to just barrel right ahead into the details.” Too often, though, she’d see only glazed looks in her audience.
It was even worse for one man she once worked with. He actually became useless as a strategist. He spent so much time pulling together data, manipulating it in Excel, and poring over results that he kept managers waiting for results. After a while, those who relied on him got tired of waiting — and tired of his overly detailed explanations about dimensions, data sources, and methods.
Now, Christine analyzes more than the data. “It pays to know your audience. If you know they’re very familiar with the underlying data, you can give more details,” she says. If they’re not familiar with the data, “You can say, ‘here’s what we’ve observed’ and ‘here’s the impact,’ then be quiet and see if you get puzzled looks.”
The essential lesson she learned: “Understanding what is meaningful is a really big deal.” A strategist understands what’s significant, and doesn’t bother with the small stuff.
For example, say you’re analyzing a pharmaceuticals market. Your company makes a drug that treats only one symptom, while some competing companies make drugs that treat that symptom plus several others. That difference makes a straight comparison — pill for pill or dollar for dollar — useless. So you have to adjust the data to allow for that difference. But managers usually don’t want to hear how you did it, only that you did take care of it.
A strategist also knows when to ignore buzzwords. A manager who liked to stay current once asked Christine, “Can you do neural networks?” Perhaps, but dare go into what that would take and you risk running out your clock.
Tags: career > Christine Muser > data analyst > strategy
