“Don’t bother me with petty distinctions between BI, analytics and decision support. I want meaning, not tools for their own sake – and here I see glimmers.”

Read it here.

Twenty tweets the first day!

Visualized data seems as natural as eating and sleeping, doesn’t it? Yet the first economic time-series wasn’t plotted until 1786, according to our patriarch of viz Edward Tufte in his 1983 book Visual Display of Quantitative Information.

What took so long? I suppose humanity really did suffer from lack of an Excel chart wizard. People had been making maps for centuries, but apparently no one had made the leap from maps to abstract quantities like time and money. That wasn’t so easy to do, after all. You can’t just scratch numbers in your clay, you have to think about it first.

It all comes back to what’s probably already a cliche: simple is hard.

Still, what took so long?

Sometimes I’m a complete idiot. I’ve been writing a monthly column for TDWI’s BI This Week since the beginning of 2011 — after a break of almost two years — and I have never posted a link. No good self promoter would have neglected to do that.

Well, here’s a link to all of them. Being a search-results page for “Cuzzillo,” it also includes a mention here and there of me. The list goes all the way back to 2007 and “Personal Data Warehouses: Challenging a Single Version of the Truth.” Back then, “single version” was still sacred.

One of the most interesting people in the wide world of information technology — Frank Buytendijk — has been hard to keep up with lately. OK, you know about his recent book. But his blog is something else, and he’s moved it again. Active minds can be like that.

Now over at BeyeNetwork, he introduces the subject of his next book, tentatively titled The Machiavellian CIO (… and other essays on how the old philosophers would view modern themes in business and IT).

What would the old philosophers have said if they would have been confronted with modern themes in IT? An ideal theme indeed. It allows me to speculate wildly on a number of topics, and blame people that have been dead for hundreds of years (at least most of them). What are they going to do? Argue with me? Unlikely.

He wants you to do what Plato can’t: argue with him. Go argue here.

A guy walks into your cube and asks you to whip up an econometric model. You’re a statistician, after all, and you’ve got a Ph.D. in something or other. You do this for lunch, he figures.

He “over-thought,” says the one whose cube such a guy walked into. Theresa Doyon has been routinely navigating datasets in the 50 to 300 million-unit range for 10 years. She’s good at all-terrain tools like SAS and KXEN. She could have produced the report he wanted. Instead, she asked him, “Why?”

They talked for an hour. What he actually needed, she discovered, was more of a descriptive report — something that gives a picture of a current situation. All he wanted to do was figure out how to allocate resources.

This is where things usually go wrong. “We’ve got all these great tools,” she says, “but I don’t see us using them as well as we could.” We could write this off as a communication problem, but she believes it’s much more than that.

It’s something like a man who walks into a kitchen with a carton of eggs and tells the cook to fry them all — when actually he’s just hungry and happened to have found eggs. Something else might suit him better. Bacon?

Business people complain that they get reports, not insights. But they’re not sure how to ask for the data. Meanwhile, analysts complain that they’re asked for a fire hose of analysis and that it’s always due yesterday. But they deliver. Asked for a report, they produce a report. Asked for a model, they produce a model.

“But that’s not what the business people really want,” says Theresa. ‘What they really want is to answer some sort of business question, like how’s my marketing doing? What can I do differently?”

“If you had a strategy and you did bite-size tests and learned as you go,” she says, “you could start to use the analytics that would really drive the insight.” We can really improve the way things are done. That’s the missing link.

At least some organizations have found that link. A home furnishings retailer she worked with recently had her work closely with marketing people as their questions and her analysis evolved.

The retailer had been suffering as ever more of its customers feared they’d soon lose their homes. The living room sets that looked so good just months before seemed to lose their appeal. Theresa’s assignment was to, in effect, come up with a silver lining.

“It was a very big and ambiguous question,” she recalls. She worked with the client on a series of projects over nearly two years as the sour economy evolved. She estimated opportunities, customer targets, and gave the marketing people she worked with critical guidance on the launch of new programs. She recalls, “It came out quite well.”

See her LinkedIn page here.

How do you approach unfamiliar data? An investment banker I talked to last week — one I know from a client’s whitepaper — rejects the “don’t think” method, advocated in my earlier post about Dan Murray. Instead, he thinks first, on paper.

“My approach is driven by having a bunch of gray hair,” says Michael Princi, managing director of ThoughtStorm Strategic Capital, a boutique investment bank and advisory firm in northern New Jersey. “I want to use my business acumen to tease out what might be the underlying issues.”

Experience counts The naive mind is prone to bad mistakes, he says. Take, for example, the 22-year-old analyst in India he employed who spotted this correlation: a firm’s revenue correlated with the number of Bobs in the workforce.

Hypothesize on paper “I first think through my hypothesis on paper,” he says. “‘It gives you a starting point.” If the model is wrong, as it often is, he just tries another one.

Test and repeat If the actual numbers are somewhat close to his expectations, he knows he’s on the right track. It’s the traditional consulting confirm-or-deny method. When the data does confirm his hypothesis, he’s able to run through the data again and again in iterations.

Test and repeat If the actual numbers are somewhat close to his expectations, he knows he’s on the right track. “It’s the traditional consulting confirm-or-deny method. It’s the quickest way I know. When the data does confirm his hypothesis, he’s able to run through the data again and again in iterations.

Does Michael Princi really analyze data differently from Dan Murray? They’ve never met, but Michael guesses not. He says of Dan, “I think he’s probably mapping it out intuitively.”

Do you have a routine for analyzing unfamiliar data? Please introduce yourself here.

Many in the business intelligence industry talk about organizational problems getting in BI’s way, but few talk about them very much.

Scratch the surface of most presentations and conversations — such as last week at the TDWI conference in San Diego — and you find people problems bobbing right up alongside data problems: indifferent executives who undermine BI, short-sighted silo keepers, and IT people who enrage business users with paternalism, to name a few top quirks. If only data were all we had to transform!

One business manger at last week’s TDWI conference in San Diego told me that one of his most daunting tasks during a recent data warehouse implementation was persuading silo managers to release their death grip. For this task, he was on his own. Couldn’t someone have briefed him on the objections he was likely to hear? Or tactics to overcome resistance?

One organization that seems to have solved its people-problem was Blue Cross Blue Shield of Kansas City. Their impressive success with Hewlett-Packard tools was based on commitment to data for strategic advantage and shrewd orchestration. They also had a steady, guiding hand from HP. For example, as Blue Cross Blue Shield built new structures, it avoided upsetting stakeholders by leaving old structures in place for 18 months. (I hope to have much more on that story in the next couple of weeks, thanks to John Santaferraro, HP senior director of marketing, business intelligence.)

Several people in the BI crowd do talk often and thoughtfully about organizational problems. Maureen Clarry, CEO of CONNECT: The Knowledge Network and longtime TDWI instructor, teaches “Power, Politics, and Partnership in Business Intelligence Projects” at every TDWI conference. Participants see for themselves how position shapes behavior. Those short-sighted silo keepers, for example, could flip into data-sharing maniacs if assigned a different position.

Jill Dychè, partner at Baseline Consulting, teaches “BI from Both Sides: Aligning Business and IT,” with strategies to avoid or pave over organizational potholes. She suggests, for example, dodging the perception that BI is “so much data loading and report provisioning.” She writes in email, “We find that the extent to which BI is viewed as a program — with platforms and tools merely components — is the extent to which BI teams are productive and visible in their companies.”

Wayne Eckerson, director of TDWI Research, also addresses these issues, most colorfully with his idea about “purple people.” They are a little bit business-blue and a little bit technology-red, and the purple coloration they acquire lets them traverse the IT-business rivalry.

Wayne spells out some important characteristics for this job, such as maturity and knowledge of technology and business domains. The best are “switch hitters,” by which he probably means to imply that they’re persuasive wherever they stand. In fact, “purple” sounds like a euphemism for another “P” word that Jill actually spells out: politician.

Bad word or not, it’s a critical function. A good politician’s essential function is to coax rivalrous parties into agreement. If that’s the kind of function Wayne sees for the purple people, then they really are, as he says, “the key to BI success” — at least at one level.

Purple may not help much at higher levels. Wayne’s knowledge of of business intelligence is far deeper than mine, but my experience elsewhere makes me think these people are just one of many keys. When I was a sort of purple person myself — in the late ‘90s, bridging an arrogant Web development group and a couple of marketing groups accustomed to full control of their media — my own skill at listening, negotiating, and arm-twisting was only one key. Another key was my boss. At first I had a strong one, later I had an indifferent one, and even later I had virtually no boss at all. I felt like my district shifted boundaries each time, my agenda with it.

One friendly executive suggested I stand up and promote the Web project around the company at any meeting that would let me. He said, “Show ‘em how great it is, and the credit will rub off on you.” Just like a politician running for office.

If I were a purple person today working in BI, where would I go after I’d exhausted training by Maureen, Jill, and Wayne? Most likely, I’d turn for inspiration to books on politics and influence, such as biographies by Robert Caro. Actually, I’ve gone there already, but only because to me politics is a good word. No, you don’t want to emulate Caro’s subjects, just clean and adapt some of the principles they used.

One thing seems clear to me: If purple people, would-be purple people, red people, and blue people are to expand the BI horizon, conversations have to go longer and deeper into the people problems. We start by ending the prissy avoidance of that word that at its best connotes people, perceptions, and compromises: politics!

Mark Albala witnessed something through a client that helps explain the cloud’s ascent: The client bought a $25,000 product, and got a bill from their technology group of almost $150,000 to install it. The client’s response: “What is this s—-?” They’re now seriously considering SaaS.

He is president of InfoSight Partners. “[SaaS] is not catching on because it’s cheaper. It’s not,” he said. “It’s not catching on because it’s more efficient. It’s not. It’s catching on because companies are tired of dealing with these technology groups.”

SaaS has what many tech groups don’t have: agility. Most large corporations, he says, have embedded, well-defined development processes that take things from conception to production, with lots of people involved for checks and balances.
Sarbanes-Oxley is a big piece of that, he says. It causes rigidity.

It’s all expensive and slow — while business people stand there, hopping with urgency.

Finally out of patience, business runs from IT and into the cloud — where they meet their needs.

But they possibly meet danger, too: multiple stovepipes.

“The understanding of why they shouldn’t establish multiple stovepipes,” he says, “is not going to be there.”

How does a responsible technology group remain responsible to its client and yet support the client? Tech groups can remember, he says, that they have to stay agile.

So many of us who feel drawn to visual analysis can’t understand why everyone can’t see the value. “Pretty pictures,” the skeptics mutter. On Eager Eyes, Robert Kosara makes important points that I haven’t seen before.

Toward the end of his post he writes, “We need a new term.” He rejects the aged and indefinite “visualization” and the baggage-laden “visual analytics.” He prefers “visual analysis.”

Whatever we call it, it’s harder to use than it seems.

You have seen the bar and pie charts, but do you actually know what they mean? Do you know how to use them to tease the relevant information out of your data? Can you handle more than two dimensions of data and still find meaningful structures? There is so much more to visual analysis than what Excel offers you.

Good, but then he’s not clear. He writes, “The key problem is that people are much more interested in clicking through interesting pictures than learning about actual analysis work done using visualization.”

Which people? He can’t mean the ones who actually analyze. He must mean the casual users, the data consumers, the armchair analysts — and they will always click through. He writes that those who value visual analysis have to fight the idea that it’s just pretty “or risk the trivialization and marginalization of visualization as an analytic tool.”

You’d think the tide was coming in an threatening a sand castle. But from everything I’ve seen, genuine visual analysis seems to be more and more popular. Even elementary visual analysis works better than the ugly alternatives.

Who are we fighting? The ones who don’t care and never will? No, they’re no more a threat than fast food is a threat to good food. To most people, fast food is good enough — and so are pie charts.

The ones to watch out for are those who sell fast food under the good food banner — the ones who’d propagate sloppy techniques and call it visual analysis. That’ll really spoil our appetite.

For more on “good food,” don’t miss the “Information Visualization Manifesto.”

In the film “Night on Earth,” Italian comic Roberto Benigni plays a taxi driver tooling around Rome one August morning at four. His flag’s up, he’s bored, and the streets are empty. “Dove sono i romani?” he asks himself, “Where are all the Romans?”

Where were all the BI people last week? Did they all go to Munich or Tehran? Or were they just resting up for the recovery?

Then I checked Twitter. Many of them had been there all the time. Still, I was bored.

Then on Saturday, Ken Rudin’s email came in announcing his new address. On Monday, LucidEra’s sad news.

At 4 on recent mornings, I suppose LucidEra execs were wide awake and staring into the dark. Mark Madsen emailed that he had an inkling, but I was surprised. This wasn’t supposed to happen. LucidEra was one of the interesting companies, one of the bright lights. In January, I had written a column for TDWI about possible expansion of the Pipeline Healthcheck. Others would go down, but not them.

On the other hand, it was a startup. They often fail, especially now. And I’m sure that chief marketing officer and co-founder Rudin and vice president of marketing Darren Cunningham will land well.

The movie is more fun. Someone finally hails the cab, a priest. The driver feels no reverence. He invents a long “confession” to amuse himself, and it works so well on the drive across town that he doesn’t notice the priest’s fatal heart attack.