Here Comes Everybody: The Power of Organizing Without Organizations author Clay Shirky says that a technology’s social effects—substitute “business” effects if you want — usually occur just when a technology has become boring. For example, email. It used to be something we talked about: “Do you have email?” “You mean the Internets?” And so on. Nowadays, everybody but John McCain uses it.

So it should be with business intelligence.

In a Harvard video, Shirky tells a story about his parents’ first date. His father borrowed his brother’s car, and on the date his mother ordered the most sophisticated drink on the menu: a root beer float. She actually hated root beer, though, and threw up in the car. Is that story about the internal combustion engine? Well, yes and no. Though those events would not have occurred without it, the boy and girl never even thought about it as events unfolded. It only enabled.

Today the only people who talk about automotive technology are backyard mechanics and industry experts. I’m happy to leave it to them.

Fortunately, we’re getting there in business intelligence with the emergence of things like LucidEra, Tableau and DataSelf. Keep the boredom coming.

I was thinking about Government 2.0 at about midnight last night—when the dark, quiet world gives way to ghosts. Then it’s easy to imagine BI tools and methods opening government to the masses.

I had been reading about open-source government last night after Letterman signed off. That’s when everyday life recedes and gives way to things that might be, could be. I call them ghosts, and you might call them memories or fantasies or even perfectly modeled government data.

In that state of mind, somewhat in this world and somewhat out of it, I asked a nearby ghost what good might come of Government 2.0.

By the way, what the hell is Government 2.0? Here’s what I understand: It’s the use of wikis, blogs, and other Web 2.0 stuff to encourage a new wave of collaboration among citizens, government officials and civil servants.

Whatever it may turn out to be, I know the concept is alive right now. A friend just got pulled out the freelance world into a comfortable government job surrounded by what he describes as very smart guys barreling forward on it. (He doesn’t work with the other type.) I’ll hear the details tonight over a St. Peter’s porter.

The best of his observations will make their way into a BI This Week story I have to write by Sunday. The rest of the story will derive from material around the Web, from a Tableau Software user, and from the ghost’s narration.

Government 2.0 could be a dream come true, and it could be a nightmare. It’ll be both.

Will it be government workers with thousands of unseen eyes looking over their shoulders? Tell me, do goldfish perform better in clear-sided bowls?

Are crowds wise enough for this task? A similar question arose before and after the American and French revolutions, when some smart people insisted that a monarchy should run things. As it turned out, they were correct—and not.

When I asked the ghost what he thought, he vanished. Lunchtime, I guess. I’ll have to try again.

At 8 o’clock Monday morning, a few hundred attendees at TDWI conference in Chicago will hear the organization’s former education director Dave Wells give his keynote, “People First: Creating a Business Intelligence Culture.” He’ll say something startling: there’s much more to BI than data.

BI is no more about data alone, he has said, than Van Gogh’s “Starry Night” is only about brush strokes. For those of you whose brains itch with a sense that BI is ultimately a social practice, this is for you.

Among other things, he will propose a new definition of business intelligence and talk about cultivating a BI culture, “not exactly a science…something more like an art form.”

I can’t be there, but this story is just beginning.

I’ve been asking known BI mavens what they make of thoughts by TDWI’s recently departed education director, Dave Wells. I summarized our Q&A a few days ago here

Among those who’ve responded, the consensus is that, as one person put it, “he’s got something there.” They’ll all be watching for it all to develop.

The BI This Week story is here.

Is the data clean enough to give us meaning? If you are one of those who insist that the war on dirty data must be won first, forgive me. I’m tired of that conversation. It sounds like backyard mechanics comparing fuel injectors when all that really matters is the commute.

I talked recently to TDWI’s just-departed education director Dave Wells. He’s been thinking, and he says, “Tweaking the details probably doesn’t make any real, significant contribution to the overall value of the insights I can gain by understanding the patterns inherent in data.”

Is it really, really going to make any difference if you can get your data from 96 percent to 98 percent pure? Will the meaning really change?

My old boss in market research had a rule of thumb: with a good sample, results usually stabilize after about 25 percent of the data have been counted.

Are we there yet? Wells says the more interesting problem these days how to soak the meaning out of what you do have. Let those who scrub keep scrubbing while the rest of us move on.

To get to the next level in BI, he says we’ll have to understand better how people think. That could involve systems thinking, how one part of an operation affects a different one. Other areas he’s looking into are design thinking and critical thinking.

Notable quotes

• One of the real looming questions I think in BI as it becomes mainstream is what do you do when analytics contradict conventional wisdom. Then it becomes a political and religious debate.
• The flaw in our current approach to analyzing things is best described by an analogy that says, “you cannot understand the impact of Van Gogh’s Starry Night by categorizing the brush strokes.”
• Maybe the value of real high impact metrics, for instance, is not in the measuring of things, but in the measuring of relationships.
• There will always be something technological to fix. Let’s go back to Starry Night. Do I need to fix that brush stroke? Or do I need to stand back and look at the big picture?

One person I’ve talked to about his ideas thinks Dave is “full of crap.” He says, “If you talk to the people who deal with data, you hear that the problem is nowhere near solved.” OK, there’s still work to be done. But rejecting Dave’s reasoning is like saying that because car design is still not perfect we can’t yet talk about driving from San Francisco to Lake Tahoe.

The full interview will be at the May 7 issue of BI This Week.

Links he suggests, with his notes:

• Systems thinking
• Design thinking: The weblog Functioning Form’s “Defining Design Thinking” post gives a good starting point, he says, with links to articles and papers. He writes, “Design thinking is a less mature discipline [than systems thinking], which makes it less robust but more interesting to explore.”
• Critical thinking: “The critical thinking community is more focused on education than on business today,” he writes, “so much of the interesting stuff there is in discovering how it does apply in business. I have no doubt that it does apply because much of critical thinking is simply another perspective on decision theory.” See the Strategies for Success page and the Foundation for Critical Thinking site.

The greatest BI show of them all opens a week from this Sunday in Las Vegas, the TDWI World Conference. Naturally, anything about Vegas and The Strip draws my attention—such as Sunday’s story about the local paparazzi.

They just can’t get a break there. Celebrities come and go, often in private garages, and the poor picture poppers mope around like neutered cats.

Las Vegas is where the money-makers have their eyes on the ball: the little guys who lay down their spare change at the roulette wheel, Keno parlor and slot machine.

At Harrah’s casinos and probably others, they do more than watch. It’s all about data, and like good card counters everywhere they keep figuring the odds on every known player.

Harrah’s customer-relationship management, aka CRM, won recognition from our very own TDWI in 2000. Harrah’s is said to know, for example, that gamblers who lose money (forget winning) for 18, 20, 24 hours or more actually give up less money than those who go for shorter stints with breaks in between.

What’s a casino keeper to do? Well, he goes down to the hard-working but about-to-be-fatigued player and presents a nice little meal voucher. “Mr. Player, we’ve noticed that you’ve been losing money for five hours straight. We’d like you to have dinner now—on the house—at Bad Restaurant With A View.”

The player, rested and fed, almost always returns to play more than he would have otherwise.

The money’s in the data.

I’ve read about an 84-year-old farmer in North Dakota who reads pig spleens the way mainstream fortune tellers read tarot. Sadly, he doesn’t service the business intelligence industry.

If he did, we might have had more fun with predictions for 2008. Now we’re stuck with these: Consolidation will continue, smaller vendors will sprout, deployment will be easier, software as a service will take off, visualization will emerge… Did I miss any? Well, who cares?

Where are all the tech-savvy clairvoyants when you need one?

A friend of mine used to swear by someone she called “the common sense psychic.” She’d get her on the phone and tell her all about the problem of the day. Yes, said the psychic, and what happened then? And then? After a while, the psychic put the phone down and came back a few minutes later. Her perception was always right on, and no wonder.

In the trade, that’s called a cold reading. You gather facts and spill them back in such a way that you cover all the bases.

Foretelling the next 12 months of BI is a warm reading. What started to happen will continue to happen.

I myself was one of those year-end wizards, but I was able to produce little contrast with everybody else. Perhaps I should have consulted a horoscope and summarized all the signs, like this.

Your star is on the rise. Changes over the next three months will inspire you to expand your horizons. Though prospects look uncertain at the beginning of the year, give it time. The surprise opportunity thrusts you into the spotlight amid roadblocks. Just keep doing what you’re doing and you’ll be fine.

Oh, hell. Let’s cut the crap and skip to the one interesting and courageous prediction from any BI leader. It comes from Mark Madsen, who looks ahead and sees that goatees are going out of style.

Imagine an executive walking around with a “dashboard” on his head. It looks like a pair of sporty sunglasses, but it does much more. Whereever he turns his head, pop-up windows tell him what he’s looking at. He doesn’t even have to ask “What’s going on in that cubicle?”

Detailed background on everyone from temps to hot-shot VPs shows up in his little glasses.

I’m just free-associating on the fantasy described in William Gibson’s 1994 novel Virtual Light. Jason Fry mentions that vision in today’s Wall Street Journal. His article is about New York cabbies complaining that the city’s new GPS system can track them—and the many other uses for tracking.

Here’s a morsel with possibly no practical value at all. It’s from last Sunday’s New York Times on what CEOs read:

“I used to tell my senior staff to get me poets as managers,” says Sidney Harman, founder of Harman Industries, a $3 billion producer of sound systems for luxury cars, theaters and airports … “Poets are our original systems thinkers,” he said. “They look at our most complex environments and they reduce the complexity to something they begin to understand.”