Search all the business literature you can and you’ll never find data analysis compared to romantic love. But, hey, why not? Love’s trajectories might hint at what the business world’s newly enabled generation of data analysts can expect.

These data analysts tend to be independent, are often creative and at least partly self-trained. They’re strapped to rockets from Tableau, Lyzasoft, Predixion, and others, tools that are at first deceptively toy-like. Aren’t they analogous to the garden variety teenager? Bothg groups revel in newly discovered tools, while both pursuits are fundamentally social — as Lyzasoft CEO Scott Davis observes about data analysis. His blog post got me thinking about this.

Everyone shows up ready to rumble. They’re fascinated with the possibilities, they experiment in private, later they have a blush of quick results followed by a long trail of self-training on finer points.

Each group’s toolset is potent and designed for early success but never early mastery. They make lots of mistakes. In love and analysis, people fall for the wrong data, mess up good data and dates, do all kinds of things they wish they hadn’t.

Without realizing, they face danger. I’ve noticed that behind most good trends comes a rotten sibling right behind it. Think of the history of other social events: Hippies begat the Summer of Love and then came Altamont. We celebrated “free love” and then came a surge of sexually transmitted diseases. Baseball begat the World Series and then came batters on steroids. PageMaker begat self-publishing but then came the ugliest lost-cat posters ever tacked on a telephone pole.

You may already wish that bad analysis would go away. Pete Warden, for one, warns of some fabulous ways people trip over new data. We could easily call this stuff “data porn” and ignore it.

But there are even more treacherous pitfalls. These potent tools can change everything in a flash (at the “speed of thought”). One minute you’re in orbit, and the next minute you wish you were dead. With sex comes the hazard of a painful breakup, and with data analysis comes the danger of unwanted speech that’s too hot for any public platform. Oops!

We have ways to deal with all that, but it’s never pleasant. The rejected lover picks up and leaves, and the analyst just finds his creative viz zapped off the cloud — by those who are themselves learning a new role.

The lover and the analyst both feel hurt, perhaps betrayed. Wasn’t each playing by the rules? Wasn’t each part of the group? Suddenly each one feels rejected for reasons that a hasty explanation doesn’t quite calm the hurt feelings.

In hindsight, we realize we shouldn’t have been surprised. Social pursuits can be like this.

By the way, who said good tools were the end of the story? Well, most vendors did. Some teenagers think so, too. But even slightly more advanced users know that technical proficiency is only the price of entry. We do the real work in many long conversations and collaborations with words, data, gestures, misunderstandings and reconciliations, and on and on.

Here the analogy breaks. The tools will keep getting better while the bodies fall apart. But the lesson’s the same: Tools enable, but conversation — better known in the business world as collaboration — is really at the heart of our work.

What will it be, a “single version of the truth” or unabated proliferation of ad hoc data? It’s a chronic dilemma, and its resolution is crucial to big-box business intelligence. Frank Buytendijk’s new book, his second one, offers a way out of this pickle.

In Dealing with Dilemmas: Where Business Analytics Fall Short (John Wiley & Sons; 2010), Buytendijk — pronounced BOW-ten-dek, according to a Dutch friend of mine — argues that the usual this-or-that, you-or-me, and now-or-then dilemmas may not be the tough choices that they seem to be.

I had known Frank Buytendijk from his two TDWI keynotes, both of which broke down old fences. Then I got to the part of the book where he takes on Michael Porter — author of the essay “What is Strategy?,” in which he defines strategy partly by what a business doesn’t do. Southwest Airlines, for one well known example, offers no reserved seats or meals. But without that distraction, it can fly you on time at a reasonable price.

Porter’s theory, I used to say, is comparable to cropping a photo: emphasize one aspect by trimming others. Then you know what the message is for once and for all, or maybe you have to interpret it, but what you need is all there. Buytendijk’s theory may be more like making a movie. The movie, too, requires the artist to decide on emphasis and exclusion, but in a movie the story plays out over time and through multiple spaces. The movie, too, has a message. But a movie goes this way and that way as it winds toward the end — like a business as it winds through its environment toward a goal.

Of course, a movie requires the artist to think harder. A movie takes shape much more slowly. I’ve only imagined Buytendijk’s principles in practice, but I think that what he prescribes is far more involved the usual strategy formulation.

Buytendijk, in fact, has fun ridiculing strategy-formation executive campouts. Inspiration may strike while they “sing songs around the campfire.” The marshmallows and scotch taste good, but the thinking doesn’t stick, the assumptions are forgotten, and the organization is left to live on slogans.

What’s required, he writes, is deeper understanding of the theory behind the business and the nature of the dilemmas that decision makers face.

While the goal (that we chose) remains intact, and the assumptions remain in place as long as they match reality, we can travel toward our goal, assessing whether options that we create and opportunities that we see fit into the framework. If so, we capitalize on them; if not, we let them go. And the moment assumptions change, we can immediately see which activities do not lead us to the goal anymore, or which activities are lacking in making it to the goal. Choices do not turn into dilemmas.

Choices don’t turn into dilemmas the way “single version of the truth” versus spreadmarts has. The Big Brother version of decision support might have been devised in a campfire sing-along — far away from those who still had work to do.

What would Buytendijk do with that problem? I think he would classify it as a “you-or-me” type. It involves one group against another, usually IT soldiers charged with enforcing a policy against rebel cells armed with spreadmarts. But if either one had a decisive victory, it might spell trouble for the organization.

He prescribes three steps: First, examine your motives. What are you really trying to do? What’s the goal? And so on. Next, communicate. Do not fall for that old slogan of ham-fisted managers, “If you’re not part of the solution, you’re part of the problem.” The solution may be found in conversation. “By being part of the solution from the start,” he writes, “the only angle you will see is your own … Acknowledging there are multiple sides to the story, even if you do not agree, is the key to reconciliation.” Finally, reconcile and synthesize. Opposites — such as love and hate, Tea Bagger and Berkeley liberal, IT soldier and spreadmart rebel — may actually be more alike than you think.

I see what he means when, at the end, he confesses to not understanding the old saying that “you can’t have your cake and eat it too.” It’s stupid, something a burned out school teacher uses to keep order. Aren’t we smarter than that? And if there’s hope for cake, there’s hope that the “single version” and the spreadmarts can live together under one roof.

Business is ultimately not technical but social, is it not? Appropriately, this book is deeply humane and intelligent — expressed in a warm, conversational voice. That alone distinguishes it from most other business books. It eases you through difficult new ways of thinking, through what I think many readers will find is new and unfamiliar territory. Decision makers who are willing to put in the effort to understand it and put it into practice, I think, will find it worthwhile.

Dealing with Dilemmas may never become a mainline classic of the kind Porter wrote. But it will certainly be a favorite among a smart and adventurous few.

Many in the business intelligence industry talk about organizational problems getting in BI’s way, but few talk about them very much.

Scratch the surface of most presentations and conversations — such as last week at the TDWI conference in San Diego — and you find people problems bobbing right up alongside data problems: indifferent executives who undermine BI, short-sighted silo keepers, and IT people who enrage business users with paternalism, to name a few top quirks. If only data were all we had to transform!

One business manger at last week’s TDWI conference in San Diego told me that one of his most daunting tasks during a recent data warehouse implementation was persuading silo managers to release their death grip. For this task, he was on his own. Couldn’t someone have briefed him on the objections he was likely to hear? Or tactics to overcome resistance?

One organization that seems to have solved its people-problem was Blue Cross Blue Shield of Kansas City. Their impressive success with Hewlett-Packard tools was based on commitment to data for strategic advantage and shrewd orchestration. They also had a steady, guiding hand from HP. For example, as Blue Cross Blue Shield built new structures, it avoided upsetting stakeholders by leaving old structures in place for 18 months. (I hope to have much more on that story in the next couple of weeks, thanks to John Santaferraro, HP senior director of marketing, business intelligence.)

Several people in the BI crowd do talk often and thoughtfully about organizational problems. Maureen Clarry, CEO of CONNECT: The Knowledge Network and longtime TDWI instructor, teaches “Power, Politics, and Partnership in Business Intelligence Projects” at every TDWI conference. Participants see for themselves how position shapes behavior. Those short-sighted silo keepers, for example, could flip into data-sharing maniacs if assigned a different position.

Jill Dychè, partner at Baseline Consulting, teaches “BI from Both Sides: Aligning Business and IT,” with strategies to avoid or pave over organizational potholes. She suggests, for example, dodging the perception that BI is “so much data loading and report provisioning.” She writes in email, “We find that the extent to which BI is viewed as a program — with platforms and tools merely components — is the extent to which BI teams are productive and visible in their companies.”

Wayne Eckerson, director of TDWI Research, also addresses these issues, most colorfully with his idea about “purple people.” They are a little bit business-blue and a little bit technology-red, and the purple coloration they acquire lets them traverse the IT-business rivalry.

Wayne spells out some important characteristics for this job, such as maturity and knowledge of technology and business domains. The best are “switch hitters,” by which he probably means to imply that they’re persuasive wherever they stand. In fact, “purple” sounds like a euphemism for another “P” word that Jill actually spells out: politician.

Bad word or not, it’s a critical function. A good politician’s essential function is to coax rivalrous parties into agreement. If that’s the kind of function Wayne sees for the purple people, then they really are, as he says, “the key to BI success” — at least at one level.

Purple may not help much at higher levels. Wayne’s knowledge of of business intelligence is far deeper than mine, but my experience elsewhere makes me think these people are just one of many keys. When I was a sort of purple person myself — in the late ‘90s, bridging an arrogant Web development group and a couple of marketing groups accustomed to full control of their media — my own skill at listening, negotiating, and arm-twisting was only one key. Another key was my boss. At first I had a strong one, later I had an indifferent one, and even later I had virtually no boss at all. I felt like my district shifted boundaries each time, my agenda with it.

One friendly executive suggested I stand up and promote the Web project around the company at any meeting that would let me. He said, “Show ‘em how great it is, and the credit will rub off on you.” Just like a politician running for office.

If I were a purple person today working in BI, where would I go after I’d exhausted training by Maureen, Jill, and Wayne? Most likely, I’d turn for inspiration to books on politics and influence, such as biographies by Robert Caro. Actually, I’ve gone there already, but only because to me politics is a good word. No, you don’t want to emulate Caro’s subjects, just clean and adapt some of the principles they used.

One thing seems clear to me: If purple people, would-be purple people, red people, and blue people are to expand the BI horizon, conversations have to go longer and deeper into the people problems. We start by ending the prissy avoidance of that word that at its best connotes people, perceptions, and compromises: politics!

Lyzasoft wasn’t among the 38 exhibitors in TDWI’s Las Vegas exhibit hall. Lyzasoft sponsored no part of the lunch, and they hired no stage magician. But their buzz was the loudest I heard over the event’s five days.

Others may have heard different buzz because buzz varies. Business intelligence elites gather every year at TDWI’s big Las Vegas event to teach, and they end up schmoozing, too. Over beer, food, and sometimes playing cards, they compare notes.

Is anyone seeking a consensus? I suppose someone might, but the interesting ones just play with ideas, reflect on what others say, make a joke, and think about it. If there’s any “truth,” it develops during a lot of talk and thought, whether it’s about politics, tofu, the future of passenger rail in America, or business. That goes for any kind of conversation, whether the medium is words or data.

In business, the conversation is somehow forgotten in favor of the data. But to Scott Davis, CEO of Lyzasoft, the conversation is critical to understanding the data. “A chart has no context at all,” he said in mid February. “The conversation is what’s really valuable.”

The conversation-free, top-down “single version of the truth” isn’t always useful for those who need to manage data for specific uses and contexts. Its “truth” may in fact be no better than Soviet planners’ forecasts of market demand for women’s lingerie. “A single version of the truth,” said Third Nature research director Mark Madsen in Las Vegas, “is true for a single beat of the corporate heart.”

Enter Lyza 2, Lyzasoft’s new version of its data-wrangling and collaboration tool made for data analysts determined to create truth for specific uses and context. The first edition of Lyza offered Excel-like personalization. In the new edition, collaboration seems to have been the guide.

You could see this year’s improvements coming in last year’s email from Lyzasoft CEO Scott Davis: “Even though they are quants, their world is personal,” he wrote. “Relationships are vital. They think in terms of ‘who do I know who knows X type of information sources?’” He could have also been talking about journalists, artists, and anyone else who has to hear signals within noise.

In the new edition, Lyza encourages fluid interactions with a variety of social-media tools: email, Twitter-like messaging, SMS messaging, bookmark collections with annotations, and other tools track and fortify discussion. Lyza lets people work easily with other smart people they trust. If “Steve” believes that “Brian’s” work is good and “George’s” work is not, he can work with only Brian’s data. It also publishes to the new tool, Lyza Commons, for even greater collaboration while retaining users’ ability to interact with data. Lyza 2 loves a good conversation.

The data and everything that happens to it gets tracked automatically. Unlike in Excel worksheets, changes are transparent. Automatic documenting allows any change to be dug up and fixed. If only the data-free conversations in politics and other parts of business had such a tool.

I was surprised to hear spontaneous praise for Lyza’s new version. eLearningCurve education director Dave Wells and Third Nature principal and one of the event’s keynote speakers Mark Madsen both did. I heard the same from several other BI experts, too. Madsen even gave a brief look at Lyza in his Executive Summit presentation on the future of BI.

I harmonize with people who appreciate Lyza at least partly because I think it’s smart to let people work the way they want to work — the way people have always worked. They prefer working with people they trust and with tools that respond. Everything else is static.

I’m sorry to tell you serious types out there, but visual analysis is often a game — in fact, one of the best games in town with Tableau Software’s visual analysis tool. Now Tableau Public is going to bring it to the masses.

In the same way that YouTube spawned a surge of new filmmakers, Tableau Public — free, running the same engine as its desktop sibling, and embedable — will bring on a new generation of data players and spectators.

I was a spectator at a data visualization conference one afternoon two years ago. Tableau Software director of visual analysis Jock Mackinlay had finished his presentation and another person had started his. Yet someone at the control board forgot to flip a switch, and Jock’s live screen remained on one of the room’s big screens. Jock assumed his screen had been hidden, and he kept playing with the data. I don’t have to tell you who seemed to have the audience’s attention until someone pointed out the problem.

The mere visual distraction was minor. Even without narration, I got caught up in the apparent drama as he tried one look at the data after another.

Not long after that, I wondered aloud to someone at Tableau about data hobbyists. I imagined people who foraged for data to analyze then publicize it to start conversations, collaboration, or duels. Data would be their raw material of choice just as scrap metal is to some sculptors or overheard conversations is to some fiction writers.

There was no such community visible then. But I realized this week that I know one now: Dan Murray, a skilled, dedicated Tableau user. He jokes that he’s a “freak” because he analyzes data from the federal budget and posts his often provocative analyses. He’s already been answered by at least one who disagrees with him.

In beta and since its February 11 launch, Tableau Public has hosted a flurry of visualizations, including these: a map of top venture capital firms investments by U.S. region; a chart showing how long it takes to build a technology empire; a history of earthquakes in Haiti; a neighborhood breakdown of housing supply in Seattle; trends in U.S. high school graduation; and studies of deprivation and marginalization in education. In most cases, spectators can become players by selecting subsets of the data to find answers to their own questions.

With popularity comes some misuse. Many of the charts will break rules, such as what happens in another kind of game, YouTube. A New York film editor I know complains that many YouTube-acculturated film editors have neglected basic editing principles. She writes that they rely so much on special effects that they “can’t put two shots together and have them work as an unembellished edit.” On Tableau Public, there will be pie charts, chart junk, and even baselines that do not start at zero. We’ll survive it.

But what’s all this got to do with the very serious practice of business intelligence?

Like monks must have done when printing presses began producing books for the masses, many priests of business intelligence will stand aside, arms folded in the aspe chapel. But I predict that before long even they will appreciate a wider, deeper pool of analytical talent ripening for training and employment.

I suspect that the new bunch will have been sharpened by the give and take of public exposition. They’ll also learn from playing in a huge community the way artists and craftspeople of all kinds improve their skills when they bump into peers every day.

This is a new clue for the future of BI. It can’t help but improve data analysis in business. So let the games begin.

With all the talk about technology, let’s pause to refresh with three basics — culture, conversation, and collaboration. These basics will take center stage in BI’s future, and they’ll help decide which tools dominate.

Lyzasoft CEO Scott Davis and I have been reading the same book, The Culture of Cities by Lewis Mumford. The great thing about Mumford, said Scott, is his mastery of so many subjects, which lets him see relationships and meaning among facts that might otherwise seem irrelevant to each other.

The intelligence that soaks up through every paragraph makes the book a thoroughly enjoyable and insightful chew.

“Think of the most intelligent people you know,” he said. “Are they intelligent because they have an encyclopedic knowledge of facts? Or is it their ability to see relationships? I would say it’s door number two, right?”

Trouble is, we can’t all be Stephen Hawking or Lewis Mumford. That calls for linking of minds. That is, collaboration and conversation. We have to find a way for normal people to find a way to see relationships and offer them.

Train people?, I suggested. Maybe, he replied, but what would that say about the state of BI? It’s natural to have conversations and collaborate. Relevant knowledge and observations bubble up as people focus on something and make associations, gradually raising the group’s insight.

But today, BI platforms make contributing difficult. “BI designers decide what reports are going to be out there, and that’s the well you can drink from.” Instead, people should be as free as in any conversation to make new syntheses, to comment, to recommend — “all the things you’d expect at a dinner party. BI doesn’t feel like a dinner party, does it?”

“I suspect,” he said, “that even though we don’t see it within formal tools, it really is happening, such as through email, spreadmarts, the water cooler and such.”

First comes culture. “If you don’t have that culture that draws people into that practice [of collaboration],” said Scott, “you can have all the tools in the world and it isn’t going to help.”

Next come tools. “There are technological things we can do to make contributing more routine.” He mentioned two mashup tools he likes: JackBe and NetVibes.

The entrenched players are going to change, he predicts. Either Cognos and BO and the others will become a lot more like JackBe and NetVibes, or else companies like JackBe and NetVibes will become the dominant players.

He said, “Ultimately, open and relational are going to win.”