Search all the business literature you can and you’ll never find data analysis compared to romantic love. But, hey, why not? Love’s trajectories might hint at what the business world’s newly enabled generation of data analysts can expect.

These data analysts tend to be independent, are often creative and at least partly self-trained. They’re strapped to rockets from Tableau, Lyzasoft, Predixion, and others, tools that are at first deceptively toy-like. Aren’t they analogous to the garden variety teenager? Bothg groups revel in newly discovered tools, while both pursuits are fundamentally social — as Lyzasoft CEO Scott Davis observes about data analysis. His blog post got me thinking about this.

Everyone shows up ready to rumble. They’re fascinated with the possibilities, they experiment in private, later they have a blush of quick results followed by a long trail of self-training on finer points.

Each group’s toolset is potent and designed for early success but never early mastery. They make lots of mistakes. In love and analysis, people fall for the wrong data, mess up good data and dates, do all kinds of things they wish they hadn’t.

Without realizing, they face danger. I’ve noticed that behind most good trends comes a rotten sibling right behind it. Think of the history of other social events: Hippies begat the Summer of Love and then came Altamont. We celebrated “free love” and then came a surge of sexually transmitted diseases. Baseball begat the World Series and then came batters on steroids. PageMaker begat self-publishing but then came the ugliest lost-cat posters ever tacked on a telephone pole.

You may already wish that bad analysis would go away. Pete Warden, for one, warns of some fabulous ways people trip over new data. We could easily call this stuff “data porn” and ignore it.

But there are even more treacherous pitfalls. These potent tools can change everything in a flash (at the “speed of thought”). One minute you’re in orbit, and the next minute you wish you were dead. With sex comes the hazard of a painful breakup, and with data analysis comes the danger of unwanted speech that’s too hot for any public platform. Oops!

We have ways to deal with all that, but it’s never pleasant. The rejected lover picks up and leaves, and the analyst just finds his creative viz zapped off the cloud — by those who are themselves learning a new role.

The lover and the analyst both feel hurt, perhaps betrayed. Wasn’t each playing by the rules? Wasn’t each part of the group? Suddenly each one feels rejected for reasons that a hasty explanation doesn’t quite calm the hurt feelings.

In hindsight, we realize we shouldn’t have been surprised. Social pursuits can be like this.

By the way, who said good tools were the end of the story? Well, most vendors did. Some teenagers think so, too. But even slightly more advanced users know that technical proficiency is only the price of entry. We do the real work in many long conversations and collaborations with words, data, gestures, misunderstandings and reconciliations, and on and on.

Here the analogy breaks. The tools will keep getting better while the bodies fall apart. But the lesson’s the same: Tools enable, but conversation — better known in the business world as collaboration — is really at the heart of our work.

It was International Workers’ Day on Saturday and the official release day of Lyzasoft’s latest product: its foray into “free.” It’s a good way to say “power to the people.”

Some people associate that slogan with protests and even violence. But I think the best paths to power usually involve well-analyzed data, whether in public life or at work. Now the Little Guy has a potent new tool to deploy.

Lyzasoft founder Scott Davis calls Lyza on Lyza Commons “the YouTube of data.” This fully functional cloud-based version of Lyza is a strong tool for office-based, home-based, cubbyhole-based, dorm-based, or public library wifi-based users and groups. Import your data from whatever sources you have, refine it, share it with to whomever you like, and even charge toll over Paypal if you want to.

“Obviously,” Scott says, “what we’re doing is saying, ‘This thing can scale.’ But instead of going for the uber-enterprise as our leading play, we’re saying that what’s unique about this technology is it can make it to everybody within a small and medium business without having to have a big IT team around.”

Lyzasoft’s second, paid tier serves customers who need private clouds. That version starts at “small” for $150 a month, seating up to 10 users and providing “plenty” of storage. Go upward through “medium” and into “large,” and you pay $2500 a month for up to 200 users.

Wait, you say. You’ve heard this “YouTube of data” thing before. Yes, just three months ago another YouTube of data launched, Tableau Public. (I wrote about it here.) Tableau, Lyza, and YouTube itself all say “power to the people” by popularizing a medium with free, easy-to-use tools and a venue. Each one’s growing crowd of Little Guys and their audiences turns into a movement that the those in executive suites can’t help but notice. At some point, YouTube and those who follow its model hope that “free” leads enough customers to “ka-ching” to yield a profit.

YouTube seems to be well on the way. Its ready-to-roll movie theater had fired the imaginations in a waiting mob. These filmmakers-to-be had been trained over years of TV and movies to understand film and crave a chance to do their own.

Is there a waiting mob of would-be data analysts? One pioneer of free analytics is skeptical. LucidEra founder Ken Rudin, now vice president of analytics at Zynga, says you need more than free tools, no matter how easy the tools are to use. He says, “Tools are only as valuable as the questions you ask.” One of his biggest hurdles was getting customers to appreciate the possibilities of analytics.

But the YouTube idea is more than tools. It’s a game and a self-reinforcing mob. The tiny films YouTube users make don’t just play as if on a jukebox, they’re scored, they’re answered, and commented on. It’s like the difference between voting in a little booth and going out on a street march. It reinforces and stimulates. Unlike most business environments, it asks people to play, which is how Lyza Commons and Tableau Public users will break out into creative and incisive data analysis.

I also hope there’s a new supply of analysts. Ken Rudin and others are hungry for them. (In fact, if you’re a data analyst and you want to work with cutting-edge technology and data in one of the world’s largest databases, email Ken today at krudin@zynga.com.)

Power to the data analysts!

Lyzasoft wasn’t among the 38 exhibitors in TDWI’s Las Vegas exhibit hall. Lyzasoft sponsored no part of the lunch, and they hired no stage magician. But their buzz was the loudest I heard over the event’s five days.

Others may have heard different buzz because buzz varies. Business intelligence elites gather every year at TDWI’s big Las Vegas event to teach, and they end up schmoozing, too. Over beer, food, and sometimes playing cards, they compare notes.

Is anyone seeking a consensus? I suppose someone might, but the interesting ones just play with ideas, reflect on what others say, make a joke, and think about it. If there’s any “truth,” it develops during a lot of talk and thought, whether it’s about politics, tofu, the future of passenger rail in America, or business. That goes for any kind of conversation, whether the medium is words or data.

In business, the conversation is somehow forgotten in favor of the data. But to Scott Davis, CEO of Lyzasoft, the conversation is critical to understanding the data. “A chart has no context at all,” he said in mid February. “The conversation is what’s really valuable.”

The conversation-free, top-down “single version of the truth” isn’t always useful for those who need to manage data for specific uses and contexts. Its “truth” may in fact be no better than Soviet planners’ forecasts of market demand for women’s lingerie. “A single version of the truth,” said Third Nature research director Mark Madsen in Las Vegas, “is true for a single beat of the corporate heart.”

Enter Lyza 2, Lyzasoft’s new version of its data-wrangling and collaboration tool made for data analysts determined to create truth for specific uses and context. The first edition of Lyza offered Excel-like personalization. In the new edition, collaboration seems to have been the guide.

You could see this year’s improvements coming in last year’s email from Lyzasoft CEO Scott Davis: “Even though they are quants, their world is personal,” he wrote. “Relationships are vital. They think in terms of ‘who do I know who knows X type of information sources?’” He could have also been talking about journalists, artists, and anyone else who has to hear signals within noise.

In the new edition, Lyza encourages fluid interactions with a variety of social-media tools: email, Twitter-like messaging, SMS messaging, bookmark collections with annotations, and other tools track and fortify discussion. Lyza lets people work easily with other smart people they trust. If “Steve” believes that “Brian’s” work is good and “George’s” work is not, he can work with only Brian’s data. It also publishes to the new tool, Lyza Commons, for even greater collaboration while retaining users’ ability to interact with data. Lyza 2 loves a good conversation.

The data and everything that happens to it gets tracked automatically. Unlike in Excel worksheets, changes are transparent. Automatic documenting allows any change to be dug up and fixed. If only the data-free conversations in politics and other parts of business had such a tool.

I was surprised to hear spontaneous praise for Lyza’s new version. eLearningCurve education director Dave Wells and Third Nature principal and one of the event’s keynote speakers Mark Madsen both did. I heard the same from several other BI experts, too. Madsen even gave a brief look at Lyza in his Executive Summit presentation on the future of BI.

I harmonize with people who appreciate Lyza at least partly because I think it’s smart to let people work the way they want to work — the way people have always worked. They prefer working with people they trust and with tools that respond. Everything else is static.

I like the sound of Government 2.0: Collaborate with citizens online and you can change government from a sewer-dwelling raccoon into a purring housecat.

Social media lets us try for a kind of politics that was impossible until now. I hope for great results. For many, Government 2.0, or “collaborative government,” will mean just “friending” a local cop. But in full flower, Government 2.0 can mean far better service, and far more government-and-citizen collaboration than ever before.

Even before we had social media, the glare of public attention was a proven antidote for bad politics. Citizens getting up their elbows in policymaking has always been another strong medicine.

Trouble is, that “sewer-dwelling raccoon” is always smarter than people think. When he’s hungry, he purrs like a housecat and covers stinky laws with high-minded names. Advertising fools just enough voters — so often complacent and impatient — to throw a new law onto the books. On and on it goes.

Such a stinky new law is just what Californians got in 2000. Proposition 34 was sold to voters as campaign finance reform. It was a ruse. (A few days ago, a report confirmed suspicions, and a major drafter of the proposition insisted he was “outraged.” Yeah, and round up the usual suspects.)

One other fix, more honest, came 100 years ago: California amended its constitution to give citizens the ballot proposition. It was the only way for voters to bypass the paralyzed Legislature and loosen the Southern Pacific Railroad’s grip. It worked. But more recently, ballot propositions have helped tie the state’s budget in knots.

In the long run, who knows how social media, visual analysis, and other tools may be used in government? What will matter most of all is who uses them. If it’s “the people,” which people?

I hope this new, pervasive politics mobilizes a new wave of smart activists — the way desktop publishing and, later, weblogs enabled editors and writers. Or the way tools like Tableau and Lyza are enabling independent-minded, creative analysts today.

As these activists learn about politics, I also hope that more citizens than ever before step up to watch, push, and verify. Such a voter would be Citizen 2.0, the real hope.

Otherwise, it’s going to be that raccoon again — this time on Twitter.

What’s the hardest part of training a new data analyst? Resetting the trainee’s mindset.

“They start out with the idea that there’s a right answer,” says Joe Mako.

Joe’s leaving his job — where about one year ago he began analyzing data — to go work for the producer of Lyza. Lyzasoft CEO Scott Davis sees him as a “prototype” of a kind of creative, resourceful analyst that Lyza was designed for. Joe will engage with other analysts to evangelize Lyza and to help new users ease into the flow.

Joe, 29 and a veteran of two Army tours in Iraq, started out on the help desk. He answered calls from within the company, an ISP. Many callers couldn’t or wouldn’t analyze their own data, so Joe did it for them. His boss also enlisted his help — and now won’t dare go without a backup.

The first people he’ll help get into the flow are the two women who’re replacing him, and he’s got to do before he starts at Lyzasoft on November 9. They’re some of only a few in the his group who applied. Most others refused the “boring” work with “ugly” data.

New users, he says, want to know, “Where’s my wizard?” There is none. “But that’s why I enjoy these tools.” He uses Lyza and Tableau primarily. “They stay out of my way. They enable me. It’s just me and the data. … That’s what’s neat. But [new users] don’t know where to start.”

“I’m handed crazy files without any structure,” he says. The first thing new users have to know is that, no matter how ugly the data may be, it really can be cleaned up. He demonstrated to his new trainees, he says, and “they were blown away.” After that, he started showing them how they can clean up data on their own.

He explained basic steps and functions. Then he showed them how to combine tools, such as how to use two functions in sequence. And deeper still.

“It takes time playing to figure out where you need to get to,” he says. “You have to just go and play. If one thing doesn’t work, you try something else.”

“I always thought that data was exact,” he says. “If not, it was garbage and I’d throw it out.” But he later learned that there’s usually only a portion that’s garbage — that somewhere within the crazy mess there’s a story. “Even if every data point is wrong, there still might be some trend you can see. If there’s a bunch of ugly data, how do you figure what he story is?” It takes a willingness to figure it out, to untangle it, to find out what’s in there.

That’s a skill, not a talent, he says. “I’ve watched [his two replacements] get it closer and closer, learning to merge other data in, to reshape it and finally produce the output.”

Closer and closer. Business will trudge ahead, training a Joe here and a Joe there until people don’t complain anymore about boring work with ugly data. Someday, many more people will welcome the chance to do this work.

I hear a story like the one I heard this week and I want to ask the apparent villain why. There must be a reasonable explanation.

At first glance, he’s like other managers I’ve known of who throttle promising work for what seems like a personal need for control. “So tell me,” I’d like to say over beers, “what were you thinking when you denied that analyst free access to that data? What’s your side of it?”

If beer didn’t work, I could imagine waterboarding, even though his explanation would then be less reliable.

Here’s the story: One young analyst’s boss wanted an answer to a simple question: How many customers are receiving premium service and paying standard rates? (Sorry, I can’t identify the company.)

The analyst had little training. He was only the most curious, persistent, and resourceful member of his department. He also knew a little SQL. But, as he puts it, “I didn’t know a dimension from a measure.” But despite scant knowledge and many obstacles, in a few months he won the company just over $500,000 in unforeseen revenue the first year.

First stop: the business intelligence system. But problems showed up quickly. While the billing database — the only one within the BI system — listed about 20,000 customers receiving premium service, the installation database listed 25,000, and the customer database listed 30,000. Each should have had the same number.

He knew why they contained bad data. Each new customer triggered a 45-step process that required customer service representatives to enter numbers in correct fields and to choose from drop-down menus. Errors came easily.

The IT manager seemed to shrug at this. He refused the analyst access to the desktop client, which would have allowed him to join the other two databases. All the analyst could do was review each record one at a time. When the analyst asked for help to learn the BI product, help was denied. Nor could he have access to the vendor’s help resources.

“They say it’s their database and they don’t want anyone running queries against it,” he said. “They’re afraid it would impact the performance. But all I want to do is pull a table out. Give me an export! I’m not asking for a complex query. It’s not sensitive data. In fact, anyone in the company can get at it. They have a strange philosophy.”

He admits his next move was sneaky, but justified. He used an SQL injection, a technique for unauthorized entry, to extract a table from the customer database. That database, he said, looked to him like a “10-year-old’s work,” and had no security built in. The administrator caught on when the analyst slipped up and ran an update. The administrator loudly threatened, the analyst said, “to have me listed as a terrorist.”

A loyal insurgent, maybe. Except for the accidental update, he never altered records. He’s only sent batches of corrected records to other departments.

“My boss was able to prove that I had done a great deal of good stuff in cleaning up the database,” he said.

The greatest benefit was identifying 2200 customers who received premium service but only paid for standard service. That list went to the billing department. Every one of those customers began paying more, at an average of about $20 a month.

Even so, approval for the relatively small amounts to buy the two tools he’s found most useful in trial versions, Lyza and Tableau, has taken months.

Insurgents often annoy veterans with their ideas and can-do spirit. I can imagine what the IT manager might say in a candid moment. “The little bastard thought he knew more than we did,” for example. “He would just mess things up.”

Or he might say, “All our data’s screwed up. You think I’m going to let him see all that, tell his boss, and let everybody know?”

He might also say, “I’m a jerk. I’ve always been one.”

Next stop: reasonable and articulate friends of mine who work in IT.

I picked up the term “creative analyst” in late June on the phone with Lyzasoft CEO Scott Davis. But what does he mean?

He described one analyst he’s known of. This guy arrived at a new job with strong recommendations for his ability to tear apart a dataset. He could slice, dice, build related charts and pivot tables — but only with canned data. That is, data someone had given him. This analyst struggled with synthesis — blending separate datasets, for example, or making a formula to derive values, or simply experimenting and asking unforeseen questions.

The ability to improvise and create something new is a “prime differentiator” among analysts, says Davis.

Many of these creative, synthesizing analysts, he says, also tend to feel they have a personal brand. They have a style of charting they prefer, for example, and they produce a distinct set of information that is uniquely attractive to their subscribers within the company.

“You can sort of think of them as publishers,” he says. “They create these things that are in some ways more useful than reports from the BI tool. And they gauge their effectiveness by how many people follow them.”

Such lists have been around quite a while. Before PCs, people did the same kind of thing in hardcopy, producing a dozen or two binders with a distribution list clipped on the cover.

There’s a future, too. Davis expects to see Enterprise 2.0 — social networking within businesses — grow fastest among these analysts. They already have the social habits: commenting, trust, wikis, etc.

He says, “A spreadmart is nothing but a primitive social networking mechanism.”

Back in February when I heard about Lyza, I thought right away of Tableau. Despite each one’s different strengths in data discovery and analysis, each appeals to the same broad group.

It’s an old group that’s getting new attention: creative analysts, or “cowboy analysts” to some. The like their data raw, not aggregated. They ask questions, forage, synthesize, analyze, and publish.

Joe Mako is one of them. Tomorrow, he’s launching a website for people like himself who use both Tableau and Lyza. Makometrics will publish every Monday morning and sometimes more often.

Joe is a network engineer at a Midwest ISP. He started at the tech support desk, where he saw how much help people needed looking at their data. “They didn’t understand exploring data,” he says. “They just don’t care.” But Joe cared enough to help with data analysis, and pretty soon someone gave him a tag line: “Make it happen with Mako.”

Posts he’s lined up so far:

• He’ll walk through data analysis problems from challenge to resolution. “I’ll be practicing something akin to the cycle of visual analysis.” (See the Tableau video “The Zen of Visual Analysis.”)
• Analysis of strengths and weaknesses of Tableau and Lyza
• Analysis of his “Visualizing Rambo Kills”: how he approached the dataset, and how he created the final result.
• Demonstrate sophisticated techniques in Lyza and Tableau. He’ll go into detail on such things as combining Lyza’s “previous” and “if” functions and the basics of summary functions like “sumcolumn” and “avgcolumn.”

Check it tomorrow (Wednesday, July 1): makometrics.com.

Swap out a few terms in a recent New York Times story about farmers’ attempt to split California, and you might see the IT vs. business saga.

A quote halfway through caught my eye: “The agricultural industry is in this mode that says, ‘You will eat what’s put in front of you,’ and that’s a very condescending view of consumers and eaters. If customers are changing their preferences, the industry needs to change its ways.” Earlier in the story, an old farmer complains, “Those Hollywood types don’t have any idea what’s going on out here on the farms.”

Big Agriculture — affectionately shortened to Big Ag — seems analogous to the enigmatic Big BI. We can map food shoppers to business users, and back again.

The vast, quietly desperate majority settle for pre-packaged solutions, whether frozen dinners or standard reports. Most see little reason to change, or else they hate it all but see no way out. Locally grown produce? New ways of looking at the data? Yeah, right.

Some turn to alternatives like LucidEra, DataSelf and Birst, the on-demand chefs. They’re for people with food in the kitchen, of any origin, who don’t know what to do with it all or don’t have time.

Then we have the do-it-yourselfers. Those who insist they “need nothing fancy” prefer Excel, perhaps with the Veg-O-Matic add-in for slicing and dicing.

What to some food users is “fancy” is to others a cause to celebrate, and tools like Tableau Desktop and Lyza stay as close as the chef’s knife.

Tableau users are the Alice Waters of the data crowd: foraging local and remote sources, tasting, combining and trying out new concepts every day on friends in the kitchen and often at dinner parties. To them, data’s more than food, it’s the fire, too.

Lyza users also forage for eventual presentation, but at heart they may be more like old-time homesteaders. They thrash, mill and grind to extract the best they can get. They’ll wallow in muddy data if it means finding something better there. They’d rather do that than let someone else do it for them.

Meanwhile, back at Big Ag, the old farmer’s been out at shopping malls testing support for his initiative to split the state in two. He says, “I’m an old hound dog. If I’m barking up a tree, I want to know how many squirrels are up there.” I think he’s barking at data, not food.

Read a pile of technology marketing and you quickly assume that you alone despise many of the words you keep hearing. They’re words like optimize, leverage, synergy, and utilize. People in this industry don’t really talk like that, do they?

Many don’t, at least not in private, and they don’t tweet like that, either. One tweet trail at Gartner BI Summit complained about exactly this kind of word — these miserable words with all the wild flavor bred out of them like factory-farm tomatoes.

On the list of suggested extinction, Jill Dychè listed optimize and fact-based. Scott Davis listed leverage, co-optition, and dot-bomb. Someone also threw in win-win, synergy, and the lovely utilize.

Mark Madsen cautioned that banning all those words would leave marketing with nothing but proper names and prepositions.

The whole discussion started off when Dychè posted a link to David Silverman’s article in Harvard Business “10 Business Words to Ban.”

Now Dave Wells has followed with a weblog post. He left the TDWI conference in Las Vegas last week with his head “afloat in buzzwords.” So many new terms every quarter, and so much ambiguity. ” Maybe its time that we define our terms and differentiate between similar sounding terms.” He goes on to list a few he’d like to see on the endangered list.

Let the movement flourish.