Quantcast

Data surfing with big data

The usual big-data story leaves out crucial bits. We hear about the “what” — big, huge data of all kinds. We hear about the “when” — now and coming soon. We hear about the “how” — Hadoop with helpers. But we almost never hear about the “who” and the “why.” Who’s bothering to analyze all this data, and why?

If we believe big data’s usual, small-bore spokespeople, the whole thing is little more than getting a big enough machine to crunch mountains of data. But if that’s all it is, then all we have is warmed over business analytics. As I endure minutes upon minutes of Hadoop-speak, I’ve often grumbled to myself that if there’s anything more to this story, I sure wish someone would cough it up.

Finally, someone has. In July, president of BI Research Colin White and director of business analytics at IBM Harriet Fryman gave a refreshing presentation at the annual Pacific Northwest BI Summit, held in Grants Pass, Oregon. Yes, big data’s a big deal.

Though Colin and Harriet listed nine conclusions, I derived my own: Big-data analytics can become more than a cost, it can become a profit center and an asset. Second, high resolution is a better way to think of big data’s function than any others I’ve heard.

Unlike most talks, they supported their thoughts with actual cases — the most interesting of which was Sears, the stumbling brick and mortar chain. In April its big data operation opened its doors as a subsidiary to non-competing retailers. MetaScale’s sole purpose is to help find meaning in big data. A cost center has become a new business.

Like most big deals in business, this one echoes the past: the story of Southern Pacific railroad’s phone system. During its long domination of the West and Southwest, it had built a vast telephone system. By the mid ’70s, a few train crews actually had early mobile phones, bigger than a pork loin. Then cash tightened in the late ’70s, and that cost center became an asset when it became Sprint’s foundation.

The data itself should be thought of differently. Big data’s per-unit value is lower and has an inverse proportion of the volume and value. While we groom and pore over transactional data, with big data we throw the stuff around with shovels. Its value is in bulk because it shows value with patterns. Much of that big data, in fact, may end up discarded.

Dare I compare big data to TV viewing? Faced with either one, we may glance, evaluate, and in a blink decide to discard one sample and dwell on the next. With a remote in hand, we say we’re “channel surfing.” A comparable willingness to load data and discard it could change the whole game of analytics. Jill Dyché said, “‘Here’s the data. Go play.’ ‘Because I can’ isn’t a good reason in data warehousing,” she said, “but in big data it’s perfectly OK.” That, she said, is a “game changer.”

To sense big data’s potential, we may again think of television. The early, vacuum-tube powered TV was monochrome and a little weird. A novelty, but a poor substitute for radio. But by the ’60s, the picture had cleared up and by the mid-’60s shows were seen “in living color.” People hurried home to catch new episodes. Today, we’ve got HD on iPads, and the effects are still unfolding. It’s been video all along, but each improvement changed applications profoundly.

If all today’s experts can do is describe big data in terms of tools they know, of course it will sound like little more than new and improved BI. Big data dares us to think much, much bigger than that. It may challenge our tools for the moment, but in the long run it’s a bigger challenge to our imagination.