Datadoodle

You are here: Home / Archives for big data

big data

Data lake: compositional or architectural?

by

Is the data lake following the typical path for new technology? Merv Adrian, research VP, data management and integration and Gartner was talking about data lakes and big data projects at the just-concluded Pacific Northwest BI and Analytics Summit. Josh Good, senior director of product marketing at Qlik asked the question.

Merv’s answer:

That’s a terrific question. We’re talking about a phenomenon of some recency which is the notion of the new platform sell. [It’s] not a new application, not a new function, but a new platform designed to replace existing ones or supplement them (usually the first until they figure out that’s not practical). And that, I think, is the larger market failure … or the blunting of the thrust that there’s this new opportunity to build new platforms.

I’m relatively convinced that people coming into the market now are not thinking about the replacement of the end to end. They are looking for parts. If they’ve gotten at all sophisticated or knowledgable about how to achieve the outcome that they presumably have defined, then they have put together in their head at least some sort of chart they can draw on the wall, which is a bunch of boxes that connect to one another with flows, and they’re identifying the APIs among them.

That’s becoming an issue especially as we move to the cloud and people start talking about services-based architectures and are thinking about the way they want to get to where they want to go is a composition exercise, not an architecture one.

Schmarzo: An over-infatuation with data

by

The “Dean of Big Data,” Bill Schmarzo, said what I think few of his peers would say on the record: “We have an over-infatuation with data.”

Yet there it was, in the Marriott lobby adjoining last week’s Strata+Hadoop conference in San Jose, CA.

Before you can decide on your data, he insists, you have to decide on your decisions. He said, “That is clearly the biggest challenge in the ‘smart’ conversation,” such as in establishing a “smart city.”

Schmarzo, who is by day Dell EMC CTO, jumped into the “smart city” conversation recently when he joined the San Jose, CA Innovation Advisory Board. His first question was to the airport CTO: what decisions do you face? From there, they could choose data. Must they spot beacons over the terminals, for example, or to use the cheaper and readily available MAC addresses? Such choices could make a difference to that CTO and any tax-starved cities on their way to “smart.”

I asked how people react when he suggests easing up on collecting data willy nilly, consuming it like junk food binges? He paused. “So far, the vast majority are relieved,” he said. “They don’t know what to do with all the data.”

“We over-complicate things, and data vendors are the most guilty,” he said. “They talk about smart this and smart that. And they say ‘You gotta have this technology and these gateways.’ But what will you do with that?”

There’s still one more thing to do: explain the decisions. How can you make the data be made interesting not only for the mayor but the citizens, too? “What I don’t know how to do,” he said, “is how to tell a story around it.”

Uber and AirBNB hint at BI’s future

by

You may have seen yesterday’s New York Times story about ratings on Uber, AirBNB, and similar outfits. Customers rate providers, but providers also rate customers. And guess what: As coarse as a five-point scale may seem, actual ratings tend to be worse. Forget the shades of gray, you’re probably either terrific or terrible.

Welcome to a chilling thought. This is what a “data driven culture” could be among the late adopters— a warning those of us who worship “data-driven culture” should be careful what we wish for. Is the failure of business intelligence’s failure to penetrate beyond the present beachhead such a bad thing? The Times story makes me wonder: Which comes first, tools or intelligence?

As important part of intelligence, we might say, is the ability to perceive subtle meaning. At least that’s the assumption underlying the design of most data analysis tools. No one needs SAS, QlikSense, or any of the others to recognize a gold mine glinting in the sun. But we do need data analysis to reveal the vein that, by the thinnest of margins, could turn a profit. And so-called “big data” offers even finer resolution on that picture, letting analysts study questions from many new aspects.

Perhaps you dispute the Doodler’s point. You say that the Uber example has nothing to do with data analysis. You say that the problem is in data quality! That would imply that those who submit data with hamfisted abandon would analyze that same kind of data with a sensitive touch — very unlikely.

What’s more likely is that, if true, the Uber and AirBNB experience portends a dark future for “data driven” cultures.

Read the Times story here. “Ratings Now Cut Both Ways, So Don’t Sass Your Uber Driver”