So many people who should know better seem to miss the point when they mention Tableau. Why? I asked BI veteran Stephen McDaniel for his thoughts — which he gave, but then went on to suggest an almost unheard of challenge: a data analysis face-off among vendors.
Consider this description by a BI analyst: “Tableau provides business analysts speed of thought visual analysis on data held in memory on their desktop machines.” All that’s fine, but it may as well have been about a whole bunch of other tools, too.
At the root of this fuzz, explained McDaniel, is that most analysts who concern themselves with tools don’t actually use the tools. They rely on demos , marketing, and hearsay.
Though much of McDaniel’s recent work has centered on Tableau — his second book is Rapid Graphs with Tableau Software and he gives training sessions around the country — he also has a long, credible trail back through BI and data mining. He was director of analytics at Netflix, and has worked with more than 50 companies in BI. His first book was SAS for Dummies.
“I love SAS,” he says. Still, he remembers his sister in law’s reaction to his book on SAS. She was not an analyst but a “people manager.” These are the ones, he says, who have hated BI because “it had been made into a priesthood.” When she had looked through the book, she said, “Oh, this is great” and put it down. But she read the Tableau book for a half hour and said, “You should come talk to some people I work with.” She had recognized what she could do with the tool.
McDaniel’s sister in law and many like her don’t care whether the data is “in memory,” they don’t see themselves as business analysts, they take “desktop” for granted, and they know “speed of thought” is just gloss.
The list of features really doesn’t matter. All that really matters is whether someone can do what needs to be done with the tool.
McDaniel imagines a throw down, a data analysis match. It would be open to any BI vendor. Each vendor would send their best people, and each team would receive a uniform set of data. Over some defined period, teams would analyze and then present the results to a panel of vendor-neutral judges.
The reward? Perhaps a signed copy of a Stephen McDaniel book, or maybe a beer, possibly both. But certainly, repute.
What do you think of the face-off idea? Please write a comment.
Great idea, but I see a slight problem with suggestion from Alex – one of the biggest advantages of Tableau (and probably a few other tools) is its speed. If analysts have all the time in the world and take a few months to set up and configure the infrastructure, then some bulky, expensive and slow tools can do some amazing things. But is it a fair comparison?
Rather than judge the tool and the data analytics ability of the BI vendor, I would rather see the same group of analysts (not affiliated with a vendor) take on each BI tool and produce the best result they could with the same data.
Combine this with user submitted entries (from the same dataset) and then I think you could begin to form some great conclusions.
I agree with Morgan Bates that a face-off between users would be much more useful: can users actually use the software? Is it intuitive? Can they quickly configure the reports they need for their business? Or will the tools drive them back to the de-facto tool of choice: Excel?
Good idea, agree with Morgan. Besides that a test with regular users would make it more realistic, it would eliminate the need of convincing the vendors to participate. This way a selected set of tools could be judged independently from the vendors.
Great idea, and I’d jump on the challenge.
One caveat, however: the “uniform” set of data needs to be real-world….messy data typing, heterogeneously aggregated, disjoint hierarchies, undocumented, mixture of normalized and denormalized, dispersed across multiple silos/applications, too big for in-memory, etc.
Just a suggestion, since we’re talking about real-world usefulness.
I am not sure that having a vendor send their specialist would really prove the deeper value of the tool. As you say, most of the BI landscapre created a “priesthood” that did not make sense for most users. I would rather see a face-off of regular users who are all starting from scratch. Can they take an unfamiliar tool to answer a series of questions about the data set.
Thanks Ted, I appreciate the work on your site. You drive home a frequently overlooked point in the business intelligence world; time is of the essence for many problems people hope to solve with BI tools. While there are apps that possess some of the key features found in Tableau, I sincerely believe that none of them can achieve exceptional insights in short order.
Feature lists are what third-party analysts and vendor evaluation scorecards often rely upon in determining “winners” in product bake-offs. While I agree that features do matter, I have learned by working with many customers that it matters even more if they enable rapid insights that can quickly answer a stream of relevant questions. Tableau has done an exceptional job of achieving this feat, all achieved via a relentless focus on the analytic workflow and user perception of information.
I would love to see multiple BI vendors accept this challenge. A third party will select two “real-world” datasets, from finance and marketing. Three business questions will be asked per dataset with just four hours to create our best solutions. This effort of each BI vendor analyst would be recorded, allowing the world to see what is truly possible in short order with these tools.
The fruits of our labor would be posted to your site. You would ask several marketing and finance subject matter experts to review the entries. Of course, none of the judges would work for a vendor involved in this contest. The results will also offer an opportunity for the world to view actual results to the same question created in the same time frame.
The winner would garner more than a copy of my book; they will have proven the worth of their product in a fair, open competition. A great prize indeed!
Ted,
Excellent idea – Stephen McDaniel’s ideas always are. Perhaps you and some cronies could document the process of getting to the “results,” with each of you assigned to one vendor. Count me in to help in some way.
MANY BLESSINGS!
Peace and All Good!
Michael W Cristiani