Long before Scott Davis made the self-service ETL tool he calls Lyza, he tried to find out how analysts really work. He remembers in particular the woman in a focus group who said, “I want to stay close to the data.”
He didn’t understand at first. The data was right in front of her, neatly summarized. But she meant all of the data, every little bit of it. She wanted to snap open a zillion-row-long window that she could scroll down to see the figures flip by. (Yes, you can; I saw it yesterday.) She wouldn’t try to read them, she’d only see their shapes. She could say, for example, “Hmm, I see that just two thirds are under 1000.” Davis calls that visualization with browse—as legitimate a use of “visualization” as any I’ve heard of.
He also thought about how people use Excel. In fact, it helps explain’s Excel’s popularity. They have the data, and they have the formulas, and you can reveal either one. If a number shows up that doesn’t look right—say it’s six figures instead of five—you just look at the formula. You say, “Oh, that’s the annual figure. I forgot to divide by twelve.”
Something similar goes on at all levels of analysis: a rapid back and forth from question to answer, back to a rephrased question, and back to an adjusted answer.
Forget the flow charts. Forget the “data train,” a metaphor I admit to having used. Analysis is more like what my labrador does when she knows there’s something good nearby. She sniffs in what looks like a random pattern until you realize she’s narrowing the range.
What drives analysts crazy about working with IT, he says, is that the data’s taken away. The conversation goes like this: the IT guy asks what the analyst wants; the analytst describes her best guess; the IT guy goes away and does it. But that may not be what the analyst really needed, and the anallyst may not realize it until the first data’s tried and proves inadequate or suggests yet another path.
I can relate, because it’s like writing. I do a lot of scribbling and writing over, and I don’t have time to explain it. If I had to tell a typist what to write, I’d write much less.
Visualize the bumper stickers: “free the analysts” but also “free IT.”
Now, Larissa T. Moss has her doubts. Perhaps she’ll sit for a demo. I’d like to hear what she says.
Barbara makes an excellent point. I remember the world of keypunching in hundreds of rows. I also want to specifically respond to her question about “making sense of connection and understanding.” Implementing BI without at least basic knowledge of basic data warehouse structure, star schema AND the use of new data visualization tools (like Tableau Desktop) the analyst would have a difficult time maintaining the data intimacy that Barbara describes. However, the good news is that this knowledge is attainable to the data analyst and that person doesn’t have to be an “IT guy.”
Ah, this post brings back memories of being a business analyst at a strategic consulting firm. Fresh out of college, toiling over an Excel model for days and nights on end, having to do ungodly acts like take mainframe printouts (with the green and white bands and perforated edges, remember?), retype the data into Excel, do the analysis, and then create pretty charts to cut and paste into PowerPoint.
I’m thankful that we’ve evolved past that point. However, it was an experience that made me love data and information. When I learned how to use Access, I really didn’t like the black box nature of it. With Excel, it was manual and painful, but you got to see how the sausage got made. You didn’t just see the numbers, you *knew* the numbers. Slight variations or unexpected surprises made hairs rise on the back of your neck, so that you’d go and root out the cause of the disturbance.
Of course, I wasn’t working with gigabyte and terabyte data stores, at which point it becomes completely flummoxing to swim in all of that data and you need something bigger, like a true BI solution. Which raises an interesting question – how can you make sense of that scale of data and still remain a sense of connection and understanding? Is the move towards greater self-service BI an answer to that, so that people who create and need that data can work with it instead of the IT person who is one step removed?