Sometimes I get annoyed by all the pious statistician-types I find all around me. They aren’t all statisticians, but there are a lot of people who raise analytics and “data-driven” to the level of a holy activity. It isn’t that I don’t like analytics. I use statistics whenever it is a cost of doing business. You’d be dumb to not take advantages of ideas like A/B testing for messy questions.
What bothers me is that there are a lot of people who use statistics as an excuse to avoid thinking. Why think about what ONE case means, when you can create 25 cases using brute force, and code, classify, cluster, correlate and regress your way to apparent insight?
This kind of thinking is tempting, but is dangerous. I constantly remind myself of the value of the other approach to dealing with data: hard, break-out-in-a-sweat thinking about what ONE case means. No rules, no formulas. Just thinking. I call this “learning from one data point.” It is a crucially important skill because by the time a statistically significant amount of data is in, the relevant window of opportunity might be gone.