Wednesday, March 27, 2013

How headlines misrepresent data

Have a look at this TIME article. The headline is "Americans Are Eating Fewer Calories, So Why Are We Still Obese?".

It goes on to cite two studies. One study shows that old people eat less fast food then young adults, and that in general food consumption is declining. This comes from the aging population I guess. The other shows a small decline (about 7%) in the calorie consumption of children. 

Neither study showed a decrease in calorie consumption by adult Americans, as far as I can tell. The naive assumption would be that consuming fewer calories would result in less obesity, but no data on the topic is presented. The headline is a complete misdirection.

The moral of the story is you should limit your observations to what the data actually says.

Thursday, March 14, 2013

BI Trends for 2013

Analysts like to provide a list of predictions for the coming year. I've never been a big fan because I think that real tipping points are pretty rare in any industry. People tend to overestimate the short-term change and underestimate long-term change.

Here are some trend ideas that are floating around the internet. There are basically three kinds:

New data sources: This mostly means Big Data, whatever that means. (Personally, I like Amazon's definition, which is "too much data to handle on a single server". Other definitions strive to include appliances like Teradata and HANA.)

New applications: Sentiment analysis, predictive analytics, collaboration

New UIs and functions: Better dashboards and visualizations, more self service and agility, voice interfaces, mobile.

New Platforms: This is mostly in memory, cloud and SaaS.

My take on all this is simple:

New data sources: I think big data is still mostly for online and mobile providers. It's true that manufacturers and retailers are trying to figure out how to make better use of the huge amounts of data their business directly and indirectly generates. But this business is still heavily dependent on boutique providers that bring a lot of domain knowledge and deep understanding of statistics with into the deal. I do not think it will have much impact on existing BI business. It's something different.

New applications: The same remarks apply to sentiment analysis as to big data. Predictive analytics is a more interesting market, but to find large, non-specialist audiences, vendors need to prove that their "black box" predictions are as reliable as the expertise of business users without explaining the math behind them.

New user interfaces and functions continue to appear, but I believe that as long as most BI companies stick to solving the easy problems, like making software look cool in a demo, and ignore the harder problems, like user-friendly data governance, there will be no big changes here. Mobile has surprised me, but it still hasn't made a big difference in the BI business.

New platforms. It is good to remember that business users don't care what platform is used, and that the most successful projects are controlled by business users. A platform is only good in the sense it delivers things like speed and convenience. It doesn't add any value per se.