August 31, 2015

Big Data’s Dirty Little Secret?

Do you know what you’re doing with your Big Data? Based on my experience, only a small group of companies really do.

There’s just one little problem, as Intel cloud platforms group vice president and general manager Jason Waxman, told investors in a webcast last week.

This is the dirty little secret about big data: No one actually knows what to do with it,” said Waxman, as ComputerWorld reported. Link (emphasis mine)

While I disagree with his statement that no one” actually knows what to do with it, I do agree with his overall premise.

It’s been my experience that many companies adhere to the Volume aspect of the Volume, Variety, and Velocity definition. Typically they’re so scared to miss any potential data point that they just save everything and clutter up their clusters.

What many companies lack is a well thought out Data Policy that builds a cluster with the Right Volume, the Right Variety, and the Right Velocity. Unfortunately, formulating that data policy takes up a lot time and requires broad input from across the organization.

Not only will it require input from the Data Science team (i.e. data analysis), but it will require input from the Line of Business (LOB) groups and corporate strategy/vision from upper management. Sounds great! Let’s get to work, right?

Just ask yourself this: when was the last time all three of these groups aligned with minimal friction?

Answer: Usually never.

Tl;dr: Right Volume > Volume

Don't forget to sign up for our monthly newsletter on Data Science and RapidMiner here!

Big Data thoughts Machine Learning

Previous post
One Year Already? There’s a reason why I’ve been neglecting this blog, it’s because I’ve been working my tail off! I’ve been so busy that I blew right through my one
Next post
Yet Another Language to Learn In today’s Advanced Analytic landscape we often hear about the mythical Data Scientist that can cure all our problems and unearth value hidden deep