What’s up with “deep data”?

“Deep data” is just another fancy phrase one particular professional coined to describe the digital information that actually matters.

For all the insight data analysis software can provide, you’re probably scrutinizing more information than what’s necessary. Some scientists may tell you to cast a wide net and pull in as big of a haul as your data warehouses can handle, but that’s not what I’m going to tell you.

Choose your sources wisely
InformationWeek contributor Jeff Bertolucci interviewed Badri Raghavan, CTO of FirstFuel, a firm that uses data analytics tools to help enterprises run buildings in a more energy efficient manner. During their discussion, Raghavan noted the importance of selecting resources, using his company’s own protocol as an example.

Basically, FirstFuel will scrutinize meter data by launching its own algorithms, which in turn identify a facility’s pain points. It can answer questions such as:

  • Which part of the building uses the most electricity?
  • Which energy components can be shut off during off-hours?
  • What kind of light bulbs do certain areas use?

“A given data stream can be very informative,” he told Bertolucci. “Conversely, you can collect a lot of data that’s not particularly insightful or informative.”

Define “informative”
Huffington Post contributor Otto Scharmer asserted that deep data helps systems, individuals and communities “see themselves,” whereas big data consists of surface-level information that “bombards” us with unwanted advertisements.

While Scharmer focuses on the social implications of using big data and discusses the empowerment of consumers, he’s also inadvertently discussing a more complex concept. Essentially, one instance of data production – say, whenever I post a comment on Facebook – is itself a huge repository of information containing more insightful meaning. Language itself is quite complex and intricate, so imagine how much you could learn about me simply by scrutinizing 10,000 posts I’ve made in under five minutes.

That’s the kind of knowledge advanced analysis tools can provide you (when they’re directed toward the appropriate sources).