From Unstructured DataJune 25, 2012, 8:29 PMStep 1: Create a monster by networking 16,000 ultrafast computer processors.Step 2: Feed the monster 10 billion images chosen at random from YouTube videos.Step 3: See what happens.What Happened: The monster taught itself to recognize cats.“We never told it during the training, ‘This is a cat.’ It basically invented the concept of a cat.” – Jeff Dean, speaking for the scientists at Google’s secretive X-LabsThe frightening part of this report is that modern computers appear to be capable of independent learning through extrapolation.The comforting part of this report is that it takes 16,000 ultrafast processors working together to do something that’s completely effortless for a human toddler.I drive 40 minutes to meet Jeffrey Eisenberg for lunch in a place that looks like it used to be a Denny’s. I hand him an advance copy of Pendulum. “Hold it up next to your face,” I said. He held it up and smiled. [click]Jeff laid the book on the table and thumbed through it, “This really turned out nice.”“So tell me what’s happening in Jeff-world.”Jeff said he was developing applications of big data for some of America’s largest companies.“What’s big data?” I asked.“You’ve been teaching Practical Applications of Chaos Theory in the Magical Worlds course for about 12 years now, right?”“Right.”“Big data is just one more use of that idea.”“How so?”“Dump huge amounts of unstructured data into a computer, then wait to see the patterns it discovers. The bigger the dataset, the more obvious the patterns.”Jeff went on to explain that ‘unstructured data’ included information from climate sensors, digital photos and videos, purchase transaction records, Tweets and other social media posts, GPS signals from cell phones, things like that.YouTube, Facebook, Twitter, traffic and security cameras and the worldwide proliferation of portable digital devices add up to this: If every book, document, spreadsheet, register, newspaper and photograph created prior to July, 2010, were digitized, they would account for only about 10 percent of the world’s data.Ninety percent of all the data in the world has been created in the last two years.According to Ed Dumbill,“Big data is data that exceeds the processing capacity of conventional database systems. The data is too big, moves too fast, or doesn’t fit the strictures of your database architectures. To gain value from this data, you must choose an alternative way to process it.”Institutions can use big data to reduce fraud and errors. Hospitals can use it to improve patient care while reducing healthcare costs. According to IBM, one health care organization used big data this year to decrease patient mortality by 20 percent. A telecommunications company reduced processing time by 92 percent and a utility company improved the accuracy of power resource placement by 99 percent.Huge organizations like these have been the first to embrace big data, but Jeff Eisenberg tells me that he and Bryan are working to make its power available to retailers and small businesses, as well.When Jeff said big data was just another practical application of chaos theory, here’s what he meant: Chaos, in science, is not randomness, but precisely the opposite. Chaos is...