Guest Post: Forget big data… here comes machine learning

Richard Harris, chief executive and founder of Intent Media explains what the future of human-machine interaction means for online travel booking Continue reading →

Richard Harris, chief executive and founder of Intent Media explains what the future of human-machine interaction means for online travel booking

The world generates a staggering 2.5 quintillion bytes of data per day. That is written 2,500,000,000,000,000,000. To give you an understanding of how much data that really is, just one character of the words you are reading right now requires 1 byte of data. 2.5 quintillion characters would fill 705 billion King James Bibles, containing more than 780,000 words each. Stacked one on top of another on the ground in front of you, those books would stretch 239,000 miles into the sky, through space, and all the way to the moon…over 400 times.

Meanwhile, back here on Earth, we would carry on generating 3.6 million Google searches, 103 million spam mails, 15 million text messages and 18 million weather forecast requests…every single minute. Is it any wonder that the phrase “big data” is thrown around as much as it is?

Data doesn’t solve problems…Machine Learning does

For all the talk about how important this gargantuan volume of data is, I’ve got a dirty little secret to tell you….data is dumb. By itself, data holds no meaning and delivers no value. In order to derive benefit from the information we produce, first we must sift through that data, determining what is relevant and what is not, recognizing patterns, and putting data to work for us through complex predictions of what it all means. And that, undoubtedly, is too difficult for even the brightest of unaided Earthling minds.

Enter the Chinese strategy game Go. “What in the world does untangling quadrillions of bytes of data have to do with a 2,500 year old game of black and white pebbles?” Go, played with 361 moveable stones, is said to have more possible positions than the Universe has atoms. The best players perform on intuition, calculating moves and predicting outcomes on a level too sublime to breach cognitive awareness. Because of this, Go makes for an especially enticing challenge to Machine Learning (ML) researchers hoping to mimic human capabilities. In 2016, Google DeepMind’s AlphaGo system defeated reigning World Go champion, Lee Sedol using a “deep neural network” trained on 30 million moves by human players. By analyzing those moves, AlphaGo independently learned to improve its own choices, effectively becoming the best Go player in the World. What’s amazing about that is that some kinds of Machine Learning are already surpassing human analytical abilities.

But Machine Learning does not show up only in the advanced research labs of data scientists, working on esoteric intellectual problems. Of the tens of billions of dollars that companies invest in Artificial Intelligence each year, about 60% is now focused on ML, propelling real-world solutions into the marketplace.

Machine Learning isn’t sorcery, but it does work remarkably well

One area where there’s a huge opportunity to be explored with Machine Learning is the travel booking industry. In 2016, online travel sales worldwide exceeded $550 billion, and are predicted to climb another $100 billion per year through 2019. But of the hundreds of millions of travelers who visit an online travel agent at any given moment, less than 5% of them actually complete a booking.

What’s going on here? How can we make sense of this contrary data?

Machine Learning to the rescue. At Intent Media, we trained our ML systems on billions of online travel booking site page views, hoping to distill useful information about how travelers really buy. What we found was that each visitor sends hundreds of individual “signals”— demographic, behavioral and contextual data points generated during an online shopping session. With Machine Learning, travel websites can use those signals to segment different types of travel shoppers and accurately predict their behavior at any point along the transaction path.

Is this visitor ready to make a purchase now? Would an enticing offer at this moment likely result in a purchase? Should we re-target them with advertising? Or is this one simply destined to be ‘the fish who got away?’ Much like the Go playing example we looked at earlier, well-trained ML systems calculate all the possible outcomes simultaneously, in real time, and execute the engagement tactic with the highest chance of driving value out of each visitor. Fed on a steady diet of data, our Machine Learning system seems to know what users intend to do, even before those users themselves. It is truly sublime.

Machine Learning isn’t sorcery, though, and the work of training ML systems is never done. After scoring users we still must A-B test ML-powered predictions against control groups over and over and over, scoring and segmenting users, placing and validating machine predictions, and incorporating the results yet again into our constantly improving machine intelligence model. Why go through such a grueling, iterative process? Simple…because it pays off. Over eight years, the travel sites we support have cashed in on Machine Learning to the tune of billions of dollars of incremental gains.

The future of machines is the future of humanity

Despite its graduation from the theoretical to the practical in recent years, however, Machine Learning is still very much in its infancy, and that’s exciting. In the coming decade, ML will become as ubiquitously and invisibly woven into the fabric of all aspects of our lives as it is in industries like travel booking today.

As for those “trips to the moon and back?” Chump change compared with the hockey stick data growth that’s on the launch deck. The volume is so large that Microsoft has been successfully experimenting with a solution to the problem of preserving all that data, using the densest known storage medium in the universe— one that can hold a full quintillion bytes in a single cubic millimeter— DNA. There’s just too much information, coming forth too rapidly to possibly be processed by humans, unaided. Machine Learning is beginning to help define entirely new ways to do live, and to do business. As Jeff Bezos pointed out recently, “We are now solving problems with Machine Learning and Artificial Intelligence that were in the realm of science fiction for the last several decades.”