Friday, February 18th, 2011

Jeopardy, Watson, and Media6Degrees

Posted by: Claudia Perlich

On Wednesday night, I had the distinct honor of a front row seat to an historic event. 500 friends and family members of IBM Research (my former employer) were packed into the cafeteria of the Thomas J. Watson Research Center – witnesses to Watson’s achievements in the quest for artificial intelligence and natural language processing. We watched as this amazing machine (and it’s sometimes hard to remember Watson is in fact only a machine) beat two of the most accomplished Jeopardy players at their own game.

Watson is even more impressive when you consider that just four years ago, the best computer-based question answering system would not have passed even the preliminary qualifiers required to become a Jeopardy contestant. That system would have been confident about perhaps 5% of the questions and answered only 30% of them correctly, while average Jeopardy champions typically answer 80% of the questions and are correct 90% of the time. Of course Watson’s competitors, Ken Jennings and Brad Rutter, are hardly your “average” Jeopardy champions.

Natural language, with all of its quirks and ambiguities, has been the holy grail of artificial and data science. In 1950, Alan Turing suggested that the ultimate confirmation of machine intelligence would be if a machine could carry on a typed conversation with a human without the human realizing he was speaking to a machine. Could Watson become the first machine to pass the Turing Test?

Watson is far from perfect – and some of his Jeopardy answers hint at his quirks. But those quirks make him almost human and a pleasure to watch (yes, I have decided Watson deserves to be personified). He is a complex synthesis of highly-parallelized hardware, terabytes of raw data, really smart language processing and, of course, machine learning.

Watson is the most public example of the amazing innovations being aided by machine learning across industries and disciplines – including medicine, fraud detection, finance, and, at Media6Degrees, even marketing. Much like Watson, we are applying machine learning techniques on top of our highly parallelized hardware and billions of data points to predict which advertisement a person might be interested in seeing.

Machine learning is much more than rule-based decision-making (if x then y). On his way to determining an answer, Watson uses natural language processing and draws inferences to synthesize insights from millions of data sources to consider hundreds of possible answers to a nuanced question. At that point, machine learning is the gatekeeper so to speak, the critical conscience of the machine. Much like a human contestant, Watson honed his Jeopardy skills by playing thousands of practice rounds over the last several years. Through practice, he learned to evaluate possible answers, determine which assessments to trust, identify a predictably reliable answer… and ultimately press his Jeopardy buzzer to show us all just how much he knows and understands.