Digital Agenda for Europe
A Europe 2020 Initiative

Research talks: Watson… two years on!

Article
Alessandro Moschitti, Assistant Professor at the Information Engineering and Computer Science Department of the University of Trento - Italy, speaks about his experiences on the Watson Jeopardy! Challenge, with Paul Hearn, Scientific Officer from the Future and Emerging Technologies Programme at the European Commission.
Share this

Alessandro Moschitti is Assistant Professor at the Information Engineering and Computer  Science Department of the University of Trento, in Italy. Before joining Trento, Alessandro held various positions at Columbia, Colorado and John Hopkins Universities in the US. His expertise relates to advanced machine learning techniques applied to natural language processing, information retrieval and data mining. Alessandro has authored or co-authored around 160 research papers and is the holder of several awards among which two IBM awards and a Google Faculty award. Alessandro is speaking here about his experiences on the Watson Jeopardy! Challenge, with Paul Hearn, Scientific Officer from the Future and Emerging Technologies Programme at the European Commission in Brussels, Belgium.

 

Paul - Trento was the only European University involved in Watson Jeopardy!, which hit the headlines last year when a computer beat human contestants to the first prize in the final of the US Jeopardy! Game Show. What was your role in the challenge?

Alessandro - IBM selected 8 universities to contribute to Jeopardy! Trento was indeed the only European partner. Having said that, quite a few European researchers also took part in US research team efforts...

I worked on two main tasks: Question Analysis and Answer Reranking. Question Analysis involved automatic classification of questions in different categories so they can be processed with analytics and models specific to each category. Answer Reranking concerned reordering of the candidate answers derived by Watson by order of highest probability. Our main scientific contribution was the use of an advanced machine learning and natural language processing approach called deepQA.

Before Jeopardy! it was unclear whether syntactic and semantic processing could really help the design of information retrieval systems. We were basically missing a clear and verifiable evaluation procedure to test the accuracy of information retrieval systems using such technology.

Source picture: CNN

 

Paul - Did the fact that Watson beat very capable human contestants tell us anything, in your view, about human and machine intelligence?

Alessandro - The definition of intelligence is pretty fuzzy! With respect to machine intelligence, the most famous and well-known definition is the Turing Test, one that Watson would definitely fail! But it is interesting that humans sometimes associate intelligence with the ability to store and retrieve information. For example, in Italy we label people with the expression "Pico della Mirandola" (after 15th Century Italian philosopher Giovanni Pico della Mirandola) when we are impressed by their intellectual capacity, typically in memorizing information. The test relates to "just" reverse reciting the text of Dante's Divine Comedy, from the end to the beginning. This can be performed by a trivial, i.e., pretty stupid, computer program!

In this perspective, Watson can be considered the most intelligent human artifact: it can access many terabytes of information and find the desired piece of information in a few seconds. The Jeopardy! challenge showed that Watson is more "intelligent" than the most skilled humans at this task.

 

Paul - I remember Watson Jeopardy! being a big story at the time. What has happened since? What can we expect next? Are computers any better today at mimicking human intelligence?

Alessandro - Watson has now been "retired" from playing games, and is now working in the medical domain. Its complex algorithms and huge computational power allow it to answer difficult medical questions by searching through and analysing millions of medical abstracts and documentation. In this area, the impact of Watson is potentially huge: it may be possible to efficiently and effectively support doctors, e.g. to find a cause of a disease, or to quickly provide preliminary diagnoses.

As regards the future, we can expect machines to get better and better at supporting the work of humans. Again, this would not mean necessarily that they will be more intelligent according to the Turing Test, but they will become more helpful in making humans more efficient and intelligent, e.g., in finding a cure for a disease, or in product design.

The challenges for the future on the machine side is to build efficient and accurate algorithms for searching, retrieving, managing and aggregating information, whereas human intelligence will be challenged in its ability to exploit the potentiality of such machines to the best extent…

 

Paul - What you are working on now?  

Alessandro - We are still working on exploiting the Reranker for new domains. In addition, I am currently managing the EternalS project, funded under the EU Future and Emerging Technologies Programme. In EternalS, we are trying to establish an emerging research network in the area of Eternal Systems. Eternal systems are flexible software applications, which can adapt to environment changes, e.g., user needs or security conditions. A number of broad research fields are challenged by the concept of eternality, including networking, security, formal methods and knowledge-based systems. EternalS is trying to weave these various fields together, and produce a research roadmap for the future. One concrete research outcome so far has been to focus on networking devices with increased ability to connect with others, using some of the machine learning, natural language processing and information retrieval concepts we explored in Jeopardy! and elsewhere.

 


Further reading:

Watson is an artificial intelligence computer system capable of answering questions posed in natural language, developed in IBM's DeepQA project by a research team led by principal investigator David Ferrucci. Watson was named after IBM's first president, Thomas J. Watson. In 2011, as a test of its abilities, Watson competed on the quiz show Jeopardy!, in the show's only human-versus-machine match-up to date. In a two-game, combined-point match, broadcast in three Jeopardy! episodes February 14–16, Watson beat Brad Rutter, the biggest all-time money winner on Jeopardy!, and Ken Jennings, the record holder for the longest championship streak (74 wins). Watson received the first prize of $1 million, while Ken Jennings and Brad Rutter received $300,000 and $200,000, respectively. Jennings and Rutter pledged to donate half their winnings to charity, while IBM divided Watson's winnings between two charities. [Extract from Wikipedia]