Robots are Now Beating Humans at Reading Comprehension

Fagjun | Published 2018-01-19 20:06

Maybe at this point, it would be easier to list the things that robots haven’t bested us at.

Robots are beating us at our own games.

Robots are beating us at our own games.


An artificial intelligence algorithm has managed to beat humans at the Stanford Question Answering Dataset (SQuAD). This test has over 100,000 questions on 500 articles, and was created by Stanford University to assess the reading comprehension skills of computers. The articles in the test come from various sources, including Wikipedia.


Now, for the first time, a machine has managed to beat human scores on the test. Chinese retail giant Alibaba’s Institute of Data Science of Technologies built an AI algorithm with the use of natural language processing and a machine learning model. The highest score achieved by humans is 82.305, while the machine managed to score 82.44 this past January 11.

Answering Questions

Multiple machines have beaten human scores on the test.

Multiple machines have beaten human scores on the test.


The test seems pretty straightforward. The human or AI program taking the test reads a passage from, for example, a Wikipedia article about The Lord of the Rings trilogy of films. The questions in the test are something like “Who is the actor who played Frodo in the movie franchise?” (It’s Elijah Wood.)


"These kinds of tests are certainly useful benchmarks for how far along the AI journey we may be," said Microsoft spokesman Andrew Pickup. Microsoft also developed an AI program that beat human scores in the test, with a test result of 82.650 points. Facebook, Samsung, and Tencent have submitted their own AI programs to the Stanford project as well.


"However,” Pickup continues, “the real benefit of AI is when it is used in harmony with humans.”


So what exactly are the benefits of machines that have better reading comprehension skills than we do? According to Luo Si, chief scientist for natural language processing at the Institute of Data Science of Technologies, “objective questions such as ‘What causes rain’ can now be answered with high accuracy by machines.”


This may mean that we'll have more interactions with robots in the future.

This may mean that we'll have more interactions with robots in the future.


Customer service may also soon be vastly different from what we’ve gotten used to. Alibaba has already put their AI to work on Singles Day, the biggest shopping event in the world. Now, this may seem like a nightmare for customer service workers, but Alibaba used machines to field several customer service questions.


However, customer service won’t be the only application that can benefit from the use of this technology. “The technology underneath can be gradually applied to numerous applications, such as customer service, museum tutorials and online responses to medical inquiries from patients, decreasing the need for human input in an unprecedented way,” says Si.

Hey! Where are you going?? Subscribe!

Get weekly science updates in your inbox!