Monday 11 November 2019

AI Reading comprehension scores

In September 2019 there was a great many blog posts about AI surpassing human reading comprehension scores.
Humans comprehension of a piece of text is 89.452%

As of June 4th 2021, AI is now hitting scores of 93.214, by IE-Net (ensemble) - RICOH_SRCB_DML

Plotting the scores on a graph:


Extrapolating the scores, we should see an AI hit 100% sometime in 2021.
Spreadsheet I created for the graph above: AI-Comprehension-score.xlsx

Original data source: https://rajpurkar.github.io/SQuAD-explorer/
Also see https://paperswithcode.com/sota/question-answering-on-squad20