Monday, 11 November 2019

AI Reading comprehension scores

In September 2019 there was a great many blog posts about AI surpassing human reading comprehension scores.
Humans comprehension of a piece of text is 89.452%

As of April 6 2020, AI is now hitting scores of 93.011, by SA-Net on Albert (ensemble)

Plotting the scores on a graph:


Extrapolating the scores, we should see an AI hit 100% sometime in 2021.
Spreadsheet I created for the graph above: AI-Comprehension-score.xlsx

Original data source: https://rajpurkar.github.io/SQuAD-explorer/