Overview
SQuAD, the Stanford Question Answering Dataset, established the modern paradigm for evaluating reading comprehension in NLP. Created by Pranav Rajpurkar at Stanford, it requires models to extract answer spans from Wikipedia passages.
What’s In It
SQuAD 1.1 contains 107,785 question-answer pairs on 536 Wikipedia articles. SQuAD 2.0 extends this with 53,775 unanswerable questions requiring models to know when a passage does not contain the answer. Questions were created by crowdworkers.
How It’s Used
SQuAD became the primary benchmark for pre-trained language model evaluation and drove the development of attention-based architectures. BERT’s success on SQuAD demonstrated the power of transfer learning for NLP.
Controversies
Critics note SQuAD’s extractive format limits it to questions answerable by copying text spans, missing abstractive reasoning. The competitive leaderboard may have over-optimized the field for this specific task format.