Overview
Natural Questions is Google’s dataset of real search queries paired with Wikipedia answers, providing the most naturalistic evaluation of open-domain question answering. Its questions reflect actual information needs typed by real users.
What’s In It
The dataset contains 307,373 training examples pairing real anonymized Google search queries with Wikipedia pages annotated with long answers and short answers. Some questions are marked as having no answer on the given page.
How It’s Used
Natural Questions evaluates open-domain QA systems. It drove development of retrieval-augmented approaches and dense passage retrieval. BERT, T5, and RAG systems report it as a key metric.
Controversies
Wikipedia-only answers limit evaluation to factoid questions. The dataset reflects Google search demographics and query patterns, potentially missing diverse information needs.