ToM QA Dataset: Evaluating Theory of Mind in Question Answering

ToM QA Dataset: Evaluating Theory of Mind in Question Answering

The ToM QA Dataset is designed to evaluate question-answering models' ability to reason about beliefs. It includes 3 task types and 4 question types, creating 12 total scenarios. The dataset is inspired by theory-of-mind experiments in developmental psychology and is used to test models' understanding of beliefs and inconsistent states of the world.

ToM QA Dataset: Evaluating Theory of Mind in Question Answering

Pengenalan Terperinci

The ToM QA Dataset, introduced in the EMNLP 2018 paper 'Evaluating Theory of Mind in Question Answering', provides a comprehensive set of scenarios to test question-answering models. The dataset includes first-order and second-order belief questions, as well as memory and reality questions, to ensure models have a correct understanding of the state of the world and others' beliefs. It is available in four versions: easy with noise, easy without noise, hard with noise, and hard without noise.

Lagi
Set data

Kata Kunci

ToM QA DatasetTheory of MindQuestion AnsweringBelief ReasoningDevelopmental PsychologyEMNLP 2018

Kongsi