Maarten Sap

Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMs

This page contains the preprocessed data for our EMNLP 2022 paper on Neural Theory of Mind.

Social IQa

The dev examples only are at socialIWa_v1.4_dev_wDims.jsonl. Relevant fields in each line:

Note, the full Social IQa dataset from


The data used on our paper, preprocessed: ToMi-finalNeuralTOM.csv: The relevant columns are:

The code to generate the stories is available at . IMPORTANT: as noted in our paper, the code generates a set of story-question-answer tuples that assume an omniscient reader despite being false-belief stories (i.e., the answer is incorrect).

Citation information

Our paper on Neural Theory of Mind:

  title={Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMs},
  author={Sap, Maarten and LeBras, Ronan and Fried, Daniel and Choi, Yejin},

The Social IQa paper:

  title={Social IQa: Commonsense Reasoning about Social Interactions},
  author={Sap, Maarten and Rashkin, Hannah, and Chen, Derek and LeBras, Ronan and Choi, Yejin},

The ToMi paper:

    title = "Revisiting the Evaluation of Theory of Mind through Question Answering",
    author = "Le, Matthew  and
      Boureau, Y-Lan  and
      Nickel, Maximilian",
    booktitle = "EMNLP-IJCNLP",
    month = nov,
    year = "2019",
    address = "Hong Kong, China",
    publisher = "Association for Computational Linguistics",
    url = "",
    doi = "10.18653/v1/D19-1598",
    pages = "5872--5877",