Maarten Sap

I am an assistant professor at CMU's LTI department with a courtesy appointment in HCII, and a part-time research scientist and AI safety lead at the Allen Institute for AI (AI2). My research focuses on measuring and improving AI systems' social and interactional intelligence, and understanding social inequality, safety, and bias in human- or AI-generated language.

Before this, I was a Postdoc/Young Investigator at the Allen Institute for AI (AI2), working on project Mosaic. I received my PhD from the University of Washington where I was advised by Noah Smith and Yejin Choi, and have interned at AI2 working on social commonsense reasoning, and at Microsoft Research working on deep learning models for understanding human cognition.
[bio for talks]

Recent updates:

March 2025 πŸ‘©β€πŸ’»πŸ€–: Super excited to give a talk at SxSW on How to Be a Smarter AI User to a full room! Read the CNet article here.

January 2025 πŸ‘¨πŸΌβ€πŸ«πŸ§ : Happy to give a talk in Artificial Social Intelligence at the Cluster of Excellence "Science of Intelligence" (SCIoI) at the Technische UniversitΓ€t Berlin.

January 2025 πŸ‘¨πŸΌβ€πŸ«πŸ“’: I'm happy to be giving a talk at the First Workshop on Multilingual Counterspeech Generation at COLING 2025 (remotely)!

December 2024 πŸ‡¨πŸ‡¦β›°οΈ: Excited to be attending my very first NeurIPS conference in Vancouver BC! I'll be giving a talk at New in ML at 3pm on Tuesday!

November 2024 : I received a Google Academic Research Award for our work on participatory impact assessment of future AI use cases.

November 2024 πŸ«‚πŸ‘¨β€πŸ«: Very excited that I now have a courtesy appointment in the Human Computer Interaction Institute!

November 2024 πŸ”πŸ§‘β€πŸŽ“: As a reminder, due to my lab being quite full already, I'm not taking any students in this upcoming PhD application cycle 😟.

[older news]


My research group:

Dan Chechelnitsky

LTI PhD student
co-advised with Chrysoula Zerva

Joel Mire

LTI MLT student

Karina Halevy

LTI PhD student
co-advised with Mona Diab

Jimin Mun

LTI PhD student

Jocelyn Shen

MIT PhD student
co-advised with Cynthia Breazeal

Akhila Yerukola

LTI PhD student

Mingqian Zheng

LTI PhD student
co-advised with Carolyn RosΓ©

Xuhui Zhou

LTI PhD student


Overarching Research Themes

*Extracted by GPT-4, there may be inconsistencies.* #### *Ethics and Human-Centric AI* My research group explores the ethical implications and responsibilities of AI systems in our society. We investigate the biases and social implications of AI technologies, as exemplified in the paper [Disparities in LLM Reasoning Accuracy and Explanations: A Case Study on African American English](https://arxiv.org/abs/2503.04099), which reveals significant bias in large language models. Another notable contribution is the paper [Mind the Gesture: Evaluating AI Sensitivity to Culturally Offensive Non-Verbal Gestures](https://arxiv.org/abs/2502.17710), addressing how AI can misinterpret or perpetuate biases through non-verbal cues. Our ongoing research includes frameworks for understanding users' perceptions of AI, as demonstrated in [User-Driven Value Alignment: Understanding Users' Perceptions and Strategies for Addressing Biased and Discriminatory Statements in AI Companions](https://arxiv.org/abs/2409.00862). #### *Narrative Structures and Empathy* My research group explores the intricacies of narratives and their emotional impacts, particularly in online contexts. We delve into how narrative styles can evoke empathy, shown in the paper [HEART-felt Narratives: Tracing Empathy and Narrative Style in Personal Stories with LLMs](https://arxiv.org/abs/2405.17633), which investigates how personal storytelling resonates with different audiences. Another key publication, [Quantifying the narrative flow of imagined versus autobiographical stories](https://www.pnas.org/doi/10.1073/pnas.2211715119), sheds light on the structural elements that differentiate types of narratives. Additionally, we explore users’ engagement with stories in various online spaces, as highlighted in [Where Do People Tell Stories Online? Story Detection Across Online Communities](http://arxiv.org/abs/2311.09679). #### *Social Intelligence in AI* My research group explores the development of socially intelligent AI agents, focusing on how these systems can better understand and interact within social contexts. Our recent work includes the paper [SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents](https://arxiv.org/abs/2310.11667), which presents a framework for assessing social reasoning in AI interactions. Furthermore, we investigate the nuances of language usage in social interactions through [Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Models](https://arxiv.org/abs/2305.14763), which tests the limitations of AI in simulating social reasoning. Our research also includes practical implementations, as seen in [ProsocialDialog: A Prosocial Backbone for Conversational Agents](https://arxiv.org/abs/2205.12688), aimed at fostering healthier interactions between users and AI systems. #### *Moderation and Toxic Language Detection* My research group explores new methodologies for detecting and moderating toxic language online, recognizing its critical importance for community safety. A major aspect of our work is highlighted in the paper [PolyGuard: A Multilingual Safety Moderation Tool for 17 Languages](https://arxiv.org/abs/2504.04377), which seeks to create a more inclusive approach to handling toxic language across diverse linguistic contexts. Additionally, we analyze how biases affect detection systems, as detailed in the paper [BiasX: 'Thinking Slow' in Toxic Language Annotation with Explanations of Implied Social Biases](https://arxiv.org/abs/2305.13589), addressing how societal biases can distort the interpretation of language. Our research framework also encompasses contextual analysis of offensive statements, evident in the study [COBRA Frames: Contextual Reasoning about Effects and Harms of Offensive Statements](http://arxiv.org/abs/2306.01985).