Maarten Sap

I am an assistant professor at CMU's LTI department with a courtesy appointment in HCII, and a part-time research scientist and AI safety lead at the Allen Institute for AI (AI2). My research focuses on measuring and improving AI systems' social and interactional intelligence, and understanding social inequality, safety, and bias in human- or AI-generated language.

Before this, I was a Postdoc/Young Investigator at the Allen Institute for AI (AI2), working on project Mosaic. I received my PhD from the University of Washington where I was advised by Noah Smith and Yejin Choi, and have interned at AI2 working on social commonsense reasoning, and at Microsoft Research working on deep learning models for understanding human cognition.
[bio for talks]

Recent updates:

April 2025 🏜️🚂: Though I will not be attending NAACL 2025, my students and collaborators will be presenting some exciting papers: Joel Mire on Rejected Dialects: Biases Against African American Language in Reward Models, Akhila Yerukola on NormAd: A Framework for Measuring the Cultural Adaptability of Large Language Models; Kaitlyn Zhou on Rel-A.I.: An Interaction-Centered Approach To Measuring Human-LM Reliance; Xuhui Zhou on AI-LieDar: Examine the Trade-off Between Utility and Truthfulness in LLM Agents.

April 2025 🦞👨🏼‍🏫: Excited to give a talk at the MIT CSAIL NLP seminar on the challenges of socially aware and culturally adaptable LLMs.

March 2025 👩‍💻🤖: It was fun to give a talk at SxSW on How to Be a Smarter AI User to a full room! Read the CNet article here.

January 2025 👨🏼‍🏫🧠: Happy to give a talk in Artificial Social Intelligence at the Cluster of Excellence "Science of Intelligence" (SCIoI) at the Technische Universität Berlin.

January 2025 👨🏼‍🏫📢: I'm happy to be giving a talk at the First Workshop on Multilingual Counterspeech Generation at COLING 2025 (remotely)!

December 2024 🇨🇦⛰️: Excited to be attending my very first NeurIPS conference in Vancouver BC! I'll be giving a talk at New in ML at 3pm on Tuesday!

November 2024 : I received a Google Academic Research Award for our work on participatory impact assessment of future AI use cases.

[older news]


My research group:

Dan Chechelnitsky

LTI PhD student
co-advised with Chrysoula Zerva

Joel Mire

LTI MLT student

Karina Halevy

LTI PhD student
co-advised with Mona Diab

Jimin Mun

LTI PhD student

Jocelyn Shen

MIT PhD student
co-advised with Cynthia Breazeal

Akhila Yerukola

LTI PhD student

Mingqian Zheng

LTI PhD student
co-advised with Carolyn Rosé

Xuhui Zhou

LTI PhD student


Overarching Research Themes

*Extracted by GPT-4, there may be inconsistencies.* #### *Ethics and Cultural Sensitivity in AI* My research group explores the importance of ethical considerations and cultural sensitivity in the development of AI systems. A key paper, [Out of Style: RAG's Fragility to Linguistic Variation](https://arxiv.org/abs/2504.08231), highlights how linguistic variation can impact the robustness of AI models. Additionally, we investigate biases against specific dialects in AI systems, as shown in [Disparities in LLM Reasoning Accuracy and Explanations: A Case Study on African American English](https://arxiv.org/abs/2503.04099). The study of non-verbal cues and their cultural implications is also paramount, exemplified by [Mind the Gesture: Evaluating AI Sensitivity to Culturally Offensive Non-Verbal Gestures](https://arxiv.org/abs/2502.17710). #### *Narrative Flow and Empathy* My research group explores the dynamics of narrative structures and their impact on empathy. We delve into how narratives are shaped and perceived, as discussed in the paper [Quantifying the narrative flow of imagined versus autobiographical stories](https://www.pnas.org/doi/10.1073/pnas.2211715119). We also analyze emotional resonance in storytelling with [HEART-felt Narratives: Tracing Empathy and Narrative Style in Personal Stories with LLMs](https://arxiv.org/abs/2405.17633), which investigates the narrative techniques that foster empathy in audiences. Furthermore, the paper [Modeling Empathic Similarity in Personal Narratives](https://arxiv.org/abs/2305.14246) adds depth to our understanding of how empathy influences narrative engagement. #### *Social Intelligence in AI Agents* My research group explores the development of AI agents that demonstrate social intelligence and can appropriately engage in social interactions. Important findings are presented in the paper [SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents](https://arxiv.org/abs/2310.11667), which assesses how AI systems can be evaluated for their social capabilities. We also investigate the limitations of social reasoning in AI through [Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Models](https://arxiv.org/abs/2305.14763), highlighting the need for improved understanding and training of AI models. Additionally, our framework for large-scale social simulations is detailed in [SOTOPIA-S4: A User-Friendly System for Flexible, Customizable, and Large-Scale Social Simulation](https://arxiv.org/abs/2502.13069). #### *Multilingual Moderation and Safety* My research group explores the challenges and strategies involved in creating effective moderation and safety tools for AI, particularly in diverse linguistic contexts. An innovative approach is detailed in [PolyGuard: A Multilingual Safety Moderation Tool for 17 Languages](https://arxiv.org/abs/2504.04377), which aims to enhance the capability of AI systems to understand and moderate language across different cultures. We also examine how neural networks can generate harmful content, as demonstrated in [PolygloToxicityPrompts: Multilingual Evaluation of Neural Toxic Degeneration in Large Language Models](https://arxiv.org/abs/2405.09373), focusing on the implications of toxicity in AI outputs. Additionally, the insights provided by [Counterspeakers’ Perspectives: Unveiling Barriers and AI Needs in the Fight against Online Hate](https://arxiv.org/abs/2403.00179) shed light on societal perceptions and the challenges posed by AI technologies.