Maarten Sap

I am an assistant professor at CMU's LTI department with a courtesy appointment in HCII, and a part-time research scientist and AI safety lead at the Allen Institute for AI (AI2). My research focuses on measuring and improving AI systems' social and interactional intelligence, and understanding social inequality, safety, and bias in human- or AI-generated language.

Before this, I was a Postdoc/Young Investigator at the Allen Institute for AI (AI2), working on project Mosaic. I received my PhD from the University of Washington where I was advised by Noah Smith and Yejin Choi, and have interned at AI2 working on social commonsense reasoning, and at Microsoft Research working on deep learning models for understanding human cognition.
[bio for talks]

Recent updates:

May 2025 πŸ§‘β€πŸ’»πŸ†: Super super excited to announce that our paper Rel-A.I.: An Interaction-Centered Approach To Measuring Human-LM Reliance received the Best Paper Runner Up award at NAACL 2025. Huge congratulations to Kaitlyn!

April 2025 πŸœοΈπŸš‚: Though I will not be attending NAACL 2025, my students and collaborators will be presenting some exciting papers: Joel Mire on Rejected Dialects: Biases Against African American Language in Reward Models, Akhila Yerukola on NormAd: A Framework for Measuring the Cultural Adaptability of Large Language Models; Kaitlyn Zhou on Rel-A.I.: An Interaction-Centered Approach To Measuring Human-LM Reliance; Xuhui Zhou on AI-LieDar: Examine the Trade-off Between Utility and Truthfulness in LLM Agents.

April 2025 πŸ¦žπŸ‘¨πŸΌβ€πŸ«: Excited to give a talk at the MIT CSAIL NLP seminar on the challenges of socially aware and culturally adaptable LLMs.

March 2025 πŸ‘©β€πŸ’»πŸ€–: It was fun to give a talk at SxSW on How to Be a Smarter AI User to a full room! Read the CNet article here.

January 2025 πŸ‘¨πŸΌβ€πŸ«πŸ§ : Happy to give a talk in Artificial Social Intelligence at the Cluster of Excellence "Science of Intelligence" (SCIoI) at the Technische UniversitΓ€t Berlin.

January 2025 πŸ‘¨πŸΌβ€πŸ«πŸ“’: I'm happy to be giving a talk at the First Workshop on Multilingual Counterspeech Generation at COLING 2025 (remotely)!

December 2024 πŸ‡¨πŸ‡¦β›°οΈ: Excited to be attending my very first NeurIPS conference in Vancouver BC! I'll be giving a talk at New in ML at 3pm on Tuesday!

[older news]


My research group:

Dan Chechelnitsky

LTI PhD student
co-advised with Chrysoula Zerva

Joel Mire

LTI MLT student

Karina Halevy

LTI PhD student
co-advised with Mona Diab

Jimin Mun

LTI PhD student

Jocelyn Shen

MIT PhD student
co-advised with Cynthia Breazeal

Akhila Yerukola

LTI PhD student

Mingqian Zheng

LTI PhD student
co-advised with Carolyn RosΓ©

Xuhui Zhou

LTI PhD student


Overarching Research Themes

Themes extracted and images generated with the OpenAI API; there may be inconsistencies.

Ethics and Responsible AI

My research group explores the complexities of ethics and bias in artificial intelligence systems. A key paper, [Mind the Gesture: Evaluating AI Sensitivity to Culturally Offensive Non-Verbal Gestures](https://arxiv.org/abs/2502.17710), investigates how AI can be sensitive to cultural nuances in communication, which is crucial for developing more inclusive models. Additionally, the work on [Not Like Us, Hunty: Measuring Perceptions and Behavioral Effects of Minoritized Anthropomorphic Cues in LLMs](https://arxiv.org/abs/2505.05660) highlights the importance of understanding how anthropomorphic designs affect user perceptions across diverse demographics. Another significant study, [Particip-AI: A Democratic Surveying Framework for Anticipating Future AI Use Cases, Harms and Benefits](https://arxiv.org/abs/2403.14791), focuses on engaging the community in discussions about AI's potential impacts, ensuring that diverse voices contribute to responsible AI development.

Narrative Exploration through AI

My research group explores how narratives shape human experience and understanding, especially in the context of AI-generated content. A recent paper, [Quantifying the narrative flow of imagined versus autobiographical stories](https://www.pnas.org/doi/10.1073/pnas.2211715119), delves into the differences between personal storytelling and constructed narratives, enhancing our comprehension of narrative structures. Another important study, [HEART-felt Narratives: Tracing Empathy and Narrative Style in Personal Stories with LLMs](https://arxiv.org/abs/2405.17633), examines how AI can influence emotional engagement in storytelling. Furthermore, the work on [The Empirical Variability of Narrative Perceptions of Social Media Texts](https://aclanthology.org/2024.emnlp-main.1113/) provides insight into the diverse ways audiences interpret narratives across digital platforms.

Social Intelligence in AI Agents

My research group explores the intersection of social intelligence and AI agent interactions. The paper [SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents](https://arxiv.org/abs/2310.11667) sets the groundwork for assessing how well AI can navigate social situations and exhibit appropriate responses. Additionally, [AI-LieDar: Examine the Trade-off Between Utility and Truthfulness in LLM Agents](https://aclanthology.org/2025.naacl-long.595/) discusses the critical balance between functionality and ethical considerations in AI conversations. Finally, our work on [SOTOPIA-S4: A User-Friendly System for Flexible, Customizable, and Large-Scale Social Simulation](https://arxiv.org/abs/2504.16122) offers a framework for engaging users in complex social scenarios using AI.

Mitigating Bias in Language Models

My research group explores strategies to reduce bias and improve fairness in language models. A landmark study, [Mitigating Bias in RAG: Controlling the Embedder](https://arxiv.org/abs/2502.17390), investigates the influence of embeddings on bias propagation in generated content. Another important contribution, [Disparities in LLM Reasoning Accuracy and Explanations: A Case Study on African American English](https://arxiv.org/abs/2503.04099), emphasizes the need to evaluate reasoning capabilities across diverse linguistic contexts. Additionally, [Rejected Dialects: Biases Against African American Language in Reward Models](https://arxiv.org/abs/2502.12858) sheds light on how language models can inadvertently reinforce stereotypes, underscoring the need for systematic bias audits.