Maarten Sap Twitter Profile Google Scholar Profile Semantic Scholar Profile

Maarten Sap

I am an assistant professor at CMU's LTI department with a courtesy appointment in HCII, and a part-time research scientist and AI safety lead at the Allen Institute for AI (AI2). My research focuses on measuring and improving AI systems' social and interactional intelligence, and understanding social inequality, safety, and bias in human- or AI-generated language.

Before this, I was a Postdoc/Young Investigator at the Allen Institute for AI (AI2), working on project Mosaic. I received my PhD from the University of Washington where I was advised by Noah Smith and Yejin Choi, and have interned at AI2 working on social commonsense reasoning, and at Microsoft Research working on deep learning models for understanding human cognition.
[bio for talks]

Recent updates:

May 2025 🧑‍💻🏆: Super super excited to announce that our paper Rel-A.I.: An Interaction-Centered Approach To Measuring Human-LM Reliance received the Best Paper Runner Up award at NAACL 2025. Huge congratulations to Kaitlyn!

April 2025 🏜️🚂: Though I will not be attending NAACL 2025, my students and collaborators will be presenting some exciting papers: Joel Mire on Rejected Dialects: Biases Against African American Language in Reward Models, Akhila Yerukola on NormAd: A Framework for Measuring the Cultural Adaptability of Large Language Models; Kaitlyn Zhou on Rel-A.I.: An Interaction-Centered Approach To Measuring Human-LM Reliance; Xuhui Zhou on AI-LieDar: Examine the Trade-off Between Utility and Truthfulness in LLM Agents.

April 2025 🦞👨🏼‍🏫: Excited to give a talk at the MIT CSAIL NLP seminar on the challenges of socially aware and culturally adaptable LLMs.

March 2025 👩‍💻🤖: It was fun to give a talk at SxSW on How to Be a Smarter AI User to a full room! Read the CNet article here.

January 2025 👨🏼‍🏫🧠: Happy to give a talk in Artificial Social Intelligence at the Cluster of Excellence "Science of Intelligence" (SCIoI) at the Technische Universität Berlin.

January 2025 👨🏼‍🏫📢: I'm happy to be giving a talk at the First Workshop on Multilingual Counterspeech Generation at COLING 2025 (remotely)!

December 2024 🇨🇦⛰️: Excited to be attending my very first NeurIPS conference in Vancouver BC! I'll be giving a talk at New in ML at 3pm on Tuesday!

[older news]

My research group:

Dan Chechelnitsky

LTI PhD student
co-advised with Chrysoula Zerva

Joel Mire

LTI MLT student

Karina Halevy

LTI PhD student
co-advised with Mona Diab

Jimin Mun

LTI PhD student

Jocelyn Shen

MIT PhD student
co-advised with Cynthia Breazeal

Akhila Yerukola

LTI PhD student

Mingqian Zheng

LTI PhD student
co-advised with Carolyn Rosé

Xuhui Zhou

LTI PhD student

Overarching Research Themes

Themes extracted and images generated with the OpenAI API; there may be inconsistencies.

Navigating Ethical AI Design

My research group explores the intersection of AI, ethics, and human-centered design to ensure that technology aligns with societal values. One important paper we focus on is [Mind the Gesture: Evaluating AI Sensitivity to Culturally Offensive Non-Verbal Gestures](https://arxiv.org/abs/2502.17710), which examines how AI systems can misinterpret cultural cues and foster harm. Additionally, we study [Mitigating Bias in RAG: Controlling the Embedder](https://arxiv.org/abs/2502.17390) to develop frameworks for AI that reduce biases. Our work is further informed by the findings in [Let Them Down Easy! Contextual Effects of LLM Guardrails on User Perceptions and Preferences](https://arxiv.org/abs/2506.00195), which highlights ways to enhance user trust through responsible AI design.

Exploring Narrative Dynamics

My research group explores the significance of narrative structures in understanding human experiences and AI interactions. We delve into the implications of narrative in technology through papers such as [Quantifying the narrative flow of imagined versus autobiographical stories](https://www.pnas.org/doi/10.1073/pnas.2211715119), which analyzes how stories impact our perceptions. Also noteworthy is [HEART-felt Narratives: Tracing Empathy and Narrative Style in Personal Stories with LLMs](https://arxiv.org/abs/2405.17633), which investigates emotional engagement in narratives produced by AI. Our exploration is further enhanced by analyzing [The Empirical Variability of Narrative Perceptions of Social Media Texts](https://aclanthology.org/2024.emnlp-main.1113/), shedding light on how narratives adapt across different platforms.

Enhancing AI Social Interaction

My research group explores the development of AI agents designed for social interactions, focusing on their capacity for social intelligence. A pivotal paper in this area is [SOTOPIA-S4: A User-Friendly System for Flexible, Customizable, and Large-Scale Social Simulation](https://arxiv.org/abs/2504.16122), which discusses how we can create more sophisticated simulations that mimic human behaviors. Another significant contribution is [Interactive Agents to Overcome Ambiguity in Software Engineering](https://arxiv.org/abs/2502.13069), addressing the potential of AI to support complex tasks. Moreover, we are interested in [Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Models](https://arxiv.org/abs/2305.14763), which evaluates how well AI systems can understand and replicate social contexts.

Advancing Conversational Technologies

My research group explores the innovative methodologies for enhancing conversational agents and their effectiveness in real-world applications. One significant work is [ProsocialDialog: A Prosocial Backbone for Conversational Agents](https://arxiv.org/abs/2205.12688), which proposes frameworks for promoting prosocial behavior within dialogue systems. We also focus on [SODA: Million-scale Dialogue Distillation with Social Commonsense Contextualization](https://arxiv.org/abs/2212.10465), aiming to capture the nuances of social reasoning in conversations. Critical insights are derived from [Aligning LLMs to Ask Good Questions: A Case Study in Clinical Reasoning](https://arxiv.org/abs/2502.14860), which explores the efficacy of AI in assisting healthcare conversations.

More about me