Maarten Sap Twitter Profile Google Scholar Profile Semantic Scholar Profile

Maarten Sap

I am an assistant professor at CMU's LTI department with a courtesy appointment in HCII, and a part-time research scientist and AI safety lead at the Allen Institute for AI (AI2). My research focuses on (1) measuring and improving AI systems' social and interactional intelligence, (2) assessing and combatting social inequality, safety risks, and socio-cultural biases in human- or AI-generated language, and (3) building narrative language technologies for prosocial outcomes.

I received my PhD from the University of Washington where I was advised by Noah Smith and Yejin Choi.
[bio for talks]

Recent updates:

August 2025 🌟: Incredibly honored to be one of 7 US recipients of the 2025 Okawa Research Grant from the Okawa Foundation!

August 2025 🧑‍🎓: Welcoming my first postdoc, Vasudha Varadarajan, to the lab!

August 2025 👨🏼‍🏫: Excited to give a (virtual) talk about Responsible AI for Diverse Users and Cultures at the Gender Bias in NLP workshop at ACL 2025!

July 2025 🧠🛡️: Five papers were accepted to COLM 2025! Highlights include HAICOSYSTEM, a framework for sandboxing safety risks in human-AI interaction; ALFA, which aligns LLMs to ask better clinical questions; and PolyGuard, a multilingual moderation tool for unsafe content. Two other papers to be released soon :)

May 2025 🧑‍💻🏆: Super super excited to announce that our paper Rel-A.I.: An Interaction-Centered Approach To Measuring Human-LM Reliance received the Best Paper Runner Up award at NAACL 2025. Huge congratulations to Kaitlyn!

April 2025 🏜️🚂: Though I will not be attending NAACL 2025, my students and collaborators will be presenting some exciting papers: Joel Mire on Rejected Dialects: Biases Against African American Language in Reward Models, Akhila Yerukola on NormAd: A Framework for Measuring the Cultural Adaptability of Large Language Models; Kaitlyn Zhou on Rel-A.I.: An Interaction-Centered Approach To Measuring Human-LM Reliance; Xuhui Zhou on AI-LieDar: Examine the Trade-off Between Utility and Truthfulness in LLM Agents.

April 2025 🦞👨🏼‍🏫: Excited to give a talk at the MIT CSAIL NLP seminar on the challenges of socially aware and culturally adaptable LLMs.

[older news]

My research group:

Dan Chechelnitsky

LTI PhD student
co-advised with Chrysoula Zerva

Joel Mire

LTI MLT student

Karina Halevy

LTI PhD student
co-advised with Mona Diab

Jimin Mun

LTI PhD student

Jocelyn Shen

MIT PhD student
co-advised with Cynthia Breazeal

Vasudha Varadarajan

LTI Postdoc

Akhila Yerukola

LTI PhD student

Mingqian Zheng

LTI PhD student
co-advised with Carolyn Rosé

Xuhui Zhou

LTI PhD student

Overarching Research Themes

Themes extracted and images generated with the OpenAI API; there may be inconsistencies.

Ethics and Responsible AI

My research group explores the complex landscape of ethics in AI, focusing on how technology can be aligned with human values and societal norms. Recent work such as [Let Them Down Easy! Contextual Effects of LLM Guardrails on User Perceptions and Preferences](https://arxiv.org/abs/2506.00195) delves into user reactions to AI safety measures, emphasizing the importance of context in ethical considerations. Additionally, [Synthetic Socratic Debates: Examining Persona Effects on Moral Decision and Persuasion Dynamics](https://arxiv.org/abs/2506.12657) investigates how different AI personas can influence moral reasoning, shedding light on the social implications of AI interactions. Through these studies, we aim to enhance understanding of how AI systems can be designed responsibly and effectively.

Narratives and Empathy

My research group explores the intricate relationships between narratives, empathy, and personal storytelling. We have investigated how narrative flow can vary between imagined and autobiographical stories in our key paper [Quantifying the narrative flow of imagined versus autobiographical stories](https://www.pnas.org/doi/10.1073/pnas.2211715119), providing insights into human cognition and memory. Another significant contribution is [HEART-felt Narratives: Tracing Empathy and Narrative Style in Personal Stories with LLMs](https://arxiv.org/abs/2405.17633), which examines how narrative styles can evoke empathy. Our work underscores the critical role narrative analysis plays in understanding human experiences and AI’s ability to simulate such interactions.

AI Agents and Social Intelligence

My research group explores the development of AI agents with enhanced social intelligence, focusing on their capabilities to interact and engage with users meaningfully. For instance, our paper [SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents](https://arxiv.org/abs/2310.11667) introduces a framework for evaluating the social competencies of AI, which is crucial for their effective deployment in real-world applications. Similarly, [Is This the Real Life? Is This Just Fantasy? The Misleading Success of Simulating Social Interactions With LLMs](http://arxiv.org/abs/2403.05020) questions the authenticity of LLMs in simulating human-like interactions, driving forward our understanding of AI’s social limitations. These findings emphasize the importance of building socially aware AI systems that can navigate complex social landscapes responsibly.

Language Models and Human Interaction

My research group explores the evolving interactions between humans and language models, particularly how they can be aligned to obtain valuable responses in various contexts. The paper [ALFA: Aligning LLMs to Ask Good Questions: A Case Study in Clinical Reasoning](https://arxiv.org/abs/2502.14860) illustrates practical applications of language models in healthcare, demonstrating their ability to enhance question-asking behaviors. Another notable paper, [Relying on the Unreliable: The Impact of Language Models' Reluctance to Express Uncertainty](https://arxiv.org/abs/2401.06730), analyzes the consequences of LLMs’ hesitance in producing uncertain statements, which can affect user trust. Our research aims to refine these interactions to make language models more intuitive and reliable tools for users.

More about me