Journal

  1. Liwei Jiang, Jena D Hwang, Chandra Bhagavatula, Ronan Le Bras, Jenny Liang, Jesse Dodge, Keisuke Sakaguchi, Maxwell Forbes, Jon Borchardt, Saadia Gabriel, Yulia Tsvetkov, Oren Etzioni, Maarten Sap, Regina Rini & Yejin Choi (2024) An Empirical Investigation of Machines’ Capabilities for Moral Judgment with the Delphi Experiment. Nature Machine Intelligence.
  2. Jocelyn Shen, Daniella DiPaola, Safinah Ali, Maarten Sap, Hae Won Park & Cynthia Breazeal (2024) Empathy Towards AI vs Human Experiences: The Role of Transparency in Mental Health and Social Support Chatbot Design. JMIR Mental Health.
  3. Maarten Sap, Anna Jafarpour, Yejin Choi, Noah A. Smith, James W. Pennebaker & Eric Horvitz (2022) Quantifying the narrative flow of imagined versus autobiographical stories. PNAS.
  4. Gregory Park, H Andrew Schwartz, Maarten Sap, Margaret L Kern, Evan Weingarten, Johannes C Eichstaedt, Jonah Berger, David J Stillwell, Michal Kosinski, Lyle H Ungar & Martin E P Seligman (2017) Living in the Past, Present, and Future: Measuring Temporal Orientation with Language. Journal of Personality.
  5. Margaret L Kern, Gregory Park, Johannes C Eichstaedt, H Andrew Schwartz, Maarten Sap, Laura K, Smith & Lyle H Ungar (2016) Gaining Insights From Social Media Language: Methodologies and Challenges. Psychological Methods.
  6. Johannes C Eichstaedt, H Andrew Schwartz, Margaret L Kern, Gregory Park, Darwin R Labarthe, Raina M Merchant, Sneha Jha, Megha Agrawal, Lukasz A Dziurzynski, Maarten Sap, Christopher Weeg, Emily Larson, Lyle H Ungar & Martin E P Seligman (2015) Psychological Language on Twitter Predicts County-level Heart Disease Mortality. Psychological Science 26(2). SAGE Publications. 159--169.
  7. Charlene A Wong, Maarten Sap, Hansen Andrew Schwartz, Robert Town, Tom Baker, Lyle Ungar & Raina M Merchant (2015) Twitter Sentiment Predicts Affordable Care Act Marketplace Enrollment. Journal of Medical Internet Research 17(2). JMIR Publications Inc..
  8. Raina M. Merchant, Yoonhee P. Ha, Charlene A. Wong, H. Andrew Schwartz, Maarten Sap, Lyle H. Ungar & David A. Asch (2014) The 2013 US Government Shutdown (#Shutdown) and Health: An Emerging Role for Social Media. American Journal of Public Health 2014. e1--e3.

Conference

  1. Joel Mire, Maria Antoniak, Elliott Ash, Andrew Piper & Maarten Sap (2024) The Empirical Variability of Narrative Perceptions of Social Media Texts. EMNLP.
  2. Xuhui Zhou, Zhe Su, Tiwalayo Eisape, Hyunwoo Kim & Maarten Sap (2024) Is This the Real Life? Is This Just Fantasy? The Misleading Success of Simulating Social Interactions With LLMs. EMNLP.
  3. Jocelyn Shen, Joel Mire, Hae Won Park, Cynthia Breazeal & Maarten Sap (2024) HEART-felt Narratives: Tracing Empathy and Narrative Style in Personal Stories with LLMs. EMNLP.
  4. Liwei Jiang, Kavel Rao, Seungju Han, Allyson Ettinger, Faeze Brahman, Sachin Kumar, Niloofar Mireshghallah, Ximing Lu, Maarten Sap, Nouha Dziri & Yejin Choi (2024) WildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language Models. NeurIPS.
  5. Jimin Mun, Liwei Jiang, Jenny Liang, Inyoung Cheong, Nicole DeCario, Yejin Choi, Tadayoshi Kohno & Maarten Sap (2024) Particip-AI: A Democratic Surveying Framework for Anticipating Future AI Use Cases, Harms and Benefits. AIES.
  6. Devansh Jain, Priyanshu Kumar, Samuel Gehman, Xuhui Zhou, Thomas Hartvigsen & Maarten Sap (2024) PolygloToxicityPrompts: Multilingual Evaluation of Neural Toxic Degeneration in Large Language Models. COLM.
  7. Maria Antoniak, Joel Mire, Maarten Sap, Elliott Ash & Andrew Piper (2024) Where Do People Tell Stories Online? Story Detection Across Online Communities. ACL.
  8. Akhila Yerukola, Saujas Vadugur, Daniel Fried & Maarten Sap (2024) Is the Pope Catholic? Yes, the Pope is Catholic. Generative Evaluation of Non-Literal Intent Resolution in LLMs. ACL.
  9. Kaitlyn Zhou, Jena D Hwang, Xiang Ren & Maarten Sap (2024) Relying on the Unreliable: The Impact of Language Models' Reluctance to Express Uncertainty. ACL.
  10. Ruiyi Wang, Haofei Yu, Wenxin Zhang, Zhengyang Qi, Maarten Sap, Graham Neubig, Yonatan Bisk & Hao Zhu (2024) SOTOPIA-π: Interactive Learning of Socially Intelligent Language Agents. ACL.
  11. Jimin Mun, Cathy Buerger, Jenny T. Liang, Joshua Garland & Maarten Sap (2024) Counterspeakers’ Perspectives: Unveiling Barriers and AI Needs in the Fight against Online Hate. CHI.
  12. Natalie Shapira, Mosh Levy, Hossein Seyed Alavi, Xuhui Zhou, Yejin Choi, Yoav Goldberg, Maarten Sap & Vered Shwartz (2024) Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Models. EACL.
  13. Xuhui Zhou, Hao Zhu, Leena Mathur, Ruohong Zhang, Haofei Yu, Zhengyang Qi, Louis-Philippe Morency, Yonatan Bisk, Daniel Fried, Graham Neubig & Maarten Sap (2024) SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents. ICLR.
  14. Niloofar Mireshghallah, Hyunwoo Kim, Xuhui Zhou, Yulia Tsvetkov, Maarten Sap, Reza Shokri & Yejin Choi (2024) Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory. ICLR.
  15. Ashutosh Baheti, Ximing Lu, Faeze Brahman, Le Ronan Bras, Maarten Sap & Mark Riedl (2024) Leftover-Lunch: Advantage-based Offline Reinforcement Learning for Language Models. ICLR.
  16. Taylor Sorensen, Liwei Jiang, Jena Hwang, Sydney Levine, Valentina Pyatkin, Peter West, Nouha Dziri, Ximing Lu, Kavel Rao, Chandra Bhagavatula, Maarten Sap, John Tasioulas & Yejin Choi (2024) Value Kaleidoscope: Engaging AI with Pluralistic Human Values, Rights, and Duties. AAAI.
  17. Akhila Yerukola, Xuhui Zhou, Elizabeth Clark & Maarten Sap (2023) ``Don't Take This Out of Context!'' On the Need for Contextual Models and Evaluations for Stylistic Rewriting. EMNLP.
  18. Yiming Zhang, Sravani U. Nanduri, Liwei Jiang, Tongshuang Wu & Maarten Sap (2023) BiasX: ``Thinking Slow'' in Toxic Language Annotation with Explanations of Implied Social Biases. EMNLP.
  19. Jocelyn Shen, Maarten Sap, Pedro Colon-Hernandez, Hae Won Park & Cynthia Breazeal (2023) Modeling Empathic Similarity in Personal Narratives. EMNLP.
  20. Jimin Mun, Emily Allaway, Akhila Yerukola, Laura Vianna, Sarah-Jane Leslie & Maarten Sap (2023) Beyond Denouncing Hate: Strategies for Countering Implied Biases and Stereotypes in Language. Findings of EMNLP.
  21. Hyunwoo Kim, Melanie Sclar, Xuhui Zhou, Ronan Le Bras, Gunhee Kim, Yejin Choi & Maarten Sap (2023) FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactions. EMNLP.
  22. Hyunwoo Kim, Jack Hessel, Liwei Jiang, Peter West, Ximing Lu, Youngjae Yu, Pei Zhou, Ronan Le Bras, Malihe Alikhani, Gunhee Kim, Maarten Sap & Yejin Choi (2023) SODA: Million-scale Dialogue Distillation with Social Commonsense Contextualization. EMNLP.
  23. Xuhui Zhou, Hao Zhu, Akhila Yerukola, Thomas Davidson, Jena D. Hwang, Swabha Swayamdipta & Maarten Sap (2023) COBRA Frames: Contextual Reasoning about Effects and Harms of Offensive Statements. Findings of ACL.
  24. Julia Mendelsohn, Ronan Le Bras, Yejin Choi & Maarten Sap (2023) From Dogwhistles to Bullhorns: Unveiling Coded Rhetoric with Language Models. ACL.
  25. Sebastin Santy*, Jenny T. Liang*, Ronan Le Bras, Katharina Reinecke & Maarten Sap (2023) NLPositionality: Characterizing Design Biases of Datasets and Models. ACL.
  26. Skyler Hallinan, Alisa Liu, Yejin Choi & Maarten Sap (2023) Detoxifying Text with MaRCo: Controllable Revision with Experts and Anti-Experts. ACL.
  27. Organizers Of QueerinAI, Anaelia Ovalle, Arjun Subramonian, Ashwin Singh, Claas Voelcker, Danica J. Sutherl, Davide Locatelli, Eva Breznik, Filip Klubicka, Hang Yuan, J Hetvi, Huan Zhang, Jaidev Shriram, Kruno Lehman, Luca Soldaini, Maarten Sap, Marc Peter Deisenroth, Maria Leonor Pacheco, Maria Ryskina, Martin Mundt, Milind Agarwal, Nyx McLean, Pan Xu, A Pranav, Raj Korpan, Ruchira Ray, Sarah Mathew, Sarthak Arora, St John, Tanvi An, Vishakha Agrawal, William Agnew, Yanan Long, Zijie J. Wang, Zeerak Talat, Avijit Ghosh, Nathaniel Dennler, Michael Noseworthy, Sharvani Jha, Emily Baylor, Aditya Joshi, Natalia Y. Bilenko, Andrew McNamara, Raphael Gontijo-Lopes, Alex Markham, Evyn Dǒng, Jackie Kay, Manu Saraswat, Nikhil Vytla & Luke Stark (2023) Queer In AI: A Case Study in Community-Led Participatory AI. FAccT.
  28. Hyunwoo Kim, Youngjae Yu, Liwei Jiang, Ximing Lu, Daniel Khashabi, Gunhee Kim, Yejin Choi & Maarten Sap (2022) ProsocialDialog: A Prosocial Backbone for Conversational Agents. EMNLP.
  29. Maarten Sap, Ronan Le Bras, Daniel Fried & Yejin Choi (2022) Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMs. EMNLP.
  30. Zhijing Jin, Sydney Levine, Fernando Gonzalez Adauto, Ojasv Kamal, Maarten Sap, Mrinmaya Sachan, Rada Mihalcea, Joshua B. Tenenbaum & Bernhard Schölkopf (2022) When to Make Exceptions: Exploring Language Models as Accounts of Human Moral Judgment. NeurIPS.
  31. Maarten Sap, Swabha Swayamdipta, Laura Vianna, Xuhui Zhou, Yejin Choi & Noah A. Smith (2022) Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detection. NAACL.
  32. Prithviraj Ammanabrolu, Liwei Jiang, Maarten Sap, Hanna Hajishirzi, Yejin Choi & Noah A. Smith (2022) Aligning to Social Norms and Values in Interactive Narratives. NAACL.
  33. Thomas Hartvigsen, Saadia Gabriel, Hamid Palangi, Maarten Sap, Dipankar Ray & Ece Kamar (2022) ToxiGen: Controlling Language Models to Generate Implied and Adversarial Toxicity. ACL.
  34. Saadia Gabriel, Skyler Hallinan, Maarten Sap, Pemi Nguyen, Franziska Roesner, Eunsol Choi & Yejin Choi (2022) Misinfo Reaction Frames: Reasoning about Readers' Reactions to News Headlines. ACL.
  35. Jesse Dodge, Maarten Sap, Ana Marasović, William Agnew, Gabriel Ilharco, Dirk Groeneveld, Margaret Mitchell & Matt Gardner (2021) Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus. EMNLP.
  36. Ashutosh Baheti, Maarten Sap, Alan Ritter & Mark Riedl (2021) Just Say No: Analyzing the Stance of Neural Dialogue Generation in Offensive Contexts. EMNLP.
  37. Alisa Liu, Maarten Sap, Ximing Lu, Swabha Swayamdipta, Chandra Bhagavatula, Noah A. Smith & Yejin Choi (2021) DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Experts. ACL.
  38. Albert Xu, Eshaan Pathak, Eric Wallace, Suchin Gururangan, Maarten Sap & Dan Klein (2021) Detoxifying Language Models Risks Marginalizing Minority Voices. NAACL.
  39. Xuhui Zhou, Maarten Sap, Swabha Swayamdipta, Yejin Choi & Noah A. Smith (2021) Challenges in Automated Debiasing for Toxic Language Detection. EACL.
  40. Xinyao Ma*, Maarten Sap*, Hannah Rashkin & Yejin Choi (2020) PowerTransformer: Unsupervised Controllable Revision for Biased Language Correction. EMNLP.
  41. Sam Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi & Noah A Smith (2020) RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models. Findings of EMNLP.
  42. Maxwell Forbes, Jena D. Hwang, Vered Shwartz, Maarten Sap & Yejin Choi (2020) Social Chemistry 101: Learning to Reason about Social and Moral Norms. EMNLP.
  43. Maarten Sap, Eric Horvitz, Yejin Choi, Noah A Smith & James W. Pennebaker (2020) Recollection versus Imagination: Exploring Human Memory and Cognition via Neural Language Models. ACL.
  44. Maarten Sap, Saadia Gabriel, Lianhui Qin, Dan Jurafsky, Noah A Smith & Yejin Choi (2020) Social Bias Frames: Reasoning about Social and Power Implications of Language. ACL.
  45. Maarten Sap*, Hannah Rashkin*, Derek Chen, Ronan LeBras & Yejin Choi (2019) Social IQa: Commonsense Reasoning about Social Interactions. EMNLP.
  46. Maarten Sap, Dallas Card, Saadia Gabriel, Yejin Choi & Noah A Smith (2019) The Risk of Racial Bias in Hate Speech Detection. ACL.
  47. Antoine Bosselut, Hannah Rashkin, Maarten Sap, Chaitanya Malaviya, Asli Celikyilmaz & Yejin Choi (2019) COMET: Commonsense Transformers for Automatic Knowledge Graph Construction. ACL.
  48. Maarten Sap, Ronan LeBras, Emily Allaway, Chandra Bhagavatula, Nicholas Lourie, Hannah Rashkin, Brendan Roof, Noah A Smith & Yejin Choi (2019) ATOMIC: An Atlas of Machine Commonsense for If-Then Reasoning. AAAI.
  49. Hannah Rashkin, Antoine Bosselut, Maarten Sap, Kevin Knight & Yejin Choi (2018) Modeling Naive Psychology of Characters in Simple Commonsense Stories. ACL.
  50. Hannah Rashkin*, Maarten Sap*, Emily Allaway, Noah A. Smith & Yejin Choi (2018) Event2Mind: Commonsense Inference on Events, Intents, and Reactions. ACL.
  51. Maarten Sap, Marcella Cindy Prasetio, Ari Holtzman, Hannah Rashkin & Yejin Choi (2017) Connotation Frames of Power and Agency in Modern Films. EMNLP.
  52. Roy Schwartz, Maarten Sap, Ioannis Konstas, Li Zilles, Yejin Choi & Noah A Smith (2017) The Effect of Different Writing Tasks on Linguistic Style: A Case Study of the ROC Story Cloze Task. CoNLL.
  53. H. Andrew Schwartz, Gregory Park, Maarten Sap, Evan Weingarten, Johannes Eichstaedt, Margaret Kern, David Stillwell, Michal Kosinski, Jonah Berger, Martin Seligman & Lyle Ungar (2015) Extracting Human Temporal Orientation from Facebook Language. NAACL.
  54. Maarten Sap, Gregory Park, Johannes C. Eichstaedt, Margaret L. Kern, David J. Stillwell, Michal Kosinski, Lyle H. Ungar & Hansen Andrew Schwartz (2014) Developing Age and Gender Predictive Lexica over Social Media. EMNLP.

Workshop

  1. Emily Allaway, Nina Taneja, Sarah-Jane Leslie & Maarten Sap (2022) Towards Countering Essentialism through Social Bias Reasoning. EMNLP workshop on NLP for Positive Impact.
  2. Zhilin Wang, Anna Jafarpour & Maarten Sap (2022) Uncovering Surprising Event Boundaries in Narratives. Workshop on Narrative Understanding.
  3. Tal August, Maarten Sap, Elizabeth Clark, Katharina Reinecke & Noah A. Smith (2020) Exploring the Effect of Author and Reader Identity in Online Story Writing: the StoriesInTheWild Corpus. Workshop on Narrative Understanding, Storylines, and Events (NUSE)@ ACL.
  4. Roy Schwartz, Maarten Sap, Ioannis Konstas, Li Zilles, Yejin Choi & Noah A Smith (2017) Story Cloze task: UW NLP System. EACL Workshop LSD Sem. 52--55.
  5. Daniel Preotiuc-Pietro, Maarten Sap, H Andrew Schwartz & Lyle Ungar (2015) Mental Illness Detection at the World Well-Being Project for the CLPsych 2015 Shared Task. NAACL Workshop on CLPsych.
  6. Daniel Preotiuc-Pietro, Johannes Eichstaedt, Gregory Park, Maarten Sap, Laura Smith, Victoria Tobolsky, H Andrew Schwartz & Lyle Ungar (2015) The Role of Personality, Age and Gender in Tweeting about Mental Illnesses. NAACL Workshop on CLPsych.
  7. H Andrew Schwartz, Johannes Eichstaedt, Margaret L Kern, Gregory Park, Maarten Sap, David Stillwell, Michal Kosinski & Lyle Ungar (2014) Towards Assessing Changes in Degree of Depression through Facebook. ACL Workshop on CLPsych. 118--125.

Demo

  1. Maria Antoniak, Anjalie Field, Ji Min Mun, Melanie Walsh, Lauren F. Klein & Maarten Sap (2023) Riveter: Measuring Power and Social Dynamics Between Entities. ACL demonstrations.
  2. Hao Fang, Hao Cheng, Maarten Sap, Elizabeth Clark, Ariel Holtzman, Yejin Choi, Noah A Smith & Mari Ostendorf (2018) Sounding Board: A User-Centric and Content-Driven Social Chatbot. NAACL System Demonstrations.
  3. H Andrew Schwartz, Salvatore Giorgi, Maarten Sap, Patrick Crutchley, Lyle Ungar & Johannes Eichstaedt (2017) DLATK: Differential Language Analysis ToolKit. EMNLP System Demonstrations. 55--60.

Other

  1. Maarten Sap (2021) Positive AI with Social Commonsense Models.
  2. Hao Fang, Hao Cheng, Elizabeth Clark, Ariel Holtzman, Maarten Sap, Mari Ostendorf, Yejin Choi & Noah A Smith (2017) Sounding Board - University of Washington’s Alexa Prize Submission. Alexa Prize Proceedings.
  3. H Andrew Schwartz, Maarten Sap, Margaret L Kern, Johannes C Eichstaedt, Adam Kapelner, Megha Agrawal, Eduardo Blanco, Lukasz Dziurzynski, Gregory Park, David Stillwell, Michal Kosinski, Martin E P Seligman & Lyle H Ungar (2016) Predicting individual well-being through the language of social media. Biocomputing 2016: Proceedings of the Pacific Symposium. 516--527.

Preprint

  1. Jing-Jing Li, Valentina Pyatkin, Max Kleiman-Weiner, Liwei Jiang, Nouha Dziri, Anne G. E. Collins, Jana Schaich Borg, Maarten Sap, Yejin Choi & Sydney Levine (2024) SafetyAnalyst: Interpretable, transparent, and steerable LLM safety moderation. arXiv.
  2. Wenkai Li, Jiarui Liu, Andy Liu, Xuhui Zhou, Mona Diab & Maarten Sap (2024) BIG5-CHAT: Shaping LLM Personalities Through Training on Human-Grounded Data. arXiv.
  3. Xuhui Zhou, Hyunwoo Kim, Faeze Brahman, Liwei Jiang, Hao Zhu, Ximing Lu, Frank Xu, Bill Yuchen Lin, Yejin Choi, Niloofar Mireshghallah, Ronan Le Bras & Maarten Sap (2024) HAICOSYSTEM: An Ecosystem for Sandboxing Safety Risks in Human-AI Interactions. arXiv.
  4. Zhe Su, Xuhui Zhou, Sanketh Rangreji, Anubha Kabra, Julia Mendelsohn, Faeze Brahman & Maarten Sap (2024) AI-LieDar: Examine the Trade-off Between Utility and Truthfulness in LLM Agents. arXiv.
  5. Xianzhe Fan, Qing Xiao, Xuhui Zhou, Jiaxin Pei, Maarten Sap, Zhicong Lu & Hong Shen (2024) User-Driven Value Alignment: Understanding Users' Perceptions and Strategies for Addressing Biased and Discriminatory Statements in AI Companions. arXiv.
  6. Jen-tse Huang, Jiaxu Zhou, Tailin Jin, Xuhui Zhou, Zixi Chen, Wenxuan Wang, Youliang Yuan, Maarten Sap & Michael R. Lyu (2024) On the Resilience of Multi-Agent Systems with Malicious Agents. arXiv.
  7. Kaitlyn Zhou, Jena D. Hwang, Xiang Ren, Nouha Dziri, Dan Jurafsky & Maarten Sap (2024) Rel-A.I.: An Interaction-Centered Approach To Measuring Human-LM Reliance. arXiv.
  8. Abhinav Rao*, Akhila Yerukola*, Vishwa Shah, Katharina Reinecke & Maarten Sap (2024) NormAd: A Benchmark for Measuring the Cultural Adaptability of Large Language Models. arxiv.