LTI 11-830 Ethics, Safety, and Social Impact in NLP and LLMs (Spring 2025)

---- ### Schedule |Week|Date|Theme|Topics|Assignments| |---|---|---|---|---| |1|01/14|Introduction|Motivation, requirements and overview|All homeworks released| ||01/16|Introduction|Project expectations and ideas|| |2|01/21|Philosophical Foundations & History|Overview: Moral theories, trolley problem, history of ethics, etc.|| ||01/23|Philosophical Foundations & History|Human subjects, IRB and crowdsourcing|Preproposals & project teams due| |3|01/28|Philosophical Foundations & History|Discussion|| ||01/30|Objectivity and Bias|Stereotypes, prejudice, and discrimination: background|Proposal is due (Monday 02/03)| |4|02/04|Objectivity and Bias|Bias in AI systems|| ||02/06|Objectivity and Bias|Discussion|[HW1](https://canvas.cmu.edu/courses/45279/assignments/797257) due| |5|02/11|Toxicity and Safety of LLMs|Overview: Hate speech, Toxicity, Hate speech detection|| ||02/13|Toxicity and Safety of LLMs|Toxicity in LMs & Toxicity in Conversations|| |6|02/18|Toxicity and Safety of LLMs|Aligning and Detoxifying LMs|| ||02/20|Toxicity and Safety of LLMs|Discussion|| |7|02/25|Toxicity and Safety of LLMs|*Guest lecture -- [Liwei Jiang](https://liweijiang.me/) and [Nouha Dziri](https://liweijiang.me/) -- LLM Alignment and Jailbreaking*|| ||02/27|Misinformation & Manipulation|Overview: Misinformation & Manipulation|[HW 2](https://canvas.cmu.edu/courses/45279/assignments/797258) due| |--|03/04|--- *Spring break* ----|---|| ||03/06|--- *Spring break* ---|---|| |8|03/11|Misinformation & Manipulation|*Guest lecture, [Saadia Gabriel](https://saadiagabriel.com/) -- LLMs & Misinformation*|| ||03/13|Midterm project check-ins|----|Midterm project check ins| |9|03/18|Misinformation & Manipulation|Discussion|| ||03/20|Privacy, Profiling, Security|Overview: Privacy and profiling|[HW 3](https://canvas.cmu.edu/courses/45279/assignments/797259) due| |10|03/25|Privacy, Profiling, Security|LLMs and Privacy|| ||03/27|Guest lecture|TBD|| |11|04/01|NLP for Social Good|NLP for social good overview, pitfalls, challenges|| ||04/03|No class -- Spring Carnival|...|| |12|04/08|NLP for Social Good|Discussion|[HW 4](https://canvas.cmu.edu/courses/45279/assignments/797260) due| ||04/10|NLP for Social Good|*Guest lecture — [Graham Neubig](https://www.phontron.com/) — Inclusive Multilingual NLP*|| |13|04/15|NLP for Social Good|*Guest lecture — [Lama Ahmad](https://www.linkedin.com/in/lamaahmad) — Perspectives from OpenAI*|| ||04/17|NLP for Social Good|*Guest lecture — [Emma Strubell](https://strubell.github.io/) — Energy Considerations in NLP*|[HW 5](https://canvas.cmu.edu/courses/45279/assignments/797261) due (*extra credit*)| |14|04/22|Final project presentations||| ||04/24|Final project presentations||Final project presentations| |15|04/29|---|---|Final reports due| ---- ### Grading Grades are based on a combination of homeworks and participation (individual) and a semester-long class project (group). All assignments are due at 11:59pm Eastern on the specified date. - **Homework assignments.** (4 assignments; 50% total) Each assignment contains a combination of coding, analysis, and discussion. For each assignment, completing the baseline requirements will obtain a passing (B-range) grade. A-range grades can be obtained through completing the open-ended “Advanced Analysis” part of the assignment. Assignments are not necessarily designed to focus on technical solutions, but instead to encourage students to think critically about the course material and understand how to approach ethical problems in NLP, while also allowing for exploration of various methodologies. - **Project.** (30%) a semester-long 3- or 4-person team project (more details below and in class). - **Participation.** (20%) classes will include discussions of reading assignments. Students will be expected to read relevant papers and participate in class discussions. Participation points can also be earned by posting interesting questions and useful answers on Canvas/Slack. ---- ### Projects A major component of this course is a team project. It will be a substantial research effort carried out by each group of students (expected group size = 3; 2-4 is acceptable). You can find some project ideas and resources [here](https://docs.google.com/document/d/1PfUU79IyiQUlbENwthkjbg4pTE-hAhXX9p7PwQLTNhg/edit?usp=sharing). Please use the "Find project groups" discussion in Canvas to find classmates with complementary interests and form groups. There will be a number of project milestones throughout the semester: - **Pre-proposal:** (2%) Brainstorming phase: pick two or three project ideas and flesh them out, in a 1-2 paragraph describing the focus area of the project along with quick-and-dirty method descriptions and stretch goals. Also, define team members. - **Proposal:** (8%) Pick a project: A 2-3 page document ([ACL format](https://github.com/acl-org/acl-style-files)) containing a literature review, concrete problem definition and bias statement, evaluation criteria, identification of baseline models, and ideas for final models. Sections should include Introduction (incl. Bias Statement), Related Work, Data, Evaluation, Baseline, Proposed Approach. Baselines should be clearly defined but do not need to be implemented yet. - **Midterm check-ins:** (2%) An in-class presentation of project and current progress. Presentation should include problem definition, baseline models and results, and description of proposed models. - **Final Presentations:** (6%) In-class presentations of the project will be held during the last week of classes. See here for [rubric and tips on how to make your final presentation](projectRubric.html). - **Final Report:** (12%) A final project report will be due the following week. We will use the 8-page ACL Rolling Review [format](https://github.com/acl-org/acl-style-files), [author guidelines](https://aclrollingreview.org/authors), and [Code of Ethics](https://www.aclweb.org/portal/content/acl-code-ethics). See [here for a rubric](projectRubric.html). ---- ### Discussion format Some of the lectures will be discussion-based. To make the discussion more lively and engaging, we will adopt reading roles, inspired by [Alec Jacobson and Colin Raffel' seminar](https://colinraffel.com/blog/role-playing-seminar.html) and Aditi Raghunathan's [15-884 class](https://www.cs.cmu.edu/~aditirag/teaching/15-884F22.html). Each student will play one of the following roles: - *Reviewer*: who discusses the strengths and weaknesses of the paper, either advocates for the paper to be accepted at a conference (e.g., [ACL, EMNLP](https://2021.aclweb.org/blog/instructions-for-reviewers/), [FAccT](https://facctconference.org/2022/reviewer_instructions.html)) or advocates for its rejection. - *Archaeologists I & II*: who determines where this paper sits in the context of previous and subsequent work. Keep an eye out for work that contradicts the takeaways in the current paper. - *Forensic archaeologist*: find and report on at least one older paper cited within the current paper that substantially influenced the current paper and at least one newer paper that cites this current paper. Do not choose any of the other papers assigned for this week. - *Community archaeologist*: choose two of the other papers assigned for this week, and discuss connections or conflicts with respect to your assigned paper. **This role will be responsible for sharing out in class** (see below). - *Academic researcher*: who proposes potential follow-up projects not just based on the current paper but also only possible due to the existence and success of the current paper. - *Visitor from the past*: who is a researcher from the early 2000s. They must discuss how they comprehend the results of the paper, what they like or dislike about the settings and benchmarks considered, and what surprises them the most about presented results. In-class discussion: - We will spend ~20-30 minutes in small groups discussing individual papers. We suggest each role goes in order and discusses their notes, while the community archaeologist takes notes (with the help of others). Each group must come up with a question they want to pose to the rest of the class to seed discussion. - Then, each of the community archaeologists will share out to the summary of their small group discussion. - Start by summarizing your paper in 2-3 minutes, explaining the main problem setup, the approach, and the takeaways. - Briefly (less than 1 minute) discuss the positives and negatives (gathered from the *Reviewer*), as well as the papers that have influenced and been influenced by the current paper (gathered from the *Forensic archaeologist*). - Briefly (less than 1 minute) describe possible follow up research projects (gathered from the *Academic researcher*) and how the paper is contextualized based on a researcher from early the 2000s (gathered from the *Visitor from the past*). - Briefly (less than 1 minute) explain which two other papers in the class are related (gathered from the *Community archaeologist*). Do not summarize these papers too much, leave that up to the group who was reading that paper. - Pose your question to the full class - After each paper, we will engage in a short full-class discussion about each paper. Either students will ask questions about the papers, or we will attempt to answer the seed question posed by that paper's group. Discussion grading: - You will be automatically assigned a role for each discussion class - **Before each discussion class**, you will post your talking points on Canvas (due the day before class). These notes should include your role, and 2-4 talking points. Make sure to go into enough depth that others can understand what you're saying (simply adding more shallow talking points will not lead to a better grade). - **After each discussion class**, you will reply to your notes and briefly discuss one reflection or point that you had not previously thought about. This can be significantly shorter than the initial post. ---- ### Policies **Late policy.** Each student will have 4 total late days that may be used for HW assignments at any point during the semester. Once the 4 days have been used up, you can still submit your assignments late, but at a loss of 20% (i.e., maximum grade is 80%), and not after the next assigment is due / after finals week. *Note: late days may not be used for project benchmarks.* **Academic honesty.** Homework assignments are to be completed individually. Verbal collaboration on homework assignments is acceptable, as well as re-implementation of relevant algorithms from research papers, but everything you turn in must be your own work, and you must note the names of anyone you collaborated with on each problem and cite resources that you used to learn about the problem. The project is to be completed by a team. You are encouraged to use existing NLP components in your project; you must acknowledge these appropriately in the documentation. Suspected violations of academic integrity rules will be handled in accordance with the [CMU guidelines on collaboration and cheating](http://www.cmu.edu/policies/documents/Cheating.html). **AI-assisted writing policy.** We will follow the [ACL 2023 policy](https://2023.aclweb.org/blog/ACL-2023-policy/) on the use of AI assistants for writing. This means you may use AI tools for checking your writing and correcting your grammar; any usage beyond that (idea generation, literature review, etc.) is strongly discouraged or forbidden, see the policy for details. **Accommodations for students with disabilities.** If you have a disability and have an accommodations letter from the Disability Resources office, we encourage you to discuss your accommodations and needs with the instructors as early in the semester as possible. We will work with you to ensure that accommodations are provided as appropriate. If you suspect that you may have a disability and would benefit from accommodations but are not yet registered with the [Office of Disability Resources](https://www.cmu.edu/disability-resources/), we encourage you to contact them via their website. ---- ### Prerequisites and required skills Since the focus is on critical thinking about AI and NLP technologies, this course is meant to be accessible to many people, with only basic experience with machine learning or natural language processing skills required. Concretely, you will need to: - Measure word and label co-occurrence statistics in an NLP corpus ([HW1](./hw1.html)) - Do basic data processing and training a neural network classifier using standard well-documented libraries ([HW2](./hw2.html)) - Use LLM APIs or run open-source LLMs if you want ([HW3](./hw3.html)) [HW1](./hw1.html) and [HW2](./hw2.html) are the most computational assignments, so feel free to take a look at those to determine whether taking this class is the right fit for you. Additionally, projects can have varying degrees of computational or algorithmic components, and can be qualitative in nature. To help you calibrate, **in the past, several HCII students have taken the course and done well**. ---- ### Note to students **Take care of yourself!** As a student, you may experience a range of challenges that can interfere with learning, such as strained relationships, increased anxiety, substance use, feeling down, difficulty concentrating and/or lack of motivation. This is normal, and all of us benefit from support during times of struggle. There are many helpful resources available on campus and an important part of a healthy life is learning how to ask for help. Asking for support sooner rather than later is almost always helpful. CMU services are available free to students, and treatment does work. You can learn more about confidential mental health services available on campus through [Counseling and Psychological Services (CaPS)](https://www.cmu.edu/counseling/). Support is always available (24/7) at: 412-268-2922. **Take care of your classmates and instructors!** In this class, every individual will and must be treated with respect. The ways we are diverse are many and are fundamental to building and maintaining an equitable and an inclusive campus community. These include but are not limited to: race, color, national origin, caste, sex, disability (visible or invisible), age, sexual orientation, gender identity, religion, creed, ancestry, belief, veteran status, or genetic information. Research shows that greater diversity across individuals leads to greater creativity in the group. We at CMU work to promote diversity, equity and inclusion not only because it is necessary for excellence and innovation, but because it is just. Therefore, while we are imperfect, we ask you all to fully commit to work, both inside and outside of our classrooms to increase our commitment to build and sustain a campus community that embraces these core values. It is the responsibility of each of us to create a safer and more inclusive environment. Incidents of bias or discrimination, whether intentional or unintentional in their occurrence, contribute to creating an unwelcoming environment for individuals and groups at the university. If you experience or observe unfair or hostile treatment on the basis of identity, we encourage you to speak out for justice and offer support in the moment and/or share your experience using the following resources: - [Center for Student Diversity and Inclusion](https://www.cmu.edu/student-diversity/): [csdi@andrew.cmu.edu](mailto:csdi@andrew.cmu.edu), (412) 268-2150 - [CMU anonymous reporting hotline](https://secure.ethicspoint.com/domain/media/en/gui/81082/index.html), (844) 587-0793

Jimin Mun

Lindia Tjuatja

Akhila Yerukola

Instructor

Maarten Sap

TAs