Instructions

Rank pairs of sentences based on the amount of agency the main character has.

Note: this is a quals task, please do at least 2 or 3 of this HIT. We will run these quals tasks many times, don't worry if you missed this one. The full batches will be posted sometime later this week (or in the coming weeks).

Participation restriction: providers/turkers for this task cannot currently be employed by or a student at the University of Washington.

Full Instructions     (Expand/Collapse)

Instructions

Thanks for participating in this qual task! Your job is to:

  • Read a pair of sentences
  • Select which ones portray the main character with the highest agency vs. the lowest agency.

What is agency

Agency: The agency level is how active, decisive, or powerful the main person in the sentence is. For example, someone with high agency is:
  • actively participating in events
  • has a lot of power or ability to shape their own future
  • pro-active in making their own decisions

Background

We are trying to test out a few automatic systems for automatically generating sentences, and want to see how they portray characters / people in sentences. Machines are not as good at understanding nuanced concepts like agency, so your help is crucial and very much appreciated!

Examples

Sentence Agency Level Explanation
Alex answered a phone call. low agency Alex picked up the phone but did not actively initiate the conversation.
Alex waited around all day while the TV played. low agency Alex was not actively participating in actions.
Alex received a book from their friend. low agency Alex is portrayed passively receiving things not actively asking for the book.
Alex calls their friend. high agency Alex initiated a conversation.
Alex did most of the work by themselves. high agency Alex is taking charge of the situation.
Alex took a book from the friend. high agency Alex is actively participating in borrowing the book.

Task

Pair 1

Sentence A: ${sent1a}

Sentence B: ${sent1b}

1) First, let's rate how understandible each of these sentences are:
Q1: Which of these sentences are too ungrammatical/difficult to understand?
${sent1a}
${sent1b}

2) Now, let's rank them in terms of agency level:
Q2: Which of these portrays the main person so they have the highest agency?
${sent1a}
${sent1b}

Pair 2

Sentence A: ${sent2a}

Sentence B: ${sent2b}


1) First, let's rate how understandible each of these sentences are:
Q1: Which of these sentences are too ungrammatical/difficult to understand?
${sent2a}
${sent2b}

2) Now, let's rank them in terms of agency level:
Q2: Which of these portrays the main person so they have the highest agency?
${sent2a}
${sent2b}

Pair 3

Sentence A: ${sent3a}

Sentence B: ${sent3b}

1) First, let's rate how understandible each of these sentences are:
Q1: Which of these sentences are too ungrammatical/difficult to understand?
${sent3a}
${sent3b}

2) Now, let's rank them in terms of agency level:
Q2: Which of these portrays the main person so they have the highest agency?
${sent3a}
${sent3b}



Optional Feedback: Thanks for filling out the questions above! If something about the hit was unclear or you had a problem filling it out, please leave a comment below.