Instructions

Thanks for participating in this new task! We are trying to test out a few automatic systems for rephrasing sentences so that characters portrayed have higher agency.
Given an sentence from a story and two automatically revised versions, please answer the following three questions about the revised sentences:

• Agency overall
  Which of the revised sentences portrays the main person so they have the highest agency overall (regardless of meaning preservation)?
• Meaning
  Which do you think is closer in meaning to the original sentence (regardless of agency change)?
• Agency and meaning similarity
  Which sentence best portrays the main person with highest agency while preserving the general content of the sentence?

Some notes:

• What is agency?
  The agency level is how active, decisive, or powerful the main person in the sentence is. For example, someone with high agency is actively participating in events, has a lot of power or ability to shape their own future, or pro-active in making their own decisions.
• What is meaning similarity?
  Two sentences are similar in meaning or overall content if they describe roughly the same event and have roughly the same interpretation. But, it's okay if a few details in the sentence get changed a bit.
• What is the revised sentence is hard to understand?
  If you cannot answer the revisions, let us know by selecting impossible to understand. If the sentence has some errors but is possible to understand, select some grammar errors.

Background

We are trying to test out a few automatic systems for automatically generating sentences, and want to see how they portray characters / people in sentences. Machines are not as good at understanding nuanced concepts like agency, so your help is crucial and very much appreciated!

Examples

Original	Revisions	Agency	Meaning	Agency & Meaning	Notes
Alex answered a phone call.	Alex called up a friend.		✔	✔	Here, both revisions give Alex more agency, yet calling a friend is closer in meaning to answering a phone call.
	Alex visited their friend's house.	✔
Alex loves football.	Alex loves watching football.		✔		Even though loving football and loving to watching are closer in meaning, Alex loves to play football portrays Alex with more agency without much overall meaning change.
	Alex loves to play football.	✔		✔
Alex won a pie-eating contest.	Alex lost a pie-eating contest.				Winning is the same as taking home a trophy.
	Alex took home the first-place trophy.	✔	✔	✔
In class, Alex often participated.	In class, Alex often listened.		✔		Even though Alex participating and listening is closer in meaning, interjecting gives Alex more agency and doesn't drastically change the events.
	In class, Alex often interjected.	✔		✔
Alex went for a walk.	They decided to go swimming.	✔			Deciding to go swim implies more agency, but strolling is closer to the original sentence.
	They strolled in the park.		✔	✔

Notes/FAQ:

• You are welcome to complete more than one HIT! As long as you take your time and read each HIT carefully, you may complete as many as you want.
• We estimate that each of these HIT's may take around 1-2 minutes. Please read carefully. (Of course, it's ok if you take slightly more or less time. We know that everyone has different reading speeds).