A natural experiment involving 5,740 participants in a MOOC ( massive open online course) has found that when students were asked to assess each other's work, and the examples were exceptional, a large proportion of students dropped the course.

In the MOOC, as is not uncommon practice, course participants were asked to write an essay and then to grade a random sample of their peers' essays. Those randomly assigned to evaluate exemplary peer essays were dramatically more likely to quit the course than those assigned to read more typical essays.

Specifically, around 68% of students who graded essays of average quality finished and passed the course, earning a certificate. Among those who graded slightly above average essays (more than one standard deviation above the class mean, 7.5/9), 64% earned a certificate. But among those who graded the best essays (those more than 1.6 SDs above the mean), only 45% earned a certificate.

These numbers can be compared to the fact that 75% of students who wrote an average essay earned a certificate, and 95% of those who wrote a 'perfect' essay, 9/9, earned a certificate. The difference between these numbers is about the same (in fact, slightly less) than the effect of grading average vs top essays.

A follow-up study, involving 361 participants, simulated this setting, in order to delve into what the students thought. Participants, recruited via Amazon's Mechanical Turk, were asked to write a minimum of 500 characters in response to a quote and essay prompt. They were told the best responses would go into a lottery to win a bonus. They were then asked to assess two very short essays (about 200 words) supposedly written by peers. These were either both well-written, or both poorly-written. This was followed by some questions about what they felt and thought, and an opportunity to write a second essay.

Unsurprisingly, those who were given exceptional essays to grade felt significantly less able to write an essay as good as those. They also decided that the ability to write an excellent short answer to such philosophical questions was not very important or relevant to them, and were much more likely not to write another essay (43% of those who read the poor essays went on to try again, while only 27% of those who read the excellent essays did so).

Until now, research has mainly focused on how students respond when peer work is of a standard that the student is likely to see as “attainable”. This research shows how comparisons that are seen as unattainable may do more harm than good.

http://www.eurekalert.org/pub_releases/2016-02/afps-sep020216.php