Higher pay brought more people back, but not better answers
With higher pay, more workers on an online platform completed follow-up. Response quality did not improve.
The key idea
Higher pay brought more people back. The answers did not clearly improve.
The study, in plain language
The experiment varied the initial payment and the increase offered for a second survey. The team then examined three outcomes: how well people responded, how they perceived the payment, and who completed follow-up.
Completing follow-up was more common among those paid more at first and offered a larger increase for the second stage. In contrast, there was no convincing evidence that payments improved overall performance. Perceptions of choice, fairness, and compensation varied by measure and stage: each question showed a different pattern.
What they found
- Higher payments did not show an improvement in overall response quality.
- Higher initial pay and an offered increase for the second stage were associated with more completed follow-ups.
- Views about having a choice and payment fairness varied by measure.
How they studied it
The study examined 359 U.S. MTurk adults in a two-stage survey. The design combined three initial payments with three multipliers offered for the second stage, six weeks later. Researchers measured performance, payment perceptions, and follow-up completion. The article reports that one case completing more than one condition was removed from the dataset.
What the study cannot show
- The study used one platform and one online task with participants who opted in. Fewer people responded to the second survey.
- The study does not show that increasing any wage fails to improve performance in real jobs or define a fair payment for everyone.
The original study
Pay for performance, satisfaction and retention in longitudinal crowdsourced research
Source under CC0 1.0. English adaptation and explanation by PaperFlow, with AI assistance. The text has been reorganized for a general audience; it is not the complete paper and does not imply endorsement by its authors.
One study alone does not represent a field's consensus. Educational content does not replace professional advice. Report a possible error.
