Andrew Gelman links to Casey Mulligan’s take on whether survey respondents should be compensated for their time. Mulligan suggests that from the researcher’s perspective, paying people might lead to better results. But he seems to miss the most important reason for paid surveys leading to better data quality: running an unpaid survey means that your refusal rate will be an increasing function of how much respondents value their time (and their impatience, and a few other thing). Your wonderful random sample is now no longer representative of the whole population. This is maybe okay, but you don’t actually know the selection process. Also, the value people put on their time is, in rough approximation, equal to their wages. Wages are a common outcome variable in analyses, and as I taught my undergraduate econometrics students last term, selecting your sample on the outcome variable means your estimates will be biased – in layman’s terms, you’ll get the wrong answer.
Mulligan’s piece reminds me of Chris Blattman’s post on this same topic last year. He discusses two possible problems with paying, raised by Aine McCarthy. The first is that people might pay less attention or participate for the wrong reasons; Blattman points out that it’s possible to run an experiment to study that stuff. They both seem unlikely to me, though.
But the second issue Blattman discusses is just bizarre to me – the idea that we might establish a social norm of paying people for their time (okay, plausible) and that that somehow is a bad thing (wait, really?). I’ve seen this suggestion from a number of corners, and I don’t get it. You’re taking something of value from someone (a chunk of their time, sometimes more than a whole day); a norm that says they should be compensated for that is a great idea. These aren’t stray cats that we’re thinking about feeding – they’re people, who have other stuff going on, and, as I’ll discuss, may feel obliged to participate.
One concern often raised by non-economists and mentioned in the comments on Blattman’s post is that paying people, especially if you pay them too much relative to their income, is coercive. Maybe it’s indicative of my training as an economist that I find that argument confusing. Are jobs “coercion”? Those mostly involve being paid enough to do something you dislike.
But that’s beside the point here in Malawi, and probably in lots of other places as well. Despite researchers’ best efforts to ensure respondents that our surveys are voluntary through the informed consent process, virtually no one refuses. I’ve heard that this is because some government surveys are legally mandatory. I would feel terrible imposing on someone who thinks they are required to participate and not at least try to make it worth their while. It seems very much like stealing to me, and an abuse of my status and power in the places where I do research.