Things I would do if I had infinity billion dollars

I occasionally daydream about all the things I would do if I had more money to run my project, like having a car to get around instead of hoping that a minibus will be going where I want to get to sometime soon. When I’m actually lucky enough to get another grant, of course, I immediately budget it toward a bigger sample size. Having enough respondents to measure effects precisely is just much more important than, say, covering the cost of my housing instead of paying for that out of my pocket.

My suspicion is that this is a pretty common position to be in, at least for graduate students. Hence when I first learned about Computer-Assisted Personal Interviewing (CAPI) – basically programming a tablet computer to automate your survey – my reaction was “who the hell can afford that?” It’s a really neat technology: you can build in data validity checks, so people don’t put impossible values in a field, and also skip rules so that your enumerators don’t go to the wrong place on a survey. Even better, you don’t need to enter the data. The Development Impact Blog has a pair of posts up about CAPI, covering its advantages and potential drawbacks. They cite a paper that showed the quality of data is higher using CAPI than traditional paper surveys.

Even though the paper is quite long, it doesn’t seem to contain the key details you’d need in order to know whether it showed anything useful: the extent of idiot-proofing in the paper survey forms and the entry templates. When I design surveys, for example, I almost never have my enumerators hand-write in codes for a given response. Imagine a question on ethnic group, where 1 is Chewa, 2 is Nyanja, 3 is Lomwe, etc. Many surveys I’ve seen have a blank space where the enumerator has to write the code value. Following my advisor’s scheme, I have check boxes for each value. This makes it much harder to screw up, and checking the surveys is way easier – you can just eyeball it, and not look to see if the code is valid. I’d like to run a John Henry vs. the Machine-type contest where I design a survey that’s totally idiot-proof and an entry template full of validity checks, and compete with a CAPI. I might be able to win based on perfecting the question phrasing and translations – people say that changing a CAPI is pretty hard, and I edit my surveys many times because translations and phrasing matter a lot.

The main downside people emphasize about CAPI is that there’s a lot of programming involved. Aine McCarthy makes this point in a post describing the survey design process. That seems like a red herring to me: properly entering and cleaning data from paper surveys should involve very similar code, since you’ll want the same validity checks and verification of skip rules. In sharp contrast, the costs are extremely different and in fairly inflexible ways. To run a CAPI I’d need to buy the devices and the software needed to program them; I’d need a secure office to store them where they can’t be stolen; I’d need insurance and/or a scheme for making sure my enumerators don’t lose, break, or make off with them; and I’d need reliable power to charge the things out in the field or I’d need to make regular trips back to a place with power. Storage is a major issue – it would probably mean commuting from an office to the field sites every day. That would add tons of time to the surveys, either ballooning costs or cutting my sample size. So CAPI isn’t going to work for me, at least this time around. Maybe someday, when I win the lottery.

6 thoughts on “Things I would do if I had infinity billion dollars”

  1. The answer to “who the hell can afford CAPI” is pretty much the same as the answer to “who the hell can afford ____” with blank here representing any slight advancement over the “graduate student slogging around a country with a bag of questionnaires” model (think: cars, computers, translators etc.) – The World Bank!

    I am pretty impressed with the amount of interviewing work that is now being done on computers instead of paper but I often wonder if this will not just give rise to new issues rather than less issues.

  2. I just found out that Kim and Tara used CAPI on iPads to run their study here. So it’s no longer just a crazy thing that IPA and the World Bank do.

    But you’re right on the money with the potential for new issues. There’s already too much programming going on that’s done by social scientists who aren’t trained programmers. Now our sloppy programming practices are going to infect the data itself, so that you can’t just go back and redo stuff; you literally won’t have another version. We need to rethink how researchers are trained, I think.

  3. True, I helped out a bit on that project – mapping things out and finding EAs for them to work in.
    Agreed on rethinking how researchers are trained… I think there is some merit to including field resaerch techniques training as an option at the undergraduate level – catch ’em early.

Leave a Reply to Jason Kerwin Cancel reply

Your email address will not be published. Required fields are marked *