Things I would do if I had infinity billion dollars

I occasionally daydream about all the things I would do if I had more money to run my project, like having a car to get around instead of hoping that a minibus will be going where I want to get to sometime soon. When I’m actually lucky enough to get another grant, of course, I immediately budget it toward a bigger sample size. Having enough respondents to measure effects precisely is just much more important than, say, covering the cost of my housing instead of paying for that out of my pocket.

My suspicion is that this is a pretty common position to be in, at least for graduate students. Hence when I first learned about Computer-Assisted Personal Interviewing (CAPI) – basically programming a tablet computer to automate your survey – my reaction was “who the hell can afford that?” It’s a really neat technology: you can build in data validity checks, so people don’t put impossible values in a field, and also skip rules so that your enumerators don’t go to the wrong place on a survey. Even better, you don’t need to enter the data. The Development Impact Blog has a pair of posts up about CAPI, covering its advantages and potential drawbacks. They cite a paper that showed the quality of data is higher using CAPI than traditional paper surveys.

Even though the paper is quite long, it doesn’t seem to contain the key details you’d need in order to know whether it showed anything useful: the extent of idiot-proofing in the paper survey forms and the entry templates. When I design surveys, for example, I almost never have my enumerators hand-write in codes for a given response. Imagine a question on ethnic group, where 1 is Chewa, 2 is Nyanja, 3 is Lomwe, etc. Many surveys I’ve seen have a blank space where the enumerator has to write the code value. Following my advisor’s scheme, I have check boxes for each value. This makes it much harder to screw up, and checking the surveys is way easier – you can just eyeball it, and not look to see if the code is valid. I’d like to run a John Henry vs. the Machine-type contest where I design a survey that’s totally idiot-proof and an entry template full of validity checks, and compete with a CAPI. I might be able to win based on perfecting the question phrasing and translations – people say that changing a CAPI is pretty hard, and I edit my surveys many times because translations and phrasing matter a lot.

The main downside people emphasize about CAPI is that there’s a lot of programming involved. Aine McCarthy makes this point in a post describing the survey design process. That seems like a red herring to me: properly entering and cleaning data from paper surveys should involve very similar code, since you’ll want the same validity checks and verification of skip rules. In sharp contrast, the costs are extremely different and in fairly inflexible ways. To run a CAPI I’d need to buy the devices and the software needed to program them; I’d need a secure office to store them where they can’t be stolen; I’d need insurance and/or a scheme for making sure my enumerators don’t lose, break, or make off with them; and I’d need reliable power to charge the things out in the field or I’d need to make regular trips back to a place with power. Storage is a major issue – it would probably mean commuting from an office to the field sites every day. That would add tons of time to the surveys, either ballooning costs or cutting my sample size. So CAPI isn’t going to work for me, at least this time around. Maybe someday, when I win the lottery.