I listen to the Freakonomics Radio podcast every week, out of a vague sense of wanting to be on top of how applied microeconomics is seen in the media, and, more important, because I need something to distract my brain as I do all the menial tasks to wrap up my fieldwork here in Malawi; eventually I just run out of other podcasts that interest me. The show is usually a slightly-less-interesting version of This American Life, rather than a real show about microeconomics, which is too bad; I think the world really needs a microeconomics podcast and I wish this one were it but it isn’t, quite. What really bothers me is their pattern of failing to talk about the interesting economics of an issue – there was one short on whether selling beer at sports venues could reduce public drunkenness, which it appears to have done in some cases, but no real explanation of why that may have happened.*
I’ve never been more frustrated with how shallow their coverage is than this week, when they spent the whole episode plugging Freakonomics Experiments, a website that helps you make decisions. The idea is that if you can’t choose whether to, say, change jobs, then you go to the site, take a survey, and then it flips a coin (presumably using a computerized pseudorandom number generator) for you to tell you which option to pick. The heart of the episode is focused on the claimed psychological benefits of flipping a coin to make decisions, and in particular on how it may be preferable to have someone else do the coin flip. I’m not convinced that the Freakonomics team actually believes that claim, but even if they do it’s not at all why they are running this website. They talk very little about the real reason, and that’s unfortunate: unlike hearing people talk about how flipping coins to make choices has improved their lives, the real reason for Freakonomics Experiments is actually interesting, and actually has something to do with economics.
The truth, which everyone has already guessed from the title, is that they are running an experiment. The interesting part is why this is necessary. Take the example of changing jobs. Suppose we want to know what the effect of changing jobs is on your income ten years down the road. We’re trying to estimate b in the equation Y = bC + e, where Y is your eventual income, C is an indicator for whether you changed jobs, and e is an error term. b tells us how much your income goes up if you change jobs versus the case where you stay in the same one. We can’t just compare people who did change jobs to people who did not, because C is not assigned at random. Indeed, headhunters are much more likely to swoop in and hire away people who are going to be worth more in the future, and hence earn more irrespective of where they work. The “coin flip” solves this problem. People who use the site aren’t sure what they want to do, and the random number generator tells them what to do. Now C is assigned at random and we can really measure what happens when you change jobs.
This is all very clever. It’s even more clever than it appears: even if people don’t always obey the random number generator, it is still a valid instrumental variable for the choice of whether to change jobs. That is, we can focus on the variation in C that is driven by the Freakonomics Experiments website – to speak very imprecisely, we can look just at those who do what the “coin” says to do – and look at the effects on that group. What disappoints me is the huge missed opportunity here to talk about the difficulties of doing research on economics (and decisionmaking more broadly) and to help the show’s listeners learn about what kinds of before-and-after comparisons are untrustworthy and why. Going with the cute story, instead of talking about the real reasons for the project, does their audience a disservice.