Last weekend I was lucky enough to attend IPA‘s second annual Researcher Gathering on Advancing Financial Inclusion at Yale University. My friend and collaborator Lasse Brune presented the proposed research design for a study we are planning with Eric Chyn on using deferred wages as a savings technology in Malawi. Our project expands builds on an earlier paper by myself and Lasse that shows that demand for deferred wages is quite high and that there are potentially-large benefits.
The conference featured lots of great talks, but I particularly liked the one Emily Breza gave. She was presenting early results from a paper called “Measuring the Average Impacts of Credit: Evidence from the Indian Microfinance Crisis” that she is writing with Cynthia Kinnan. (Here is a link to an older version of the paper.) One of the major results of the RCT revolution in development economics is a robust literature showing that microcredit – small loans targeted at the very poor, at below-market interest rates – has very limited economic benefits. This was a fairly surprising result, and at odds with both the priors of microfinance practitioners and previous non-experimental research.
Randomized evaluations of microcredit generally follow the following format. You find an area where microcredit doesn’t exist, and get a firm to expand to that area. But instead of providing it to everyone in the area, you convince the firm to randomly select people to offer the product to.
Breza and Kinnan turn that logic on its head. They instead look at markets that were already actively served by microcredit, where the supply of credit was drastically reduced. This reduction happened because of the 2010 Andhra Pradesh (AP) Microfinance Crisis, where a wave of suicides by indebted people caused the AP state government to ban microcredit in the state. Breza and Kinnan don’t study AP in particular; instead, they exploit the fact that some (but not all) microcredit suppliers had exposure AP through outstanding loans there, which defaulted en masse. This reduced their liquidity sharply and forced them to cut back on lending. So if you happened to live in an area largely served by microcredit firms that were exposed to AP, you suffered a large decline in the availability of microloans.
This clever research design yields effects on economic outcomes that are much larger than those estimated in traditional RCTs.* Is that surprising? I don’t think so, because we are studying market goods – those provided through reasonably well-functioning markets, as opposed to things like primary education in the developing world where markets are limited or non-existent.** Any randomized evaluation of a market good necessarily targets consumers who are not already served by the market.
In an RCT we can get the estimated effect of receiving a good by randomizing offers of the good and using the offer as an instrument for takeup. It’s well-known that these local average treatment effects are specific to the set of “compliers” in your study – the people induced to receive the good by the randomized offer. But usually the nature of these compliers is somewhat nebulous. In Angrist and Evans’ study of childbearing and labor supply, they are the people who are induced to have a third kid because their first two kids are the same sex (and people tend to want one of each). Are those people similar to everyone else? It’s hard to say.
In the context of market goods, however, the compliers have a specific and clear economic definition. They are the consumers who firms find it unprofitable to serve. Here is a simplistic illustration of this point:
These are the subset of all consumers with the lowest willingness-to-pay for the good – so we know that they experience the lowest aggregate benefits from it.*** RCTs can only tell us about their benefits, which are plausibly a lower bound on the true benefits across the population. To learn about the actual average treatment effects, in this context, we need a paper like Breza and Kinnan’s.
So were the randomized evaluations of microcredit even worth doing at all? Absolutely. They tell us about what might happen if we expand microcredit to more people, which is policy-relevant for two reasons. First, the priced-out people are the next group we might think about bringing it to. Second, we have a decent sense that markets are not going to serve them, so it’s up to governments and NGOs to decide do so. That decision requires understanding the incremental benefits and costs of expanding microcredit, as compared with all other potential policies.
A broader conclusion is that any study of a good or product that might be provided by markets needs to confront the question of why it is not being provided already – or, if it is, why it is worth studying the benefits of the product for the subset of people whom markets have deemed unprofitable.
*An exception is another paper by Breza and Kinnan, along with Banerjee and Duflo, which finds some long-run benefits of random microcredit offers for people who are prone to entrepreneurship.
**This is distinct from the concept of public vs. private goods. Education is not a public good per se as it is both excludable and rival, but it is typically provided by the state and hence standard profit motives need not apply.
***Leaving aside Chetty’s point about experienced utility vs. decision utility.