Reading, Fighting and 'rithmetic... RCT battles

I don't often find myself on the same side of an issue as Jeffrey Sachs, but he has taken on the randomistas, and their use of Randomized Controlled Trials (RCTs) in development. As my bleeps know, I have some issues with RCTs as well. (See "If RCT's Could Kill" and "Is It Right to Have the Poor Pay?"). So while my reasons are different than Sachs's, we agree that RCTs are not necessarily a valid, or useful, way to evaluate development programs.

Round 1: New York Times article by Joe Nocera on Sachs
Round 2: Sachs responds
Round 3: Dean Karlan weighs in on Freakonomics site

While Sachs has been charged with being an "ineffectual utopian" by another development economist, William Easterly, I think the randomistas are dangerous utopians. And I prefer the company of ineffectual utopians over dangerous ones.

The randomistas have taken a tool appropriate to medicine, and applied it to cultural behavior, and appear to believe that it will have similar predictive ability. When one tests a medicine in Detroit, one can be fairly confident that it will also work (or not) in Delhi because people are genetically very similar. Not so with culture.  Can one generalize to rural Brazil from tests of people's willingness to use free bed nets in Kenya, or on gender impacts of microfinance in India? If these programs do not work in these areas, they may be cause for caution, but they are not predictive of failure elsewhere. But the randomistas, and their followers, are quick to proclaim that these interventions don't work. Large scale damnations for petty sins.

Furthermore, it seems utopian to think that if people prefer to get something for free (like bed nets), that manufacturers will oblige by continuing to provide that something for free, forever. One has to consider the system over the long term. As Paul Polak has said well, we cannot "donate people out of poverty." Development needs markets, and markets need price signals.

This issue matters, and is this fight is among smart, experienced, well-intentioned and respected people. While it could seem like an academic debate, it has to do with real issues and real people. It isn't just a "wonk war." Which types of development programs work? Which don't? How does one test the effectiveness of a program?  Can one predict whether a program that is successful in one area (or not) will be successful (or not) in another area?

These are multi-billion dollar questions, with potential effects on millions of people. Poor decisions based on improper analysis by "searchers" using RCTs can be more dangerous than development efforts by "planners" that ignore the needs and desires of the beneficiaries. Neither are optimal, of course, but one is worse. Dean Karlan states a concern for future children, and I do too. The ones that don't have the advantage of  the mosquito nets, financial access, and cook stoves that might have been available to their families, but for the mis-use of RCT results.