Measure for measure

RESEARCHERS are as sure as they can be of anything that a particular combination of cash transfers, assets (usually cows) and training lifts people out of deep poverty. That is not just a triumph for the NGOs that came up with the formula. It is also another coup for the randomised controlled trial (RCT), and its controversial but increasingly common use in the assessment of social policies.

In the 1920s Ronald Fisher, an English statistician, came up with a way to measure the effects of fertiliser on potatoes. Rather than spreading it on entire fields and studying the result, he divided fields into small plots and decided randomly which ones would be treated. Since external influences (such as soil quality and shade) would not differ systematically between treated and untreated plots, any difference in yields would probably be due to the fertiliser. Fisher’s ideas were seized on by statisticians doing quality control in munitions factories during the second world war. After the war ended, they were taken up in medicine.

These days RCTs are routine in development economics. Like potato plants, people eligible for help are randomly assigned to treatment and control groups. The difference between the two groups’ outcomes is used to assess the intervention’s effectiveness. The Abdul Latif Jameel Poverty Action Lab in Massachusetts, known as J-PAL, has started or completed 689 RCTs. Earlier this year it was estimated that the number of published papers on development involving formal impact evaluations (often but not always RCTs) rose more than tenfold from 2000 to 2012.

Emerging-world governments are commissioning RCTs partly because others are doing so, says Esther Duflo of J-PAL. As the trials spread, experienced researchers are becoming easier to find. It helps that some studies have had striking results. J-PAL attracted a lot of publicity, and some resentment, when it assembled evidence that microlending barely boosts consumption.

RCTs are being used more often to assess social policy in America, France and Scandinavia. In Britain there has been a flurry evaluating educational innovations—including one called “Teensleep”, in which the treatment group starts school at 10am (the idea is that they will learn more after a lie-in). Often the results are underwhelming, though no less important for that. A randomised experiment in New York showed that paying all teachers in a school more if their pupils do better in tests does not raise attainment.

But the emerging world remains far ahead. In Britain and America RCTs are mostly used to test minor policy tweaks, not fundamental reforms. In poorer countries, by contrast, they are being used to design welfare systems and measure corruption. Oriana Bandiera, who studied the BRAC programme in Bangladesh, is discussing with Zambia’s government the possibility of doing a randomised trial of political decentralisation. The technical difficulties will be immense. Still, she says, persuading Zambia to sign up is much easier than it would be to get Britain to give it a try.