They also served

“I BECAME a statistician because I was put in prison,” says Claus Moser. Aged 92, he can look back on a distinguished career in academia and civil service: he was head of the British government statistical service in 1967-78 and made a life peer in 2001. But statistics had never been his plan. He had dreamed of being a pianist and when he realised that was unrealistic, accepted his father’s advice that he should study commerce and manage hotels. (“He thought I would like the music in the lobby.”)

War threw these plans into disarray. In 1936, aged 13, he fled Germany with his family; four years later the prime minister, Winston Churchill, decided that because some of the refugees might be spies, he would “collar the lot”. Lord Moser was interned in Huyton, near Liverpool (pictured above). “If you lock up 5,000 Jews we will find something to do,” he says now. Someone set up a café; there were lectures and concerts—and a statistical office run by a mathematician named Landau. One day Lord Moser sat with him at lunch. By the meal’s end he had agreed to be the mathematician’s assistant—and found his vocation.

Released after a few months on account of his youth (aged just 17, he should never have been interned), he studied statistics instead of commerce at the London School of Economics, and graduated with a stellar first. That, and his fluent German, earned him a place in a top-secret group based in newly liberated Versailles that visited the sites of RAF bombing raids to find out whether pilots’ reports of the damage they had done matched reality. (On the whole, yes; a little exaggeration.) After the war he became an academic. When in 1967 the prime minister, Harold Wilson, asked him to become head of the government statistical service, the recognition felt particularly sweet: just two years earlier he had been refused a sabbatical in the same branch of public service because of his former status as an enemy alien.

Random walks

Some war stories are tales of derring-do; others are more cerebral. Both types specialise in plot twists: wars knock people off one path and onto another. Lord Moser was one of dozens of young men in Britain who would never have become statisticians had it not been for the war, who helped win it for Britain and its allies—and who went on to transform statistics.

Wars are when governments find the money and will to make things happen, says Eileen Magnello, a historian of statistics. As it mobilised and supplied troops, grew and rationed food, and retooled factories to make shells, planes and tanks, Britain’s government realised that its knowledge of the country’s society, economy and manufacturing skills was not up to the task—and that statisticians could help.

“When I was called upon to clothe the army in May 1939, the War Office had no statistical evidence to assist me,” wrote Frederick Marquis, first Earl of Woolton and minister of food for much of the war, in 1946. “I had the greatest difficulty in arriving at any figures that would show how many suits of uniform and how many boots were involved.” Estimates of losses in action or from attacks on supply dumps “were little more than guesses”. He turned to Major Greenwood, an epidemiologist at the London School of Hygiene and Tropical Medicine. “In those urgent months before the war [he helped me] to create a statistical basis on which the operations of this aspect of the war effort could be built.”

Ministries drove Churchill to distraction with conflicting figures on crucial subjects such as shipping tonnage

In 1939 Churchill, then first Lord of the Admiralty, asked Frederick Lindemann, a distinguished scientist, to set up a statistical arm of the civil service. When Churchill became prime minister the following year the “S Branch” became his personal statistical service, and a central office was created. The Royal Statistical Society (RSS) had proposed something similar in 1919, only to be told it would be impracticable. But now the alternative was unbearable. Ministries drove Churchill to distraction with conflicting figures on crucial subjects such as shipping tonnage. The central office, the war cabinet was told, would produce “a regular series of figures on a coherent and well-ordered basis…that will be accepted and used without question”. The new statisticians worked on government accounts; rationing (which ensured no Britons starved and greatly improved the diet of the poorest third); manpower surveys; the “pay as you earn” system of taxation (which raised the cash needed to wage the war); and the Beveridge Report on social insurance that later led to the founding of the welfare state.

The lady tasting tea

Karl Pearson is regarded as the father of modern statistics. In 1911 he founded the world’s first statistics department, at University College London (UCL), aiming to convert statistics from “the playing field of dilettanti and controversialists into a serious branch of science”. A freethinker, proto-feminist and believer in “war with inferior races” as essential to improving what eugenicists called the “human stock”, he put the study of variation within populations, and correlations between characteristics, on a firm intellectual footing. (Readers who have done a first course in statistics will know him from the chi-squared test.)

Ronald Fisher, at first Pearson’s protégé and then, as two big egos clashed, his enemy, had turned down a post in Pearson’s department in 1919 and gone instead to Rothamsted, an obscure agricultural research station in Hertfordshire that wanted someone to make sense of its records from 76 years of crop trials. As he did what he described later as “raking over the muck-heap”, Fisher realised that many had been too poorly done to prove anything at all.

So he set out to understand how such work could be done most efficiently, and without the results being swamped by external influences such as the weather. His answer was to vary several factors of interest simultaneously, and randomly assign different treatments to different plots. He criticised the conventional wisdom that “we must ask Nature few questions, or, ideally, one question, at a time. The writer is convinced that this view is wholly mistaken. Nature, he suggests, will best respond to a logical and carefully thought out questionnaire.”

He was the first person to think systematically about when to discount chance as the explanation of a trial’s result. He used a parodically English example: that of the “lady tasting tea”. A woman claims she can tell by the taste whether milk has been poured into a cup before the tea or after it. How often does she have to be tested, and how often must she be right, to be believed?

He reasoned that when comparing treatments (such as pouring the milk first or last), a researcher should look at the probability that a difference at least as big as that observed would have occurred if the treatments were the same—for example, if the lady could not really distinguish milk first from milk last by taste. If that probability is very small, he argued, the observed difference can be taken to represent something real: the lady genuinely can taste the difference. (Those readers who recognised “chi-squared” will know this as a simple version of a significance test.)

Legend has it that Fisher’s un-named lady was Muriel Bristol-Roach, a biologist at Rothamsted. David Salsburg, a retired statistician who called his history of 20th-century statistics “The Lady Tasting Tea” in homage to the story, says he heard from an associate of Fisher that the test had actually been carried out one sunny afternoon—and that she had been right every time.

Not just the perfect cup of tea, but feeding Britain, which before the war had imported most of its food, was at stake. Under Fisher’s leadership Rothamsted had become a statistical powerhouse, and in 1940, Frank Yates, formerly his deputy and now running Rothamsted, summarised the results of all previous fertiliser trials. His findings were used to rewrite the rules governing fertiliser rationing, and to balance imports of food and fertiliser, given the cost and danger of shipping.

Surprisingly, Fisher was not directed to war work, though by 1939 he was well known in scientific circles: he seems to have been unfairly suspected of fascist leanings. Perhaps his greatest contribution to the war effort was via his two books: “Statistical Methods for Research Workers”, published in 1925, and “The Design of Experiments”, published a decade later, which fell into the hands of many of those who ended up doing statistical war work.

One of them was George Box, a young sergeant from Gravesend who had left school aged 16 in 1936 to work as an assistant to a chemist in a sewage plant. This background brought him to Porton Down, where the army was studying the effects of poisons such as mustard gas. Sergeant Box knew enough to tell his superiors that they needed a statistician—and that he had once tried to read a book by someone called Fisher, though he had not understood it.

“Well, you read the book so you’d better do it,” came the reply. In his engaging memoir, “An Accidental Statistician”, Box describes testing treatments for poison-gas victims. He wrote to Fisher, by then at Cambridge, to ask for help. Fisher invited him to visit. “The Army, however, did not have a procedure to send a sergeant to see a professor at Cambridge, so they made out a railway warrant that said I was taking a horse there.”

Volunteers at Porton Down would receive six drops of mustard-gas liquid on different parts of their arms. The challenge was to unpick variations in healing time caused by treatments from those caused by natural variation between individuals. Different substances were tested as ointments, adopting Fisher’s approach to crop trials; Box found that they all made matters worse and established what is still the correct treatment for mustard-gas blisters: clean them, dress them and leave them alone.

Experimental subjects

As the war progressed and the value of statistics became clearer, some Cambridge mathematics students were drafted to statistical work. David Cox, a grammar-school boy from Birmingham, was among them. In 1944 he went to the Royal Aircraft Establishment, a research institute in Farnborough. “The naive assumption was that if you were good enough to get a good degree in mathematics you would pick up statistics in a week or two,” he says. He was part of a group that analysed the distribution of German bombs falling on London each day. They concluded that the Germans were trying to destroy the docks but missing.

Sir David also worked on quality-control in the manufacture of aircraft components, and the calculation of the distribution of stresses on aircraft in flight. “The aim was to load planes up to the point that the wings were about to drop off,” he says. The research meant the RAF dropped more bombs, and brought more pilots safely home, than it would have otherwise.

A personally momentous project concerned the strength of a joint in metal made with “spot” rather than continuous welds. The probability that it holds depends on the probability that each spot does: if some fail, the others may or may not be strong enough to bear the extra load. He described the relationship between the strengths of the individual welds and the whole in a way that he now he describes as “correct but extremely clumsy”.

One day, after the war had ended but he had not yet been demobbed, Sir David found a paper giving an “extremely elegant and beautiful” proof of the same result, but in the context of the strength of wool and its individual strands. As he left the library he saw an ad for a job as assistant to the author, Henry Daniels, who had worked in the ministry of aircraft production and was returning to his pre-war post with the Wool Industries Research Association in Leeds. Sir David had intended to return to Cambridge, but on impulse he applied, setting his life on a different course.

The Cambridge students’ main destination was the Ministry of Supply, which in 1942 set up a unit, SR 17, to train industry and munitions factories in quality-control techniques. The terrible toll German attacks were taking on merchant shipping meant that as much as possible of everything had to be made locally; much of it by novice workers in new or adapted factories. Shoddy products were inefficient—and shoddy explosives dangerous. The young proto-statisticians learned the basics and passed them on: how to take a random sample; how large it should be; when to discard a batch on the basis of that sample.

Aged 18 when he joined, Peter Armitage was the youngest of the SR 17 recruits. He worked on what are now called “sequential methods”—analysing samples as they are taken rather than waiting until they are all gathered and doing the analysis in a single go. If production is going swimmingly—or indeed, catastrophically—the first few items reveal this, and sampling can be abandoned; only if the situation is less clear-cut does sampling need to continue. Wartime scarcity made this efficiency gain well worthwhile.

In “Statistics in the United Kingdom, 1939-45”, published in 1985, George Barnard and Robin Plackett, SR 17 alumni, recalled both such weighty matters and the group’s strong esprit de corps. Its young members kept records of their arrival times and measured punctuality by calculating how much they varied. They also founded a society, the Worshipful Order for the Unbuttoning of Taylor, dedicated to the study of the three jacket buttons that one of their number always kept done up.

Scatter plot

“Peace finally returned, and the statistical scene in the United Kingdom had been completely transformed,” wrote Barnard and Plackett. “No other method would have produced these changes in only six years.” Dozens of clever young people had been taught a fast-changing new subject—and in many cases done original research. Even routine work was elevated by the urgency and camaraderie of the war effort—and even the fact that they were new to the field. “A lot of the work was statistically boring,” Sir David says now. “But the point is that I didn’t really know anything.”

“After the war the section exploded like a London bomb into missionary statistical occupations all over the country,” wrote Geoffrey Jowett, one of the SR 17 alumni, in 1990. “In convincing others that we had a good product to sell we convinced ourselves.” He went on to do research in quality control at United Steel Companies in Sheffield, and then to Sheffield University, and thence to universities in Australia and New Zealand.

Box went to study with Egon Pearson, Karl’s son and successor as professor of statistics at UCL, and then to Imperial Chemical Industries, where he helped chemists design and analyse experiments. He later moved to the University of Wisconsin to found a statistics department. After four years at the Wool Industries Research Association Sir David Cox went on to an equally distinguished academic career. The pair wrote a paper together years later for the pleasure of the joint authorship: “Box and Cox”—the name of a popular farce in which two lodgers, one of whom works by day and one by night, rent the same room.

Many other war statisticians had equally illustrious careers. Interviews published in Significance, the magazine of the RSS, over the past decade contain many variations on a theme: an introduction to the subject during war service, followed by research in industry or academia. As time went by the statistical revolution spread into new fields, in particular to medicine.

Professor Armitage was recruited to the London School of Hygiene and Tropical Medicine by Austin Bradford Hill, who had studied the incidence and nature of civilian casualties during the war. Where to advise the population to take shelter during a raid? Were they safer upstairs, downstairs or outside? The work took tact, he recalled later: “One had to be careful in a post-raid inquiry that one did not infuriate shocked and perhaps homeless persons with what might well appear to be damn-fool questions.”

When he heard about Professor Armitage’s war work on sequential methods, Hill asked him to apply them to trials of medical treatments, in which Hill was a pioneer. (He was also the joint author, with Richard Doll, a doctor, of the famous 1950 paper that established the link between smoking and lung cancer.) Analysing the results of a trial bit by bit, rather than all at once when it was finished, meant it could be stopped straight away if it became clear that the new treatment was so good that everyone should be getting it, or indeed useless or even dangerous. Implementing this simple-sounding idea, now standard in medical trials, requires great statistical sophistication—and has saved many lives.

America, too, had seen big statistical advances during the war. Strong university research groups were founded, and the attempt to improve manufacturing spread quality-control methods through industry. Some important techniques were developed on both sides of the Atlantic around the same time—in particular the basics of sequential analysis. But in continental Europe statistical progress had been halted, or sometimes worse. René Carmille, a French administrator who had created a novel punch-card system for semi-automated analysis of population data to help with mobilisation against the Nazis, attempted to subvert its later use by the Vichy government to identify Jews. He was found out and died in Dachau. Only Britain experienced the dire need of all-out war footing, but was still stable enough for good statistical research.

“A consequence of Britain’s war effort was that some brilliant abstract thinkers gained a deep knowledge of industrial processes,” says Sir David Spiegelhalter, a medical statistician. (Barnard, whose pre-war research had been in very pure mathematics, after the war designed the British standard for testing condoms.) The collaboration fostered by the war effort between statisticians and industrialists, factory workers, government officials and researchers in other fields, together with the democratic post-war spirit, turned out self-effacing, intellectually generous types without whom statistics might have remained a riven discipline, as it had been during Pearson and Fisher’s inter-war feud. “Wars mix people up, but also bring them closer together,” says Sir David. “They were a unique and inspiring generation.”