Ignorance is bliss

AS ALL fans of crime fiction know, DNA is the gold standard of forensic science. Or is it? Itiel Dror, a cognitive psychologist at University College, London, thinks this doctrine of infallibility needs to be questioned. His problem is not with the technology itself, but with the way it is deployed. For he has gathered evidence that DNA examiners' interpretations of their results are, at least in complex cases, open to subjectivity and bias.

When America's National Academy of Sciences produced a report on the state of forensic science in 2009, it criticised many of the methods then in use. Citing earlier research by Dr Dror, the report's authors stated, for example, that fingerprint examiners' claims of zero error rates were scientifically implausible. DNA, however, was spared their criticism. Now Dr Dror and Greg Hampikian, a forensic biologist at Boise State University in Idaho, have published a study in Science & Justice that suggests all is not shipshape in the domain of the double helix either.

Do Not Adulate

Dr Dror's and Dr Hampikian's experiment presented data from a real case to 17 DNA examiners working in an accredited government laboratory in North America. The case involved a gang rape in the state of Georgia, in which one of the rapists testified against three other suspects in exchange for a lighter sentence, as part of a plea bargain. All three denied involvement, but the two DNA examiners in the original case both found that they could not exclude one of the three from having been involved, based on an analysis of swabs taken from the victim.

As is almost always true in forensic-science laboratories, these examiners knew what the case was about. And their findings were crucial to the outcome because in Georgia, as in many other states, a plea bargain cannot be accepted without corroborating evidence. However, of the 17 examiners Dr Dror and Dr Hampikian approached—who, unlike the original two, knew nothing about the context of the crime—only one thought that the same suspect could not be excluded. Twelve others excluded him, and four abstained.

Though they cannot prove it, Dr Dror and Dr Hampikian suspect the difference in contextual information given to the examiners was the cause of the different results. The original pair may have subliminally interpreted ambiguous information in a way helpful to the prosecution, even though they did not consciously realise what they were doing.

And DNA data are ambiguous more often than is generally realised. Dr Dror thinks that in about 25% of cases, tiny samples or the mixing of material from more than one person can lead to such ambiguity. Moreover, such is DNA's reputation that, when faced with claims that the molecule puts a defendant in a place where a crime has been committed, that defendant will often agree to a plea-bargain he might otherwise not have accepted.

This one example does not prove the existence of a systematic problem. But it does point to a sloppy approach to science. According to Norah Rudin, a forensic-DNA consultant in Mountain View, California, forensic scientists are beginning to accept that cognitive bias exists, but there is still a lot of resistance to the idea, because examiners take the criticism personally and feel they are being accused of doing bad science. According to Dr Rudin, the attitude that cognitive bias can somehow be willed away, by education, training or good intentions, is still pervasive.

Medical researchers, by contrast, take great care to make drug trials “blind”, so that neither the patient nor the administering doctor knows who is receiving the drug being tested, and who is getting a control drug or placebo. When someone's freedom—and, in an American context, possibly his life, as well—is at stake, it surely behoves forensic-science laboratories to take precautions that are equally strong.