Spilling the beans

“I’M THE one who looks the patient in the eye and tells them the trial is beneficial,” says Tim Crater, a research physician at the Hutchinson Clinic in Kansas. Dr Crater runs drug tests for large pharmaceutical firms. He says volunteers are interested in more than just the promise of payment. “A lot of people want to help, they are altruistic to a certain degree and want to advance science.” Dr Crater’s experience is typical. Those who participate in trials often believe that they are, in a small way, contributing to the advancement of medicine and that any suffering on their part will help others.

Unfortunately, this is not always the case. Though pertinent trials carried out by companies do have to be reported to those responsible for licensing drugs and medical devices, there is no obligation on firms to make them public. That means such trials cannot be scrutinised by outsiders. The licensing authorities look at them, of course, so anything approved should, in theory, be safe, and have at least some beneficial effect. But the practitioners who go on to use them do not know all the details.

Some estimates suggest the results of half of clinical trials are never published. These missing data have, over several decades, systematically distorted perceptions of the efficacy of drugs, devices and even surgical procedures. And that misperception has sometimes harmed patients.

In America, where most of the world’s drugs first receive approval, the law was changed in 2007 to try to deal with this problem. Trials, with the exception of early safety evaluations, are supposed to be registered on a website, clinicaltrials.gov. Then, within a year of the completion of data collection, their results are supposed to follow suit. That, at least, is the theory. Practice seems different.

A unfined mess

Earlier this year, a report in the New England Journal of Medicine combed through clinicaltrials.gov, looking to see how quickly after completion trials were reported. It found that, after the legal maximum of a year was up, just 17% of those paid for by industry had had their results published. Drug firms were not, though, the worst offenders. Only 8.1% of trials paid for by the National Institutes of Health, the American government’s main conduit for medical-research money, were reported within a year. And just 5.7% of the ones paid for by other government agencies and academic institutions were (see chart). Moreover, even though the Food and Drug Administration (FDA), the agency which monitors the website, has the power to fine companies that do not comply, it has never actually done so.

The assumption is that the missing trials tend to be those that show drugs in a less flattering light. Drug companies have an obvious incentive to play such trials down. Academic researchers, keen for glory and their next grants, may often prefer not to waste time on writing up results which show little effect. And journals are also culpable, for they notoriously favour papers which demonstrate that something works over those which demonstrate that it does not. As a result, many medicines look better than they are.

Ben Goldacre, a British doctor and author, is one of the instigators of an international campaign called AllTrials, which is designed to force researchers to publish all of their clinical trials. One drug he talks about is an antidepressant called reboxetine (branded as Edronax by its makers, Pfizer) that he used to prescribe to his patients. He says a couple of trials were published showing that this is as good for depression as any other drug. However, unpublished data, collected in trials involving three times as many people as participated in the published trials, showed it was not.

Something similar is true in the case of another class of antidepressant—selective serotonin-reuptake inhibitors (SSRIs), such as Prozac. These were introduced in the late 1980s, but it was not until 2006 that an analysis of all clinical-trial data on SSRIs, submitted to the FDA over the years, showed that their use by adolescents was associated with an increase in the risk of suicide. That fact might have been discovered earlier if all those data had been publicly available. An independent review in 2008 showed that 94% of published trials of SSRIs suggested they produced positive results, whereas this was the conclusion of only 51% of the (more numerous, but often unpublished) trials submitted to the FDA.

Even when no medical harm is done, financial harm can be. Since 2006, the British government has spent £424m ($660m) stockpiling Tamiflu, an antiviral drug, in order to anticipate an influenza pandemic. At the time the decision was taken, 60% of the trial data about this drug remained unpublished. Those data have now been analysed, and that analysis has raised questions about Tamiflu’s effectiveness in reducing hospital admissions, and thus about whether creating the stockpile was money well spent.

Nor is it just drug trials that are plagued by non-publication. In 1994 a study on surgery for bowel cancer found that, among those whose tumours returned, a second visit to the operating theatre to remove the resurgent carcinoma made no difference to life expectancy. Had this information been made public then, rather than as it was in 2014, countless very ill patients could have been spared surgical procedures. Also, and just as importantly, the wider question of how useful repeat surgery of this sort is on other parts of the body would have been raised sooner.

Opening up

There are signs, though, that real change is afoot. AllTrials, and the groups behind it, such as Sense About Science, a British charity, have helped shape legislation that will regulate clinical trials in Europe when it comes into force in 2016. That law, like the existing American one, will require drug trials to be registered and their results to be published. In America, meanwhile, regulators are proposing to write rules that will clarify and extend the scope of the existing legislation, in order to remove the wriggle-room that has made them hard to enforce.

Another reason to think companies might actually toe the line this time around is that their shareholders are increasingly concerned about the missing data. Helena Viñes Fiestas, who studies companies’ financial sustainability at BNP Paribas Investment Partners, a big bank, says many owners of drug-company shares support the push for publication, and are asking the firms in which they invest to publish plans that will make sure trials past, present and future are registered and their results reported.

Cynics may be surprised by this shareholder desire for transparency, since it could result in lower sales. But long-term investors would prefer the truth straight away, to reduce the level of risk in their portfolios. According Ms Viñes Fiestas, around 30% of a drug company’s value is tied to the results of trials.

There is also a more direct risk to firms that do not publish all their trial results. Between 2007 and 2014, the 21 large pharma companies that Ms Viñes Fiestas follows incurred collective fines of $40 billion. Her studies showed that around half of this sum was a consequence of a lack of clear reporting of side effects, which are often missing because trials containing these sorts of data go unpublished.

Sense About Science is working on an index, to be published later this year, that will rate pharma firms according to the extent of their commitment to publish all trials. This will, no doubt, be of interest to shareholders.

The future, then, may be more transparent. But the past remains opaque. It is likely that most drugs in use today have a few skeletons in their cupboards, in the form of unpublished trials, but no one (except those who actually ran them) knows how many. A few firms have started to open their archives. GlaxoSmithKline has published the results of all trials completed since its formation (via a merger) in 2000. Pfizer, founded in 1849, goes back to 2007. Others, though, do nothing.

Earlier this year the Institute of Medicine, the national academy of America’s doctors, said in a report that sharing data from old trials offers both risks and benefits, and should be considered case by case. One of the things the institute worried about was whether participants in old trials, which often happened before technological change made the question relevant, had given appropriate consent for the sort of sharing of their data which modern standards of transparency demand.

As that observation shows, digging up the past has its dangers. But the advantages of knowing the truth about past trials mean it is worth trying to overcome these. In any case, there is no excuse for not dealing with the trials of the future. Only if that is done will those such as Dr Crater, who perform the actual testing, be able to continue to look patients in the eye and tell them that their contribution really is making a difference.