Clinical Trial Prestidigitation - Making Old Data Look Worse So New Data Looks Better

Prestidigitation, also known as sleight of hand or legerdemain, is a form of stage magic which uses dextrous manipulation and misdirection to distract the viewer from seeing it.

The example, reported by Bloomberg on 26 February, 2014, is discussed late because the nature of the trick meant it took me a while before I understood how it was done.

Background

Pradaxa, a new anticoagulant, was approved by the US Food and Drug Administration (FDA) in 2010 for use by patients with atrial fibrillation to prevent strokes.  We discussed previously the release of documents during litigation that showed how marketers from its manufacturer, Boehringer - Ingelheim, attempted to manipulate the results of a clinical study of the drug.  The study results initially seemed to suggest that blood tests should be used to monitor therapy with it.  However, that would have contradicted a key marketing message, that Pradaxa was more convenient than warfarin because it did not require blood tests for monitoring.

Litigation by patients alleging that Boehringer-Ingelheim concealed the severity of the bleeding risks caused by the drug
continues.  The Bloomberg report dealt with how the company responded to requests by the FDA for data about these risks.  
 
The Question from the FDA

As per Bloomberg,

In the Pradaxa case, the regulators asked Boehringer to compare fatal bleeding reports received by the FDA against the number of patients using the drug in order to establish a death rate. The request was made to allow the FDA to 'evaluate the need for modifications to the Pradaxa label or future study,' according to a letter made public as part of the court files.

The company reported back that 6.1 of every 10,000 patients who used the drug after approval developed fatal bleeding, the court documents showed. The regulators also asked Boehringer to reanalyze results from an earlier study used to gain approval to see how that rate compared.

So, in other words, the FDA wanted to assess the rate of bleeding reflected by reports received after the drug had gone on the market (that is, the FDA was evaluating data from post-marketing surveillance.)  To do so, it wanted to calculate the proportion of patients who took the drug who developed fatal bleeding.  Then it wanted to compare this rate to the rate of fatal bleeding found in the previous controlled trials of the drug done by the manufacturer to secure drug approval.  Note that these trials are also considered the major source of clinical research evidence about the benefits and harms of the drug.

The major source of clinical evidence about Pradaxa (dabigatran) was the RE-LY trial (Randomized Evaluation of Long-Term Anticoagulation Therapy), sponsored by Boehringer-Ingelheim, published in the New England Journal of Medicine [Connolly SJ, Ezekowitz MD, Yusuf S et al.  Dabigatran versus warfarin in patients with atrial fibrillation.  N Engl J Med  2009; 361:1139-1151.  Link here.]   The trial reported that the rates of life-threatening bleeding were 1.8% per year for warfarin, 1.45% for high dose dabigatran, and the rates of major bleeding of 3.36% versus 2.21%.  However, the trial report did not include rates of fatal bleeding per se, nor the rates of patients with major bleeding who then died.  

Presto Chango

According to Bloomberg, here is how Boehringer-Ingelheim responded to the question from the FDA about the rate of fatal bleeding.

That effort produced two separate analyses by the company, according to the unsealed court documents. One, looking only at people whose primary cause of death was bleeding, found 5.8 of 10,000 patients died per year. The other, which included anyone who had a major bleeding event and died for any reason, found a rate of 19.5 fatal events per 10,000 patients per year, the documents show.

 The report sent to the FDA, though, contained only the analysis indicating the death rate from the earlier research was much higher than the numbers seen after approval, according to the court filings. Potentially, such a finding could head off any after-market action by the FDA because the data showed the drug was safer after it was approved and widely used.   

So, to recapitulate, Boehringer-Ingelheim used results from the RE-LY trial to persuade the FDA that its new drug was safe and effective.  However, the company did not include, at least in the results reported publicly, any measure of fatal bleeding in patients given the drug.  It now appears that the rate of major bleeding in patients who subsequently died was high, perhaps high enough to have raised questions about the safety of the drug prior to approval, had it been revealed.

However, after the drug had been approved, when the question was about rates of severe bleeding after the drug had been marketed, the company furnished an analysis of data from the clinical trial that now showed a very high bleeding rate. That was the major misdirection, the sleight of hand.. It would make the rate of fatal bleeding in the post-marketing surveillance data appear low. 

But apparently this data also kept the eyes of the FDA focused on the question of bleeding in the post-marketing period.  Did no one in the FDA notice that while this high rate of bad bleeding in the trial made the post-marketing surveillance look good, it also raised questions about whether the drug should have been thought to have been safe enough to be approved in the first place?  


By the way, Bloomberg also reported that Boehringer-Ingelheim performed a trick for European regulators, 

Selective disclosure of Pradaxa information may have camouflaged a serious safety signal, according to an assessment report from European Medicines Agency’s Committee for Medicinal Products for Human Use in August 2012. That agency is the European Union’s drug regulator.

Boehringer also didn’t give the EMA the figures showing that real-world bleeding rates were higher when the committee conducted a similar review and published a report in August 2012. The panel’s analysis, which concurred with Boehringer’s, concluded that Pradaxa’s bleeding risks among the general population were 'substantially less' than the rates seen in the study the drugmaker used as the basis for approval. It didn’t have the analysis showing the risk was higher using a different approach, according to the unsealed files.

'If the reporting rate in the post-marketing phase had been higher than in' Boehringer’s study, 'there would clearly have been a safety concern,' agency officials wrote.

Summary

We have discussed a variety of ways that companies that sell health care goods (like drugs or devices) and services may manipulate clinical research to make their products look better than they may actually be.  It appears that these companies may hire very clever people who are very knowledgeable about the design of clinical research to game the system to aid marketing, but meanwhile obfuscate the clinical research data that clinicians and patients rely on to make clinical decisions in the best interests of patients, and that policy makers rely on to make decisions in the best interests of the public.  So far, the hands of the corporate marketers seem to be quicker than the eyes of the clinicians, patients, and policy makers.

I wonder, though, how those very clever people sleep well at night knowing that their talents for legerdemain are being used to increase revenues and make corporate insiders rich, at the expense of patients, and of patients' health?

As I have said before, and most recently in the context of discussing the previous example of manipulation of clinical research to support Pradaxa marketing,  I strongly advocate that those who author authoritative systematic reviews, meta-analyses, and clinical practice guidelines base their work on extremely rigorous, skeptical reviews that assume the likelihood that all commercially sponsored published clinical research has been manipulated (and that research that even post manipulation could not be twisted to support marketing many have been suppressed)  Reviews, meta-analyses, and guidelines that were not so based on extremely critical review should also be viewed with a jaundiced eye. 

Perhaps it is possible to devise legal and regulatory methods to at least make such manipulation more transparent.  Maybe this case, however, should again suggest that clinical research, that is, research on human beings, should be completely separated from those with vested interests in selling products or services which could be better hyped were the research turn out in their favor. 

Hat tip to 1BoringOldMan.