Sovaldi, a Quantum Leap... Backwards to the Days Before Randomized Controlled Trials?

The Sovaldi (sofosbuvir - Gilead) media circus is continuing.  The New York Times just reported that sales of the new drug for hepatitis C were about $3.5 billion for the last quarter, which should intensify the kerfuffle over its US price ($1000 per pill, $84,000 for a 12 week course of medication).

Meanwhile, reports of its wondrous properties continue to appear in medical journals.

The latest was announced this way in Bloomberg,

Gilead Science Inc Solvadi, controversial because of its price, helps cure hepatitis C in people with HIV, according to researchers who say the drug has the potential to limit a top cause of death in these patients.

In a study of 223 HIV-infected patients, Solvadi combined with ribavirin cleared the most common US strain of hepatitis C in 76 percent of newly treated patients over 24 weeks. Only seven participants stopped treatment because of side effects and there were no adverse effects on HIV treatment, according to a report in the Journal of the American Medical Association.

Not unexpectedly, the study investigators were thrilled,

This is 'the first clinical trial to demonstrate that we can cure hepatitis C in patients with HIV co-infection without the use of interferon,' said Mark Sulkowski, the study author and medical director of the John Hopkins Infectious Disease Center for Viral Hepatitis in Baltimore. 'It represents a transformative step in our approach to this therapeutic area.'

Furthermore, as noted in a Medscape report,

In an accompanying editorial,(1) Michael Saag, MD, also of the University of Alabama at Birmingham, said the finding is a 'quantum leap' forward for people coping with both viruses.

What Clinical Trial?

I have posted previously about the controversy over the high price of Sovaldi.  I noted that most of the arguments, even from people most incensed about paying $1000 per pill, assumed that the drug was nearly miraculous, e.g., "a triumph," or "a revolution" in medicine.  Yet my reading suggested that the published evidence from clinical studies of the drug was in fact weak.  Critical reviews of the evidence from groups in the US, the UK, and Germany (look here, here, and here) also questioned the evidence backing Sovaldi.  Yet doubts about the drug's benefits versus harms have yet to influence the public discussion, which is focused on whether $1000 a pill is too much to charge for a miracle drug.

So I thought that "quantum leap" demonstrated in the latest trial(2) would bear further scrutiny.

The most important feature of the "trial" is how its design is described,

In this multicenter, open-label, nonrandomized, uncontrolled phase 3 trial, all patients received 400 mg of sofosbuvir (Gilead Sciences) administered orally once daily along with ribavirin (Ribasphere, Kadmon) administered orally twice daily, with doses determined according to body weight....

Let me make very clear, in this "trial," all patients got the same therapy.  There was no comparison group.  The study description is therefore inherently self-contradictory.  In fact, this study was a case-series, not a clinical trial.  It had no control group.

The classic Users Guides to the Medical Literature suggests that the first question a critical reviewer should ask about a study to assess a treatment is "was the assignment of patients to treatment randomized?"(3) The rationale was that studies of patients for whom clinicians made treatment decisions could easily be confounded by the reasons clinicians used to make these decisions.  Clinicians are trained to individual treatment decisions, and thus patients prescribed a treatment are likely to greatly differ from those not assigned that treatment.  The ways they differ could have as much, or more effect on their outcomes than the treatments themselves.  Although complex statistical adjustment methods could be used to try to compensate for these differences, it is rarely possible to be assured that all important differences have been taken into account. 

However, this still assumes a comparison between patients getting different treatments, or treatment versus no treatment.  In the case-series of sofosbuvir, there were no patients getting alternative treatments.  Thus the study itself could not be used to make comparative assessments of sofosbuvir.  

Of course, one could imply comparisons with the results of other studies.  However, such comparisons would not only be susceptible to confounding by reasons for treatment decisions as above, but to bias due to  differences between the current case series and previous studies, e.g., in characteristics of patients recruited, details of treatments given, study settings, and changes in patients, diseases, and treatments over time.   

Thus, a very basic rule in critical review is thou shalt not draw any conclusions about therapy from case-series unless the results are truly extraordinary, i.e., patients clearly doomed to die suddenly become completely well.  There were no such extraordinary results in the case-series of patients with HIV treated with Sovaldi for hepatitis C. 

The latest study of Sovaldi for patients with HIV and hepatitis C is virtually useless to assess whether Sovaldi might be the preferred therapy for such patients.   

Comparable Results?

Yet in an accompanying editorial(1), Dr Michael S Saag asserted that this non-trial 

demonstrates the efficacy of the all-oral regiment of the directly acting agent sofosbuvir plus ribavirin in patients with HCV who are coninfected with HIV

To demonstrate efficacy means to show that the treatment caused good effects.  But without a control group, it is not possible to assess causation.

Also,

The PHOTON-1 trial demonstrates that the all-oral combination of sofosbuvir plus ribavirin yielded results comparable with those of standard pegylated-interferon-ribavirin-vased regimens for each of the genotypic groups, thereby demonstrating that an all-oral combination could achieve similar outcomes as those derived from an injectable therapy.

However, this case-series could not assess comparability, because it included no comparison.


There Goes the Neighborhood - Throwing Out Skeptical Evaluation in the Rush to Promote "Innovation"

For years, the randomized, and usually double-blind, controlled trial has been considered the scientific cornerstone of the evaluation of new treatments.  Yet in the rush to bring new "innovative" treatments to market, presaging often tremendous revenues for their manufacturers, suddenly the randomized controlled trial does not seem so necessary.  In the US, the Food and Drug Administration (FDA) no longer seems to require randomized controlled trials to demonstrate efficacy of new drugs deemed to be "breakthroughs."   As noted by Steinbrook and Redberg in the July, 2014, issue of JAMA Internal Medicine(4)

these drugs [for hepatitis C] were approved quickly based on the surrogate endpoint of sustained viral response rate; they were studied in limited populations without long-term follow-up.  Under the US Food and Drug Administration's breakthrough therapy designation, the marketing for sofosbuvir did not include requirements for randomized clinical trials.

Yet without such trials, how could one ever determine if the drug is really so efficacious as to constitute a "breakthrough?"  


As we noted earlier, the two real randomized controlled trials of sofosbuvir did not assess its effects long term, even though it is a drug meant to treat a chronic disease, and particularly did not assess whether it affect clinical outcomes, that is, reduce rates of cirrhosis, liver failure, or liver cancer.  In fact, there has never been a controlled trial meant to assess whether any treatment of hepatitis C affects these outcomes.  Furthermore, while the first author of the case-series above, and many others have claimed the Sovaldi cures hepatitis C, it has never been shown that SVR equals cure.  As Gluud et al wrote in a letter to Lancet,(5)

The claim of cure rests solely on sustained virological response (SVR).  Since some patients who achieve SVRs still go on to develop end-stage liver disease and might die from the interventions, this concept is not correct and might even be wrong in principle.

However, despite all the words written, the concernthat Sovaldi (and possibly other new drugs for hepatitis C) might not be wondrous quantum leaps, because there is really no good evidence to that effect, has remained anechoic.  Not only does the evidence about the benefits and harms of this new drug need to be part of the discussion whether its price is excessive, there needs to be a larger discussion of whether in our rush for "innovation," we will throw out the baby of scientifically valid assessment of new therapies with the bathwater.

Finally,  the Sovaldi case is a signal example of how our health care system is awash in marketing hype and public relations buzz that has swamped rational skeptical thinking about logic and evidence.  That marketing and PR is ever enriching managers while it will send the rest of us, health care professionals included, to the poor house.  And all the money we spend will not buy us the promised miracles and triumphs.

As we have said until blue in the face, true health care reform would bring some skeptical thinking and regard for evidence and logic into the health policy discussion.  

 References

1. Saag MS. Quantum leaps, microeconomics, and the treatment of patients with hepatitis C and HIV coinfection. JAMA 2014; 312: 347. doi:10.1001/jama.2014.7735.  Link here.

2. Sulkowski MS et al.  Sofosbuvir and ribavirin for hepatitis C for patients with HIV coninfection.  JAMA 2014; 312: 353.  doi:10.1001/jama.2014.7734.  Link here.
3. Guyatt GH, Sackett DL, Cook DJ et al. Users' Guides to the Medical Literature: II. How to Use an Article About Therapy or Prevention A. Are the Results of the Study Valid? JAMA. 1993; 270(21): 2598-2601. doi:10.1001/jama.1993.03510210084032. Link here.
4.  Steinbrook R, Redberg R. The high price of the new hepatitis c virus drugs.  JAMA Int Med 2014; 174: 1172.  Link here. 
5.  Gluud C, Koretz R, Gurusamy K. Hepatitis C: a new direction, but an old story? Lancet 2014; 383: 2122-2123.  Link here.