Conflicted View on the Pitfalls of Government-Sponsored Comparative Effectiveness Research

Many have raised concerns that physicians too often favor the latest, most expensive, most high-technology, and sometimes most invasive treatments, contributing to the high costs and often bad outcomes of health care. One solution would be better research comparing the various possible treatments for particular problems. Yet some worry that this approach, too, has its problems.

The Wall Street Journal just printed a commentary warning about the dangers of government-sponsored studies comparing treatment effectiveness, written by Dr Scott Gottlieb, described as "a physician and resident fellow at the American Enterprise Institute, ... a former senior official at the Food and Drug Administration and the Centers for Medicare and Medicaid Services."

Dr Gottlieb raised some good points about pitfalls to be expected when comparing alternative treatments. These include:

• The need to do very large (and hence expensive) studies to have sufficient statistical power to determine if the treatments really are different, "detecting small clinical differences between two active drugs, such as whether one pill lowers blood pressure more than another, requires very large studies...."
• The tendency of investigators not to share results, making them harder to verify.
• The danger that results that fit preconceived notions will be too readily accepted.
• The danger that government-funded studies might be influenced by the government's need to control medical costs, and thus favor cheaper treatments. "Medicare is no ordinary payer: It dictates decisions made in the private market. So as the government begins tying its own payment decisions to the results of its own studies, there's a great temptation to selectively interpret data and arbitrarily release results. Clearly, this obvious conflict of interest demands even more outside scrutiny and transparency than has been the usual fare when it comes to government research."

Of course, these also are pitfalls of studies comparing active treatment to placebo, and of studies funded by not-for-profit organizations or commercial firms.

In fact, Dr Gottlieb's essay seemed to do its best to raise suspicions about government-funded comparative effectiveness research while neglecting that the alternatives may be worse.

The evidence-based medicine paradigm, to which I obviously subscribe, suggests that physicians should make decisions about treatment based on a critical review of the best available evidence from clinical research, informed by their knowledge of biology and the medical context, and taking into account patients' preferences and values.

Where can physicians get the best available evidence from clinical research now, in the absence of many government-funded studies of comparative effectiveness? In most cases, the best they can do is to look at results of studies sponsored by the manufacturers of the drugs and devices they evaluated, often compared to placebo.

Dr Gottlieb used a few examples of government-sponsored comparative studies to illustrate their potential problems. But most of these problems have affected commercially sponsored studies even more severely.

Dr Gottlieb asserted that early results of the Women's Health Initiative "were rushed to print with a cleverly orchestrated PR blitz." Yet pharmaceutical, biotechnology, and device companies make huge marketing efforts to rapidly publicize results of the trials they sponsor that are favorable to their products. (This is so glaringly obvious that it needs no web-link.)

Dr Gottlieb accused the researchers who ran the Women's Health Initiative of refusing "to share bottom-line results, even with outside academics or the companies that manufactured the drugs." Well, at least the researchers had their own data. There have been noteworthy cases of researchers running commercially sponsored studies who were unable to access the data they collected, which were kept secret by the commercial sponsor. See, in particular, the case of Dr Aubrey Blumsohn, who was unable to analyze the data he collected in a trial of Actonel because the company's sponsor, Proctor & Gamble, refused to let him have it. (See this link on Dr Blumsohn's blog for numerous posts, and our most recent posts here and here.) And we have posted about a New England Journal of Medicine study(1) which showed that US medical schools are often quite willing to let commercial research sponsors maintain control of the data collected by academic researchers.

Dr Gottlieb worried that studies funded by the government might be biased to favor less expensive drugs. Yet there is copious evidence that commercially sponsored studies favor their sponsors' projects.(2-3)

Dr Gottlieb worried that government-funded comparative effectiveness studies might be too small to generate precise enough results. Yet there is considerable evidence that the design and implementation of commercially-funded are often manipulated to increase the likelihood that the studies' results will favor the sponsors' products. (4-6)

Finally, and most ironically, Dr Gottlieb worried that Medicare sponsorship of studies would constitute a conflict of interest. We have posted before about previous commentaries by Dr Gottlieb in which he failed to reveal his past ties to industry, including his past consulting work for a large public relations firm that has many pharmaceutical companies as clients, and a firm that recruits physicians to advise about particular health care related investments. Dr Gottlieb also failed to reveal his apparent current role as a "counsel" to a major pharmaceutical company.

Most recently, Dr Gottlieb was just appointed to the board of directors for a biotechnology firm, Molecular Insight Pharmaceuticals. Thus he now has responsibility to the share-holders of this firm to maximize its profits, profits which presumably would be at risk if a government-sponsored comparative effectiveness study were to find that one of its products were not as good as expected.

I would be more inclined to credit the comments of people who have financial ties to health care corporations if they revealed these ties when making public comments. It amounts to stealth health policy advocacy for someone with multiple ties to the pharmaceutical and biotechnology industries to write in the guise of a think-tank intellectual.

ADDENDUM (4 September, 2007) - See also Aubrey Blumsohn's comments on the Scientific Misconduct Blog here.

References

1. Mello MM, Clarridge BR, Studdert DM. Academic medical centers' standards for clinical-trial agreements with industry. N Engl J Med 2005; 352: 21. (available here)

2. Bekelman JE, Li Y, Gross CP. Scope and impact of financial conflicts of interest in biomedical research: a systematic review. JAMA 2003; 289: 454-465.

3. Lexchin J, Bero LA, Djulbegovic B, Clark O. Pharmaceutical industry sponsorship and research outcome and quality: systematic review. Brit Med J 2003; 326: 1167-1170.

4. Bero LA, Rennie D. Influences on the quality of published drug studies. Int J Technol Assess 1996; 12:209-237.

5. Brophy JM. Selling safety - lessons from muraglitazar. JAMA 2005; 294:2633-5.

6. Smith R. Medical journals are an extension of the marketing arm of pharmaceutical companies. PloS Medicine 2005; 2: 364-366.