Friday, August 31, 2007

Conflicted View on the Pitfalls of Government-Sponsored Comparative Effectiveness Research

Many have raised concerns that physicians too often favor the latest, most expensive, most high-technology, and sometimes most invasive treatments, contributing to the high costs and often bad outcomes of health care. One solution would be better research comparing the various possible treatments for particular problems. Yet some worry that this approach, too, has its problems.

The Wall Street Journal just printed a commentary warning about the dangers of government-sponsored studies comparing treatment effectiveness, written by Dr Scott Gottlieb, described as "a physician and resident fellow at the American Enterprise Institute, ... a former senior official at the Food and Drug Administration and the Centers for Medicare and Medicaid Services."

Dr Gottlieb raised some good points about pitfalls to be expected when comparing alternative treatments. These include:
  • The need to do very large (and hence expensive) studies to have sufficient statistical power to determine if the treatments really are different, "detecting small clinical differences between two active drugs, such as whether one pill lowers blood pressure more than another, requires very large studies...."
  • The tendency of investigators not to share results, making them harder to verify.
  • The danger that results that fit preconceived notions will be too readily accepted.
  • The danger that government-funded studies might be influenced by the government's need to control medical costs, and thus favor cheaper treatments. "Medicare is no ordinary payer: It dictates decisions made in the private market. So as the government begins tying its own payment decisions to the results of its own studies, there's a great temptation to selectively interpret data and arbitrarily release results. Clearly, this obvious conflict of interest demands even more outside scrutiny and transparency than has been the usual fare when it comes to government research."

Of course, these also are pitfalls of studies comparing active treatment to placebo, and of studies funded by not-for-profit organizations or commercial firms.

In fact, Dr Gottlieb's essay seemed to do its best to raise suspicions about government-funded comparative effectiveness research while neglecting that the alternatives may be worse.

The evidence-based medicine paradigm, to which I obviously subscribe, suggests that physicians should make decisions about treatment based on a critical review of the best available evidence from clinical research, informed by their knowledge of biology and the medical context, and taking into account patients' preferences and values.

Where can physicians get the best available evidence from clinical research now, in the absence of many government-funded studies of comparative effectiveness? In most cases, the best they can do is to look at results of studies sponsored by the manufacturers of the drugs and devices they evaluated, often compared to placebo.

Dr Gottlieb used a few examples of government-sponsored comparative studies to illustrate their potential problems. But most of these problems have affected commercially sponsored studies even more severely.

Dr Gottlieb asserted that early results of the Women's Health Initiative "were rushed to print with a cleverly orchestrated PR blitz." Yet pharmaceutical, biotechnology, and device companies make huge marketing efforts to rapidly publicize results of the trials they sponsor that are favorable to their products. (This is so glaringly obvious that it needs no web-link.)

Dr Gottlieb accused the researchers who ran the Women's Health Initiative of refusing "to share bottom-line results, even with outside academics or the companies that manufactured the drugs." Well, at least the researchers had their own data. There have been noteworthy cases of researchers running commercially sponsored studies who were unable to access the data they collected, which were kept secret by the commercial sponsor. See, in particular, the case of Dr Aubrey Blumsohn, who was unable to analyze the data he collected in a trial of Actonel because the company's sponsor, Proctor & Gamble, refused to let him have it. (See this link on Dr Blumsohn's blog for numerous posts, and our most recent posts here and here.) And we have posted about a New England Journal of Medicine study(1) which showed that US medical schools are often quite willing to let commercial research sponsors maintain control of the data collected by academic researchers.

Dr Gottlieb worried that studies funded by the government might be biased to favor less expensive drugs. Yet there is copious evidence that commercially sponsored studies favor their sponsors' projects.(2-3)

Dr Gottlieb worried that government-funded comparative effectiveness studies might be too small to generate precise enough results. Yet there is considerable evidence that the design and implementation of commercially-funded are often manipulated to increase the likelihood that the studies' results will favor the sponsors' products. (4-6)

Finally, and most ironically, Dr Gottlieb worried that Medicare sponsorship of studies would constitute a conflict of interest. We have posted before about previous commentaries by Dr Gottlieb in which he failed to reveal his past ties to industry, including his past consulting work for a large public relations firm that has many pharmaceutical companies as clients, and a firm that recruits physicians to advise about particular health care related investments. Dr Gottlieb also failed to reveal his apparent current role as a "counsel" to a major pharmaceutical company.

Most recently, Dr Gottlieb was just appointed to the board of directors for a biotechnology firm, Molecular Insight Pharmaceuticals. Thus he now has responsibility to the share-holders of this firm to maximize its profits, profits which presumably would be at risk if a government-sponsored comparative effectiveness study were to find that one of its products were not as good as expected.

I would be more inclined to credit the comments of people who have financial ties to health care corporations if they revealed these ties when making public comments. It amounts to stealth health policy advocacy for someone with multiple ties to the pharmaceutical and biotechnology industries to write in the guise of a think-tank intellectual.

ADDENDUM (4 September, 2007) - See also Aubrey Blumsohn's comments on the Scientific Misconduct Blog here.


1. Mello MM, Clarridge BR, Studdert DM. Academic medical centers' standards for clinical-trial agreements with industry. N Engl J Med 2005; 352: 21. (available here)

2. Bekelman JE, Li Y, Gross CP. Scope and impact of financial conflicts of interest in biomedical research: a systematic review. JAMA 2003; 289: 454-465.

3. Lexchin J, Bero LA, Djulbegovic B, Clark O. Pharmaceutical industry sponsorship and research outcome and quality: systematic review. Brit Med J 2003; 326: 1167-1170.

4. Bero LA, Rennie D. Influences on the quality of published drug studies. Int J Technol Assess 1996; 12:209-237.

5. Brophy JM. Selling safety - lessons from muraglitazar. JAMA 2005; 294:2633-5.

6. Smith R. Medical journals are an extension of the marketing arm of pharmaceutical companies. PloS Medicine 2005; 2: 364-366.


Anonymous said...

I'd like to write an editorial about the pitfalls of paying much attention to anything Gottlieb has to say.

Anonymous said...

I find the question of financial disclosure in medicine very interesting. With the passage of Sarbanes Oxley, a short time ago, even casual references of company's stocks or performance must carry a disclaimer of personal ownership or financial relationship. Often this is seen as SOX with the information following. There is at the very least a tacit acceptance that financial involvement, even at the smallest level, will influence behavior.

We contrast this with Pharma's contention that all studies are done fairly with no preconceived notion of outcomes or marketability of product. Then we have those such as Dr. Gottleib stepping forward with undisclosed financial ties ti industry, and with great fanfare, decrying the governments role in medication selection.

We then learn on the following day (8/30/2007) via The Wall Street Journal that the SEC has asked for 200 firms to detail top executive pay. Leading the list is Pfizer, Schering-Plough, Brysto-Myer -Squibb, and the insurance company Prudential, who would include drug stocks in it's portfolio.

One has to ask: Why is this transparency on the financial side not available on the medical side of these transactions? We see time and time again people speaking from the medical perspective with undisclosed financial ties to the products or companies pe=roducts they are touting.

Financially we see drugs touted as they enter Phase I trials and can follow their progress, including marketing plans, and extrapolate sales programs, all the way to market introduction. Often we find that financial disclosure has forced negative drug reactions into the public view prior to it's release to the medical community, Elidel's black box warning was noted six months prior to FDA action in part of the financial guidance for Novartis.

My frustration is that we are not dealing with a level playing field when Pharma is able to make public statements without disclosing it's, or it's surrogates, financial involvement such as we see with Dr. Gottlieb.

As a side note, appearing on the same page of the WSJ, we see that Boston Scientific will pay $16.8M out of an estimated $706M to 35 states and the District of Columbia for hiding design flaws in defibrillators.

SOX I do not personally own any of the stocks listed in this comment, although I do own funds that invest in these and other health related companies.

Steve Lucas

Anonymous said...


Steve says "Why is this transparency on the financial side not available on the medical side of these transactions?"

But the problem of treating medicine differently goes further, because even WITHIN the financial and the legal arena medical scientific deception is treated differently (a special low standard) from most other kinds of analagous deception.

So if an oil company were to lie about the size of their oil reserves (to drive stock prices) somehow the financial regulators treat that very differently from the identical case of a drug company telling lies about a drug to achieve exactly the same end - to drive stock prices. At the very least they would regard it as "wrong". Even in the case of Enron, when it was known what had happened, the financial market regarded what had happened as somehow "wrong".

I suppose the difference is that pharmaceutical research deception does not in fact affect the markets much (and in that sense it is not in fact "wrong"). And the reason it does not affect the markets (much) is that the deception is carried out with the complicity of government and the perpetrators are effectively immunized by government.

So when one considers (as a financier would) that the ethical need is to disclose things that might have consequences, there is in fact no ethical problem at all.


Greg Pawelski said...

What needs to be done is to remove the profit incentive from the choice of drug treatments. Patients should receive what is best for them and not what is best for their physicians. Let's take physicians out of the retail pharmacy business and force them to be physicians again!