Pseudo-Evidence Based Medicine Should be a Global Health Concern

We have frequently advocated for evidence-based medicine (EBM) , that is, medicine based on judicious use of the best available evidence from clinical research, critically reviewed and derived from systematic search, combined with biomedical knowledge and understanding of patients' values and preferences.  However, EBM risks being turned into pseudo-evidence based medicine due to systematic manipulation and distortion of the clinical research evidence base.  Dr Wally Smith wrote about pseudo-evidence based medicine, which he defined as "the practice of medicine based on falsehoods that are disseminated as truth," in the British journal Clinical Governance (Smith WR. Pseudoevidence-based medicine: what it is, and what to do about it. Clinical Governance 2007; 12: 42-52. Also see this post).  

Now it appears that this issue is causing concern in the Cochrane Collaboration, the main voluntary international group promoting EBM.  An article from December, 2011 in the Indian Journal of Medical Ethics outlined why we need to question the trustworthiness of the global clinical evidence base.  (See Tharyan P. Evidence-based medicine: can the evidence be trusted.  Ind J Med Ethics 2011; 8: 201-207.  Link here.)  It merits further review.

The Role of Vested Interests

Dr Tharyan, its author, emphasized that a major threat to the integrity of the clinical research data base is the influence on clinical research of those with vested interests in marketing particular products, e.g., drugs, devices, etc.

The motives for conducting research are often determined by considerations other than the advancement of science or the promotion of better health outcomes. Many research studies are driven by the pressure to obtain post-graduate qualifications, earn promotions, obtain tenured positions, or additional research funding; many others are conducted for financial motives that benefit shareholders, or lead to lucrative patents.

The Importance of Pervasive Conflicts of Interest

Early on, the author discussed how the distortion of the clinical research data base arises from the pervasive web of conflicts of interest in medicine and health care:

This hijacked research agenda perpetuates further research of a similar nature that draws more researchers into its lucrative embrace, entrenching the academic direction and position statements of scientific societies and academic associations. Funders and researchers are also deterred from pursuing more relevant research, since the enmeshed relationship between academic institutions and industry determines what research is funded (mostly drugs at the expense of other interventions), and even how research is reported; thus hijacking the research agenda even further away from the interests of science and society.

The article then goes on to show how the influence of vested interests may distort the design, implementation, analysis, and dissemination of research.

Distortion of Research Design and Implementation

The Research Question

Dr Tharyan noted

The majority of clinical trials conducted world-wide are done to obtain regulatory approval and a marketing licence for new drugs. These regulations often require only the demonstration of the superiority of a new drug over placebo and not over other active interventions in use. It is easier and cheaper to conduct these trials in countries with lower wages, lax regulatory requirements, and less than optimal capacity for ethical oversight. It is therefore not surprising that the focus of research does not reflect the actual burden of disease borne by people in the countries that contribute research participants, nor address the leading causes of the global burden of disease. Some 'seeding' trials, conducted purportedly for the purpose of surveillance for adverse effects, are often only a ploy to ensure brand loyalty among participating clinician-researchers.

Insufficient Sample Size

The article stated,

Many trials do not report calculations on which the sample size was estimated, often leading to sample sizes insufficient to detect even important differences between interventions (for primary, let alone secondary outcomes)

I would add that such small trials are particularly bad at detecting important adverse effects of the interventions being promoted.

Excessively Stringent Enrollment Criteria

As Dr Tharyan wrote,

Most RCTs funded by industry and academia are designed to demonstrate if a new drug works, for licensing and marketing purposes. In order to maximise the potential to demonstrate a 'true' drug effect, homogenous patient populations; placebo controls; very tight control over experimental variables such as monitoring, drug doses, and compliance; outcomes addressing short term efficacy and safety; and methods to minimise bias required by regulatory agencies are used to demonstrate if, and how, the drug works under ideal conditions.

The problem is that very few patients in clinical practice resemble those enrolled in such trials, so the generalizability of the trials' results is actually dubious. Ideally, clinicians practicing evidence-based medicine could refer to trials that include patients similar to those for whom they care.

Practical or pragmatic clinical trials are designed to provide evidence for clinicians to treat patients seen in day-to day clinical practice, and evaluate their effectiveness under 'real-world' conditions. These trials use few exclusion criteria and include people with co-morbid conditions, and all grades of severity. They compare active interventions that are standard practice, and in the flexible doses and levels of compliance seen in usual practice. They utilise outcomes that clinicians, patients, and their families consider important, such as satisfaction, adverse events, return to work, and quality of life). Recommendations exist on their design and reporting , but such trials are rare.

Comparisons to Placebo, not the Other Interventions Clinicians Might Realistically Consider

Per the article,

Industry sponsored trials rarely involve head-to head comparisons of active interventions, particularly those from other drug companies, thus limiting our ability to understand the relative merits of different interventions for the same condition.

The result may thus be a number of studies showing particular interventions appear to be better than nothing, but few if any studies that would help clinicians decide which intervention would be best for a particular patient.

Inappropriate Comparators

However,when studies are done comparing the intervention of interest to other active interventions, the details of the choice of comparators are often managed so that the comparators are likely to appear to be worse.

Even if active interventions are compared in industry-sponsored trials, the research agenda has devised ways in which the design of such trials is manipulated to ensure superiority of the sponsor’s drug. If one wants to prove better efficacy, then the comparator drug is a drug that is known to be less effective, or used in doses that are too low, or used in non-standard schedules or duration of treatment. If one wants to show greater safety, then the comparator is a drug with more adverse effects, or one that is used in toxic doses. Follow up, also, is typically too short to judge effectiveness over longer periods of time.

Meaningless Outcomes

A favorite tactic used in the design of trials influenced by vested interests is to choose outcomes that are likely to show results favorable to the product being promoted, but have no meaning for patients or clinicians. There are three ways this is commonly done.

The first is to use rating scales that are very sensitive to small perturbations, but not meaningful in terms of how patients feel or function:

The choice of outcome measures used often ensures statistically significant results in advance, at the expense of clinically relevant or clinically important results. Outcomes likely to yield clinically meaningless results include the use of rating scales (depression, pain, etc.). These scales yield continuous measures usually summarised by means and standard deviations, rather than the dichotomous measures clinicians use such as: clinically improved versus not improved. These rating scales, however extensively validated, are hardly ever used in routine clinical practice. A difference of a few points on these scales results in statistically significant differences (low p values), that have little clinical significance to patients.

Another is to use surrogate outcomes,

Other outcomes commonly used are surrogate outcomes; outcomes that are easy to assess but serve only as proxy indicators of what ought to be assessed, since the real outcome of interest may take a long time to develop. These are mostly continuous measures that require smaller sample sizes (blood sugar levels, blood pressure, lipid levels, CD4 counts, etc.). These measures easily achieve statistical significance but do not result in meaningful improvements (reduction in mortality, reduction in complications, improved quality of life) in patients’ lives, when the interventions are used (often extensively) in clinical practice.

Finally, there are composite outcomes,

The use of composite outcomes, where many outcomes (primary, secondary and surrogate outcomes) are clubbed together (e.g.: mortality, non-fatal stroke, fatal stroke, blood pressure, creatinine values, rates of revascularisation) as a single primary outcome, can also mislead. Such trials also require smaller sample sizes, and increase the likelihood of statistically significant results. However, if the composite outcome includes those of little clinical importance (lowered blood pressure, or creatinine values), the likelihood of real benefit (reduction in mortality, or strokes, or hospitalisation) and the potential for harm (increase in non-fatal strokes or all-cause mortality) are masked.

Distortion of Analysis

Relative Instead of Absolute Risks

A favorite trick is to emphasize relative risks rather than absolute risks. For example if a trial reduces the risk of a bad outcome from 2 of 10,000 patients (0.02%) to 1 of 10,000 patients (0.01%), the relative risk reduction is 50%, but only 1 of 10,000 patients experienced a benefit.

The use of estimates of relative effects of interventions, such as relative risks (RR) and odds ratios (OR) with their 95% confidence intervals, provides estimates of relative magnitudes of the differences and whether these exclude chance, as well as if these differences were nominal or likely to be clinically important.

However, even relative risks can be misleading since they ignore the baseline risk of developing the event without the intervention. The absolute risk reduction (ARR) is the difference in risk of the event in the intervention group and the control group, and is more informative since it provides an estimate of the magnitude of the risk reduction, as well the baseline risk (the risk without the intervention, or the risk in the control group). Systematic enquiry demonstrates that on average, people perceive risk reductions to be larger and are more persuaded to adopt a health intervention when its effect is presented as relative risks and relative risk reduction (a proportional reduction) rather than as absolute risk reduction; though this may be misleading.

Sub-Group Analysis

Another statistical trick used to present favourable outcomes for interventions is the use of spurious subgroup analyses, where observed treatment effects are evaluated for differences across baseline characteristics ( such as sex, or age, or in other subpopulations). While they are useful, if limited to a few biologically plausible subgroups, specified in advance, and reported as a hypothesis for confirmation in future trials; they are often used in industry-sponsored trials to present favourable outcomes, when the primary outcome(s) are not statistically significant.

A major statistical point is that the more ways one subdivides the study population, the more likely it is to find a difference between those receiving different interventions by chance alone.

Distortion of Dissemination

Poor Exposition

As noted by Dr Tharyan,

Evidence also shows that the published reports are not always consistent with their protocols, in terms of outcomes, as well as the analysis plan, and this again is determined by the significance of the results. Harms are very poorly reported in trials compared to results for efficacy; and are also often suppressed or minimised.

Ghost-Writing

Also, those with vested interests may try to ensure maximally favorable dissemination by employing ghost-writers,

Other tactics used to influence evidence-informed decision making include ghost-writing, where pharmaceutical companies hire public-relations firms who 'ghost-write' articles, editorials, and commentaries under the names of eminent clinicians; a strategy that was detected by one survey in 75% of industry-sponsored trials, where the ghost author was not named at all, and in 91% when the ghost author was only mentioned in the acknowledgement section. Detecting such conflicts of interest is difficult, since they are rarely acknowledged due to the secrecy that shrouds the nexus between academia and industry in clinical trials.

Industry-sponsored trials often place various constraints on clinical investigators on publication of their results; these publication arrangements are common, allow sponsors control of how, when, and what is published; and are frequently not mentioned in published articles.

Ghost-writers under the direct control of those with vested interests could more efficiently bias their writing in favor of their sponsors than could even academics constrained by contractual obligations to sponsors.

Suppression of Research

When all else fails, the most crude example of distorted dissemination is suppression of studies that despite all the manipulations noted above fail to produce favorable results for the product being promoted:

A considerable body of work provides direct empirical evidence that studies that report positive or significant results are more likely to be published; and outcomes that are statistically significant have higher odds of being fully reported, particularly in industry funded trials.

By the way, the latest demonstration of suppression of research was a study by Turner et al (Turner EH, Knoepflmacher D, Shapley L. Publication bias in antipsychotic trials: an analysis of efficacy comparing the published literature to the US Food and Drug Administration database. PLoS Medicine 2012; 9(3): e1001189. Link here) It showed that of 24 trials of atypical antipsychotics registered in the US FDA database, 15/20 (75%) of the published trials were positive, that is, had results in favor of the sponsors' drugs according to the FDA review, but only 1/4 (25%) of the unpublished trials were positive. In other words, 15/16 of positive trials were published, but only 5/8 non-positive ones were.

Summary

Thus, Dr Tharyan provided a nice summary of many of the ways that the clinical research data base can be manipulated or distorted to serve vested interests. Such distortions risk the transformation of evidence-based medicine into pseudo-evidence based medicine.

We have repeatedly discussed (see links above) most of these issues. Because we are located in the US, and speak mainly English, this blog may have given the impression that the lack of trustworthiness of the clinical research evidence base is primarily a US problem. The article by Tharyan emphasizes, however, that it is a global problem.

While this problem now seems to have the attention of the Cochrane Collaboration, it seems to be relatively anechoic in global health circles. It appears to be no more prominent on the agendas of global health organizations than is health care corruption (look here.) Yet the two issues are highly related. Most of the distortions in the global clinical evidence database may be driven by conflicts of interest. Conflicts of interest are risk factors for corruption. Distortions of the global clinical research data base that lead to use of expensive, but ineffective or dangerous interventions when other options would work as well or better leads to needless suffering and death, and by unnecessarily raising the costs of care, decreases access, especially for the poor.

Also note that conflicts of interest may be one reason that all these problems remain so anechoic (look here).

True global health care reform requires addressing health care corruption, but also conflicts of interest and their role in the distortion and manipulation of the clinical research data base. I still live in hope that that some academic health care institutions, professional societies, health care charities and donors, and patient advocacy groups will gain enough fortitude to stand up for accountability, integrity, transparency, and honesty in health care.

BLOGSCAN - Wandering in the Wilderness

On the 1 Boring Old Man blog, "Mickey," the anonymous blogger, a retired academic psychiatrist, posted about how he figured out the extent that health care dysfunction affected psychiatry while wandering the "wildnerness" outside of academic psychiatry.  He proposed that what needs to die is "the shameful mockery many have made of their own rallying cry - 'evidence based medicine'…"  Dr Wally Smith would have called it pseudo-evidence based medicine.   (And thanks for the mention of Health Care Renewal.)

Has Ghostwriting Infected The "Experts" With Tainted Knowledge, Creating Vectors for Further Spread and Mutation of the Scientific Knowledge Base?

I could have titled this piece "have the deceivers been themselves deceived?"

Recent revelations have shown that biomedical ghostwriting is a scourge that some physicians and scientists -- who are clearly lacking ethics, insight and perhaps an understanding of psychiatric defense mechanisms such as rationalization -- engage in all too often, as it taints the corpus of worldwide scientific literature.

(I covered the issues related to ethical and justified claims of provenance for scientific writing in my post "Wyeth: Ghostwritten Papers Fake, But Accurate" here along with a discussion of why lay leadership of biomedical R&D organizations was not a good idea. Such leaders cannot vouch for the scientific fairness and accuracy emanating from their own organizations.)

One should examine the evidence in a first-person manner. The New York Times has made available secret documents and emails secured by Senator Grassley on Wyeth's ghostwriting practices. These documents were unsealed by a judge in a personal injury lawsuit and chronicle Wyeth's use of a medical writing company to prepare articles for publication in medical journals.

Wyeth said all the articles are "scientifically accurate" and that "it is a common practice in the pharmaceutical industry to work with such firms" (and, by implication, to falsify claimed authorship). As justification for clearly unethical practices, this is a rather poor argument, being a classic logical fallacy known as the Appeal to Common Practice:

Description of Appeal to Common Practice
The Appeal to Common Practice is a fallacy with the following structure:

1. X is a common action.
2. Therefore X is correct/moral/justified/reasonable, etc.

That said, these documents raise an important question of a systemic nature:

Have the scientific ghostwriting deceivers themselves been deceived regarding biomedical reality by these practices?

Might they genuinely believe they are contributing to the scientific literature based on their own erroneous notions, the latter based upon a scientific literature contaminated by other ghostwriters?

To illustrate this point, in the New York Times online document pool, we discover that Dr. Gloria Bachmann, Associate Dean for Women's Health and Professor of OB/GYN and Medicine at UNDNJ-Robert Wood Johnson Medical School was presented with a tidy ghostwritten article entitled "The Importance of Treating Vasomotor Symptoms" (in peri- and post-menopausal women) by authors "TBD" (to be decided):

(Original ghostwritten draft. Authors "TBD." Click to enlarge)

about which Bachmann wrote the following in an email to the true authors' organization:

"Outline is excellent as written. I would like to try for publication as a review article in Obstetrics and Gynecology. We would have to change the title to "Vasomotor Symptoms: Evidence Based Treatment Options" or a title that emphasizes the data driven writing style."

Here is an image of the email:

(Approval email. Click to enlarge)

She found the content OK and agreed to review it.

Apparently, only minor changes were made. (Bachmann should have received accreditation in the acknowledgments section for reviewing the article, and/or a position far down the authorship hierarchy if she actually added some material).

Instead, the reverse occured.

The published document appeared with Bachmann as first (and only) author after Bachmann submitted it to a medical journal under her name, itself unethical. See what "Authors TBD" morphed into in the image below:

(Article submission cover page. Click to enlarge)

Here is how the true authors, medical writers for DesignWrite, the medical writing and communications company retained by Wyeth to write the article, received credit:

(How the "ghosts" received credit for writing the paper. Click to enlarge)

Unbelievable.

Bachmann has since stated that she believed she was adding to science by engaging in this deceit. From the New York Times:

The article, a nearly verbatim copy of the DesignWrite draft, appeared in 2005 in The Journal of Reproductive Medicine, with Dr. Bachmann listed as the primary author. It described hormone drugs as the “gold standard” for treating hot flashes and was less enthusiastic about other therapies.

The acknowledgments thanked several medical writers for their “editorial assistance,” not disclosing that those writers worked for DesignWrite, which charged Wyeth $25,000 to generate the article.

Dr. Bachmann, who has 30 years of research and clinical experience in menopause, said she played a major role in the publication by lending her expertise. Her e-mail messages do not reflect contributions she may have made during phone calls and in-person meetings, she said. ["May have made?" This gossamer excuse is embarrassing to me, as an academic myself - ed.]

“There was a need for a review article and I said ‘Yes, I will review the draft and make sure it is accurate,’ ” Dr. Bachmann said in an interview Tuesday. “This is my work, this is what I believe, this is reflective of my view.”…

She is quite off base regarding scientific authorship and "this being her work", but what also needs examination is her "lending her expertise" and her agreeing with the paper as "reflective of her view."

The paper itself lists many references, for example these:

(References: click to enlarge)

Here is, then, the dilemma, another major problem posed by ghostwriting:

How many of the cited references, which contributed to Bachmann's beliefs regarding these drugs, were themselves ghostwritten and potentially biased, or otherwise influenced by drug companies?

For that matter, how many studies with different ("negative") conclusions on these subjects were suppressed from publication by industry pressure? 

Bachmann's own "expert opinions" about these matters may be based upon false or misleading information she's read in other journals, the percentage of which are genuine vs. ghostwritten or industry-influenced being an unknown at this point in time.

While I believe Bachmann is most likely rationalizing away the unethical nature of attribution without fair and justifying contribution, perhaps fueled by additional motives of academic portfolio building and perhaps present and future potential pecuniary gain, it is possible she is being genuine about agreeing with the "science" in the paper.

However, her agreement may be predicated upon her own tainted expertise -- tainted by reading ghostwritten or industry-influenced articles in her area of specialty.

Will we ever be able to peel back all the layers of the ghostwriting onion to get to the core of impartial and objective scientific articles related to drugs and medical devices? Perhaps not, but the practice must stop going forward.

Tainted literature creates tainted scientific knowledge, the carriers of which may then further taint the knowledge base (with the best of intentions and with firm belief in the fairness and accuracy of their activities, of course).

Practitioners of evidence-based medicine may be unwittingly practicing "evidence-tainted medicine", or "pseudo-evidence based medicine" as described by others on this blog.

Such are the systemic risks of the scourge of ghostwriting.

-- SS

Aug. 31, 2012 Addendum:

I was made aware of the following 2011 paper thanks to a post by Dr Howard Brody on the Hooked: Ethics, Medicine and Pharma blog:  "Why Does Academic Medicine Allow Ghostwriting?  A Prescription for Reform" by Jonathan Leo, Jeffrey R. Lacasse, and Andrea N. Cimino

The paper also observes:

Due to its secretive nature, it is difficult to quantify how many invisible corporate authors haunt the medical literature.

The paper is worth reading in its entirety and as of this writing is available free at the link above.

-- SS

Pseudo-Evidence Based Medicine Threatens Health Care Reform Based on "What Works"

As I posted yesterday, the increasingly noisy debate about health care reform in the US has not dealt much with the issues we often discuss on Health Care Renewal. These include problems in how health care organizations are led which threaten physicians' and other health care professionals' core values using tactics including perverse incentives, deception, and intimidation.
Last night, however, President Obama held a news conference mostly devoted to health care issues, in which he stressed the importance of changing not just how health insurance works, but how health care decisions are made. As Newsweek's "The Gaggle" blog reported,

Can I guarantee that there are going to be no changes in the health care delivery system? No. The whole point of this is to try to encourage changes that work for the American people and make them healthier. The government already is making some of these decisions. More importantly, insurance companies right now are making those decisions. And part of what we want to do is to make sure that those decisions are being made by doctors and medical experts based on evidence, based on what works, because that's not how it's working right now.

So what the President seems to be advocating is making health care more evidence-based, perhaps in the formal sense of evidence-based medicine.

As a card-carrying evidence-based medicine advocate, I certainly agree, but let me reiterate that evidence-based medicine is not just medicine based on some sort of evidence. As Dr David Sackett, and colleagues wrote [Sackett DL, Rosenberg WM, Muir Gray JA, Haynes RB, Richardson WS. Evidence-based medicine; what it is and what it isn't. BMJ 1996; 312: 71-72. Link here. ]

Evidence based medicine is the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients. The practice of evidence based medicine means integrating individual clinical expertise with the best available external clinical evidence from systematic research. By individual clinical expertise we mean the proficiency and judgment that individual clinicians acquire through clinical experience and clinical practice. Increased expertise is reflected in many ways, but especially in more effective and efficient diagnosis and in the more thoughtful identification and compassionate use of individual patients' predicaments, rights, and preferences in making clinical decisions about their care. By best available external clinical evidence we mean clinically relevant research, often from the basic sciences of medicine, but especially from patient centred clinical research into the accuracy and precision of diagnostic tests (including the clinical examination), the power of prognostic markers, and the efficacy and safety of therapeutic, rehabilitative, and preventive regimens.

Furthermore,

Evidence based medicine is not 'cookbook' medicine. Because it requires a bottom up approach that integrates the best external evidence with individual clinical expertise and patients' choice, it cannot result in slavish, cookbook approaches to individual patient care. External clinical evidence can inform, but can never replace, individual clinical expertise, and it is this expertise that decides whether the external evidence applies to the individual patient at all and, if so, how it should be integrated into a clinical decision.

One can find other definitions of EBM, but nearly all emphasize that the approach is designed to appropriately apply results from the best clinical research, critically reviewed, to the individual patient, taking into account that patient's clinical characteristics and personal values.

So far, so good. I believe the proper application of "real" (as described above) evidence-based medicine has the potential to improve patient outcomes while moderating health care costs. However, we have pointed out how problems arising from concentration and abuse of power in health care threaten the evidence-based medical ideal.

First, there are major problems with the development of the sort of clinical research evidence required by the EBM process. We have discussed how clinical research is frequently manipulated by those with vested interests in producing results that favor the products or services that they sell. The critical review process inherent in EBM is meant to cope with less than optimally designed and implemented research. However, the process was designed to cope with honest mistakes and inevitable trade-offs, not deliberate manipulation by vested interests.

Worse, we have discussed how vested interests may engineer the suppression of research when manipulation fails to produce the desired results. The EBM process assumes that the research on which decisions should be based is an unbiased sample of research that was done (and done to advance science, not commercial or ideological interests). When research whose results are unwanted by vested interests is suppressed, the resulting distortion of the evidence base may irretrievably bias the EBM process.

Finally, we have posted about how vested interests have distorted the discussion, dissemination, and teaching of the results of clinical research. They may develop systematic stealth marketing campaigns, often employing supposed "key opinion leaders," who are paid on the side by marketers, and using "medical education and communication companies"as marketing fronts whose publication strategies include deceptive tactics such as "ghost-writing."

Thus, a rising tide of "pseudo-evidence based medicine" threatens to overwhelm even the most conscientious physicians trying to practice evidence-based medicine.

So, while I applaud President Obama's advocacy of reforming health care to emphasize what the best evidence suggests really works, I do not think this effort will get far unless we deal with the rising tide of pseudo-evidence based medicine. As a minimum, we need full and detailed disclosure of all the relationships among vested interests and medical research and education, and a much greater role for research and education that is not subsidized by corporations bent on using research and education to market their products and services.

Another Misuse of EBM

We agree with Roy Poses' comment on the recent JAMA article about EBM, particularly regarding some of its unfortunate "misuses." In the article that stands as the defining credo of Health Care Renewal, Roy wrote:

Most physicians believe that medicine should be based on science. Evidence based medicine, (EBM) ‘‘the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients’’ [74], is a refinement of this view. Meanwhile, the scientific basis of medicine is under increasingly severe attack.

We'd like to take this opportunity to identify a misuse of EBM that, ironically, contributes to the attack on the scientific basis of medicine, and that authors Montori and Guyatt didn't consider--probably because they are unaware of it. We explain it in the following letter, submitted to JAMA in response to their article. We discuss it at more length on the other blog to which we contribute, Science-Based Medicine, referenced from the letter:

To the editor:

In their recent article, “Progress in Evidence-Based Medicine,” Drs. Montori and Guyatt include important caveats about misuses of EBM. They mention trials funded by industry. They list several, documented problems with such trials, and correctly warn that these can result in erroneous interpretations by “unsophisticated users of the medical literature.” They correctly worry about “abandoning appropriate skepticism.”

There is a related misuse of EBM that leads to erroneous interpretations even by sophisticated authors, and that calls attention to systematic problems in EBM: trials of highly implausible hypotheses. Thus, regarding the homeopathic preparation Oscillococcinum, the Cochrane Collaboration writes:

Though promising, the data were not strong enough to make a general recommendation to use Oscillococcinum for first-line treatment of influenza and influenza-like syndromes. Further research is warranted…[1]

The data, however, are not promising, and further research is not warranted. The reason is that for highly implausible treatments, existing knowledge easily trumps modestly “positive” trial data. In the case of homeopathy, each of its tenets has been more than adequately disproved, and its claim that the potency of a “remedy” increases with each dilution of the original substance, continuing well past the point at which not a molecule of that substance remains in the preparation, contradicts physical reality.[2]

Such considerations reveal at least two, related problems with the evidence of EBM as it pertains to clinical trials. The first is explicit: by relegating “physiology, bench research or ‘first principles’” to Level 5 of its “Levels of Evidence” hierarchy, EBM fails to acknowledge that consistency with firmly established knowledge is a necessary, even if insufficient, basis for accepting a hypothetical treatment as effective.[2]

The second problem is part of the culture, if not the letter of EBM: by eschewing Bayesian in favor of “frequentist” statistics, its practitioners can excuse themselves from justifying estimates of prior probability, and erroneously conclude that trial data tell us more about reality than they do.[3-5]

In January, 2008, several colleagues began the blog Science-Based Medicine, referenced below.* The name, which could just as well have been “knowledge-based medicine,” is a frank recognition of the EBM weaknesses described above. Because of our interests, Science-Based Medicine’s content is predominantly, though not entirely, concerned with implausible health claims. Its larger message, however, is that these claims call attention to fundamental ways in which EBM can be improved, such as to justify its own claim to being an arbiter of real evidence.

Kimball C. Atwood, IV

Wallace I. Sampson

10/19/2008

* Steven Novella, David Gorski, Harriet Hall, Wallace Sampson, Kimball Atwood

[1] Vickers A, Smith C. Homoeopathic Oscillococcinum for preventing and treating influenza and influenza-like syndromes. Cochrane Database of Systematic Reviews 2006, Issue 3. Art. No.: CD001957. DOI: 10.1002/14651858.CD001957.pub3.

[2] Atwood K. Homeopathy and Evidence-Based Medicine: Back to the Future Part V. Science-Based Medicine 2008 (Feb. 8). http://www.sciencebasedmedicine.org/?p=42. Accessed October 19, 2008

[3] Atwood K. Prior Probability: the Dirty Little Secret of “Evidence-Based Alternative Medicine.” Science-Based Medicine 2008 (Feb. 15). http://www.sciencebasedmedicine.org/?p=48. Accessed October 19, 2008

[4] Goodman SN. Toward evidence-based medical statistics. 1: The P value fallacy. Ann Intern Med. 1999;130:995-1004.

[5] Goodman SN. Toward evidence-based medical statistics. 2: The Bayes factor. Ann Intern Med. 1999;130:1005-13.

BLOGSCAN - Duplicate Publication about an SSRI

On the Clinical Psychology and Psychiatry Blog, the anonymous blogger described an interesting example of apparent dual publication on a selective serotonin reuptake inhibitor (SSRI) anti-depressant drug, duloxetine (Cymbalta, manufactured by Eli Lilly). Important aspects of the story include how both articles draw conclusions that go well beyond the (same) data in their enthusiasm for the drug in question, and how the second article incorporated an author not found in the first, an author who also happened to be the co-editor in chief of the journal in which the second article appeared, and the leader of an important medical society. This author has been the subject of previous posts (like this one), on Health Care Renewal.

Dr Marcia Angell on Reforming a "Broken System" of Clinical Research

This week's Journal of the American Medical Association (3 September, the paper version of which seems to arrive early at my house) is notable for having three commentaries on relationships among physicians, academic medicine, and industry. One, by David Rothman and Susan Chimonas, seemed rather optimistic, while two, by Dr Arnold Relman and by Dr Marcia Angell, were quite dark.

The latter [Angell M. Industry-sponsored clinical research: a broken system. JAMA 2008; 300: 1069-1071. Link here when available] should be required reading for anyone who worries about the issues we raise on Health Care Renewal. Dr Angell provided an excellent summary of how commercial "sponsors" have come to dominate clinical research, and thus ensure that the resulting studies favor their vested interests. My only criticism of her summary of the data was that it may actually have been too optimistic. For example, Dr Angell focused almost exclusively on pharmaceutical companies' influence on research, while the problem is even larger. Biotechnology companies, device manufacturers, and multiple miscellaneous commercial suppliers of health care goods and services also may influence research. Dr Angell's admittedly extensive list of the kinds of relationships among researchers and industry (grant support, paid consulting, service on speakers' bureaus and advisory panels, and stock ownership) was incomplete, leaving out, for example, the even more acute conflicts of interest generated by service on companies' boards of directors and service as companies' officers.

But those are quibbles. Most striking was Dr Angell's summary:

Looking at this picture altogether, it would be naive to conclude that bias is only a matter of a few isolated instances. It permeates the entire system. Physicians can no longer rely on the medical literature for valid and reliable information. This is the conclusion I reluctantly reached toward the end of my 2 decades as an editor of the New England Journal of Medicine, and it has been reinforced in subsequent years. Clinicians just do not know anymore how safe and effective prescription drugs really are, but these products are probably nowhere near as good as the published literature indicates.

I am afraid I must concur, and also concur with her summary of the effects of the substitution for valid clinical research of pseudo-evidence favoring the vested interests of "research" sponsors:

Physicians learn to practice a very drug-intensive style of medicine.

Physicians are also led to believe that the newest, most expensive brand-name drugs are superior to older drugs or generics, even though there is seldom any evidence to the effect because sponsors do not usually compare their drugs with old drugs at equivalent doses.

Physicians learn to use drugs for off-label uses without good evidence of effectiveness.

Keep in mind that the same considerations apply to devices, so that physicians also learn to practice a test- and procedure-intensive style of medicine; are led to believe that the newest tests, devices, and procedures are superior to older ones without any evidence comparing the new to the old; and physicians learn to extend the use of tests, devices, and procedures far beyond what the evidence would support.

In addition, I agree with Dr Angell's advocacy of an independent body that would administer all clinical trials of drugs. Dr Angell suggested it be an Institute within the NIH. I think it might be possible to form a not-for-profit body that would be independent of government that would do the same thing. Furthermore, that body, or another in parallel should also administer all clinical trials of diagnostic tests, devices and procedures, for analogous reasons.

Finally, Dr Angell urged academic investigators and their institutions to "be at the forefront of efforts to reform the system of clinical research and not leave it to the government and legal profession." However, as Dr Angell previously noted, a majority of medical school faculty have financial ties to drug, device, biotechnology and other health care companies, as do a majority of department chairs. As we have noted, some top leaders of academic medicine serve also as top leaders of for-profit health care corporations (see this post), and other corporations whose products have profound effects on health, even tobacco companies (see this post). Many academic leaders make a small fortune working on the side for such companies. It may be too much to expect that such leaders would be at all interested in reforming a system that has so effectively lined their pockets. For better or worse, we may need a wholesale change in the leadership of academic organizations before we can expect any real cooperation from them in transforming clinical research from being mainly a marketing exercise to once more being real clinical science.

ADDENDUM (5 September, 2008) - See also comments by Dr Howard Brody on the Hooked: Ethics, Medicine and Pharma blog.

The NIH Trial to Assess Chelation Therapy (TACT): an unethical study sponsored by taxpayers

Two occasional HCR bloggers are co-authors of an article published this week:

Why the NIH Trial to Assess Chelation Therapy (TACT) Should Be Abandoned

Kimball C. Atwood IV, MD; Elizabeth Woeckner, AB, MA; Robert S. Baratz, MD, DDS, PhD; Wallace I. Sampson, MD

Medscape J Med. 2008;10(5):115. ©2008 Medscape
Posted 05/13/2008

Abstract:

The National Institutes of Health (NIH) Trial to Assess Chelation Therapy (TACT) was begun in 2003 and is expected to be completed in 2009. It is a trial of office-based, intravenous disodium ethylene-diamine-tetra-acetic acid (Na2EDTA) as a treatment for coronary artery disease (CAD). A few case series in the 1950s and early 1960s had found Na2EDTA to be ineffective for CAD or peripheral vascular disease (PVD). Nevertheless, a
few hundred physicians, almost all of whom advocate other dubious treatments, continued to peddle chelation as an office treatment. They claim that chelation dramatically improves symptoms and prolongs life in 80% to 90% of patients. In
response, academics performed 4 controlled trials during the 1990s. None favored chelation, but chelationists repudiated those findings.

We have investigated the method and the trial. We present our findings in 4 parts: history, origin and nature of the TACT, state of the evidence, and risks. We present evidence that chelationists and their organization, the American College for Advancement in Medicine, used political connections to pressure the NIH to fund the TACT. The TACT protocols justified the trial by misrepresenting case series and by ignoring evidence of risks. The trial employs nearly 100 unfit co-investigators. It conflates disodium EDTA and another, somewhat safer drug. It lacks precautions necessary to minimize risks. The consent form reflects those shortcomings and fails to disclose apparent proprietary interests. The trial's outcome will be unreliable and almost certainly equivocal, thus defeating its stated purpose.

We conclude that the TACT is unethical, dangerous, pointless, and wasteful. It should be abandoned.

The entire article is available at: http://journal.medscape.com/viewarticle/570625_print

You may be asked to "register"; don't worry, it's free. The article is very long, but the Introduction, Executive Summary, Discussion, and Conclusion are reasonably succinct and make the important points. Readers who want to learn more details, who want to see more evidence for our assertions, or who are compelled by an odd fascination with crackpotism (my own weakness) will want to read more.

I've posted a similar announcement on Science-Based Medicine.

"A Catastrophic Failure" - More on the Ketek Trial Debacle

We previously posted (also here, here and here) about the ill-fated clinical trial, study 3014, of the antibiotic telithromycin (Ketek) made by Sanofi-Aventis, run by Pharmaceutical Product Development Inc. (PPD). Problems with the trial included fabrication of data at one clinical site, and allegations of manipulation of data at another. The physician in charge of the first site was convicted of mail fraud, and the physician in charge of the second had his license suspended. Although the results of this trial were never published, it still crept into the clinical literature: it was cited in a review article in the New England Journal of Medicine.

The case is now the subject of a US congressional investigation. As reported by the Associated Press (via the International Herald Tribune),

Democrats in the U.S. House of Representatives are threatening to hold a member of President George W. Bush's Cabinet in contempt for not turning over documents in a probe of a Sanofi-Aventis antibiotic.

Dingell, who chairs the House Energy and Commerce Committee, said he would support holding Health and Human Services Secretary Michael Leavitt in contempt for refusing to turn over FDA briefing documents subpoenaed by the committee.

The documents were used to prepare FDA Commissioner Andrew von Eschenbach for his appearance before lawmakers last year. Von Eschenbach testified the FDA did not use the flawed safety study to approve Ketek. Dingell and other Democrats say that statement may be untrue.

Robert West, an FDA agent who first investigated Ketek, said he tried to get permission in 2002 to look into whether Aventis was aware of fraudulent data when it submitted the study. West said his request was blocked by senior FDA officials, although he said he did not know which ones.

"A catastrophic failure" was how another FDA investigator, Douglas Loveland, described the company's handling of the study.

"The decision-making process Aventis used to investigate these problems was illogical and ineffective and it could have led them to come to the wrong conclusion," Loveland told House lawmakers.

The investigator stopped short of saying Aventis, Sanofi-Aventis' corporate predecessor, knew the results were false when they submitted them but said it "should have known." Loveland said the results contained all the hallmarks of sham data, including forged signatures and crossed out information.

Sanofi President for Research and Development Paul Chew testified that the company submitted the study in "good faith."

The company was not able to spot the fake patient results....

It seems clear that this case involved a very sloppy clinical trial sponsored and run by a pharmaceutical company. Such sloppy science breaks promises made to human subjects of clinical research that their participation would go to advance science and improve health care. Sloppy trials that produce dubious data will do neither. Furthermore, sloppy science that produces dubious results provides patients and doctors pseudoevidence rather than the best possible evidence relevant to decisions about health care interventions, and hence may also lead to bad decisions that harm patients.

It is bad enough that clinical trials sponsored by pharmaceutical, biotechnology, and device companies are often sloppy, and designed to show their products in the most favorable light, rather than provide valid and unbiased data testing hypotheses most important for patients' and physicians' decisions. This case now raises the more disturbing possibility that high government officials entrusted with protecting the public's health and safety may have hid the sloppiness and unreliability of the Ketek trial. Time and further investigation will tell if this possibility is more than speculation. But the latest events in the Ketek saga clearly show the need for more open and transparent clinical research on drugs and devices, and more open and transparent regulation of such companies by the government.

Who Decided to Change the Outcome Variable of the ENHANCE Trial of Ezetimibe?

We have posted before (as have many others, see below) about problems with the ENHANCE trial of ezetimibe (Zetia, by Schering-Plough, and one component of Vytorin, by Merck), and how the trial was designed and implemented seemed meant to increase the likelihood of a favorable result for the sponsors' interests. Particularly controversial was the sponsors' decision to change the definition of the trial's outcome variable after the data was collected. That decision was then reversed.

Today, Matthew Herper, writing in Forbes, described how the decision to change the outcome variable was made. First, the decision was made apparently entirely without any involvement by the ostensible principle investigator of the study.

Forbes has learned that the lead investigator charged with conducting the study, John J. P. Kastelein of the University of Amsterdam, was not even present when the companies made the controversial decision to change its goals, an unusual circumstance in such situations.

It's 'shocking' that Kastelein would not be party to discussion of the ENHANCE trial, says Harlan Krumholz, a cardiologist at Yale University. 'There should be a scientific committee that's independent running a study. He should be taking a leadership role.'

Asked about Kastelein not being present, Lee Davies, a Schering-Plough spokesman, said only, 'I will confirm that this was an independent expert panel.' Kastelein could not be reached for comment Wednesday or Thursday, but in the past he has said he disagreed with the company's decision to change the endpoint of the trial. In an interview with The Wall Street Journal in December, he said he breathed a 'sigh of relief' when it was changed back.

Next, Herper listed the members of the supposedly independent panel.

The list of those on the panel, obtained by Forbes, is filled with well-regarded experts in the field and should satisfy critics left wondering who was behind the recommendation to change the goals. It's not a wide-ranging committee of experts on clinical trials, but instead a group with expertise on studying the arteries with imaging technology. Four are top experts in the field of using ultrasound to take pictures of artery plaque. David Orloff, as a former Food and Drug Administration official, has expertise in exactly what regulatory challenges might be posed by changes to a clinical trial.

The panel: J. Robin Crouse and Gregory W. Evans of Wake Forest University, who conducted a study using Crestor, a rival cholesterol drug from AstraZeneca; David G. Orloff, medical director of Medpace, a company that conducts clinical trials for drug firms and the former head of the FDA division that handled the approvals of Zetia and Vytorin; Michiel L. Bots of the Julius Center for Health Sciences and Primary Care University Medical Center Utrecht, Netherlands, who has worked on similar studies and James H. Stein of the University of Wisconsin, who has conducted studies on the treatment of heart disease in patients with HIV and how various substances affect artery walls.

But is this panel really so independent? Using PubMed, I searched for conflict of interest disclosures made by its members. Guess what - you can see this one coming, can't you -

• J. Robin Crouse - "Dr Crouse reported receiving grant or salary support from Merck, Merck-Schering Plough, Pfizer, AstraZeneca, and Kos Pharmaceuticals; and giving lectures for Merck, Merck-Schering Plough, Pfizer, AstraZeneca, Abbott, and Kos Pharmaceuticals." (1)
• Gregory W. Evans - "Mr Evans reported receiving grant support and honoraria from AstraZeneca, Organon, and Pfizer; and being a consultant to AstraZeneca and Pfizer."(1)
• David G. Orloff - "David G. Orloff, MD, serves as a consultant to Amylin, AstraZeneca Pharmaceuticals LP, Bristol-Myers Squibb, Glaxo-SmithKline, ISIS, Merck & Co., Inc., Novo Nordisk, Pfizer Inc, Roche, Takeda Pharmaceuticals, Vivus, and Wyeth; and owns stock in Abbott Laboratories Inc., Amgen, Genentech, Genzyme, Johnson & Johnson, Eli Lilly, Merck & Co., Inc., Pfizer Inc, and Wyeth."(2)
• Michiel L. Bots - "Dr Bots reported receiving study grants for studies on carotid intima-media thickness and/or honoraria for professional input on carotid intima-media thickness issues from AstraZeneca, Icelandic Heart Foundation, Organon, Pfizer, the Netherlands Heart Foundation, the Netherlands Organisation for Health Research and Development, Servier, and Unilever."(1)
• James H. Stein - " Dr Stein has received research or grant funding from AstraZeneca, Bristol-Myers Squibb, Kos, LipoScience, PreMD, and Sanofi-Aventis, in addition to serving as a consultant to or on the speakers’ bureau for Kos, LipoScience, Merck, Pfizer, Schering-Plough, and Takeda."(3)
  

I presume that all the panel members were paid by Merck or Schering-Plough for their service on it. But all five panel members had other financial arrangements with multiple pharmaceutical companies, and three had other financial arrangements with Merck, or both Merck and Schering-Plough. So this panel may have been independent of the principal investigator of the study, but it did not seem independent of the two corporate sponsors of the study.

Thus, the case of the ENHANCE trial of ezetimibe appears to be an even stronger example of how commercial sponsors of clinical research may try to manipulate studies in ways likely to produce results favorable to their products and vested financial interests. We have posted frequently about cases in which a sponsor manipulated the design, implementation, analysis, or dissemination of clinical research to increase the likelihood that the results would favor the sponsor's product (e.g., recently here and here). One important study showed the contractual basis for such manipulation when studies are done in academic medical environments (see post here). In extreme cases, sponsors have attempted to suppress the research results entirely when they turn out to be unfavorable to their interests (e.g., see recent post here). Such manipulation, of course, is bad for patients and doctors, since it impedes their ability to make decisions about tests, treatments etc based on the best clinical evidence. It also betrays the trust of people who volunteer to participate in research thinking that the goal is to improve health care and advance science.

Although this case is striking, and has received considerable attention in the media and the blogsphere, it has been largely ignored in the health care and medical literature, the dominant source of information for physicians and health care professionals. On the Scientific Misconduct Blog, Dr Aubrey Blumsohn cataloged how extensively this case has been discussed on blogs, listing 35 related posts in the top 100 medical blogs since November. But he could find not a single mention of it in the best known English language medical journals, the British Medical Journal, Lancet, JAMA, or the New England Journal of Medicine. He asked, "One wonders how long our key medical journals will retain any credibility as honest impartial portals for discussion of science and the principles of good medicine." Thus this case also seems to be yet another example of what we have called the anechoic effect, the failure of stories about questionable ethics or bad leadership in health care to create the echoes one might expect (see post here).

See also coverage on the PharmaLot blog, updated here.

References

1. Crouse JR, Raichlen JS, Riley WA, Evans GW, Palmer MK, O'Leary DH, Grobbee DE, Mots ML, for the METEOR Study Group. Effect of rosuvastatin on progression of carotid intima-media thickness in low-risk individuals with subclinical atherosclerosis: The METEOR Trial. JAMA. 2007;297:1344-1353. (Link here.)

2. Orloff DG. Regulatory considerations in the development of high-density lipoprotein therapies. Am J Cardiol 2007; 100[suppl]: 10N-14N.

3. Keevil JG, Cullen MW, Gangnon R, McBride PE, Stein JH. Implications of cardiac risk and low-density lipoprotein cholesterol distributions in the United States for the diagnosis and treatment of dyslipidemia: data from National Health and Nutrition Examination Survey 1999 to 2002. Circulation. 2007;115:1363-1370. (Link here.)

A Skeptical Look at a Dire Warning: Is Medicare "Willing to Let Seniors Suffer and Die Sooner from Cancer?"

Yesterday's Providence Journal included an op-ed by Robert Goldberg, entitled "Fewer Drugs, Fewer Seniors," featuring this dire warning:

Now, to save money, America’s Medicare program is willing to let seniors suffer and die sooner from cancer....

How in the world could Medicare do that?

As a first step, the Center for Medicare and Medicaid Services (CMS), which runs Medicare, decided to restrict reimbursements for erythropoiesis stimulating agents (ESAs) — drugs that prevent the fatigue from chemotherapy-induced anemia.

In this case, CMS came up with a new standard for paying for new drugs called the Precautionary Principle. This principle, developed by environmental extremists in Europe nearly 20 years ago, states that anything unknown is unsafe and should not be used. CMS asserts that it does not know enough about the safe use of ESAs beyond achieving a specific hemoglobin level. So it won’t pay for anything above that point. No exceptions. If seniors feel fatigued or confused they can pass the hat like Chantelle Hill.

And why in the world would they do that?

The CMS decision has nothing to do with safety and everything to do with cost.

But before hitting the panic button, consider how the author of the op-ed, Robert Goldberg, identified as "Vice President of the Center for Medicine in the Public Interest," tried to support his contention.

First, he ignored the most relevant evidence while criticizing less relevant studies.

CMS says that ESAs cause cancer and heart problems beyond hitting a hemoglobin target of 10 g/dL. But that claim is based largely on two studies that examined the experimental use of extremely high dosages of ESAs in patients without cancer.

Goldberg ignored six studies that used different doses of different erythropoetin products (or ESAs, to use his terminology) to achieve different target hemoglobin levels in patients with various kinds of cancer. Four were published, and all six were summarized last June in the New England Journal of Medicine [Khuri FR. Weighing the hazards of erythropoiesis stimulation in patients with cancer. N Engl J Med 2007; 356:2445-2448. Link here.] Five of the studies showed shorter average survival in patients given erythropoetins. One showed increased progression of disease and a statistically insignificant trend towards decreased survival. This set of studies raise the concern that erythropoetins, rather than preventing early death, may hasten early death.

Next, Goldberg turned to the US Food and Drug Administration (FDA) for support.

The Food and Drug Administration, its European counterpart the European Medicines Agency, and physician groups such as the American Society of Clinical Oncologists recognized the value of these studies in guiding clinical decisions but not restricting them.

It is very rare for the FDA to restrict use of a drug already on the market. Actually, the FDA has emphatically warned about the possibility that erythropoetins may hasten death, rather than increase survival. The FDA considered the six studies summarized in the Khuri article above, and issued the following official "black box" warning for these drugs (for example, see label for Epogen below).

WARNINGS: Erythropoiesis-Stimulating Agents

Use the lowest dose of EPOGEN® that will gradually increase the hemoglobin concentration to the lowest level sufficient to avoid the need for red blood cell transfusion (see DOSAGE AND ADMINISTRATION).

EPOGEN® and other erythropoiesis-stimulating agents (ESAs) increased the risk for death and for serious cardiovascular events when administered to target a hemoglobin of greater than 12 g/dL (see WARNINGS: Increased Mortality, Serious Cardiovascular and Thromboembolic Events).

Cancer Patients: Use of ESAs
• shortened the time to tumor progression in patients with advanced head and neck cancer receiving radiation therapy when administered to target a hemoglobin of greater than 12 g/dL;
• shortened overall survival and increased deaths attributed to disease progression at 4 months in patients with metastatic breast cancer receiving chemotherapy when administered to target a hemoglobin of greater than 12 g/dL;
• increased the risk of death when administered to target a hemoglobin of 12 g/dL in patients with active malignant disease receiving neither chemotherapy nor radiation therapy. ESAs are not indicated for this population.

Finally, Goldberg asserted,

Seniors who are stronger and don’t have to spend time shuttling to the hospital are better equipped to endure newer and more powerful combinations of cancer drugs in the elderly. Indeed, a recent study conducted by Columbia University’s Frank Lichtenberg found that an aggressive chemotherapy regimen for seniors extends life expectancy by 8 to 12 months.

Lichtenberg's study seems so irrelevant that I wonder if it was raised as a deliberate distraction. Goldberg's original warning was about restrictions on payments for erythropoetins, not about chemotherapy. That Lichtenberg's study is not yet published, and that Lichtenberg is an economist at the Columbia Business School, just add to the irrelevance.

So to summarize, Robert Goldberg's dire warning that Medicare is out to make sure, as implied by the title of his article, there are "Fewer Drugs, Fewer Seniors," seemed to be based on a mixture of half-truths and irrelevancies. So why did he make this warning?

I don't know, but what the Pro Jo article did not reveal is some relevant background information about the ties of Goldberg's organization to pharmaceutical companies. We have noted before that Goldberg works for the Center for Medicine in the Public Interest, whose president is Senior Vice President for Global Health Affairs at the big public relations firm Manning, Selvage and Lee. Manning, Selvege and Lee has many big pharmaceutical accounts, as listed on the CommuniqueLive.com site. This suggests at least an indirect conflict of interest, favoring the interests of the pharmaceutical industry, that should have been revealed in this op-ed article. Furthermore, in a post on CMPI's DrugWonks.com blog, Goldberg acknowledged "CMPI accepts grant from drug companies -- and we do so proudly because they actually invent things that help people."

We have previously posted about other stealth health policy advocacy coming from CMPI (here and here.)

If CMPI really accepts money from pharmaceutical and biotechnology companies "proudly," its officers should not be afraid to make this and any other financial ties they may have to these companies clear when they write op-eds in major newspapers. That might at least alert readers who are not trained in evidence-based medicine to be a bit skeptical about the apparently scientific, but actually mostly irrelevant references with which they adorn their writing, and to wonder how much relevant evidence they did not bother to include.

That health care in the US is now in a crisis is now a cliche. Finding our way out of this crisis requires logical discussion based on evidence, not half-truths and irrelevancies promulgated by people with hidden financial ties.

Data About Ezetimibe Adverse Effects Suppressed?

Here we go again. Last week, an article by Alex Berenson in the New York Times reported that Merck and Schering-Plough conducted trials of ezetimibe (Zetia, by Schering-Plough, and a component of Vytorin, by Merck) "that raise questions about its risks to the liver," but never publicly reported their results.

Partial results of the studies, alluded to in documents on the Food and Drug Administration’s Web site, raise questions about whether Zetia can cause liver damage when used long term with other cholesterol drugs called statins.

A Schering executive, when asked by a reporter about the unpublished studies, confirmed their existence. But the executive, Dr. Robert J. Spiegel, said the companies had not considered the studies scientifically important enough to publish their findings. Some may eventually be published, he said.

The unpublished Zetia studies, devised as safety tests, would not prove the drug’s effectiveness. But they would give the public more information about Zetia’s potential risks. All the unpublished studies covered periods at least one year in length and were intended to show whether long-term use of Zetia might pose dangers that short-term use did not.

Most of the studies about Zetia in which Merck and Schering have published the results covered periods of only 12 weeks — not enough time for liver problems to develop in most patients.

The unpublished studies, conducted from 2000 to 2003 according to the F.D.A. documents, were not listed on the industry Web sites where companies are supposed to register the results of all drug trials that were ongoing after October 2002. The New York Times discovered references to the studies in briefing papers on the F.D.A. Web site.

Together those studies cover several thousand patients who took Zetia along with statins for one to two years. The statins include Lipitor and Crestor, as well as Zocor, which is usually prescribed generically as simvastatin and is the statin used in the Vytorin pill. Doctors often add Zetia to a low dosage of a statin, because Zetia reduces cholesterol in a different way than the statins do and leads to deeper overall cholesterol reductions.

'We’re pretty comfortable that people don’t have trouble tolerating Zetia,' said Dr. Spiegel, the chief medical officer of the Schering-Plough Research Institute, Kenilworth, N.J.

I agree more with this response by Dr Harlan Krumholz, from Yale.

We keep telling people we want to practice evidence-based medicine, and what we keep finding out is that much of the evidence is obscured.

There is important evidence, but it’s not in public view. It’s hidden from investigators.

As I have noted now it seems ad infinitum, to make the best decisions about patient care, physicians and patients need access to the best relevant evidence from clinical studies. Concealing evidence about, say, the toxicities of a particular drug may lead to bad decisions, specifically, prescribing the drug when its possible harms outweigh its risks.

In addition, suppressing the results of clinical research betrays the trust of human research subjects who were participating because they thought the research might advance science and/or patient care. Obviously, suppressed research can do neither.

This is one more anecdote suggesting that either clinical research sponsored by the companies who make the products the research is meant to evaluate needs much stronger regulation, or such research should be taken entirely out of the hands of the companies who stand to benefit if its results turn out a certain way.

See also comments in the Clinical Psychology and Psychiatry Blog, the Hooked: Ethics, Medicine and Pharma blog (with clinical and clinical epidemiologic background), the Medical Evidence Blog (suggesting a boycott of ezetimibe), and the Scientific Misconduct Blog (warning, contains adult language.)

Haunted Health Care: the Scope of Ghost Management

An important new article in PLoS Medicine expanded thinking about the involvement of pharmaceutical companies (and possibly other health care corporations) in the shaping of clinical research. [Sismondo S. Ghost management: how much of the medical literature is shaped behind the scenes by industry? PLoS Med 4(9): e286 doi:10.1371/journal.pmed.0040286]

Sismondo defined ghost management of medical research and publishing:

when pharmaceutical companies and their agents control or shape multiple steps in the research, analysis, writing, and publication of articles. Such articles are 'ghostly' because signs of their actual production are largely invisible—academic authors whose names appear at the tops of ghost-managed articles give corporate research a veneer of independence and credibility. They are 'managed' because those companies shape the eventual message conveyed by the article or by a suite of articles.

Sismondo thus went beyond the issue of ghost-writing, the contribution of unnamed authors who may be influenced, or hired, by pharmaceutical companies or other organizations with vested interests. (See recent post here.) He also went beyond the issue of how pharmaceutical companies and other research sponsors may attempt to influence the design and implementation of research studies, the analysis of their results, and/or how the results are reported. (See this post.)

As Sismondo noted,

It has been repeatedly and firmly established that pharmaceutical company funding strongly biases published results in favor of the company's products. Ghost management amplifies that bias, because when one set of commercial interests exerts influence at multiple stages of research, writing, and publication, it will shape the resulting article. In turn, bias affects medical opinion and practice, and ultimately, patients.

Sismondo also attempted to estimate the prevalence of ghost management. This would have been predicted to be a difficult task, because,

it is not in the interests of writers, authors, or sponsors and their agents to reveal ghost management processes; hence a number of the published accounts of ghost management have stemmed from legal proceedings and investigative journalism.

So he set out to review existing evidence and find some new evidence.

First he noted the article by Healey and Cattall, based on documents produced during legal proceedings, that explained how Pfizer Inc ghost managed publications of multiple articles on the drug sertraline [Zoloft], producing 85 manuscripts, and accounting for an important part of the literature on this one drug. (See previous post here.)

An important contribution by Sismondo was what he called his "supply-side analysis."

A survey in 2001 identified 182 MECCs in the United States, up from 153 in 1998. A number specialize in producing, placing, and tracking journal articles, known in the trade as 'publication planning' or 'strategic communication planning.' While these firms hide details of their work—from potential critics and competitors—they also energetically promote themselves and their services.

I spent six hours searching web pages for MECCs offering publication planning or similar or overlapping services to the pharmaceutical industry, and found 23 (list available from the author). This is not an estimate of the number of such firms, but indicates how common they are. There may be many more firms providing publication planning, including some not uncovered in this search, and some not advertising these services on the Internet.

In a primer on publication planning, the director of one MECC defines the activity as: 'gaining product adoption and usage through the systematic, planned dissemination of key messages and data to appropriate target audiences at the optimum time using the most effective communication channels'.

Complete Healthcare Communications (CHC) claims on its banner that it 'has honed the systems and skills needed to develop the intellectual heart of pharmaceutical marketing—the publication plan. The result for your product? A continuum of awareness, interest, and prescriber confidence.'

CHC includes among its clients Pfizer, Sanofi-Aventis, Ortho Biotech, Wyeth, Schering-Plough, Shire, AstraZeneca, and other pharmaceutical companies.

Other agencies offer very similar services. As described in an article by three of its managers, the Medical Knowledge Group starts publication planning with a phase of exploring 'key messages' and 'author/journal options' before designing any publications to incorporate those messages

Another MECC, Envision Pharma, says that 'data generated from clinical trials programs are the most powerful marketing tools available to a pharmaceutical company.' Envision will work from early on in the process to ensure 'consistent message dissemination,' will plan and track the 'data dissemination plan,' and will produce 'scientifically accurate, commercially focused abstracts, posters, and primary and secondary publications'

Several of the publication planning firms identified are owned by major publishing houses. For example, Excerpta Medica is 'an Elsevier business' and writes that its 'relationship with Elsevier allows… access to editors and editorial boards who provide professional advice and deep opinion leader networks.'

Wolters Kluwer Health draws attention to its publisher Lippincott Williams & Wilkins, with 'nearly 275 periodicals and 1,500 books in more than 100 disciplines,' and to Ovid and its other medical information providers, emphasizing the links it can make between its different arms

Vertical integration is attractive in the industry as a whole: at least three of the world's largest advertising agencies own not only MECCs, but also CROs.

Ghost management of medical journal publications is clearly a substantial business, employing thousands of marketers, writers, and managers. It is large enough that the industry has established the International Publication Planning Association.

The conclusion is that

Given the amount of data that pharmaceutical companies control, the number of publication planning agencies that openly advertise on the Internet, the number of medical writers, the existence of two associations for publication planners, and meetings organized and reports written for them, we can conclude that ghost management is common.

Why is it so important?

Articles in medical journals have real effects upon physician prescribing behavior, which is why pharmaceutical companies invest so much in their publication. Journal articles are heavily used in detailing, to validate claims and rebut worries. Even independent of detailers, responsible physicians and medical researchers search the literature to gather evidence about the best treatments. Published scientific articles are the sources of medical information with the highest authority. Systematic reviews and meta-analyses almost all start with the published literature—so even fully independent reviews are influenced by ghostly activities.

It is clear that ghost management is a major part of the production of pseudoevidence-based medicine.

Sismondo's article is an important contribution to our knowledge of how research on human beings is manipulated into marketing and propaganda, if not outright disinformation. Such practices break the trust of research subjects who thought they were participating to advance science and medical care. Such practices mock physicians who think they are trying to make decisions based on clinical science. Such practices can harm patients who are subjected to tests and treatments based on hype and manipulation rather than unbiased data.

Sismondo ended with some suggestions for alleviating the problem, but noted,

There are no straightforward solutions, short of large changes to the nature of medical publishing and/or research, changes that would effectively sequester pharmaceutical company funding from research and publishing....

We have now said frequently that there are many reasons to think about whether organizations who have a vested interest having clinical research produce particular results should be banned from funding or other involvement in research on human beings. The probably pervasive practice of ghost management is another strong reason to consider this seemingly extreme step.

In the absence of such a drastic prohibition, all I could suggest is a much strengthened and more comprehensive disclosure policy. Many would now agree that all clinical trials should be registered. Such registration should include details of any involvement of pharmaceutical, biotechnology, device and similar companies in the design and implementation of the research, and the analysis and reporting of its results. Any involvement by contract research organizations, for-profit institutional review boards, medical education and communication companies, and the like should be disclosed in trial registration.

Furthermore, there should be full disclosure of all involvement by any of these organizations accompanying any publication or presentation of study results, including publications in peer-reviewed journals, abstract presentations, etc.

Furthermore, the penalties for concealing involvement of any of these organizations should be severe, all the way up to considering failing to disclose as potential criminal fraud.

Allowing ghost management to continue victimizes people who volunteer to participate in research, mocks physicians who try to base their clinical decisions on the best available evidence, and hurts patients.

Hat tip to the Medical Humanities Blog, and one of our corps of Health Care Renewal scouts.

BLOGSCAN - More Criticism of Study Linking Rising Youth Suicide and "Black Box" Warnings for SSRIs

On the Clinical Psychology and Psychiatry blog, this post dissects that study which purported to link the decreasing use of selective serotonin reuptake inhibitor (SSRI) anti-depressants, after the US Food and Drug Administration (FDA) mandated a "black box" warning for these drugs, circa 2004, with increasing youth suicide rates (in 2004). It turns out that the study's methods appear rather weak. Questions had already been raised about the study (see earlier post on Clinical Psychology and Psychiatry) , which was also not corroborated by reports that the suicide rate declined again in 2005 (see another earlier post). We previously noted Merrill Goozner's critical take on the same study in GoozNews, in which he also alluded to the conflict of interest of the study's author that was widely ignored in its previous media coverage.