What If the Institute of Medicine Wrote a Report and Nobody Followed it? - the Case of the Standards for Developing Trustworthy Guidelines

For over 20 years, clinical practice guidelines (CPGs) have been touted to improve health care quality and control costs.  Enormous numbers of guidelines have been developed, but with seemingly little impact on health outcomes.  While some of those leading health care organizations have predictably blamed individual practitioners for obstinately ignoring or challenging guidelines, there is increasing evidence that maybe the guidelines themselves are part of the problem.

 An Example of a Guideline that Apparently was Not Trusted

One example Dr Wally Smith and I have taught in our recurring mini-course on why physicians fail to follow guidelines (and otherwise appear not to practice in accord with others' wishes) is that of the guidelines on management of depression in primary care.  Most existing guidelines urge physicians to screen patients for (presumably mild-to-moderate) depression and treat them aggressively, with emphasis on the use of the newer anti-depressants.  These guidelines, in turn, were based on numerous published randomized clinical trials that showed that these drugs were safe and efficacious.  Yet multiple studies showed that physicians failed to follow these guidelines, and various attempts to improve their adherence did little.  So for years the assumption was that physicians at best experienced practical and system barriers to follow these guidelines, and at worst were ill-informed or irrational. 


However, information that came out gradually during the early part of the 21st century suggested that perhaps the problem was within the guidelines, not the health care professionals.  First, documents produced during New York Attorney General Eliot Spitzer's lawsuit against GlaxoSmithKline about the marketing of one of these drugs (Paxil, paroxetine) suggested that the company had suppressed clinical trial data that reflected poorly on the drug (See Kondro W.  Drug company experts advised staff to withold data about SSRI use in children.  Can Med Assoc J 2004; 170: 783.  Link here.)    These suspicions were later fleshed out  by consideration of documents further disclosed in litigation (e.g., see this post and its links).  Then several studies, most particularly that by Erick Turner and colleagues, showed that numerous trials of new anti-depressants had been suppressed, that is, never published (Turner et al. Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med 2008; 358:252-260.  Link here).  When the results of these trials were added to those that were published, the efficacy of anti-depressants was no longer so clear.  So maybe the guidelines that physicians did not follow were not trustworthy, and should not have been followed in the first place.

Would IOM Standards to Improve Guideline Trustworthiness Help?

So in 2011, the prestigious Institute of Medicine released a report on the development of  better standards to produce more trustworthy guidelines (Clinical Practice Guidelines We Can Trust.  Link here.)  We posted about that report here, but noted that it was receiving little other attention, an example of the anechoic effect. 

A few weeks ago, an article appeared documenting a study meant to assess the the trustworthiness of clinical practice guidelines published soon after the IOM report.  Its title telegraphs the results. ( Kung J, Miller RR, Mackowiak PA. Failure of clinical practice guidelines to meet Institute of Medicine standards: two more decades of little, if any progress.  Arch Intern Med 2012.  Link here.)

Methods and Results

The investigators selected a random sample of 114 individual guidelines available during June, 2011 stratified by 26 clinical topics. The versions of the guidelines used were those archived in the National Guideline Clearinghouse (NGC) maintained by the Agency for Healthcare Research and Quality (AHRQ).


The goal of the study was to "examine adherence to the IOM standards" by guidelines published after the standards were published.  Actually, the study only assessed adherence to 18 of the 25 standards espoused by the IOM (because the remaining seven were "too vague and subjective to be analyzed.")

Furthermore, the criteria used to determine if a specific guideline met each of the three items above were rather lax:

In evaluating each guideline summary, care was taken to be as liberal as possible in considering that a standard was met when the individual guideline summary provided any information pertaining to that particular standard.

Nevertheless, using these lax standards to only evaluate adherence to 18/25 guidelines, the authors found that "the overall median number of IOM standards satisfied (out of 18) was 8 (44.4%) ....  Fewer than half of the guidelines surveyed met more than 50% of the IOM standards."

An examination of the details of the study's methods reveals things are even worse than that.

Analyzing the Study's Methods to Find that Things Are Worse Than They Seem

Review of the study's methods show that they provided a very optimistic view of adherence to the IOM standards.  As noted above, the study did not look for adherence to all of the IOM standards.  Moreover, those they did consider were simplified and made less rigorous.  For example, Standard 2 from the IOM on management of conflict of interest (COI) was:

  STANDARD 2
Management of conflict of interest (COI)

2.1
Prior to selection of the Guideline Development Group (GDG), individuals being considered for membership should declare all interests and activities potentially resulting in COI with development group activity, by written disclosure to those convening the GDG.
- Disclosure should reflect all current and planned commercial (including services from which a clinician derives a substantial proportion of income), non-commercial, intellectual, institutional, and patient/public activities pertinent to the potential scope of the CPG.

2.2
Disclosure of COIs within GDG
- All COI of each GDG member should be reported and discussed by the prospective development group prior to the onset of their work.
- Each panel member should explain how their • COI could influence the CPG development process or specific recommendations.

2.3
Divestment
- Members of the GDG should divest themselves of financial investments they or their family members have in, and not participate in marketing activities or advisory boards of, entities whose interests could be affected by CPG recommendations.

2.4
Exclusions
- Whenever possible GDG members should not have COI.
- In some circumstances, a GDG may not be able to perform its work without members who have COIs, such as relevant clinical specialists who receive a substantial portion of their incomes from services pertinent to the CPG.
- Members with COIs should represent not more than a minority of the GDG.
- The chair or co-chairs should not be a person(s) with COI.
- Funders should have no role in CPG development.

However, the study boiled all this down to three items:
-  COIs stated
-  Chair has COI
-  Co-chairperson has COI

Thus the study did not address standards requiring full and complete disclosure (not just some disclosure) of all COIs; consideration of how the COIs might influence the particular guideline; divestment of specific types of conflicts of interest, that is, financial investments, and cessation of participation in marketing activities or advisor boards; minimization of conflicts of all members of the committee; and barring of participation of funders in guideline development.

Even so, the guidelines assessed did a very poor job upholding even these few liberalized standards regarding conflicts of interest.  Of the guidelines assessed, less than half, 46.8% provided ANY disclosure of conflicts of interest.  Those written by sub-specialty societies were particularly opaque in this regard.  Less than one-third, 29.3%, provided any disclosure.  Thus the majority of guidelines assessed were not at all transparent about conflicts of interest affecting the guideline development process.

Furthermore, of the 46.8% of all the guidelines which made any disclosures of conflicts of interest, 71.4% admitted their chair people HAD a conflict of interest.  Thus, only (0.468 * [1 - .714]) = 13.3% provided assurance that they fulfilled the single requirement (from standard 2.4 above) that the chair person did not have a COI.  For the guidelines written by sub-specialty societies, by a similar calculation, only 12.2% provided an assurance that the chair had no COI.  (The proportions providing assurance that the co-chair people had no COI were even lower.)  Thus the vast majority of guidelines did not clearly follow two straight-forward standards for minimizing the effects of conflict of interest, that the guideline committee chair and co-chair should not have any relevant conflicts. 

Given the miserable results concerning even minimal adherence to some of the IOM report's conflict of interest standards, it is likely that almost no published guidelines from 2011 came close to fulfilling the full set of IOM standards.  Despite the best efforts of the IOM, it appears that guideline developers have not progressed at all towards providing trustworthy guidelines. 

Summary

An editorial (Shaneyfelt T. In guidelines we cannot trust.  Arch Intern Med 2012) accompanying the article by Kung and colleagues summarized its results thus:

The same problems that have plagued guideline development continue to plague guideline development; namely, their variable and opaque development methods, their often conflicted and limited panel composition, and their lack of significant external review by stakeholders throughout the development process.   As a result, the trustworthiness of guidelines is limited.

While guidelines may have seemed to be a promising method to improve health care in the early 1990s, they have failed to live up to that promise.  Shaneyfelt was not optimistic they would improve in the future:

I am not optimistic that much will improve.  No one seems interested in curtailing the out-of-control guideline industry.

On the other hand, in my humble opinion, it is not that on one is interested in better guidelines.  It would clearly be in the best interests of patients and the public, and of health care professionals who care about the quality of their practice and the outcomes of their patients to curtail that industry.  The issue is why patients', the public's, and professional's interests were ignored.

Neither Shaneyfelt nor Kung et al discussed why there has been so little attention to patients' and the public's health, and to health care professionalism in all this.  For a quick answer, we do not have to look far on Health Care Renewal. 


In fact, the IOM report on guideline development from 2011 was a serious challenge to the powers that be in health care.  In particular, it challenged the cozy relationships that had grown up among the organizations that undertook guideline development and the health care professionals on guideline panels on one hand and organizations that stand to gain were specific guidelines to favor their products, services, and agendas on the other.  The standards mandated transparency and honesty about conflicts of interest affecting guideline committees and the organizations which assembled them, and if upheld would have greatly reduced these relationships.

Now it turns out that the guideline standards have been honored mainly in the breach.  Of course, these standards, while increasing trustworthiness, would have cost a lot of medical societies considerable commercial funding, and would have cost a lot of health care professionals on guideline panels considerable personal wealth.  These standards would probably also have cost a lot of companies whose products and services were addressed by guidelines to lose revenue.  So it is not surprising that the IOM standards were ignored.  Their implementation would have cost too many people who are financially benefiting from the status quo too much money.  And these people, that is, leaders of professional societies dependent on commercial outside funding, health care professionals and academic used to financial support from commercial interests, and health care corporations are good at making sure their interests are not ignored, even if their interests conflict with those of patients, the public, and well-intentioned health care professionals.  

So, the flouting of the well reasoned IOM guideline standards adds one more reason for patients and the general public to distrust modern health care and all those who "deliver" it, even to distrust well-intentioned health care professionals who have not been able to distinguish themselves from their colleagues who are too happy to help commercial interests while taking commercial money.  If health care professionals want to regain the public's trust, they could do worse than publicly declaring their intention to show that their practice in the future will be guided by trustworthy guidelines based on clinical research evidence and knowledge of biomedical science, drawn up by health care professionals independent of commercial interests.

BLOGSCAN - Television Advertising Revenue and the Anechoic Effect

We have often discussed the anechoic effect, how cases involving or discussions of the topics we address on Health Care Renewal, the concentration and abuse of power in health care, fail to produce any responses, or echoes.  Two recent blog posts discussed one way in whicht the anechoic effect might be generated.

A post by Dr Steven Greer on CurrentMedicine.TV, enlarged upon by Alison Bass on the Alison Bass blog, discussed a segment on 60 Minutes yesterday that dealt with the evidence that anti-depressant drugs may not be efficacious for mild to moderate depression.  Since this evidence is about four years old, the question is why it has only made it to the main-stream media now?  Both Dr Greer and Ms Bass think it may be because the patents on most of the newer, mainly selective serotonin reuptake inhibitor (SSRI) type anti-depressants have run out.  Therefore, their manufacturers may no longer be interested in using the clout they derive from paying millions for television advertising to keep programs critical of these drugs off the air.  The implication is that large health care organizations may often use threats to withdraw advertising to forestall criticisms of their products or their agendas in the media, hence increasing the anechoic effect.

ADDENDUM (27 February, 2012) See also comments on the 1BoringOldMan blog. 

Quintiles to More Heavily Directly Invest in the Drugs Which it Evaluates - But Not to Worry, Says an Expert?

The Associated Press just reported on the latest trend in commercially sponsored clinical research, direct investment by contract research organizations (CROs) in the development of specific drugs by particular pharmaceutical or biotechnology companies. Here is the background in the article:

Pharmaceutical companies previously did all their own research and testing and submitted the results to the Food and Drug Administration for approval to put a medication on the market.

Since the 1990s or so, Big Pharma has farmed out more and more of that testing to companies like Quintiles Transnational Corp., which grew quickly to meet the demand.

Now Quintiles is branching out...

Now Durham, N.C.-based Quintiles is the world's biggest contract research organization, and more than ever, it's using its deep pockets to entwine itself with the pharmaceutical companies, fronting them cash and services for a piece of the profits once a drug is approved.

Quintiles, which was created in 1982, upped the ante last month, saying it wants to invest even more in potential drugs as pharmaceutical companies grapple with the down economy. Quintiles is hoping its new investments will produce enormous payoffs in the future,....

Quintiles' decision to become more aggressive in taking investment risks comes at a time it's betting drug makers can use the resources it can offer, both cash and services.

'They're both currency for managing risk and having skin in the game,' said Ron Wooten, a Quintiles executive vice president who heads the company's capital investment group.

Controversy over creating the drug investment unit in 2000 was key in pushing Quintiles founder Dennis Gilling and several private-equity firms to buy out shareholders for $1.75 billion and take the company private again in 2003. Wall Street analysts complained they couldn't estimate what the drug investments were worth, prompting Gilling and others to decide the company's stock was undervalued.

The private company no longer publishes its profits. It claims revenues of about $3 billion a year. That's also about what Quintiles has invested in other drug companies.

Furthermore,

Quintiles appears to be uniquely positioned to both invest in and test potential medications. None of the half-dozen major shareholder-owned contract research organizations will have a drug-investment division after Wilmington, N.C.-based Pharmaceutical Product Development Inc. spins off its unit in a few months, Coldwell said.

The payoffs for the risk of developing a drug can be huge.

In 2002, Quintiles took a chance on Eli Lilly and Co.'s antidepressant Cymbalta. Quintiles said it invested $125 million in cash, then spent another $400 million on a sales force of more than 550 after the product launch. Cymbalta is now Lilly's second-biggest drug. Quintiles said it's brought in more than $700 million so far for a stream of royalties lasting to 2012.

Japan's Eisai Co. is hoping Quintiles' money and experience helps it double the number of potential treatments it can put through testing simultaneously

The article raised a concern that "the buffer between drug makers and testers are again blurred."

But not to worry.... Here is the response from Quintiles,

Wooten, the Quintiles executive, doesn't see his company's increased interest in bringing drugs to the market affecting its testing objectivity. Instead Quintiles is the outsider turning a hard eye on which pharmaceuticals will become profitable.

'It allows you to really get to the essence of the data that you're looking at instead of what you hope it says,' he said. There isn't 'the natural bias that you would have with your own babies.'

Furthermore, the AP reporter got reassurances from a noted academic expert in the area,

The FDA preserves the integrity of testing in a landscape already pockmarked with potential conflicts of interest that include company-sponsored academic research and investigators testing drugs while owning shares in companies that own the drug, said Kenneth Getz, a senior fellow at Tufts University's Center for the Study of Drug Development.

The FDA's view is that any company with a financial stake in a potential drug will face similar scrutiny from the agency.

'It's assumed,' FDA spokeswoman Karen Riley wrote in an e-mail, 'that drug/device/biologic companies who sponsor trials of pending products have a conflict because they will gain if the product is approved.'

Besides, pharmaceutical companies have navigated the tricky territory that comes with testing, Getz said. So have the companies like Quintiles that have taken on the outsourced work, he said.

'These potential conflicts of interest are ever-present, and the real question is how do companies manage these potential conflicts effectively,' he said. 'These are massive, massive companies that have thousands of staff that are not generally aware of a conflict of interest at a corporate level. They are paid to do a quality job that meets internal and regulatory guidelines or they are not going to keep their jobs.'

Here on Health Care Renewal, we have discussed how the conflicts of interest that are now pervasive in health care can adversely affect clinical care, teaching, and research.  We have looked at how clinical research sponsors, who have a vested interest in having the results favor the products and services they sell, may manipulate the design, implementation, analysis, and dissemination of research, and when all else fails, suppress unfavorable results.  We have also looked at problems with the ethics and quality of research done by contract research organizations, who are, after all, paid by companies again with vested interests in selling particular products or services.

Given all that, one may argue that having contract research organizations invest in particular drugs or devices which they are also evaluating through clinical research may not make things much worse.  However, at the least, it seems to me that it further obfuscates conflicts of interest that may lead to biased research results, dishonoring the patients who volunteered for the trials, and biasing the information on which patients and physicians make decisions.

Finally, I should note how Quintiles new investments in specific drugs was defended in the article.  The article quoted Quintiles executive vice president Ron Wooten suggesting that Quintiles might be able to be objective in making the original decision to invest in a drug. He conveniently did not address what might happen after the investment was made, when Quintiles was performing, analyzing, and reporting clinical research on a drug or device in which it had invested.

The article also quoted Kenneth Getz, the Senior Fellow at Tufts, who seemed unconcerned about the issue.  What the article did not mention is that the Tufts Center for the Study of Drug Development, at which Mr Getz is based,

receives unrestricted grants from pharmaceutical and biotechnology firms, as well as companies that provide related services to the research-based industry (e.g., contract research, consulting, and technology firms). These grants represent approximately 40% of Tufts CSDD’s operating expenses.

per the Center's web-site. Another page on the web-site lists Mr Getz's qualifications,

Kenneth A. Getz is a senior research fellow at the Tufts CSDD. Considered an expert on the investigative site and CRO markets, R& D management, ECT solutions adoption and clinical research volunteerism,....

Mr. Getz is the founder and chairman of CISCRP ....

He holds an MBA from the J.L. Kellogg Graduate School of Management at Northwestern University and a bachelor's degree, Phi Beta Kappa, from Brandeis University. Mr. Getz worked for over seven years in management consulting, where he assisted biopharmaceutical companies in planning and implementing drug development strategies. He is also the founder and former CEO of CenterWatch.

Note that Mr Getz seems to have no training in biomedical science, medicine, or health care.  Furthermore, CISCRP (the Center for Information and Study on Clinical Research Participation), has an advisory board (see their web-site here) that includes representatives of pharmaceutical companies (Vertex Pharmaceuticals, Biogen Idec, and Roche), and contract research organizations, (PPD Inc, and particularly, in this case, Quintiles). The vast list of the Center's "supporters" include a catalog of drug and biotechnology companies (including Abbott Vascular, Amgen, AstraZeneca, Biogen Idec, Daiichi Sankyo Pharmaceutical Development, Eli Lilly, Forest Laboratories, Genentech, Genzyme, GlaxoSmithKline, Janssen, Johnson and Johnson, Merck, Novo Nordisk, Novartis, Ortho-McNeil, Pfizer, Roche, Sankyo, Sanofi-Aventis, Serono, Shire, Takeda, and Wyeth), and contract research organizations, (including Covance, Parexel, PPDI, and notably again, Quintiles.)

So the other lesson from this bit of news is how the information the public, health care professionals, and policy makers  receive about health care policy is frequently shaped by people with horses in the race, but that such influences are rarely disclosed.  This article included an interview with a single apparently academic expert who provided reassurance that the conflicts of interest discussed in the article were not really so worrisome.  This expert, however, failed to disclose his own extensive, albeit somewhat indirect ties to a variety of corporations that might benefit from the conflicts discussed as the main topic of the article, including corporations that were directly involved in them (Quintiles and Eli Lilly).  Maybe because the reporter thought that the expert was unbiased, there was no attempt to find a contrasting opinion.

So I say again, again, again, health care professionals, policy  makers, and people in general need to be extremely skeptical of most of the apparently unbiased pronouncements made about medical and health care policy issues in the media.  It would be a small step toward a more transparent discussion if those who expressed opinions would also fully disclose in detail their relevant conflicts of interest.

The Lexapro Marketing Plan Was Meant to Promote Marketing (Surprise?)

Last week, Gardiner Harris writing for the NY Times noted that the US Senate Special Committee on Aging had made public part of Forest Laboratories' Fiscal Year 2004 Marketing Plan for the drug Lexapro (escitalopram oxalate), an anti-depressant. The document is available here.

Review of this plan revealed the marketing department's various activities, including activities that others might have believed were educational, scientific, or had some other high minded purpose.

Continuing Medical Education

Overall, one "promotional objective" was to "Maintain SRI category leadership in total number of medical education events (including CME symposia, speaker promotion, teleconferences, and peer selling programs)"

One "critical issue" was to "increase Med Ed efforts: more sponsorships of CME, increased level of speaker programs, maintain level of teleconferences and peer selling."

Under "Marketing Tactics" was a long section on "Continuing Medical Education." It covered various venues for CME such as internet/electronic CME, "sponsoring symposia at major meetings," "regional CME symposia" which would "serve a number of medical specialties," "sponsorhips of scientific sessions," etc

Production of Scholarly Articles

The "Publications" section of "Marketing Tactics" noted "publications will be geared toward psychiatrists, PCPs, .... Articles will appear in several formats, including original reports, review articles, and journal supplements."

Thought Leaders and Consultants

Under the "Continuing Medical Education" section of "Marketing Tactics," and then in the "Advisor Relations" section, there were numerous references to what "thought leaders" would do, including
- "present new data" at "symposia at major meetings"
- act "as advisors to Lexapro in order to obtain critical feedback and recommendations on educational and promotional strategies and tactics."
- sit on the "Lexapro Exectuive Advisory Board" to "keep our advisors apprised of the commercial development ... of escitalpram."
- sit on the "Primary Care Advisory Board" to "obtain critical feedback and recommendations on educational and promotional strategies and tactics...."

Role of Medical Centers

Under the CME section of "Marketing Tactics," we see that "academic health centers" would help develop "regional CME symposia."

Under the "Sponsorship" section are plans to fund the "Professional Relations Group in their mission of establishing mutually beneficial long-term relationships with appropriate professionals and associations." Specific plans included funding the Department of Psychiatry and Behavioral Sciences at Emory

Medical Societies

The "Sponsorship section" also noted that these relationships "will also provide the basis for advocacy development and issues management, and will establish an appropriate environment for commercial and policy activities."

Under the CME section of "Marketing Tactics, we see that "medical associations and professional societies" would help develop "regional CME symposia" Also in this section is the note that "smaller, more prestigious societis do not accept industry-sponsored symposia." So instead, "sponsorship of a study groups or plenary sessions is recommended. Marketing will work with the professional relations group regarding potential opportunities."

Appendix VII, "Professional Associations of Priority" noted funding provided for guideline development by a "collaboration between APA, AAFP and ACP" for "chronic depression in primary care practice;" by AAGP for guidelines for late-life depression; multiple guidelines developed by AMDA;

The appendix also noted that Forest supports ACNP "annual programming, and is a founding sponsor of it's newly created International College of Geriatric Pscyhopharmacology (ICGP)," is a "Corporate President's Circle Sponsor of AAFP," became a "corporate sponsor of ACP in FY03," became "a Corporate Sponsor of AMDA for the last few years,"

Also, it noted that Forest has "expanded its involvement" with APA for "lobbying of State Health Departments...."

Disease Advocacy Organizations

Also in Appendix VII, Forest was identified as a "Corporate Sponsor of NAMI," a "major Corporate Sponsor of NMHA," and a "major Corporate Sponsor of DBSA."

Summary

We have often heard from pharmaceutical, biotechnology, device and other health care corporations that they are only involved in education to disseminate accurate information for the good of society. We have often heard from physicians and academics who consult for such corporations that their advice is sought about clinical, scientific, and technical issues. We often hear from academic medical institutions, medical associations and disease advocacy groups that the money they get from such corporations does not influence the content of their educational and scientific work.

Yet here we see, in considerable detail, that in the case of one drug company's promotional efforts for one drug,
- The marketing plan from the marketing department paid for medical education as a "promotional objective," that is, to market, not to educate.
- Thought leaders and consultants were again paid by marketing to market, and sometimes to provide opinions about "promotional strategies" and "commercial development."
- Medical associations are funded by marketing "for commercial and policy activities."

This suggests that health care corporations develop financial relationships with physicians, academics, academic institutions, medical and professional associations, and disease advocacy groups to support marketing first.

This allows the corporations to advance marketing disguised as education, research, and other high-minded and apparently selfless activities by professionals, not-for-profit organizations, and dedicated inviduals. Such marketing, of course, is DECEPTIVE and DISHONEST. It also is in conflict with the professionals' ideals, and the missions of the medical and professional assocations and disease advocacy groups.

The physicians, other health care professionals, and not-for-profit organizational leaders involved may rationalize these activities as consistent with their mission and professionalism, but such rationalizations are at best self-delusion.

No one knows how representative the Lexapro marketing plans are of the marketing of other drugs, devices and health care services. The only way to find out would be to force many other corporations' marketing plans into public view. The Lexapro documents suggest that society would benefit if many more marketing plans were made public, but that such publication might generate a wave of revulsion about how deceptive marketing of health care goods and services has become, and the extent that health care professionals have betrayed their professional ideals, and academic medical institutions, professional and medical societies, and patient advocacy groups have betrayed their missions.

I submit that we will not truly reform health care without making the marketing of health care goods and services honest, getting health care professionals to give up their financial relationships with health care corporations to reclaim their professionalism, and getting academic medical institutions, professional and medical societies, and patient advocacy groups to give up their financial relationships with health care corporations to reclaim their missions.

See additional comments on how the marketing plan was meant to promote CME by Dr Daniel Carlat on the Carlat Psychiatry Blog.

High Costs and Poor Outcomes - A Lesson About the Suppression of Research

Last week, blogger Alison Bass was one of the few to comment on a new article on the risks of antidepressant medications published electronically last week by the British Medical Journal.[1] The article adds to the clinical evidence about these specific drugs. However, it also should remind us about why our health care costs seem to continue to inflate disproportionate to any improvements in health care outcomes.

Background - Why Don't Doctors Diagnose and Treat Depression According to the Evidence?

Here is the background. Since the early 1990s, based on numerous published clinical trials that showed that the newer (mainly SSRI, that is, selective serotonin reuptake inhibitor) antidepressants seemed to be quite safe and effective, authoritative guidelines have urged physicians to screen patients for depression, and have a low threshold for beginning treatment, particularly with these then new drugs. (For example, see the 1993 AHCPR guidelines,[2] and the Ambulatory Care Quality Alliance Recommended Starter Set[3].) Such guidelines targeted primary care physicians as well as psychiatrists. As a primary care physician, I often felt guilty that I didn't screen all patients, including those who did not outwardly appear depressed, for depression, and was reluctant to be the primary prescriber of antidepressants for depressed patients. I was not alone. There was evidence that many primary care physicians failed to follow these apparently evidence-based guidelines.

Suspicions that SSRIs Were Not as Effective or Safe as Thought

However, in 2004, internal documents from one drug company (Glaxo Smith Kline), published by the Canadian Medical Association Journal, suggested that the company had suppressed a study which failed to show its SSRI was effective for adolescent patients.[4] Also, then New York state attorney general Elliot Spitzer sued the company alleging it had suppressed such data.[5] Considerable anecdotal information thereafter came to light suggesting that drug companies may have suppressed data from clinical research (that is, studies done on patients, not in the laboratory) which tended to show the drugs were less effective, or more dangerous than previously thought. Many began to suspect that the drugs increased the risk of suicidal thoughts or behavior, even though one would think that if they were effective in treating depression, they should decrease such risks.

How Does Uncovering Suppressed Trials Change the Evidence About the Benefits and Harms of SSRIs?

Then, an analysis of trials of SSRIs given to adolescent patients, which included unpublished as well as published trials, did show they increased the risk of suicide.[6] But what about adult patients?

In 2008, Erick Turner and colleagues published a meta-analysis that included numerous trials of SSRIs and other antidepressants given to adult patients that had never been published, but whose results had been submitted to the US Food and Drug Administration (FDA).[7] When those trials were included in the analysis, the drugs seemed less effective than was previously believed.

The new meta-analyis by Stone et al[1] suggested how adding evidence from the suppressed trials improves our knowledge about the possible harms of these drugs. Stone et al specifically addressed whether the drugs increased the risk of suicidal thoughts or actions for adult patients. It was based on data from 372 trials that included 99,231 patients. "Most of the studies were unpublished; those that had been published in some form seldom contained information concerning suicidality in the publication." Its main result was that SSRI drugs approximately doubled the risk of suicidal behavior for patients less than 25 years old, but decreased the risk for older patients.

Suppressing Trials Distorted the Evidence-Base

Thus, the combined results of several meta-analyses suggests that SSRIs are neither as safe nor as effective for adult patients as they were previously touted to be. The drugs used to appear to be more safe and effective because the drug companies that sponsored (paid for) clinical trials of their own products suppressed the publication of studies whose results did not show their products in a favorable light. The guidelines once thought to be so authoritative are probably much to enthusiastic about the use of drugs to treat depression. It may be that those physicians who resisted pressure from these guidelines and other efforts by would-be quality improvers to diagnose more depressed patients and treat them with SSRIs weren't so dumb.

Health Policy Implications

The recently published meta-analysis of the risk of suicidal ideation or behavior due to SSRIs for adult patients[1] adds not just to the clinical evidence about the treatment of depression, but to the evidence that suppression of research by those with vested interests in its results may not be rare. There is no reason to think that suppression of research about other medical interventions used for other clinical problems does not occur just as often.

This underscores why we as a society should not have handed over control of how new drugs and devices are evaluated to the companies that stand to profit by selling them. The apparent benefits and safety of most drugs and devices brought to the market in recent years may have been exaggerated by the ability of the corporations who sponsored the research to suppress those studies that were least favorable to their products. Thus, it is very likely that doctors have over prescribed newer (and more expensive) drugs and devices based on erroneous beliefs about their safety and effectiveness, beliefs supported by an evidence base which was distorted by suppression of studies whose results did not show the results their commercial sponsors wanted to see. Furthermore, we have likely collectively paid too much for these drugs and devices based on these same erroneous beliefs. Thus, the distortion of the clinical evidence base caused by putting the makers of medical products in charge of their evaluation has likely led to increased costs and worse outcomes for patients.

Health care reformers serious about wanting to improve patient outcomes and reduce costs ought to be at the forefront of a movement to make clinical research more honest. All clinical trials completed and submitted to government regulators ought to be published in some form, so that the data they contain is available to guide clinical decisions. Going forward, all studies should be published in some form in a timely manner to prevent future research suppression. There ought to be better regulation of research sponsored, and de facto run by companies who have vested interests in the results being favorable to their products. It would not be unreasonable to prevent companies with such vested interests from running or otherwise directly influencing clinical research meant to evaluate their own products.

But here in the States, we are probably too diverted by yelling at each other about whether a "public option" is vital or nefarious to worry about such issues.. Meanwhile, expect our health care to continue to get more expensive, less accessible, and less good for patients.

ADDENDUM (25 August, 2009) - Also, see the comments by Dr Howard Brody on the Hooked: Ethics, Medicine and Pharma Blog

References

1. Stone M, Laughren T, Jones ML et al. Risk of suicidality in clinical trials of antidepressants in adults: analysis of proprietary data submitted to US Food and Drug Administration. Brit Med J 2009; 339: b2880. Link here.
2. Depression Guideline Panel. Depression in Primary Care: Volume 2 - Treatment of Major Depression. Rockville, MD: US Department of Health and Human Services, Public Health Service, Agency for Health Care Policy and Research, 1993. Link here.
3. Ambulatory Care Quality Alliance. Recommended Starter Set: Clinical Performance Measures for Ambulatory Care. http://www.ahrq.gov/qual/aqastart.htm
4. Kondro W. Drug company experts advised staff to withhold data about SSRI use in children. Can Med Assoc J 2004; 170: 783. Link here.
5. Wadman M. Sptizer sues drug giant for deceiving doctors. Nature 2004; 429: 589. Link here.
6. Hammad T, Laughren T, Racoosin JA. Suicidality in pediatric patients treated with antidepressant drugs. Arch Gen Psychiatry 2006;63:332-9. Link here.
7. Turner EH, Matthews AM, Linardatos E et al. Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med 2008; 358: 252-260. Link here.

JAMA Editors Try Attacking the Messenger

Early this month, yet another story surfaced about allegations of undisclosed conflicts of interest affecting the author of a highly publicized clinical trial. This story has now taken such an odd twist that it seems worth discussing in some detail. Let me first try to present some relevant facts, derived from published articles, mainly peer-reviewed, which do not seem to be in dispute, in chronological order.

2005 - In a multi-author review article on mood disorders in the medically ill, Dr Robert G Robinson, of the University of Iowa, disclosed that he served on the speakers bureau of Forest Laboratories.(1) In an article in Stroke: Clinical Updates, he made a similar disclosure.(2)

28 May, 2008 - Dr Robinson was the first author of an article published in JAMA that described a randomized controlled trial comparing placebo, problem-solving (cognitive talking) therapy, and escitalopram (Lexapro, Forest Laboratories) in the prevention of depression in patients who have had strokes.(3) Patients who received placebo had a higher rate of depression (11 major, 2 minor cases, 22.4%) than patients who received escitalopram (3, 2, 8.5%) or who received problem-solving therapy (5,2, 11.9%). At the end of the article, Dr Robinson disclosed, "over the past 5 years, Dr Robinson reports serving as a consultant to the former Hamilton Pharmaceutical Company and Avanir Pharmaceutical Company," but made no disclosure about any financial relationship with Forest. In a news article published the day before, Dr Robinson was quoted as saying, "I think every stroke patient who can tolerate an antidepressant should be given one to prevent depression," but did not advocate the use of problem solving therapy.(4)

15 October, 2008 - In a letter to JAMA, Lacasse and Leo asked whether Robinson et al had done an analysis directly comparing problem-solving therapy and escitalopram, and noted that the reported incidence data for stroke in the two treatment groups suggest that the difference in treatment results "does not appear to be either clinically or statistically significant."(5) Robinson et al responded that there was no statistically significant differences between the groups.(6)

5 March, 2009 - In a rapid response section in the British Medical Journal, Leo and Lacasse raised their concerns about the interpretation of the results of the 2008 trial, and also stated that "during a subsequent internet search we were surprised to learn that four years previously the lead author had been listed on the speaker's bureau for Forest. The omission, however innocent or mistaken, is disturbing; neither the JAMA article nor subsequent media accounts noted that the lead author had served on the speaker's bureau for the manufacturer of Lexapro."(7) Their citation was to our reference 2.

11 March, 2009 - In a letter to JAMA, Robinson and Arndt reported that the financial disclosure for their 2008 article was incomplete, "resulting from erroneous recollection of the appropriate dates for speaking presentations sponsored by pharmaceutical companies...."(8) Dr Robinson disclosed receiving honoraria for two presentations in 2004, and serving on the Forest Laboratories speakers bureau "in 2004 and perhaps 2005." No editorial comment accompanied this letter.

So far, this seems to be a familiar story about an author who seemed to be excessively enthusiastic about a product of the company with whom he had had a financial relationship, but reluctant to disclose this relationship. But wait,

20 March, 2009 - A rather extraordinary editorial was published electronically in JAMA on 20 March, 2009.(9) Let me review its main points, section by section.

After acknowledging the attention conflicts of interest now receive, and briefly describing the 2008 study and the subsequent letter by Lacasse and Leo, the editorial stated that JAMA editors had received a communication from Leo on or after 16 October, 2008 which "indicated he had evidence that Robinson had not reported in his article that he had served on the speakers bureaus for pharmaceutical companies."

The editorial then devoted several paragraphs explaining the "due diligence" JAMA editors used to investigate this "allegation." They noted the "sensitive nature of these investigations," which required them to conduct them "confidentially," and again insisted "these investigations into undisclosed conflicts of interest are time-intensive and require careful attention." Thus, from the time Leo sent his "allegations," (apparently 16 October, 2008), it took until 31 January, 2009 to get a letter from Robinson et al acknowledging his undisclosed conflicts, and until 11 March, 2009 to publish it, a total of five months.

However, the concerns with confidentiality, and the repeated emphasis on the need for unusually painstaking investigation seemed disconnected to the particular case. It is clear that it may take quite a bit of time and effort to investigate some allegations of undisclosed conflicts of interest, especially when the allegations are vague, but the alleged conflicts are severe. However, in this case, it should have taken trivial effort to find Dr Robinson's previous, published disclosures. (It took me about 5 minutes of internet searching to find the two 2005 articles.) Once (easily) discovered, the disclosures in the 2005 articles starkly contrast with the disclosures, or lack thereof in the 2008 article. Furthermore, since these disclosures were already in the public domain, there should have been no concerns with confidentiality.

Nonetheless, the editors then asserted:

While the confidential investigation of unreported conflicts of interest is under way, we consider involvement of third parties— such as Leo had done by his posting on the BMJ site and by contacting the media—to be a serious ethical breach of confidentiality that not only potentially damages our ability to complete a fair and thorough investigation (of the specific issue that Leo had brought to our attention), but also potentially damages JAMA’s reputation by the insinuation that we would fail to do so.

This makes no sense. There was no need for confidentiality, because the allegations were not based on whistle-blowing or confidential information. They were based on published articles, articles which clearly disclosed Dr Robinson's financial ties to Forest Laboratories. Anyone who bothered to do a simple Google search could have found that Dr Robinson's disclosure in his 2008 JAMA article did not agree with his disclosure in several 2005 articles. Lacasse and Leo used no confidential information whatsoever in making their allegations. Actually, they were not making "allegations," but simply pointing out that an author had published statements which were inconsistent. Since Lacasse and Leo were simply pointing discrepancies among published articles, how could they have breached "confidentiality," ethically or not?

Second, there was no need for a complex or prolonged investigation. The discrepancies were apparent as soon as one viewed the published articles side by side.

The third point was a non sequitur, and a classic example of blaming the messenger for the message. JAMA had published an article. Other journals published other articles. Publishing these articles put them in the public domain. If comparison of these articles in the public domain were to reflect badly on on of their publishers, it would not be the fault of making the comparison.

Regardless of the illogical nature of their concerns with non-existent confidentiality, assertions that it takes a long, complex investigation to demonstrate the obvious, and their aspersions on the messenger for the message, the JAMA editors then revealed that:

A telephone conversation intended to inform Leo that his actions were inappropriate transformed into an argumentative discussion, as Leo continued to refuse to acknowledge any problems with his actions, even after he was informed that the investigation was completed and was advised to read the upcoming March 11 issue of JAMA (where the letter of explanation and apology from Robinson and the formal correction were in press).

Furthermore, the editors proposed punishing Leo, first by effectively banning him from publishing in the pages of JAMA,

Leo also was informed that, if his actions represented his apparent lack of confidence in and regard for JAMA, he certainly should not plan to submit future
manuscripts or letters for publication.

Then, by complaining to his supervisor

However, since Leo apparently did not appreciate the serious implications of his actions, despite our attempts to explain, we felt an obligation to notify the dean of his institution about our concerns of how Leo’s actions were potentially damaging to JAMA’s reputation. We sought the dean’s assistance in resolving this issue involving a member of the faculty of his institution, to assure there would be no need to publicly identify that faculty member. No dean wants his or her institution implicated in a publication reflecting improper behavior by a faculty member.

Finally, they seemed bitterly offended that the British Medical Journal would publish anything that disagreed with anything printed in JAMA:

In addition, we were dismayed that BMJ would post the article by Leo with the allegations against Robinson and the negative insinuations about JAMA, without at least contacting JAMA to verify the veracity of the report.

Again, rather than raising unsubstantiated allegations, Leo and Lacasse's letter to the BMJ simply showed the discrepancy between the disclosures in Robinson and colleagues' 2008 JAMA article and those in one of his 2005 articles. Both articles are publicly available. What useful information about the letter above and beyond that provided by viewing the discrepant articles could the JAMA editors have provided?

While the JAMA editors castigated Leo for unethical behavior, and sought to punish him for it, they said nothing negative about Robinson. Yet it was Dr Robinson who was supposed to disclose his conflicts to JAMA, but who managed to forget conflicts that he had disclosed before. Why would would it be more unethical to point out publicly available evidence about failure to disclose conflicts of interest than to conceal these conflicts in the first place?

This story is saddening. JAMA has published many important and useful articles on conflicts of interest, and other matters relevant to Health Care Renewal. We have probably favorably cited more editorials by JAMA on these topics than those in any other journal. Yet now the journal's leadership seems to have somehow lost their way. Instead of trying to constructively respond to criticism, they now seem intent on punishing the critics. I hope they find their compass soon, before an important medical institution really is irreparably damaged.


Note: another detailed analysis of this case is here in the Hooked: Ethics, Medicine and Pharma blog. Here is Professor Leo's response to the JAMA ediorial.

ADDENDUM (24 March, 2009) - see also the extensive analysis of this case on the Respectful Insolence blog, and the comments here on the Effect Measure blog, here on Gooznews, and here on KevinMD.

References

1. Evans DL, Charney DS, Lewis L et al. Mood disorders in the medically ill: scientific review and recommendations. Biol Psychiatry 2005; 58: 175-189. [Link here.]
2. Robinson RG, Zorowitz RD. Pseudobulbar affect and stroke. Stroke: Clinical Updates 2005; XV (Jan-Feb). [Link here.]
3. Robinson RG, Jorge RE, Moser DJ et al. Escitalopram and problem-solving therapy for prevention of poststroke depression: a randomized controlled trial. JAMA 2008; 299: 2391-2400. [Link here.]
4. Elias M. Study: antidpressants help stroke victims. USA Today, May 27, 2008.
5. Lacasse J, Leo J. Escitalopram, problem-solving therapy, and poststroke depression. JAMA 2008; 300: 1757-1758.
6. Robinson RG, Jorge RE, Arndt S. Escitalopram, problem-solving therapy, and poststroke depression. JAMA 2008; 300: 1758-1759.
7. Leo J, Lacasse J. Clinical trials of therapy versus medication: even in a tie, medication wins. Brit Med J, 2009: 338: b464. [Link here.]
8. Robinson RG, Arnd S. Incomplete financial disclosure in a study of escitalopram and problem-solving therapy for prevention of poststroke depression. JAMA 2009; 301: 1023-1024. [Link here.]
9. DeAngelis CD, Fontanarosa PB. Conflicts over conflicts of interest. JAMA 2009; 301: [Link here.]

BLOGSCAN - Annals of Stealth Marketing

On the Reidbord's Reflections blog, Dr Steven Reidbord recounted how he was invited to speak at a big and glitzy event, part of the national tour for "GOAL" (go on and live). He was first enthused, but when he found that the event was sponsored by a pharmaceutical company, and organized by a public relations firm, he realized he had been "duped" and backed out, to the great annoyance of his chairman, who saw the event as worthwhile "public outreach." Add this to the annals of stealth marketing. Hat tip to the Carlat Psychiatry Blog.

Thoughts on Charles Nemeroff's Not So Excellent Adventure

The saga of Dr Charles Nemeroff was most recently discussed on Health Care Renewal here. We had first posted about his failure to disclose relevant conflicts of interest relating an article he wrote for a journal he also edited here. Other posts about Nemeroff's questionable behavior are here, here. Nemeroff has also starred in numerous posts on the Clinical Psychology and Psychiatry Blog, amongst others.

Now the tale of how Nemeroff raked in hundreds of thousands of dollars as a paid speaker on behalf of drug marketers, and denied these earnings while he ran a US government funded project meant to evaluate some of the products of his commercial sponsors, has splashed across major newspapers. There has been a lot of good discussion about the case in the blogsphere. (We await, of course, discussion in scholarly medical and health care journals, if it ever appears.)

I did think it was worth weighing in again just to underline some important points about this affair.

It has become common for academic leaders to earn more hawking commercial products than from teaching, research, or patient care. The database unearthed by Senator Grassley just of the speaking fees earned by Dr Nemeroff giving speeches promoting GlaxoSmithKline products is breath-taking. As noted by the anonymous blogger on the Clinical Psychology and Psychiatry Blog, Nemeroff was earning on average more than $20K a month from GlaxoSmithKline alone. That was from only one company. As that blogger also noted, Nemeroff at one point was a consultant for 18 companies, if I counted right, and gave paid talks for 4. Here is the actual relevant part of the disclosure statement.

[Nemeroff] has been a consultant to Abbott Laboratories, Acadia Pharmaceuticals, Bristol-Myers Squibb, Corcept Therapeutics, Cypress Bioscience, Cyberonics, Eli Lilly and Co, Entrepreneur’s Fund, Forest Laboratories, Inc, GlaxoSmithKline, i3 DLN, Janssen Pharmaceutica, Lundbeck, Otsuka America Pharmaceutical, Inc, Pfizer Pharmaceuticals, Quintiles Transnational, UCB Pharma, and Wyeth-Ayerst Laboratories; has been on the speakers bureau for Abbott Laboratories, GlaxoSmithKline, Janssen Pharmaceutica, and Pfizer Pharmaceuticals;

At most medical schools, bringing in money is more important than academic integrity, or anything else. We have previously posted about how one medical school leader explained that faculty who are "taxpayers," that is, bringers of money, are more valued than all others.

The case of Nemeroff shows how big-time taxpayers are now above the law at medical schools. As Dr Howard Brody explained it on the Hooked: Ethics, Medicine and Pharma Blog

[Nemeroff] sent a confidential letter to the dean of the medical school at Emory in May 2000, listing the dozen corporate advisory boards in which he sat. He then ticked off the grants and endowments that those firms had paid to the Department of Psychiatry at Emory, and added, 'Part of the rationale for [the companies'] funding our faculty in such a manner would be my service on those boards.' Translation--you mess with my cozy relationships with these companies, and the industry gravy train to Emory dries up. The threat is only slightly veiled, that should Emory decide to take any serious action against Nemeroff for his unreported conflicts of interest, he could easily jump ship to a more permissive med school, taking a lot of his captive research faculty and all of his industry funding with him.

Lying is an acceptable way to maintain an academic medical leadership position. Senator Grassley's letter to Emory documented this instance:

On several occasions during the life of this grant, it appears that Dr. Nemeroff
failed to report to Emory that he was participating actively on the speaker’s bureau for GSK. For instance, in an email regarding his outside activities dated October 1, 2003, Dr. Nemeroff wrote:
'…I have to dig up the agreement and send it to you, GSK no standing
contract, I chair their ad board 2-3 times per year and I am paid per board
meeting at a standard rate of $5K per weekend.'
However, and based upon information in our possession, in 2003 GSK
paid Dr. Nemeroff about $119,000 in speaking fees and expenses.

$15,000 does not remotely equal $119,000. Stating he was paid the former when he was paid the latter goes beyond "misspeaking."

Bullying helps too. As Dr Brody put it,

Nemeroff is pleased to bear the nickname 'Boss of Bosses.' He has a reputation for wielding tremendous power in psychiatry, especially taking advantage of his leverage with the big drug firms, and is ruthless in attacking those whom he doesn't like or who threaten him. One such event is described in some detail in HOOKED, the hiring and then subsequent firing of David Healy as head of a psychiatric research institute at the University of Toronto, due to Healy saying bad things about Prozac, whose manufacturer, Eli Lilly, was at the time considering a major grant to Toronto. Healy cheerfully sued Toronto and won, meaning that all the correspondence related to the firing is now in the public domain. In HOOKED I focused on Toronto's spineless behavior, but it is also interesting to note that almost certainly, Nemeroff was in the background pulling all the strings that led to Healy's dismissal.

Not only was this a case of bullying, of course, but it was also a case of the trampling of academic freedom, in this case, apparently to support the continuing relationship between pharmaceutical companies, "key opinion leaders," and medical schools, so profitable to these parties, but so detrimental to the integrity of academic medicine and of the clinical research data base.

The case of Charles Nemeroff's not so excellent adventure illustrates the sorry current state of the leadership of academic medicine. As Dr Brody put it,

In today's world, a medical school like Emory looks at all the pluses and minuses of having a guy like Nemeroff as a powerful chair, and decides that the pluses outweigh the minuses. His publication record is stellar (mostly ghostwritten of course), he brings in huge research grants, and people in his specialty all over the world want to kiss the hem of his garments. What has to change, in the regulation and the culture of the academic medical center, so that it becomes a no-brainer that having a guy like this on your faculty is a net loser?

Without such change, the public will fairly start to judge academic medicine's integrity as only slightly better than that of the garbage hauling industry, and that may be too insulting to the garbage hauling industry. It is sad beyond words that the current leadership of these once proud institutions have lead them to this pass, just for a few dollars more.

ADDENDUM (8 October, 2008) - See also comments by Dr Daniel Carlat on the Carlat Psychiatry blog here and here, and by Alison Bass on the Alison Bass blog here.

BLOGSCAN - Duplicate Publication about an SSRI

On the Clinical Psychology and Psychiatry Blog, the anonymous blogger described an interesting example of apparent dual publication on a selective serotonin reuptake inhibitor (SSRI) anti-depressant drug, duloxetine (Cymbalta, manufactured by Eli Lilly). Important aspects of the story include how both articles draw conclusions that go well beyond the (same) data in their enthusiasm for the drug in question, and how the second article incorporated an author not found in the first, an author who also happened to be the co-editor in chief of the journal in which the second article appeared, and the leader of an important medical society. This author has been the subject of previous posts (like this one), on Health Care Renewal.

Attacking "Side Effects" with Logical Fallacies, Version 2

Side Effects by Alison Bass continues to generate controversy. As described by a reviewer in the New England Journal of Medicine,(1) the book

used the case of Paxil to expose the unsavory and self-serving relationships among members of the pharmaceutical industry, psychiatrists, and members of the FDA.

The reviewer concluded,

Bass's riveting and well-researched account of these disturbing ties should be widely read by members of the medical profession, many of whom continue to believe, despite all evidence to the contrary, that they are immune to the influence of drug companies.

We previously posted about how the book inspired one reviewer to unleash a series of logical fallacies in an apparent attempt to discredit Bass and Side Effects. Now a review by AJ Gelenberg has appeared in Health Affairs, the respected scholarly journal of health care and health policy research, which also employed its share of logical fallacies.(2)

Ad Hominem

One of Galenberg's main arguments was that Side Effects treated as black and white a story that is really about shades of gray. Furthermore, he alleged that "its hyperbole and 'villains-versus-heroes' style do no justice to a pressing public health issue." From there, he tried to discredit two of its "heroes."

One hero, a psychiatrist, admitted to a series of boundary violations with a patient that caused me to blanch. But no matter; he’s a good guy, and the author declares his intent to be pure. Another of her bigger-than-life heroes is Eliot Spitzer. The prepublication version of Bass’s book I read was written before Spitzer’s resignation as New York’s governor. I wonder if her heroes get redacted when their feet turn out to be made of clay.

Bass actually used a number of pages to address the allegations of malpractice made against the "hero" psychiatrist. But the psychiatrist's role in the book was the early raising of the hypothesis that selective serotonin uptake inhibitor (SSRI) anti-depressants may lead to suicidal ideation. Whether that hypothesis is true, and whether or not other people sought to manipulate or suppress evidence relevant to it were the real questions Side Effects addressed. The psychiatrist's conduct with a particular patient are irrelevant to their answers.

Similarly, although Elliot Spitzer did oversee the lawsuit against GlaxoSmithKline, the manufacturer of Paxil, his conduct years later that lead to his resignation as Governor of New York had nothing to do with this lawsuit or the issues it raised. Furthermore, Side Effects focused on how his staff investigated the relevant facts and pursued the lawsuit. Spitzer's role on the ground was minimal.

Thus, a single paragraph in Galentberg's Health Affairs review employed the ad hominem fallacy twice. This fallacy may be defined one "in which a claim or argument is rejected on the basis of some irrelevant fact about the author of or the person presenting the claim or argument. Typically, this fallacy involves two steps. First, an attack against the character of person making the claim, her circumstances, or her actions is made (or the character, circumstances, or actions of the person reporting the claim). Second, this attack is taken to be evidence against the claim or argument the person in question is making (or presenting)."

Guilt By Association

Later, Galenberg wrote,

I worry that Bass’s book can do harm with its broad-brush smear.

So,

Scientologists could find support for their cruel and misinformed agenda of laying waste an area of medical care that is tragically stigmatized.

Of course, anything written by anyone could be misinterpreted by someone else, or used in conjunction with yet more logical fallacies to provide erroneous support of an unrelated position. I submit that implying that Scientologists, not a group discussed or addressed in Side Effects, would be so involved was an attempt at asserting guilt by assocation. This may be defined as one "in which a person rejects a claim simply because it is pointed out that people she dislikes accept the claim."

Slippery Slope

Galenberg's worry noted above lead to a dire warning,

Worst of all, patients and their families could turn away from needed attention and further compound the neglect of psychiatric disorders.

Galenberg did not explain how Bass' book, devoted as it was to the case of the marketing and research of single drug, could undermine all of psychiatric care. Thus, this appears to be a slippery slope, defined as a fallacy "in which a person asserts that some event must inevitably follow from another without any argument for the inevitability of the event in question."

Conclusion

What Galenberg's review did not do is meaningfully address the facts presented in the book, and Bass' main interpretation of them. Side Effects argued that clinical research evidence specifically about the safety of SSRIs for children and adolescents, not adults was suppressed and manipulated. In fact, there is now considerable evidence that SSRIs may have risks for children and adolescents that were not appreciated until recently, and may not be very efficacious for younger patients. For example, the systematic review by Whittington et al included published and unpublished data from randomized controlled trials of SSRIs in children and adolescents.(3) Its abstact concluded "published data suggest a favourable risk-benefit profile for some SSRIs; however addition of unpublished data indicates that risks could outweigh benefits of these drugs (except fluoxetine) to treat depression in children and young people. The systematic review of published trials by Fergusson et al concluded there was "an association between suicide attempts and the use of SSRIs."(4)

Galenberg concluded on the regretful note,

I wish that Alison Bass had been more credible and responsible in presenting this dilemma and the underlying facts.

However, Galenberg never provided evidence or clear arguments that disputed the "underlying facts." Rather, this review, like the one by Herrmann, seemed at best to show the reviewer's distaste for Side Effects, rather than the source of that emotion.

Galenberg did allude in his review to his background,

I have consulted to the pharmaceutical industry, given lectures they have funded, and taken educational and research funds from them through my university.

He did not specify the companies for which he worked, the topic or purpose of the consultations, the nature of the lectures he gave, or the research he pursued. However, a quick search revealed that one of his colleagues in such research pursuits was Dr Martin Keller, one of the "villains," to use Galenberg's term, in Bass' book. (For example, see this article by Keller et al from 2007.[5]) The disclosures in that study included,

Dr. Gelenberg is a consultant to Eli Lilly, Pfizer, Best Practice, AstraZeneca, Wyeth, Cyberonics, Novartis, Forest, and GlaxoSmithKline; has stock options with Vela Pharmaceuticals; and has received research grants (to the University of Arizona) from Novartis.

Thus, this seems to be yet another in our series of cases of confused defenses of financial entanglements among industry and academics. Such confusion seems likely to appear in arguments made by people who themselves may have relevant conflicts of interest. As Joe Collier said, " people who have conflicts of interest often find giving clear advice (or opinions) particularly difficult."(6)

I would submit that Dr Gelenberg's review perhaps unintentionally supports the argument that physicians' and academics' financial entanglements with organizations with vested interests in promoting products, services, or ideologies should not merely be disclosed and managed, but should be materially reduced, if not eliminated.


References

1. Friedman RA. Side effects: a prosecutor, a whistleblower, and a bestselling antidepressant on trial. N Engl J Med 2008; 358: 2852. Link here.

2. Gelenberg AJ. Warning: Side Effects may include distorted vision. Health Aff 2008; 27: 1193-4. Link here.

3. Whittington CJ, Kendall T, Fonagy P, Cottrell D, Cotgrove A, Boddington E. Selective serotonin reuptake inhibitors in childhood depression: systematic review of published version unpublished data. Lancet 2004; 363: 1341-1345.

4. Kirsch I, Deacon BJ, Medina-Huedo TB, Scoboria A, Moore TJ, Johnson BT. Initial severity and antidepressant benefits: meta-analysis of data submitted to the Food and Drug Administration. PLoS Med 5(2): e45 . Link here.

5. Keller MB, Trivedi MH, Thase ME, Shelton RC, Kornstein SG, Nemeroff CB, Friedman ES, Gelenberg AJ et al. The prevention of recurrent episodes of depression with venlafaxine for two years (PREVENT) study: outcomes from the 2-year and combined maintenance phase. J Clin Psychiatr 2007; 68: 1246-1256. Link here.

6. Collier J. The price of independence. Br Med J 2006; 332: 1447-9. Link here.

Attacking "Side Effects" with Logical Fallacies

Concerns about manipulation and suppression of the clinical research literature are gaining more traction. Recently, Alison Bass published Side Effects, a book that explored one of the more vivid cases. As described by a reviewer in the New England Journal of Medicine,(1)

She has used the case of Paxil to expose the unsavory and self-serving relationships among members of the pharmaceutical industry, psychiatrists, and members of the FDA.

The reviewer concluded,

Bass's riveting and well-researched account of these disturbing ties should be widely read by members of the medical profession, many of whom continue to believe, despite all evidence to the contrary, that they are immune to the influence of drug companies.

So it should be no surprise that Side Effects has inspired criticism from those less skeptical about such relationships. The most prominent whack was taken at it by Mark Herrmann's review in the Wall Street Journal, which deployed an interesting variety of logical fallacies to defend the pharmaceutical industry from Ms Bass' critique. Allow me to walk through them.

Appeal to Popularity

The nature of this article becomes apparent early. It started with the premise, not defended with evidence, that selective serotonin reuptake inhibitor (SSRI) anti-depressants are an unqualified social good,

Prozac improved the nation's mood when it came on the market in 1987. Earlier antidepressants had caused many side effects and were potentially lethal in overdose. Prozac appeared to be both a godsend and a blockbuster. It was effective, easy to administer and less likely to be used by depressed patients as a means to commit suicide.

This sets up the implication that any book daring to criticize such a beneficial product ought to get little credence. Thus Herrmann appears to be using an appeal to popularity, a logical fallacy defined as "a claim is accepted as being true simply because most people are favorably inclined towards the claim. More formally, the fact that most people have favorable emotions associated with the claim is substituted in place of actual evidence for the claim."

Straw Man

Next, he noted one of the earliest criticisms of SSRIs.

Soon enough a Harvard professor, Martin Teicher, published reports of six patients who developed suicidal thinking while taking Prozac, and the Church of Scientology campaigned against it.

In 1991, the Food and Drug Administration convened a committee of experts to study the matter. As it turned out, Mr. Teicher's six patients had been deeply depressed for years, medicated with multiple drugs and in several cases had attempted suicide before they first ingested Prozac. The Scientologists rested their arguments on anecdotes, not data. The committee concluded that Prozac did not trigger suicides in adults.

This is the only time that Mr Herrmann dealt with clinical evidence in his review. Note that he started with the most preliminary evidence about harms of SSRIs that were not immediately apparent after they were first marketed. At most, Teicher's anecdotes raised the hypothesis that SSRIs may be more hazardous than they originally appeared. They were hardly definitive evidence, just a clue that more evidence needed to be sought.

Mr Herrmann sought to refute this very preliminary evidence,

A pooled analysis of nearly 100,000 patients, conducted by the FDA in 2006, showed no increased suicidality for adults ages 25 to 64.

That is the last Mr Herrmann has to say about any evidence from clinical research pertaining to the efficacy and safety of SSRIs.

What he left out, however, was that Side Effects argued that clinical research evidence specifically about the safety of SSRIs for children and adolescents, not adults was suppressed and manipulated. In fact, there is now considerable evidence that SSRIs may have risks for children and adolescents that were not appreciated until recently, and may not be very efficacious for younger patients. For example, the systematic review by Whittington et al included published and unpublished data from randomized controlled trials of SSRIs in children and adolescents.(2) Its abstact concluded "published data suggest a favourable risk-benefit profile for some SSRIs; however addition of unpublished data indicates that risks could outweigh benefits of these drugs (except fluoxetine) to treat depression in children and young people. The systematic review of published trials by Fergusson et al concluded there was "an association between suicide attempts and the use of SSRIs."(3)

Thus, Mr Herrmann was setting up a straw man, defined thus, "the Straw Man fallacy is committed when a person simply ignores a person's actual position and substitutes a distorted, exaggerated or misrepresented version of that position. "

(Parenthetically, there is also controversy about the efficacy of these drugs in the treatment of adults, although this was not the focus of Ms Bass' book. See for example the meta-analysis by Kirsch et al using published and unpublished data which suggested that the drugs improve depression only slightly compared to placebo, and then only for the most depressed patients.[4])

Guilt by Association

Additionally, by grouping Dr Teicher's observations with the Scientologists' objections, Mr Herrmann seemed to be deploying guilt by association, defined as "a fallacy in which a person rejects a claim simply because it is pointed out that people she dislikes accept the claim."

Appeal to Ridicule

Ms Bass used Dr Teicher's early observations of patients given Prozac only to set the stage for her main argument. Her crucial points were that people working on behalf of GlaxoSmithKline and its predecessors suppressed and manipulated data from clinical research on Paxil (paroxetine) to make the drug appear more efficacious and less hazardous, and thus sell more of it. As my brief summary of recent published systematic reviews above implies, when some of the suppressed studies were unearthed and their data considered in addition to the published studies, SSRIs appear less efficacious and more hazardous than was previously thought.

Mr Herrmann then sought to trivialize the importance of the data suppressed as merely "inconvenient information," and to minimize GlaxoSmithKline's efforts to suppress it,

The suppressed data that Bass was addressing, particularly about rates of suicidality in children and adolescents taking Paxil, was considerably more than "inconvenient." The use of this term appears to be little more than sarcasm. Thus, Herrmann seemed to be making an appeal to ridicule, defined as "the Appeal to Ridicule is a fallacy in which ridicule or mockery is substituted for evidence in an "argument."

Red Herring

Then he asserted,

In the event, GlaxoSmithKline settled out of court and, as soon as the matter of transparency was raised, published online all of the studies of Paxil that had been submitted to the FDA, including those that showed no significant difference between the effects of Paxil and a placebo.

The implication that GSK jumped to respond immediately after the question of "transparency" was raised does not fit the facts. The GSK registry noted by Herrmann's review contains unpublished trials of paroxetine dating back at least to 1988. Questions about suicide risk due to SSRIs, specifically Prozac, were raised by Teicher in 1990. GSK published the registry as part of the settlement of a lawsuit, and then only reluctantly. Bass documented how GSK attorneys opposed a fully transparent registry as part of a settlement in 2004. Herrmann's facile language provided no facts to counter the assertion that GSK did all it possibly could to delay release of data from clinical trials that was unfavorable to its product. His argument thus appears to be a red herring, defined as " a fallacy in which an irrelevant topic is presented in order to divert attention from the original issue."

Another Straw Man

Next, Mr Herrmann then asserted that Bass was hostile to all drug company sponsored research,

For Ms. Bass the judgment of researchers, together with their data and claims, are untrustworthy if they have received money from drug companies to finance clinical trials. She is particularly hard on a professor of psychiatry from Brown University who has defended Paxil.

Actually, Bass' criticisms were far more focused. She made detailed allegations that the "professor of psychiatry" manipulated data in a particular, key trial of paroxetine in children and adolescents. She charged that after the drug failed to produce significant improvements in the pre-selected main outcome measures, the study investigators post-hoc looked for other measures until they found some by which the drug appeared efficacious. Furthermore, she charged that the investigators dropped trial patients who exhibited suicidal ideation without recording these adverse effects. (See our recent post discussing an article published after Side Effects was written that corroborates these allegations.[5]) Thus, Herrmann's appeared to raise another straw man argument.

More Red Herrings

Mr Herrmann wound up his review with a tin of red herrings. For example, he asked,

And why shouldn't companies seek advice from the best scientific minds and pay them for their efforts?

However, Bass' book discussed relationships among drug companies that went far beyond paying people to give "advice." Although spokespeople for the pharmaceutical industry have claimed that the companies' relationships with physicians and academics amount to nothing more than consulting the best and brightest for guidance, there is evidence that industry recruits "key opinion leaders," (KOLs) a title given by Ms Bass to the "professor" above, to help market the companies' products, and that KOLs early recruitment may be based on their existing sympathy to the companies and their marketing objectives. (See this post and related article by Moynihan.[6])

Herrmann's last point was that Bass

said not a word about how society suffers when the FDA approves new drugs too slowly, depriving patients of life-improving and life-saving medicine

However, the book hardly advocated a general slowdown of drug approval. In fact, as the New England Journal of Medicine reviewer pointed out, her point was more not to slow down drug approval, but to speed up recognition "the FDA knew of the risks all along but procrastinated" in warning of them.

Conclusion

Herrman concluded with the opinion that

"Side Effects" belongs to a genre of investigative journalism that involves talking to plaintiffs, their lawyers and their expert witnesses, taking their stories as gospel and denigrating the opposing view because corporate money (apparently less pure than money from the plaintiffs' side) supposedly has a corrupting effect.

"Side Effects" is lively and well-written, but readers should be warned that they may have an adverse reaction: a deep disquiet that only half the story has been told.

Maybe Mr Herrmann would benefit from the use of a mirror. His supposed review seems mainly an uncritical defense of actions that seemed more related to one pharmaceutical company's pursuit of profits than the promotion of science or the benefit of patients. At the end of his review, the Wall Street Journal noted "Mr Herrmann is a lawyer in Chicago whose firm has defended drug manufacturers in product liability cases." Thus, his review may be likened to a legal brief on behalf of a client whose conduct may be hard to defend.

References

1. Friedman RA. Side effects: a prosecutor, a whistleblower, and a bestselling antidepressant on trial. N Engl J Med 2008; 358: 2852. Link here.

2. Whittington CJ, Kendall T, Fonagy P, Cottrell D, Cotgrove A, Boddington E. Selective serotonin reuptake inhibitors in childhood depression: systematic review of published version unpublished data. Lancet 2004; 363: 1341-1345.

3. Fergusson D, Doucette S, Glass KC, Shapiro S, Healy D, Hebert P, Hutton B. Association between suicide attempts and selective serotonin reuptake inhibitors: systematic review of randomised controlled trials. Brit Med J 2005; 330: 396. Link here.

4. Kirsch I, Deacon BJ, Medina-Huedo TB, Scoboria A, Moore TJ, Johnson BT. Initial severity and antidepressant benefits: meta-analysis of data submitted to the Food and Drug Administration. PLoS Med 5(2): e45 . Link here.

5. Jureidini JN, McHenry LB, Mansfield PR. Clinical trials and drug promotion: selective reporting of study 329. Int J Risk Safety Med 2008; 20: 73-81. Link here.

6. Moynihan R. Key opinion leaders: independent experts or drug representatives in disguise? Brit Med J 2008; 336: 1402-3. Link here.

SSRIs, Stealth Marketing, and Public Radio

We have previously posted about instances of stealth marketing in public television (here and here.) Now Shannon Brownlee and Jeanne Lenzer have published "Stealth Marketers: Are Doctors Shilling for Drug Companies on Public Radio" in Slate. Their take-home message was:

A few weeks ago, devoted listeners of public radio were treated to an episode of the award-winning radio series The Infinite Mind called 'Prozac Nation: Revisited.' The segment featured four prestigious medical experts discussing the controversial link between antidepressants and suicide. In their considered opinions, all four said that worries about the drugs have been overblown.

The radio show, which was broadcast nationwide and paid for in part by the John D. and Catherine T. MacArthur Foundation, had the air of quiet, authoritative credibility. Host Dr. Fred Goodwin, a former director of the National Institute of Mental Health, interviewed three prominent guests, and any radio producer would be hard-pressed to find a more seemingly credible quartet. Credible, that is, except for a crucial detail that was never revealed to listeners: All four of the experts on the show, including Goodwin, have financial ties to the makers of antidepressants. Also unmentioned were the "unrestricted grants" that The Infinite Mind has received from drug makers, including Eli Lilly, the manufacturer of the antidepressant Prozac.

We don't know just how much funding or when the show last received it, since neither Goodwin nor the show's producers responded to repeated requests for interviews.

In addition,

Goodwin is on the board of directors of Center for Medicine in the Public Interest, an industry-funded front, or "Astroturf" group, which receives a majority of its funding from drug companies.

Regarding the financial ties of panelist Peter Pitts

CMPI President Peter Pitts was one of Goodwin's three guests for 'Prozac Nation.' We don't know which companies fund his group because when we asked him, Pitts said, 'I don't want to go into that.'

Pitts has another title that might have been relevant to The Infinite Mind; he is the senior vice president for global health affairs at the PR firm Manning Selvage & Lee, which represents Eli Lilly Inc., GlaxoSmithKline, Pfizer, and more than a dozen other pharmaceutical companies. Yet on the show, Pitts was identified only by his title as 'a former FDA official.'

Regarding Dr Andrew F Leuchter:

a professor of psychiatry at UCLA who has received research money from drug companies including Eli Lilly Inc., Pfizer, and Novartis.

Finally, regarding Dr Nada Stotland:

Nada Stotland, president-elect of the American Psychiatric Association, has served on the speakers' bureaus of GlaxoSmithKline and Pfizer.

Note that neither Dr Goodwin nor the show's producers agreed to be interviewed by Brownlee and Lenzer.

In my humble opinion, it's great that more and more instances of apparent stealth marketing are being exposed in the media.

It may be that the people introduced as distinguished experts on this show who asserted that the suicide risks of antidepressants were overblown were not the least bit influenced by their financial ties to pharmaceutical companies which make antidipressants. Or maybe they were influenced?

But at least they should have revealed these ties to their listeners. Maybe had the show's producers felt compelled to reveal such conflicts of interest, they might have thought about at least adding some experts who did not have such ties.

Finally, as we have said before, physicians and researchers who are in a position to influence how the media discusses medicine and health care should, at a minimum, fully and completely disclose any financial arrangements they have with organizations with vested interests affected by such media discussions.

ADDENDUM (12 May, 2008) - See also this post by Ed Silverman on PharmaLot, and especially the voluminous and vociferous comments.