Friday, July 18, 2008

Attacking "Side Effects" with Logical Fallacies

Concerns about manipulation and suppression of the clinical research literature are gaining more traction. Recently, Alison Bass published Side Effects, a book that explored one of the more vivid cases. As described by a reviewer in the New England Journal of Medicine,(1)

She has used the case of Paxil to expose the unsavory and self-serving relationships among members of the pharmaceutical industry, psychiatrists, and members of the FDA.
The reviewer concluded,
Bass's riveting and well-researched account of these disturbing ties should be widely read by members of the medical profession, many of whom continue to believe, despite all evidence to the contrary, that they are immune to the influence of drug companies.

So it should be no surprise that Side Effects has inspired criticism from those less skeptical about such relationships. The most prominent whack was taken at it by Mark Herrmann's review in the Wall Street Journal, which deployed an interesting variety of logical fallacies to defend the pharmaceutical industry from Ms Bass' critique. Allow me to walk through them.

Appeal to Popularity

The nature of this article becomes apparent early. It started with the premise, not defended with evidence, that selective serotonin reuptake inhibitor (SSRI) anti-depressants are an unqualified social good,
Prozac improved the nation's mood when it came on the market in 1987. Earlier antidepressants had caused many side effects and were potentially lethal in overdose. Prozac appeared to be both a godsend and a blockbuster. It was effective, easy to administer and less likely to be used by depressed patients as a means to commit suicide.

This sets up the implication that any book daring to criticize such a beneficial product ought to get little credence. Thus Herrmann appears to be using an appeal to popularity, a logical fallacy defined as "a claim is accepted as being true simply because most people are favorably inclined towards the claim. More formally, the fact that most people have favorable emotions associated with the claim is substituted in place of actual evidence for the claim."

Straw Man

Next, he noted one of the earliest criticisms of SSRIs.

Soon enough a Harvard professor, Martin Teicher, published reports of six patients who developed suicidal thinking while taking Prozac, and the Church of Scientology campaigned against it.

In 1991, the Food and Drug Administration convened a committee of experts to study the matter. As it turned out, Mr. Teicher's six patients had been deeply depressed for years, medicated with multiple drugs and in several cases had attempted suicide before they first ingested Prozac. The Scientologists rested their arguments on anecdotes, not data. The committee concluded that Prozac did not trigger suicides in adults.

This is the only time that Mr Herrmann dealt with clinical evidence in his review. Note that he started with the most preliminary evidence about harms of SSRIs that were not immediately apparent after they were first marketed. At most, Teicher's anecdotes raised the hypothesis that SSRIs may be more hazardous than they originally appeared. They were hardly definitive evidence, just a clue that more evidence needed to be sought.

Mr Herrmann sought to refute this very preliminary evidence,

A pooled analysis of nearly 100,000 patients, conducted by the FDA in 2006, showed no increased suicidality for adults ages 25 to 64.
That is the last Mr Herrmann has to say about any evidence from clinical research pertaining to the efficacy and safety of SSRIs.

What he left out, however, was that Side Effects argued that clinical research evidence specifically about the safety of SSRIs for children and adolescents, not adults was suppressed and manipulated. In fact, there is now considerable evidence that SSRIs may have risks for children and adolescents that were not appreciated until recently, and may not be very efficacious for younger patients. For example, the systematic review by Whittington et al included published and unpublished data from randomized controlled trials of SSRIs in children and adolescents.(2) Its abstact concluded "published data suggest a favourable risk-benefit profile for some SSRIs; however addition of unpublished data indicates that risks could outweigh benefits of these drugs (except fluoxetine) to treat depression in children and young people. The systematic review of published trials by Fergusson et al concluded there was "an association between suicide attempts and the use of SSRIs."(3)

Thus, Mr Herrmann was setting up a straw man, defined thus, "the Straw Man fallacy is committed when a person simply ignores a person's actual position and substitutes a distorted, exaggerated or misrepresented version of that position. "

(Parenthetically, there is also controversy about the efficacy of these drugs in the treatment of adults, although this was not the focus of Ms Bass' book. See for example the meta-analysis by Kirsch et al using published and unpublished data which suggested that the drugs improve depression only slightly compared to placebo, and then only for the most depressed patients.[4])

Guilt by Association

Additionally, by grouping Dr Teicher's observations with the Scientologists' objections, Mr Herrmann seemed to be deploying guilt by association, defined as "a fallacy in which a person rejects a claim simply because it is pointed out that people she dislikes accept the claim."

Appeal to Ridicule

Ms Bass used Dr Teicher's early observations of patients given Prozac only to set the stage for her main argument. Her crucial points were that people working on behalf of GlaxoSmithKline and its predecessors suppressed and manipulated data from clinical research on Paxil (paroxetine) to make the drug appear more efficacious and less hazardous, and thus sell more of it. As my brief summary of recent published systematic reviews above implies, when some of the suppressed studies were unearthed and their data considered in addition to the published studies, SSRIs appear less efficacious and more hazardous than was previously thought.

Mr Herrmann then sought to trivialize the importance of the data suppressed as merely "inconvenient information," and to minimize GlaxoSmithKline's efforts to suppress it,

The suppressed data that Bass was addressing, particularly about rates of suicidality in children and adolescents taking Paxil, was considerably more than "inconvenient." The use of this term appears to be little more than sarcasm. Thus, Herrmann seemed to be making an appeal to ridicule, defined as "the Appeal to Ridicule is a fallacy in which ridicule or mockery is substituted for evidence in an "argument."

Red Herring

Then he asserted,
In the event, GlaxoSmithKline settled out of court and, as soon as the matter of transparency was raised, published online all of the studies of Paxil that had been submitted to the FDA, including those that showed no significant difference between the effects of Paxil and a placebo.
The implication that GSK jumped to respond immediately after the question of "transparency" was raised does not fit the facts. The GSK registry noted by Herrmann's review contains unpublished trials of paroxetine dating back at least to 1988. Questions about suicide risk due to SSRIs, specifically Prozac, were raised by Teicher in 1990. GSK published the registry as part of the settlement of a lawsuit, and then only reluctantly. Bass documented how GSK attorneys opposed a fully transparent registry as part of a settlement in 2004. Herrmann's facile language provided no facts to counter the assertion that GSK did all it possibly could to delay release of data from clinical trials that was unfavorable to its product. His argument thus appears to be a red herring, defined as " a fallacy in which an irrelevant topic is presented in order to divert attention from the original issue."

Another Straw Man

Next, Mr Herrmann then asserted that Bass was hostile to all drug company sponsored research,
For Ms. Bass the judgment of researchers, together with their data and claims, are untrustworthy if they have received money from drug companies to finance clinical trials. She is particularly hard on a professor of psychiatry from Brown University who has defended Paxil.
Actually, Bass' criticisms were far more focused. She made detailed allegations that the "professor of psychiatry" manipulated data in a particular, key trial of paroxetine in children and adolescents. She charged that after the drug failed to produce significant improvements in the pre-selected main outcome measures, the study investigators post-hoc looked for other measures until they found some by which the drug appeared efficacious. Furthermore, she charged that the investigators dropped trial patients who exhibited suicidal ideation without recording these adverse effects. (See our recent post discussing an article published after Side Effects was written that corroborates these allegations.[5]) Thus, Herrmann's appeared to raise another straw man argument.

More Red Herrings

Mr Herrmann wound up his review with a tin of red herrings. For example, he asked,
And why shouldn't companies seek advice from the best scientific minds and pay them for their efforts?

However, Bass' book discussed relationships among drug companies that went far beyond paying people to give "advice." Although spokespeople for the pharmaceutical industry have claimed that the companies' relationships with physicians and academics amount to nothing more than consulting the best and brightest for guidance, there is evidence that industry recruits "key opinion leaders," (KOLs) a title given by Ms Bass to the "professor" above, to help market the companies' products, and that KOLs early recruitment may be based on their existing sympathy to the companies and their marketing objectives. (See this post and related article by Moynihan.[6])

Herrmann's last point was that Bass
said not a word about how society suffers when the FDA approves new drugs too slowly, depriving patients of life-improving and life-saving medicine
However, the book hardly advocated a general slowdown of drug approval. In fact, as the New England Journal of Medicine reviewer pointed out, her point was more not to slow down drug approval, but to speed up recognition "the FDA knew of the risks all along but procrastinated" in warning of them.

Conclusion

Herrman concluded with the opinion that

"Side Effects" belongs to a genre of investigative journalism that involves talking to plaintiffs, their lawyers and their expert witnesses, taking their stories as gospel and denigrating the opposing view because corporate money (apparently less pure than money from the plaintiffs' side) supposedly has a corrupting effect.

"Side Effects" is lively and well-written, but readers should be warned that they may have an adverse reaction: a deep disquiet that only half the story has been told.

Maybe Mr Herrmann would benefit from the use of a mirror. His supposed review seems mainly an uncritical defense of actions that seemed more related to one pharmaceutical company's pursuit of profits than the promotion of science or the benefit of patients. At the end of his review, the Wall Street Journal noted "Mr Herrmann is a lawyer in Chicago whose firm has defended drug manufacturers in product liability cases." Thus, his review may be likened to a legal brief on behalf of a client whose conduct may be hard to defend.

References

1. Friedman RA. Side effects: a prosecutor, a whistleblower, and a bestselling antidepressant on trial. N Engl J Med 2008; 358: 2852. Link
here.

2. Whittington CJ, Kendall T, Fonagy P, Cottrell D, Cotgrove A, Boddington E. Selective serotonin reuptake inhibitors in childhood depression: systematic review of published version unpublished data. Lancet 2004; 363: 1341-1345.

3. Fergusson D, Doucette S, Glass KC, Shapiro S, Healy D, Hebert P, Hutton B. Association between suicide attempts and selective serotonin reuptake inhibitors: systematic review of randomised controlled trials. Brit Med J 2005; 330: 396. Link
here.

4. Kirsch I, Deacon BJ, Medina-Huedo TB, Scoboria A, Moore TJ, Johnson BT. Initial severity and antidepressant benefits: meta-analysis of data submitted to the Food and Drug Administration. PLoS Med 5(2): e45 . Link
here.

5. Jureidini JN, McHenry LB, Mansfield PR. Clinical trials and drug promotion: selective reporting of study 329. Int J Risk Safety Med 2008; 20: 73-81. Link
here.

6. Moynihan R. Key opinion leaders: independent experts or drug representatives in disguise? Brit Med J 2008; 336: 1402-3. Link
here.

6 comments:

Robyn Clothier said...

For background to the Jureidini et al paper on Study 329: http://healthyskepticism.org/documents/PaxilStudy329.php

Anonymous said...

Before Tim Russert. Before the Vytorin debacle. Before the questions of Lipitor marketing. Medpundit in an April 17, 2005, piece (medoundit.blogspot.com) questioned the use of statins on a wide basis. "Perspective: Often, in promoting a new treatment or intervention, doctors will bandy about the term "number needed to treat."

Sydney's conclusion was, that while promoting the positive, they did not disclose the numbers treated needlessly.

This received little note at the time. In a later rant Sydney reflected on a scathing letter from a cardiologist questioning the office practice of not prescribing statins for all patients. The cardiologist had no knowledge of past allergic reactions, or this patients personal desires.

This seems to be part of the pharma marketing program of: Everybody is doing it! or What is the downside?

There appears to be a growing number of drugs that are given, out of hand, to patients without questioning their need or efficacy. Troubling is the growing practice of anti-psychotics being given to children. Time and time again those promoting this therapy have ties to industry that should lead, even the most ardent supporter, to question their use.

We as a country are facing ever growing medical cost. We have the highest medical cost, as a percentage of GDP, of any country in the world. Yet we continue to allow the pharmaceutical industry to set our medical standards, when they have a very visible financial conflict.

Only when the medical community holds drug companies to a higher standard, will we as patients, see the true benefits of new drug discoveries. Only when we as patients question the need for every prescription, and demand generics when possible, will we start to break this juggernaut.

Steve Lucas

Anonymous said...

Typo: The web address for Medpundit is medpundit.blogspot.com.

Steve Lucas

Emily said...

Some treatments are available but for many it is a case of management rather than cure.

MedInformaticsMD said...

An amazing post. Influential people lacking medical experience and ability to craft a rational argument on pressing scientific and social issues abound today, to the detriment of us all.

I wonder if Mr. Herrmann's legal briefs on other matters are as replete with fundamental logical errors. (If not, it raises the issue that the logical errors in the WSJ op ed could have been purposeful.)

I've shared my thoughts with Mr. Hermann as follows:

------------------------------

Dear Mr. Herrmann,

While the WSJ gets wide exposure, healthcare blogs are getting significant exposure among Washington policy makers.

The following post appeared on Healthcare Renewal, a top-25 health care blog, regarding your WSJ op ed "The Plaintiff Was Unhappy":

Attacking Side Effects with Logical Fallacies

The post is at this link and I have attached it below as text as well.

While I had nothing to do with this posting by Roy Poses, it pretty well eviscerates your position. I invite you to learn from it.

In my conversations with congressional research staff, I use pieces such as yours as examples of misleading propaganda written by non-medical amateurs, in advancing their interests, to the detriment of the public health.

Finally, if you want to write on medical topics for a public audience, where the cross-examination will partly take place in the Court of Public Opinion and is now facilitated by the internet, I suggest you attend medical school.

MedInformaticsMD

(former Director of Scientific Information Resources and The Merck Index, Merck Research Labs)

Sara said...

Sorry I didn't find your response to this review sooner. Here's how I put it in a letter to the editor which I also attempted to post on the Opinion Journal forum on line but it was not approved:
Dear Sir,
Mr. Herrmann accuses Alison Bass of being biased and only presenting the “plaintiff’s side” but Hermann himself summarizes the history of Prozac not only in a biased fashion but also in a distorted and inaccurate one, omitting several salient facts, such as the report by Gardiner Harris in the New York Times in August of 2002 that 6 of the 9 advisory committee members who voted in favor of Prozac’s safety in 1991 said that they would have voted against it on the basis of subsequent information. It’s a low blow to bring up the Scientologists – what do they have to do with this argument at all? Alison Bass never mentions them. It just demonstrates that Hermann is operating at a knee jerk level with personal inferences rather than substantive arguments.

He has adopted the pharmaceutical company line that the “underlying disease” is responsible for suicide or suicidal behavior, not an adverse reaction to the drug, even though it has been demonstrated repeatedly that even healthy volunteers, and people who have been prescribed Prozac for off label conditions as benign as losing weight or pulling their hair, have also become intensely suicidal. It’s oh so easy to blame the victim rather than the treatment when we are dealing with the “mentally ill.” Oh yes, there are “anecdotes” galore – just listen to the testimonies of mothers, spouses, children, and friends that were given at the FDA (in 1991, 2004 and 2006) including my own, describing out of character and violent, bizarre behavior in loved ones leading not only to suicide, but also to murder. Why is this honestly given and heartfelt “evidence” dismissed when it is apparent that clinical researchers bend over backwards to conceal adverse reactions, or the reasons for drop out in their company-sponsored trials? There isn’t even a consistent means to measure suicidality; often this reaction has to be self reported even to be included. Reactions to withdrawal from prior drug treatments in the wash-out phase or after the test drug portion of the trial is over are often misrepresented as placebo events in an effort to put the new “active” drug in a better light. No, Keller’s problem is not that he accepted corporate funding but that he lied, concealed and drew conclusions in direct contradiction to the evidence (as incomplete as it was). Doctors then went on to make prescribing decisions in clinical practice on the basis of this deceit that cost children their lives. This is what is at the heart of Alison Bass’s story.

It’s a joke to claim that evidence of increased suicidality on antidepressants has not been consistently demonstrated in adults. No wonder Alison Bass didn’t include this “exculpatory adult data;” whenever anyone tries to portray data leading to this conclusion it only takes a tiny scratch below the service to realize important details have been omitted or distorted, such as the fact, in many trials, that half the participants have been given sedatives or tranquilizers to quiet the agitation and disturbance caused by the antidepressants.

Mr. Herrmann undoubtedly operates in the tradition of Nina Gussack (now leading the defense of Zyprexa), the Eli Lilly hired lawyer in Ms. Bass’s book who unfairly eviscerates Martin Teicher for a kind, but perhaps too personal gesture towards a troubled patient and nearly ruined his career as well as his private life. It is on the basis of this personal detail that he got discredited for his prescient work on Prozac. If he had been taken seriously and allowed to continue his research countless lives might have been saved. But personal gestures of generosity and a concern for the welfare of patients are outside the ken of lawyers who defend pharmaceutical giants long ago corrupted by the dazzle of blockbuster drugs that do nothing but prey on the fears and insecurities of an increasingly dysfunctional society.