Friday, December 01, 2006

Another Post-Modernist Attack on EBM: Because EBM is Not a "Vehicle" for Anyone's Political Program?

I make no secret of my support for evidence-based medicine (EBM). According to the Users’ Guides to the Medical Literature, at first the term described "an attitude of ‘enlightened scepticism’ toward the application of diagnostic, therapeutic, and prognostic technologies in ... day-to-day management of patients." Thus, "the goal is to be aware of the evidence on which one’s practice is based, the soundness of the evidence, and the strength of inference the evidence permits."[1] Another way to look at EBM is as a process, [2] e.g.:
1. translation of uncertainty to an answerable question;
2. systematic retrieval of best evidence available;
3. critical appraisal of evidence for validity, clinical relevance, and applicability;
4. application of evidence in practice; and
5. evaluation of performance.

EBM advocates understand that performing this process, however, is difficult and fraught with barriers and pitfalls. These include the shortage of coherent, consistent scientific evidence; difficulties in applying evidence to the care of individual patients; the need to develop new skills; limited time and resources; and the paucity of evidence that evidence-based medicine "works."[3] For discussions of how to cope with such difficulties and barriers, see these relevant articles.[4-6]

Yet EBM has produced negative reactions, some particularly intense, beyond what its well-described short-comings would seemingly inspire. These seem to be getting more frequent airings in the medical and health care literature, and in the media. See, for example, our previous posts on how EBM has been labelled "microfascist," both in a scholarly article (post here), and in an op-ed in US News and World Report (post here).

The latest example of such negative reactions is in an article recently published in Social Science Medicine.[7] This article by Goldenberg, entitled, "On Evidence and Evidence-Based Medicine: Lessons from the Philosophy of Science," like the one by Holmes et al,[8] took a post-modernist stance, with particular accent on feminism.

Like Holmes and colleagues, Goldenberg wrote in an opaque post-modernist style. She made the expected appeals to authority, but although these included the apparently obligatory bow to Foucault, she emphasized feminist authorities, such as Lorraine Code, Harraway, and Nelson. She quoted assertions by such luminaries as truth, without explanation or justification.

Like Holmes and colleagues, Goldenberg seemed to misunderstand EBM, and made arguments based on misunderstandings. For example, she equated EBM to logical postivism.

However, the apparent obviousness of EBM can and should be challenged on the grounds of how 'evidence' has been problematised in the philosophy of science. In this paper, I argue that evidence-based practices maintain an antiquated understanding of evidence as 'facts' about the world in the assumption that scientific beliefs stand or fall in light of the evidence. This understanding of evidence is explicitly positivist....

She then skewered logical positivism, citing how observation is theory-laden, and how theories are underdetermined. By thus discrediting logical positivism, she discredited EBM.

The formal methods promoted by EBM to replace so called 'traditional' medicine's over-reliance on intuition, habits, and unsystematic clinical experience appear to repeat the misplaced effort to separate science from values.

Note that most EBM proponents, of course, advocate the explicit incorporation of values into the EBM process, but they do distinguish values from science.

Goldenberg also wrote, citing Nelson as an authority,


The basic tenets of EBM, it seems, rest on the unquestioned authority of scientific evidence, a position that is now out of step with current post-postivistic thinking.

That might seem almost funny to an EBM proponent, given the amount of effort spent in the EBM process to rigorously critique evidence. EBM, in my humble opinion, is exactly the opposite of accepting "the unquestioned authority of scientific evidence."

Then, like Holmes and colleagues, Goldenberg accused EBM of badness. Although she did not label it "microfascism," her accusation was that EBM is harmful to women.


Feminist critiques of science are driven by a deep concern that the abstractions made in the names of scientific objectivity, generalisability, and predictability harm women. These tendencies appear to resurface in the practice of EBM.

And again,

Feminist insight reveals that the practices of EBM are marked by potential or actual gender bias, which has led at least one critic to argue that EBM is bad for women's health.

(And Goldenberg never questioned that critic's argument.)

Some of these accusations seem to be based on confusion between the messenger and the message. Goldenberg blamed EBM for insufficient representation of women in controlled clinical trials, "because evidence-based clinical decision-making, policy determinations, and the formulation of clinical guidelines rely upon existing clinical research, the movement reflects any gaps or biases that currently exist in medical research." Again, Goldenberg completely discounted how EBM has critiqued current clinical research for, among other things, not using patient populations whose results might widely generalize to the population at large.

Why are post-modernists taking such whacks at EBM lately?

Goldenberg may have provided a clue. She asserted,

A feminist philosophy of science is explicitly political, as science is recognised to be a vehicle for feminism's emancipatory programme.

I propose that one important reason that EBM may generate opposition is that EBM potentially challenges ideas, programs, and products in which people believe, or in which people have vested interests. If you really believe gizmo X works, or if you make a lot of money selling gizmo X, you may not be enthused about a rigorous review of the evidence that suggests that maybe gizmo X doesn't work so well.

In fact, EBM advocates suggest that EBM teaches people such new skills as "distinguishing evidence from propaganda (advertisement); probability from certainty; data from assertions; rational belief from superstition; and science from folklore."[2] Teaching people to distinguish evidence from propaganda and advertisement could offend the vested interests which increasingly dominate health care.

Goldenberg may have been particularly offended because EBM may not reliably support "a feminist philosophy of science [which] is explicitly political." Although Goldenberg asserted that "science is recognised to be a vehicle for feminism's emancipatory programme," EBM is not a vehicle for anyone's political program, no matter how well-intentioned.

Although some aspect of Goldenberg's political program may be worthwhile, for all I know, (which is little, since she did not explicitly describe this program), I think it is valuable to have a process like EBM which is hard for political advocates, as well as corporate executives and government bureaucrats, to manipulate.

Post-Script

Post-modernism has important influence in the humanities, and to some extent, the social sciences, in many US and other Western countries' academic institutions. Although post-modernists often claim that their work is supported by vast numbers of authorities and multiple weighty tomes, there have been a few well-written efforts to de-bunk the field. See in particular:
Hicks, SRC. Explaining Postmodernism: Skepticism and Socialism from Rousseau to Foucault. Tempe, AZ: Scholargy Publishing, 2004.
Koertge N. A House Built on Sand: Exposing Postmodernist Myths about Science. New York: Oxford University Press, 1998.
Sokal A, Bricmont J. Fashionable Nonsense: Postmodern Intellectuals' Abuse of Science. New York: Picador USA, 1998.
They did a much better job than I could do explaining why post-modernism is, as per Sokal's title, fashionable nonsense. It is too bad it has been slipping into medicine and health care, and now attacks EBM, which may be one of our few bulwarks against the economic vested interests that increasingly dominate medicine and health care.

References

1. Guyatt G, Rennie D, editors. Users’ Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. Chicago: AMA Press, 2002. P. xiv.
2. Dawes M, Summerskill W, Glasziou P, Carabellotta A, Martin J, Hopayian K et al. Sicily statement on evidence-based pratice. BMC Medical Education 2005; 5: 1.
3. Straus SE, McAlister FA. Evidence-based medicine: a commentary on common criticisms. Can Med Assoc J 2000; 163: 837-841.
4. Haynes B, Haines A. Barriers and bridges to evidence based clinical practice. Brit Med J 1998; 17: 273-6.
5. Poses RM. Money and mission? - addressing the barriers to evidence-based medicine. J Gen Intern Med 1999; 14: 262-264.
6. Norman GR. Examining the assumptions of evidence-based medicine. J Eval Clin Practice 1999; 5: 139-147. 7. Goldenberg MJ. On evidence and evidence-based medicine: Lessons from the philosophy of science. Social Science & Medicine 2006; 62(11):2621-2632.
8. Holmes D, Murray SJ, Perron A, Rail G. Deconstructing the evidence-based discourse in health sciences: truth, power and fascism. Int J Evid Based Healthc 2006; 4: 180-186.

6 comments:

Anonymous said...

Roy, you're absolutely right.

However, I think the scientific community has overcomplicated this issue.

Let me articulate how I think this issue will play out in the business and political world:

McGlynn, NEJM and a host of other studies suggest that guidelines get followed about 55% of the time in the U.S.

The philosophy behind Six Sigma (a major quality improvement methodology) is to get your defect rate down around 1-2 per million. This represents 6 standard deviations, hence the name.

55% compliance in healthcare translates to 450,000 defects per million. This is not good. Wennberg's work at Dartmouth points out that most of this variability cannot be explained....and "doctor judgment" or "patient preference" don't cut it as explanations.

Bottom line: arguing against EBM doesn't pass the straight face test. It's just not a credible.

InformaticsMD said...

Another good book that speaks about these issues is Carl Sagan's book "The Demon-Haunted World: Science as a Candle in the Dark."

It has some good chapters on idenitfying and debunking pseudoscience.

Also, don't forget physicist Alan Sokal and the "Social Text Affair", where a faux paper opining that science was just a "social contruct" humorously entitled "Transgressing the Boundaries: Toward a Transformative Hermeneutics of Quantum Gravity" was accepted with open arms for publication in Social Text, a "postmodern" journal on social and cultural phenomena focusing on "gender, sexuality, race, and the environment."

It was a brilliant expose of foolishness of the highest order.

Anonymous said...

If you dig a bit further, you find that Six Sigma splits the problem. First, there is a three sigma (1,000 ppm) on setting the right goals. It explicitly expects that the goals will be imperfect and need both measurement and adjusting. Furthermore, goals include expected variance. For example, the mechanical specification of a part will specify a tolerance. Variances within tolerance are not failures. Then there is another three sigma on the ability of the system to meet those goals.

The medical "guidelines" are goals. They often lack metrics and almost never specify accepted tolerance. Nor is there much explicit analysis of the quality of those goals. EBM is not yet that mature.

So it is hard to say what the "55% compliance" means. It is easy to argue that medicine is far from meeting the six sigma goal, given that it does not even meet the six sigma starting points. But, it is misleading to indicate a degree of failure. It should not be disturbing that six sigma end goal has not been reached. That is the end goal, where you start to celebrate full victory. It takes decades.

CL Psych said...

Wow, that was fun reading. Training docs to think scientifically (as "fascist" as it may sound) has the potential to get docs to examine evidence more closely before selecting treatments. All the arcane theoretical arguments in the world (and apparently the critics have used most of them) do nothing but attempt to obscure the fact that decisions, especially involving patient welfare, should be based on evidence.

If docs were to receive better training in interpreting research in med school, which should be part of EBM reform, then they would be less amenable to manipulation via misleading speakers, journal articles in which optimistic writing glosses over stats showing poor results, all sorts of misleading so-called continuing medical education and the like. Once docs have a firmer grasp on understanding research, they will be in better shape to follow the evidence.

Brilliant post, by the way, as is customary on the site.

InformaticsMD said...

Can't disagree that EBM has its operational flaws. I think, though, that the scientific method's the best we have for maximizing the odds of good outcomes, ideologically speaking. I would not say the scientific method is flawed because there are bullies, criminals, the greedy and others who manage to obtain employment in biomedicine. I do think we need to correct for such issues as Aubrey suggests. OTOH I do think pseudoscience/postmodernism are inherently flawed towards maximizing the odds of good outcomes- with the added bonus of also suffering, in practice, from the Dark Side of human nature.

I believe we also do need to consider "applying evidence to how the scientific method is operationally applied" because there is room for refinement. See, for example, the PLoS article "Why Most Published Research Findings Are False" at
http://medicine.plosjournals.org/perlserv?request=get-document&doi=10.1371%2Fjournal.pmed.0020124 .

The PLoS article is a critique of the scientific method as it is currently applied, written from a scientific perspective, and calls attention to what I'd term 'overconfidence in evidence that is really not good evidence.'

InformaticsMD said...

Typo in previous link (for some reason a trailing period was added, causing it to fail). The correct link to the aforementioned PLoS article is here.