Clinical Practice Guidelines: Still Conflicted After All These Years

Background: Untrustworthy Clinical Practice Guidelines

Since the 1990s, clinicians have been exhorted to follow clinical practice guidelines (CPGs) to improve their decision-making and patients' outcomes.  When Dr Wally Smith and I started teaching a short course (often at the Society for Medical Decision Making meetings) about changing physician behavior, we naively set out to improve decisions by increasing adherence to such guidelines.  We first thought that physicians' apparent shortcomings in guideline adherence were due to their lack of knowledge of the guidelines and the underlying evidence, and to their human cognitive limitations.

That approach, however, failed to yield ways to improve adherence.  After a while, it occurred to us that there might be other reasons physicians did not follow guidelines, including their lack of trust of the guidelines or the evidence supposedly incorporated within them.  We learned that integrity of  the clinical evidence base has been severely degraded due to the manipulation of clinical studies by those with vested interests, and the outright suppression of trials for which manipulation does not produce the desired results.

Furthermore, as we became less naive, we learned that the integrity of the guideline development process was also suspect, again due to the influence of those with vested interest.  No wonder physicians were suspicious of guidelines, and resisted adhering to them.

In 2011, the prestigious Institute of Medicine released a report on the development of  better standards to produce more trustworthy guidelines (Clinical Practice Guidelines We Can Trust.  Link here.)  We posted about that report here, but noted that it was receiving little other attention, an example of the anechoic effect.  The IOM report cited conflicts of interest (COIs) affecting the guideline development process as a major reason such guidelines might be untrustworthy, and suggested a variety of ways to reduce COIs affecting guidelines.

These included in particular:
- Guideline committee members should disclose their conflicts of interest in detail, and these should appear in the guidelines (Standard 2.1, 2.2)
- The number of conflicted committee members should be minimized, and in no case should be a majority of the committee (2.3, 2.4)
- Funders should have no role in guideline development (2.4)

This week, an original article(1) and editorial(2) in PLoS Medicine discussed how guidelines published since the report was produced addressed conflicts of interest.

Article Summary

Below I summarized the article, but presented results somewhat differently than did the authors.

Design

Campsall et al did a cross-sectional study of clinical practice guidelines that appeared in the US Agency for Healthcare Research and Quality (AHRQ) National Guideline Clearinghouse in 2012 from national or international organizations, excluding guidelines produced by organizations restricted to allied health professionals, guidelines produced by corporations, or guidelines that had no specific recommendations.  They obtained information from the guidelines, the organizations' websites, and surveys sent to the organizations.

Study Population

The study population was 290 guidelines produced by 95 organizations. The response rate for the surveys was 68%.  The organizations were primarily professional societies (67%) or disease advocacy groups (21%).

My comment is that therefore this study assessed guidelines mainly produced by non-profit organizations which ostensibly uphold health care professionals' values and/or primarily support patients' interests.

Industry Funding

63% of organizations reported receiving funds from biomedical companies.  (It was not clear whether the remainder denied receiving such funding, or provided no information about funding.)

My comment is that the majority of such ostensibly "do-gooder" organizations are nonetheless at least somewhat supported by commercial firms that market health care products, presumably mainly drugs or devices.

Conflict of Interest Policies

Please note that the authors provided results to emphasize the positive, for example, they provided the proportion of organizations which provided specific types of information about COIs or had particular policies to reduce COIs.  In most cases, I simply did subtractions to emphasize the negative.  Furthermore, the authors further emphasized the positive by choice of denominator, as noted below.

20% of organizations did not provide any information about conflict of interest policies, and 7% asserted the existence of such policies, but did not disclose them.  A majority, 55%, did not have disclosed policies that specifically dealt with with conflicts of interest affecting guidelines.

My comment is thus that a majority of these do-gooder organizations did not apparently even begin to address, at least in terms of clear, transparent written policy, the recommendations of the IOM report on trustworthy guidelines.

Measures to Improve Guidelines Trustworthiness by Decreasing Influence of Conflicts of Interest

The authors provided the number of organizations with disclosed policies that addressed particular issues of concern to the IOM in their report on trustworthy guidelines, but made the proportions of such organizations seem larger by using as a denominator the number of organizations that disclosed their entire policies, rather than the total number of organizations which produced the guidelines.

Therefore, they found that of the 60 (out of 95 total) organizations that fully disclosed their conflict of interest policies:
98% required committee members to disclose conflicts of interest;
90% required review of conflicts of interest before guideline publication; 
68% required that a majority of guideline committee members be free of conflicts of interest;
92% reported publishing committee members' conflicts of interest within guidelines;
78% did not allow direct industry funding of guideline development
82% did not allow industry participation in selection of committee members
67% did not allow industry to review guidelines prior to publication.

That all sound good, but consider what happens when one uses the total sample of organizations in the denominators.

Therefore, the article also found that of all 95 organizations that produced the guidelines of interest:

38% did not require CPG committee members to disclose conflicts of interest
43% did not require review of committee members' conflicts of interest prior to guidelines development
57% did not require that a majority of guideline committee members be free of conflicts
44% did not report publishing committee members' conflicts of interest within guidelines
48% did not report preventing direct industry funding of guidelines
51% did not report preventing industry participation in selection of committee members
61% did not report preventing industry review of guidelines prior to publication.

My comment is that thus the results suggested that substantial numbers of organizations that produced CPGs did not demonstrate that they were committed to measures that would improve guidelines trustworthiness by reducing the influence of conflicts of interest.

Disclosure of Conflicts of Interest Within Guidelines

Again, it was necessary to recalculate some proportions using the full population of guidelines reviewed as the denominator.

Of the 290 guidelines reviewed:
65% "included disclosure statements regarding direct funding and support"
37% specifically disclosed the absence or presence of direct funding from biomedical companies
However,
49% did not disclose committee members' financial relationships
99% did not disclose relevant financial relationships of the organization (that is, institutional conflicts of interest) that produced the guidelines as a whole

My comment is that substantial numbers of published guidelines were not clearly produced so as to increase their trustworthiness by reducing the influence of conflicts of interest on them. 

Did the Guidelines Follow the Organizations' Stated Policies?


The authors commendably provided information about whether the guidelines produced by organizations that disclosed conflict of interest policies followed those polices.  They found:

16% of organizations "that reported that committee member conflicts of interested were published in guidelines" produced guidelines that did not include committee members' COI disclosure

61% of organizations that "reported the majority of committee members must be free of conflicts of interest" produced guidelines by committees with majorities of conflicted members

6% of organizations that "reported that industry partners were not permitted to directly fund clinical practice guideline development" produced guidelines which disclosed such funding.

My comment is that thus even of the organizations that stated they had policies in place to improve the trustworthiness of the guidelines they produced by reducing the influence of conflicts of interest, they allowed guidelines to be produced that did not follow those policies.

Authors' Conclusion

Financial relationships between organizations that produce clinical practice guidelines and the biomedical industry appear to be common. These relationships are important because they may influence, through guideline usage, the practice of large numbers of healthcare providers. We believe that to effectively manage conflicts of interest, organizations that produce clinical practice guidelines need to develop robust conflict of interest policies that include procedures for managing violations of the policy, make the policies publicly available, and disclose all financial relationships with biomedical companies.

Editorialists' Conclusion

The editorial by Bastian(2) was sharper in tone.  It included an observation that reinforced my confession of naivete above:

With hindsight, I think those of us encouraging better methodology for guideline development in the 1990s took the issue of disclosure of financial interests too much for granted. It seemed so self-evident, it got barely a mention even in national policy on guideline development

She also noted that the Campsall study did not address all the ways conflicts of interest can influence guideline development:

Stelfox and colleagues focus particularly on the organizational conflicts of interest of guideline producers and their policies. They examine the financial interests of the organizations, but not of the individuals employed within those organizations. This same blind spot is evident when it comes to policies about committee members; the financial interests of the organizations that individuals represent tend to be disregarded. Yet these can be substantial, including for patients’ organizations.

She noted that the failure of guidelines to report their developers' conflicts of interest is a continuing problem.  The rate of failure to report found by Campsall et al was

about the same rate that Taylor and Giles found in 2004 and Norris and colleagues found in 2010.

Finally, she emphasized that we are still a long way from having guidelines that health care professionals and patients can trust:

Guideline processes without adequate financial conflict management have to become unacceptable to a far wider circle. They need to become unacceptable to influential committee members, to the medical journals that lend so many guidelines additional standing and reach, and to the membership of the professional societies that produce them. Until that happens, for guidelines as for clinical research, it’s a case of caveat lector: let the reader beware.

My Summary

So should physicians trust clinical practice guidelines?  At least this article suggests they ought to be very very skeptical of them.  The IOM report meant to improve the trustworthiness of practice guidelines seems to be honored mainly in the breach.  The likelihood that any given guideline was produced so as to reduce the influence of conflicts of interest on it is low.  Even organizations that ostensibly put professional values and patients first in their efforts to develop guidelines seem to often be financially beholden to companies that want to sell drugs and devices.  Physicians and patients ought to be concerned that most new clinical practice guidelines may be as much about marketing commercial products as improving medical practice or patients' outcomes.

The current US presidential race has made it evident that we have a lot of disgruntled citizens, many of whom believe our system is rigged to favor the "establishment."  I suggest that the more we know about clinical practice guidelines, the more it appears that the health care system is rigged to favor certain parts of the "establishment," the big corporations that market drugs and devices, and their paid part-time hands within medical societies and patient advocacy groups who have turned these ostensibly idealistic, do gooder organizations into part-time marketing machines.

The huge and complex web of individual and institutional conflicts of interest that binds much of the health care system, the government, and industry may be good for the insiders, but is stifling improvement in our dysfunctional health care system.  True health care reform would first expose these conflicts, then reduce or better yet, eliminate them, and make health care more about helping patients and less about making money by marketing commercial products.

Musical Interlude

To dispel the darkness a bit, Paul Simon, "Still Crazy After All These Years," 1992 acoustic version:





References

1.  Campsall P et al.  Financial relationships between organizations that produce clinical practice guidelines and the biomedical industry: a cross-sectional study.  PLoS Medicine 2016; DOI;10.1371/journal.pmed.1002029    Link here.

2.  Bastien H. Nondisclosure of financial interest in clinical practice guideline development: an intractable problem? PLoS Medicine 2016;  DOI;10.1371/journal.pmed.1002030  Link here.