Unintended errors with EHR-based result management: a case series, and a special pleading for health IT

As I wrote at "Report of an AMIA special task force on challenges in ethics, safety, best practices, and oversight regarding HIT", articles in the premier journal of Medical Informatics, the Journal of the American Medical Informatics Association (JAMIA) on real and potential downsides of health IT appear to be becoming a trend.

Another article just appeared in JAMIA as the result of a study of healthcare IT related errors: "Unintended errors with EHR-based result management: a case series"; Thomas R Yackel and Peter J Embi; JAMIA 2010 17: 104-107; doi: 10.1197/jamia.M3294.

The article presents a series of health IT-related errors and categorizes them systematically, and thus adds to our knowledge on the issue of cybernetic clinical test results management. It also makes recommendations for increased vigilance and remediation.

The abstract is below (access to the article itself requires a JAMIA subscription.)

ABSTRACT

Test result management is an integral aspect of quality clinical care and a crucial part of the ambulatory medicine workflow. Correct and timely communication of results to a provider is the necessary first step in ambulatory result management and has been identified as a weakness in many paper-based systems. While electronic health records (EHRs) hold promise for improving the reliability of result management, the complexities involved make this a challenging task. Experience with test result management is reported, four new categories of result management errors identified are outlined, and solutions developed during a 2-year deployment of a commercial EHR are described. Recommendations for improving test result management with EHRs are then given.

The article begins:

Over a 2-year period from 2005 to 2007, coinciding with the first 2 years of a planned 3-year deployment of the ambulatory EHR to multiple practice sites, the vast majority of laboratory result routing events functioned as intended. However, seven error types were identified as causing a substantial delay or disruption in result delivery to providers’ electronic inboxes [no statement is made about patient harm or "close calls" that may have resulted - ed.] and led to further investigations and case finding by our group.

Upon analysis, these seven error types were logically grouped into four distinct error categories: (1) interface and results routing logic errors, (2) provider record issues, (3) EHR system settings, and (4) system maintenance.

This was at OHSU, a leading institution in medical informatics, not at some organization that's a newcomer to health IT.

Each of the "error categories" is described in some detail. The article then makes recommendations for improved systems, which sound simple, but are going to be far more resource intensive on a national scale than meets the eye:

1. Develop fault-tolerant systems that automatically report delivery
failures.
2. Use robust testing to find rare errors that occur both within and between systems.
3. Implement tracking mechanisms for critical tests, such as cancer screening and diagnostics.
4. Deliver results directly to patients.

I find myself uncomfortable with the possible human resource costs of implementing the recommendations, especially on a national scale. These costs would be over and above the hundreds of millions per institution and the hundreds of thousands per private doctor already spent, or planned to be spent.

My other issue regarding the article (my main issue, actually) is its editorializing for a product, health IT, in a scientific article, and making a special pleading for the technology.

The next to last paragraph of the article appears more of an editorial, perhaps to make vendors comfortable, than a scientific statement of fact supported by the article:

Finally, while it might be tempting to attribute the errors noted above to the use of a particular health information system or even Health IT in general, an examination of the cases reveals that most of these errors actually resulted from local configuration and implementation decisions rather than to the technologies themselves. Indeed, the authors believe that these cases further support the emerging truism [wow! This is news to me - ed.] that errors related to Health IT are in most cases the result of human error in the implementation of new information and communication systems into our existing complex healthcare environments.[10] Therefore, we contend that the main lesson arising from these cases is that care must be taken by those responsible for implementing health information systems to remain aware of the kinds of errors that might occur and monitor for the unexpected consequences that will undoubtedly take place, but not to avoid use of such systems that likely have the capacity for far greater benefit than harm, if implemented and monitored properly.

In this paragraph the authors state: "... while it might be tempting to attribute the errors noted above to the use of a particular health information system or even Health IT in general, an examination of the cases reveals that most of these errors actually resulted from local configuration and implementation decisions rather than the technologies themselves."

Sept. 2011 addendum: the article was written before Dr Jon Patrick's expose of examples of the internal flaws of commercial health IT, as here: link, link.

As for "rather than the technologies themselves", technologies themselves are never a problem by themselves, even the atomic bomb. In a reductio ad absurdum, which is maybe not so absurd, it took a B29 Superfortress to drop two A-bombs; the bombs could have been deactivated and put in a museum instead.

However, consider a poorly designed A-bomb that could unpredictably go "BOOM" - now that would be a problem.

While I agree some errors are due to mismanaged implementation, in the article, no differentiation is made of design issues vs. implementation (i.e., local configuration and implementation decisions). Yet fundamental design is crucial, according to industry leaders and non-industry experts, in areas that cannot be vastly improved by local configuration decisions:

HIMSS's former Chairman of the Board admits the technology remains experimental:

... We’re still learning, in healthcare, about that user interface. We’re still learning about how to put the applications together in a clinical workflow that’s going to be valuable to the patients and to the people who are providing care. Let’s be patient. Let’s give them a chance to figure out the right way to do this. Let’s give the application providers an opportunity to make this better;

While HIMSS itself admits in this 2009 PDF that

"Electronic medical record (EMR) adoption rates have been slower than expected in the United States, especially in comparison to other industry sectors and other developed countries. A key reason, aside from initial costs and lost productivity during EMR implementation, is lack of efficiency and usability of EMRs currently available";

While the National Research Council (the highest scientific authority in the U.S.) last year reported that:

"Current Approaches to U.S. Health Care Information Technology are Insufficient" and that the technology "does not support clinicians' cognitive needs." The study was chaired by Medical Informatics pioneers Octo Barnett (Harvard/MGH) and William Stead (Vanderbilt);

It is very difficult if not impossible to make a clinical IT silk purse out of a poorly designed sow's ear, no matter how many sound "local configuration and implementation decisions" are made.

Further, it is stated in the JAMIA article that human errors in implementation as the cause of health IT woes are an "emerging truism".

Making the case that some observation reflects a "truism" is a powerful claim. Such a claim deserves more than one reference, but here's what we have:

"... the authors believe that these cases further support the emerging truism that errors related to Health IT are in most cases the result of human error in the implementation of new information and communication systems into our existing complex healthcare environments" [10].

10. Ash JS, Berg M, Coiera E. Some unintended consequences of information technology in health care: the nature of patient care information system-related errors. J Am Med Inform Assoc 2004;11:104–12.

Perhaps the term "truism", emerging or otherwise, should be avoided in 2010 regarding errors related to health IT.

The authors contend, presumably from the above observations that:

... we contend that the main lesson arising from these cases is that care must be taken by those responsible for implementing health information systems to remain aware of the kinds of errors that might occur and monitor for the unexpected consequences that will undoubtedly take place

"Might occur?" How about "that do occur" - as in the paper? Above all, these involve patients.

Unexpected consequences - these involve patients, too.

My relative was nearly killed by "unexpected consequences:" of health IT in May 2010. Perhaps that makes me less cavalier about health IT.

In fact, the certainty that UC's will "undoubtedly take place" reaffirms that these are still experimental technologies.

I remind that it might be best to focus on fundamental design issues before expensive systems are put into place that can cause errors and unexpected consequences, because these are mission critical systems involving live patients who have not, incidentally, been afforded informed consent to the use of these medical devices in their healthcare.

Another editorial comment follows:

[the lesson is that those responsible should remain aware] but not to avoid use of such systems that likely have the capacity for far greater benefit than harm, if implemented and monitored properly

Once again, this is an editorial and value judgment. Who knows if ultimately health IT has a capacity for far greater benefit than harm? If these systems will have predictable, unexpected consequences, how do we know that? Why should critical-thinking practitioners not avoid such systems for now until a better understanding of how to design them to improve usability and support clinician cognition is achieved?

Why put patients at risk en masse as part of a national experiment when studies even at advanced HIT sites show fundamental problems that could harm or kill?

I argue this paper and others that are "emerging" on the downsides and lack of ROI of health IT make the case for great caution and slowness (i.e., avoidance) in their adoption.

Yet the authors seek special accommodation for this technology, something that is perhaps unprecedented with (unregulated) medical devices of unknown risk.

The lesson is actually that we need to slow down with HIT; reboot and start to solve the problems of this technology before national rollout attempts.

This is the ethical position regarding any experimental medical technology that is proving risky at a level not clearly known.

-- SS