Friday, December 16, 2005

UPI picks up on pitfalls of bad clinical computing

A recent CPOE article in Journal of Pediatrics ("Unexpected Increased Mortality After Implementation of a Commercially Sold Computerized Physician Order Entry System", PEDIATRICS Vol. 116 No. 6 December 2005, pp. 1506-1512) has been picked up by United Press International (UPI).

Note these comments by AMIA official Paul Tang at the end of the UPI piece:

Paul Tang, incoming chair of the American Medical Informatics Association, said he thought there would be less risk with computerized healthcare systems in the future if the software were implemented by specialists.

"Anywhere along the line, from designing the system to its implementation, the process needs to be handled by medical informatics specialists . After all, you don't let general practitioners dispense powerful chemotherapy drugs," Tang told UPI.

A good analogy. I and others in healthcare informatics have been writing about this for a number of years (e.g. my website on informatics leadership of clinical IT). It's interesting to see what was originally thought of as a provocative assertion about the need for cross-disciplinary specialists to lead clinical IT is now becoming mainstream.

Of course, the Journal of Pediatrics article received pushback. It was critiqued on its methodology and its inferences in the medical informatics community itself and by the vendor, a part of which is presented in the UPI piece below:

IT-related deaths highlight tech needs

By ASTARA MARCH (covers healthcare technology for UPI)

WASHINGTON, Dec. 12 (UPI) -- Patient deaths at the University of Pittsburgh Children's Hospital blamed on a computerized physician order entry (CPOE) system point to the troubling possibility that electronic healthcare systems designed to save lives are, in some cases, having the opposite effect.

A key to correcting the problem might be to better tailor software systems to existing hospital networks and to assure that the systems are designed and implemented by specialists, experts say.

According to an article published in the December issue of the Journal of Pediatrics, after the Pittsburgh hospital installed a CPOE system the mortality rate of children
admitted to the hospital after being transported from another facility increased from 2.80 to 6.57 percent.

The CPOE system installed in the hospital at the time was Cerner Corporation's PowerOrder.

The deaths -- all linked to delays in obtaining medication -- were blamed on CPOE's software-design problems and faulty implementation.

Yong Han, of the University of Michigan Medical School in Ann Arbor and lead author of the journal article, told United Press International he was especially concerned about the software aspect of the problem.

"Medical software directly impacts patient care, for better or worse, and there are no checks and balances for it right now," said Han, who formerly worked at the Pittsburgh hospital.

"I could get together with my friends at MIT, whip up a program, put it on the market and no one could stop me," he said. [I wrote about this issue here in 1998 or so. - ed.]

"Drugs or medical devices must go through some degree of inspection and evaluation to determine whether the manufacturer's claims are substantiated. There's no such process for medical-software programs at the current time, and I think there needs to be," Han argued.

Although she declined to comment on Han's assessment of the system, Terri Steinberg, clinical applications manager for the Alfred I. duPont Hospital for Children in Wilmington, Del., told UPI that her facility's own experience with the software system at issue and with similar programs, has been problem-free.

"We have used the same software program as (that used at) Han's hospital since 1999 with no difficulty," she said.

Steinberg also stressed the importance of the human element when using the complex computer systems. "When you implement software in a medical environment, you must address your institution's workflow process and support the people who are using the system. Human beings are the primary determinants of failure or success of any kind of software implementation," she said.

For its part, the software's maker pointed to what it called flaws in Han's study. James Fackler, Cerner's director of critical care, charged that the data was unreliable because it compared 13 months of mortality data before the CPOE system was installed with only five months of data afterwards.

"Pediatric critical care is very seasonal. In pediatric ICUs, a mortality rate of 7 percent over six months is often followed by a mortality figure of 1 to 2 percent during the next six months. I would like to see figures for the full 13 months after our system was installed to be sure the effects were due to IT problems and not the normal fluctuations hospital pediatrics units experience," he said.

Paul Tang, incoming chair of the American Medical Informatics Association, said he thought there would be less risk with computerized healthcare systems in the future if the software were implemented by specialists.

"Anywhere along the line, from designing the system to its implementation, the process needs to be handled by medical informatics specialists. After all, you don't let general practitioners dispense powerful chemotherapy drugs," Tang told UPI.

Tang said serious problems can occur at three places in a software system.

"If the software needs too many clicks or takes too long to accomplish a task, that has to be corrected. Then there's software configuration. The vendor and the customer have to integrate the software with the hospital's system so they work well together," he said.

"The third place problems can occur is training staff and physicians to use the system. That's the customer's responsibility and it's often neglected," Tang said

I reproduce some of my comment to the informatics community below:

... This study is dealing with children, of course, and is perhaps a flag that much more detailed study of these systems, especially in socially-sensitive environments such as pediatrics, need to be performed.

For if this study's findings are not just due to serendipity and do reflect some underlying causation, the medical, ethical and legal issues could be enormous.

The health IT industry needs to realize this, and that some of their executives might end up on the witness stand during litigation as defendants. They (and we) need to do as much as possible to both sponsor such studies, and encourage others to do them, rather than discourage or disparage them - as was done in the example of Univ. of PA sociologist Ross Koppel [who wrote the JAMA article "Role of Computerized Physician Order Entry Systems in Facilitating Medication Errors"], where Ross was accused in print of being "disingenuous", I believe. In medicine, there is no legitimate excuse for protecting a company at patients' expense.

... Quite seriously, I think the question of why systems that do not fit the clinical environment well are still designed and sold, and why systems that might be OK from a design perspective are implemented without clinician and patient needs front and foremost, needs to be asked.

As per Joan Ash's research and " Most hospitals don't use latest ordering technology " in late 2003:

Computers programmed to screen out errors and standardize physicians' orders for prescriptions, tests and other care have been a source of hope in reducing medical errors and improving patient safety. The problem is that most hospitals aren't using this technology, known as computerized physician order entry (CPOE) ... Reducing medical errors gained a sense of urgency in 2000 when the Institute of Medicine reported on the issue. The institute found that medical errors may be responsible for up to 98,000 deaths in hospitals and cost the U.S. health care system approximately $38 billion per year.

Despite a wide array of quality, policy and financial incentives to use CPOE, fewer than 10 percent of American hospitals make it completely available to their physicians. This was among the findings of a study conducted by researchers in the Oregon Health & Science University School of Medicine and recently published in online edition of the Journal of the American Medical Information Association (JAMIA).

[Use of CPOE] is not yet widespread because it has a reputation for being difficult to implement successfully. Patient care information systems like CPOE ... can create unintended or "silent" errors, according to a separate study conducted by the same author in the Netherlands and Australia.

The study's authors divide these silent errors into two main types: errors during data entry and retrieval, and errors in the communication and coordination process. Both types of errors occur because the systems simply don't take into account the work atmosphere most health care professionals experience, according to Ash, also lead author of this study.

"Many information systems simply don't reflect the health care professional's hectic work environment with its all too frequent interruptions from phone calls, pages, colleagues and patients," Ash said. "Instead these are designed for people who work in calm and solitary environments. This design disconnect is the source of both types of silent errors."

The screen itself can cause errors. Choices that appear too close together result in ordering the wrong tests or sending orders for the wrong person. If a system is rigidly structured, it causes users to focus closely on entering details and switching from screen to screen to enter information.

"Some patient care information systems require data entry that is so elaborate that time spent recording patient data is significantly greater than it was with its paper predecessors," the authors wrote. "What is worse, on several occasions during our studies, overly structured data entry led to a loss of cognitive focus by the clinician. "

Calm and solitary environments, indeed.

How are systems that ignore the healthcare workplace's realities finding their way into real products? How is this possible? While the workflow of the National Security Agency might be secretive, the realities of the medical work environment are certainly not. Who are the CPOE designers, exactly, and what are their backgrounds? How could investor dollars have been spent in such a fashion as to ignore the fundamental realities of clinical settings? How could IT companies have designed and implemented systems that "led to a loss of cognitive focus by the clinician" and created error?

Again, the question is: why does this occur, and what factors permit it to happen?

-- SS


Anonymous said...

Your link to your website: "informatics leadership of clinical IT" appears to be broken.

InformaticsMD said...

Typo. Fixed.

-- SS