Upenn: Not What The Doctor Ordered

The University of Pennsylvania has penned an excellent article in JAMA (March 9, 2005): Role of Computerized Physician Order Entry Systems in facilitating Medication Errors , Koppel et al (free article). A summary of the article was in today's Philadelphia Inquirer ("Not what the Doctor Ordered"). The authors studied a CPOE system at the Hospital of the University of Pennsylvania operational from 1997 to 2004 -- and since replaced.

Many IT and workflow system-related problems are cited as causing a situation where information technology implemented with the best of intentions was actually causing error. Among the problems cited were "loss of data, time and focus when CPOE is nonfunctional ... crashes are common." "Inflexible ordering screens." "Viewing one patient's medications may require 20 screens." "lack of coordination among information systems [causing antibiotic renewal failure]."

These, among other issues cited, are simply stunning. I was involved in implementing the exact same CPOE system (TDS) at Yale-New Haven Hospital in 1992. It appears that hospital IT is truly an island, with inadequate cross-institutional problem-sharing or corporate memory.

These issues are depressingly reminiscent of similar clinical IT observations I documented via an Internet collaboration with other medical informaticists starting in 1998 at the website "Sociotechnologic issues in clinical computing: Common Examples of Healthcare IT Failure."

Here's an excerpt from the Philadelphia Inquirer article (registration required for full access):

Not What The Doctor Ordered
By Susan FitzGerald, Inquirer Staff Writer
Computerized prescription-ordering systems are promoted as the answer to preventing medication errors in hospitals, but a new study shows the technology also can cause mistakes.
The study at the Hospital of the University of Pennsylvania found that computer systems that allowed doctors to order drugs electronically, rather than writing orders, could lead to a variety of errors, including requesting drugs for the wrong patient or at an incorrect dosage. "What is supposed to be the great solution is itself
a source of errors," said Ross Koppel, a Penn sociologist who led the study.
While computerized systems "do offer some real protection," Koppel said, the study identified 22 types of medication errors that could result from the technology.

I penned a reply to the authors and to JAMA, which I reproduce here:

--------------

Re: Role of Computerized Physician Order Entry Systems in Facilitating Medication Errors” (JAMA, March 9, 2005)

Dear Dr. Koppel,

As a formally-trained Medical Informatics specialist formerly directing IT implementation in healthcare (Yale-New Haven Hospital, Christiana Care Health System after CIO Ward Keever's departure to HUP, and pharmaceuticals at Merck), I found the article “Role of Computerized Physician Order Entry Systems in Facilitating Medication Errors” (JAMA, March 9, 2005) fascinating. Congratulations on an excellent and useful study.

I would, however, have liked to see a discussion of a critical element that I feel may be at the root cause of many clinical IT problems. The root issue I have in mind was hinted at in the statement “the researchers were not involved in CPOE system design, installation or operation.”

In my 1998 JAMA letter “Barriers to Computerized Prescribing” (JAMA.1998; 280: 516-517), I identified a lack of leadership by clinical experts, especially those with dual formal training in medicine and information technology (i.e., Medical Informatics), in the design, evaluation, acquisition, implementation, revision, and system life cycle processes in clinical IT.

Instead, these processes are customarily led by those of a Management Information Systems (business computing) background in both vendor environments and provider environments. By leadership I mean the direct control of resources, staffing, skill sets, personnel evaluation, and major decisions impacting the practice of medicine. Clinicians as “consultants who know something about computers” is proving an inadequate model for success of complex clinical IT, as your experience suggests. I submit that most, if not all, of the errors you observed from CPOE use had as a root cause an inadequate leadership expertise of the system designers, evaluators and implementers. These personnel did not have bad intentions, of course, but due to a basic misalignment of skillsets required for leadership in clinical IT the observed problems may result.

A stunning example of this phenomenon on a large scale was the recent $450 million hospital IT failure at the Bay Pines, Florida VA Hospital. Half a billion dollars was wasted due to the system designers and implementers admittedly not having sufficient understanding of hospitals and healthcare, a debacle detailed by the Google search http://www.google.com/search?hl=en&lr=&q=Bay+Pines+VA+hospital+computer+system+failure .

On a national scale, the UK’s multibillion dollar national electronic medical records initiative faces similar problems due to “lack of engagement by clinicians in the early stages of the programme” (“NHS joint IT chief resigns after six months in the job”, http://www.computerweekly.com/articles/article.asp?liArticleID=133699 ).

As Joan Ash, Ph.D. noted at Oregon Health & Science University in a 2003 study “Most hospitals don't use latest ordering technology” (http://www.eurekalert.org/pub_releases/2003-11/ohs-mhd112403.php ), "many information systems simply don't reflect the health care professional's hectic work environment with its all too frequent interruptions from phone calls, pages, colleagues and patients. Instead these are designed for people who work in calm and solitary environments … some patient care information systems require data entry that is so elaborate that time spent recording patient data is significantly greater than it was with its paper predecessors," the authors wrote. "What is worse, on several occasions during our studies, overly structured data entry led to a loss of cognitive focus by the clinician."

Calm and solitary environments, indeed.

How are CPOE and other clinical IT systems that ignore the healthcare workplace's realities finding their way into real products? How is this possible? While the workflow of the National Security Agency might be secretive, the realities of the medical work environment are certainly not. Who are the CPOE designers, exactly, and what are their backgrounds? How could investor dollars have been spent in such a fashion as to ignore the fundamental realities of clinical settings? How could IT companies have designed and implemented systems that "led to a loss of cognitive focus by the clinician" and created error?

Unfortunately, the answer to these questions is that design and implementation of CPOE systems and other clinical IT are being led by Management Information Systems business-computing personnel, who generally design and implement systems for "calm and solitary" business office environments, instead of those with both IT and clinical expertise such as Medical Informatics specialists.

I detailed a number of case studies of healthcare IT failures through inadequate leadership models by business computing personnel at my old website “Sociotechnologic issues in clinical computing: Common examples of healthcare IT failure” at http://home.aol.com/medinformaticsmd/failurecases.htm . You will likely find these cases of interest. The IT model of "If it's information, we do it" starts to fall apart and impede progress in such organizationally and sociologically-complex environments as medicine.

Clearly, clinical IT leadership models are defective and must be changed as an initial step before any of the recommendations you make in your article’s conclusion can be realized.

Sincerely,

Scot M. Silverstein, MD

Former faculty, Yale School of Medicine, Center for
Medical Informatics, Director of Clinical Informatics, Christiana Care Health
System, Wilmington, DE, and Director of Scientific Information Resources &
The Merck Index, Merck & Co., Inc.

--------------

(In my letter to JAMA, unfortunately, I forgot to include perhaps the most stunning of recent, costly CPOE missteps, "Doctors Pull Plug on Paperless System" at Cedars-Sinai Medical Center in Los Angeles, AMA News, Feb. 17, 2003. An excerpt:


As I wrote above, It appears that hospital IT is truly an island, with little cross-institutional sharing or memory. I sometimes wonder if Gilligan's Island is not an inappropriate metaphor for this phenomenon.

The author's JAMA article is quite welcome as a start to closing the gaps in the world of clinical IT. These articles are all too rare; these issues are often highly whitewashed in the hospital IT press. There is also resistance to such articles from many quarters. However, either an organization is in control of its computing - and studies its IT issues and mistakes - or the IT is in control of the organization. That's an adaptation of the old UNIX adage "either you're in control of your system, or it's in control of you."

Those who oppose exposure of clinical IT's flaws, when the critique is geared towards learning and correction , are firmly rooted in a dysfuntional territorial mentality that has no place in a scientific field such as medicine. It may or may not have had its place in "data processing" shops of the past; however, such beliefs have no place in 21st century healthcare.

It must be remembered that hospitals exist so that clinicians can take care of patients, not so that IT personnel, staff and vendor companies can have easy computing jobs and lucrative contracts.

(Note: I worked with an IBM/370-165 mainframe in the past at the assembly language level. As I recall, the TDS CPOE system ran on an a mainframe successor to the 370's that was of utmost industrial reliability, and the software was accessed via character-based terminal emulators (e.g., VT100) running on PC's. How such a system can have "common crashes" is somewhat of a mystery to me.)

-- SS

"Cedars-Sinai Medical Center in Los Angeles turned off its computerized physician order entry system in January, after hundreds of physicians complained that rather than speeding up and improving patient care, it actually slowed down the process of filling their orders -- assuming those orders didn't get lost in the system ... Cedars-Sinai's decision was extraordinary but not unique. David Classen, MD, of First Consulting Group, says he knows of at least six other hospitals that have pulled paperless systems in the face of physician resistance and other problems ... "They poorly designed the system, poorly sold it and then jammed it down our throats and had the audacity to say everybody loves it and that it's a great system," [Cedars-Sinai physician user Dr. Dudley] Danoff said."