Monday, December 13, 2010

NIST Provides Healthcare IT Industry with Remedial Undergraduate Computer Science Education

The National Institute of Standards & Technology (NIST) has published a guide entitled:

NIST Guide to the Processes Approach for Improving the Usability of Electronic Health Records

It is available free at this link in PDF: (hat tip to an AMIA colleague for posting the URL on an AMIA mailing list.)

The NIST was commissioned by HHS/ONC to study Health IT issues such as usability and report on them.

I find the publication both welcome, and pitiable.

As I started to read ch. 6, for example, I observed material that is suitable for undergraduate computer science instruction:

6. User-Centered Design Process in EHRs

User-centered design is a bedrock principle for creating usable systems and devices. [You don't say? - ed.] One of the most common reasons why systems are poorly designed is that designers and developers fail to engage users in appropriate ways at appropriate times. [Hear that, my young Paduan learners? - ed.] At its core UCD is a process that relies on systematic understanding of users and their environments, and iterative design and testing based on user performance objectives. (Details on usability testing are provided in Section 9.)

UCD has been shown to be effective in many fields. In aviation, for example, this method has been used to develop cockpit navigation displays for low-visibility surface operations. [22] By taking the limitations and capabilities of the flight crew into account, navigation errors have decreased by almost 100%. The adoption of UCD has also been shown to be effective in the design of personal computers. When working on a redesign of the laptop computer, a UCD process was employed. Users were asked to offer feedback about the current model and to offer input about ways to improve the current design. User-centered design was successful in increasing market share, brand equity, and customer satisfaction. [23] In fact, user-centered design has been elevated to an ISO standard. [24] UCD serves to engineer improved human performance into a system or device, and has been crystallizing for several decades as a design philosophy. [25]

While there is no singular model of UCD, the instantiations embody the following principles:

  • Understand user needs, workflows and work environments
  • Engage users early and often
  • Set user performance objectives
  • Design the user interface from known human behavior principles and familiar user interface models
  • Conduct usability tests to measure how well the interface meets user needs
  • Adapt the design and iteratively test with users until performance objectives are met

[HHS needs ONC to commission NIST to provide schooling for the HIT industry on these bons mots? - ed.]

As an iterative process, UCD is a cycle that serves to continually improve the application. For each iteration, critical points and issues are uncovered which can be improved upon and implemented in subsequent releases. An illustration of the UCD process is included in Figure 1.

"User centered design process in EHR's." Undergraduate-level computer science 101 instruction from NIST for the healthcare IT industry? (click to enlarge)

Here is Figure 1:

User centered design 101 (click to enlarge)

I might even have used this in teaching high school students about computer programming.

Readers can download the entire report at the above URL.

I find this publication, or, rather, the need for it to exist at all in 2010, remarkable. Absurd and an embarrassment, in fact. Master of the Obvious [1] material that apparently was not so obvious to this industry.

This is after all a multi-billion dollar industry making claims its products will "revolutionize medicine" and other exceptional claims (but without exceptional evidence). These claims have been pushed so hard that many tens of billions of dollars (with penalties) have been earmarked to either entice -- or coerce -- physicians and hospitals to use these products.

What has this industry, including vendors and highly paid management consultants and contractors, been doing, exactly, for the past thirty+ years?

What have been their product design and development practices, such that leaders of their own trade group HIMSS (as I pointed out in other posts) opine we should be "patient" for them to figure it all out about how to do health IT better and they need more time, and that the technology does not support its users properly due to lack of efficiency and usability of EMRs currently available? (As at my July 2010 post "The National Program for Healthcare IT in the U.S., and the Elephant in the Living Room".)

That HHS needs NIST to provide undergraduate level remedial teaching to the multibillion dollar health IT industry is a very poignant commentary indeed on the priorities of that industry regarding engineering rigor, talent management, attention to safety, and other factors affecting human lives.

These observations speak strongly to the need for regulation for this industry, for a talent management and trade association shakeup of major proportions, and most especially for an awakening of our government servants to exactly what the real situation is with respect to HIT on the ground.

12/14/10 addendum: it struck me that the Guide might need another chapter. I suggest a chapter entitled:

"Listening to informatics experts when they say your product will kill people, instead of firing them."

-- SS

[1] This was another pithy line from my early medical mentor, pioneering cardiothoracic surgeon Victor Satinsky, MD at Hahnemann Medical College.


Anonymous said...

I too thought this publication was pathetically trying to cover for the abject neglect to consider usability, and the failure to assure that these devices are safe. The congressional committee that oversees NIST has failed as well, colluding with the HIT industry to avoid the regualtion that these devices require.

InformaticsMD said...

The congressional committee that oversees NIST has failed as well, colluding with the HIT industry to avoid the regualtion that these devices require.

As I stated in the post, I think the release of this document (which I presume NIST felt was needed) speaks more for the need for regulation.

If Pharma were still unregulated and needed lectures from NIST on how to make safe medications in 2010, what would happen?

-- SS

Live it or live with it said...

The HIT vendor business desire is to lock the users into their system not make their system better for users.

The recent PCAST report made the lock in problem front and center.

While I am not sure about their suggested solution, if the administration takes the PCAST report seriously, it would send the right sort of shock wave through the system.

InformaticsMD said...

The HIT vendor business desire is to lock the users into their system not make their system better for users.

Which implies, that they don't really give much of a damn about patients.

My own mother, gravelly injured by an EMR interfering with clinician communications, is one victim of these attitudes.

Live it or live with it said...

Well they have no duty to the patient the way things are now. The duty is borne by the hospital and doctor, as is the liability. It seems though that the hospital and doctors don't understand the true liability. Do you think some high profile lawsuits would change things? I am no fan of lawsuits, but in this case, it may be the only way.

Anonymous said...

I do not know about you but I am overloaded with the pages of gibberish that obscure the facts of the case. I found two mistakes out of f cases today, amidst the rubble of output of the EMR. One was a dose of nitro for cp of 4mg (sic) s/l prn, and the other was an assumption by a shrink consult that the patient was on wellbutrin for depression, when it had been discontinued 24 hours earlier. These errors are so widespread that medical care is dangerous from EMRs and CPOEs and can not be trusted.

This is sickness. NIST and its House Committee on Science and Technology is rather unscientific.