NIST (The U.S. National Institute of Standards and Technology) has recently issued a draft report "NISTIR 7804: Technical Evaluation, Testing and Validation of the Usability of Electronic Health Records." It is available at http://www.nist.gov/healthcare/usability/upload/Draft_EUP_09_28_11.pdf (PDF).
The report is quite negative concerning the low usability of today's commercial health IT, and recommends significant improvements that have been standard practice in other mission- and life-critical IT sectors for decades.
I will have more to write about this report, but I found a passage and footnote early in the report striking.
Of note, from the new NIST draft report:
This passage, from page 10:
The EUP (EHR usability protocol) emphasis should be on ensuring that necessary and sufficient usability validation and remediation has been conducted so that use error [3] is minimized.This passage describes what I have termed a mission hostile user experience.
[3] “Use error” is a term used very specifically to refer to user interface designs that will engender users to make errors of commission or omission. It is true that users do make errors, but many errors are due not to user error per se but due to designs that are flawed, e.g., poorly written messaging [or lack of messaging, e.g., no warnings of potentially dangerous actions - ed.], misuse of color-coding conventions, omission of information, etc.
"Blame the user" as the default, reflex reply to clinical IT-related medical errors, and the "hold vendor harmless for defects" clauses that facilitate this excuse are now heading to the junkpile - in the clinic, hospital, and courtroom.
It is my hope that the the Wild West, free-for-all, cavalier, get out of jail free days of the health IT industry are coming to a close.
Hint to health IT industry: you may actually need to invest in people who know what they're doing, instead of hiring the cheapest labor possible. (See, for example, my Aug. 2010 post
"EPIC's outrageous recommendations on healthcare IT project staffing" for more on that issue.)
-- SS
Oct. 25, 2011 Addendum:
This comment (#5, October 25th, 2011 at 12:38 pm) by "a practicing front-end designer and application developer" over at HIStalk is quite interesting. The writer points out the differences between designers and developers, and opines that:
If you’re going to improve vendor design, it has to begin with an internal commitment to value designers and what they contribute to the product development process. They can not be an afterthought. There are very few companies with this mindset. Almost all HIT companies are developer-driven, so the first thought, and one that is promoted in this report, is to turn developers into designers. This will not work! A developer and a designer require two fundamentally different skill sets that are not easily transposed. Developers are trained to think rationally and analytically; design requires empathy for the user.
In health IT, empathy for the patient as well, I might add.
-- SS
The CPOE devices are of high errorgenicity, as Dr. Koppel first reported in 2005 in a paper published in JAMA.
ReplyDeleteIn the experience of many users, all components of the EHR triad, ie the electronic library of records and the CPOE and CDS devices are error promoting, ie errorgenic.
The CPOE device is the most errorgenic and should be removed from the market place by the FDA.
Even Leapfrog, an industry group top heavy with IT vendors, an early proponent of CPOE, has now realized that its leap was premature.
Finally, NIST, a respected national authority is making an attempt to reduce abject dangers of these devices.
Is it not shocking that it has taken two decades for an agency to awaken to the fact that the EHRA and HIMSS was promoting sales of meaningfully unusable devices.
How many patients have been killed during this experiment?
With due respect to the hard work of NIST staff on producing this document, I know that for NIST be involved at all is purely due to politics.
ReplyDeleteThis is an end run by the vendors and their political influence ((on former and recently resigned (under duress) Congressman Wu of Oregon and others)) that enabled the HITECH Bill to be written by the House Committee on Science and Technology.
This resulted in elevating the NIST to an authoritative position (NIST is under the Congressional direction of that Committee) on HIT, while other leaders, including Secretary Sebelius marginalized the FDA by threat, keeping it from enforcing the F D and C Act.
Did NIST even know at the time that its responsibility was over devices that were not approved by the FDA? Nope.
To make matters worse, NIST does not have any enforcement or regulatory authority when it comes to medical devices.
The FDA has been conducting usability evaluations of medical devices for years.
ReplyDeleteWhy isn't the FDA actually evaluating the devices?
Why is this NIST lip service to usability being done so many years after the devices have already been in use? This is only a "draft". How many more years before the devices are meaningfully usable, and not an impediment to safe and efficacious care?
The vendors have heard complaints for years about this, and quashed them.
The trial lawyers should file suit against the vendors, rather than the lowly doctors who are forced to use them, and then suffer retaliation if they complain of the deaths.
The shield of HIT mysticism has been pierced by this document.
ReplyDeleteLet the truth be known.
Every death at every hospital that has deployed these error promoting devices is suspect and should be investigated by an independent quality organization.
These deaths from HIT are more egregious than what the stent doctors did in ringing the cash register with unneeded stents in Maryland and Pennsylvania.
Anonymous October 23, 2011 9:04:00 PM EDT writes:
ReplyDeleteWhy is this NIST lip service to usability being done so many years after the devices have already been in use?
See my posts regarding "putting the cart before the horse" at http://hcrenewal.blogspot.com/2010/07/meaningful-use-final-rule-have.html and http://hcrenewal.blogspot.com/2010/10/cart-before-horse-again-institute-of.html where I ask similar questions.
-- SS
Anonymous October 23, 2011 8:18:00 PM EDT writes:
ReplyDeleteThis is an end run by the vendors and their political influence ((on former and recently resigned (under duress) Congressman Wu of Oregon and others)) that enabled the HITECH Bill to be written by the House Committee on Science and Technology.
That could indeed be the case. Wu apparently was a big fan of health IT and its industry. He may also have been a fan of something else entirely. From Wikipedia:
On July 22, 2011, The Oregonian reported that an 18 year old woman accused Wu of an aggressive, unwanted sexual encounter. The teenager is the daughter of a longtime friend and campaign donor. Wu admitted the encounter but said it was consensual.[10] House Minority Leader Nancy Pelosi called for an ethics investigation into the allegations.[11] Wu initially indicated that he would not resign but would also not seek reelection in 2012.[12] Several days later, however, Wu announced he would resign following resolution of the 2011 US debt ceiling crisis. [2] He resigned on August 3, 2011.[13]
-- SS
As someone who works in UX in HIT, I thank you for raising the awareness of this. I'm pleased at the attempt by NIST to start holding companies accountable for the horrible designs out there. However, I am worried they will not have enough teeth to enforce this - I think the FDA has to make the leap into HIT.
ReplyDeleteI hope the day comes where there is accountability for usability in HIT, like the FDA has done with medical devices. Until then, we will continue to lose the battle to improve the UX of HIT software.
Occupy the ONC!
ReplyDelete