Thursday, February 16, 2012

Hospitals and Doctors Use Health IT at Their Own Risk - Even if "Certified"

Due to my observations of confusion about health IT certification [1], and due to vague or incomplete seller language that could be misinterpreted by buyers (perhaps by design), I recently asked several ONC-ATCBs (HHS's Office of the National Coordinator for Health IT-Authorized Testing and Certification Bodies) the following.

I sent this question via email to their "questions" email addresses:

"Is EHR certification by an ATCB a certification of EHR safety, effectiveness, and a legal indemnification, i.e., certifying freedom from liability for EHR use of clinical users or organizations? Or does it signify less than that?"

One ONC-ATCB provided the following in response to my request for information.

From: Trivedi, Amit V (ICSA Labs)
Sent: Thursday, February 16, 2012 11:22 AM
To: Scot Silverstein
Subject: RE: Form submission from: Contact Us

Hello Scot,

Thanks for your email. Certification by an ATCB signifies that the product or system tested has the capabilities to meet specific criteria published by NIST and approved by the Office of the National Coordinator. In this case the criteria are designed to support providers and hospitals achieve "Meaningful Use." A subset of the criteria deal with the security and patient privacy capabilities of the system.

Here is a list of the specific criteria involved in our testing:

In a nutshell, ONC-ATCB Certification deals with testing the capabilities of a system, some of them relate to patient safety, privacy and security functions (audit logging, encryption, emergency access, etc.).

What was suggested in the email below (freedom from liability for users of the system, etc.) would be out of scope for ONC-ATCB testing based on the given criteria. [I.e., certification criteria - ed.] I hope that helps to answer your question.



Amit Trivedi
Program Manager - Healthcare
ICSA Labs, an Independent Division of Verizon Business

Mar. 5, 2012 Addendum:  I also received a response from another ONC-ATCB, the Drummond Group:

From: Joani Hughes (Drummond Group)
Sent: Monday, March 05, 2012 1:06 PM
To: Scot Silverstein
Subject: RE: EHR certification question

Per our testing team:

It is less than that. It does not address indemnification although a certification could be used as a conditional part of some other form of indemnification function, such as a waiver or TOA, but that is ultimately out of the scope of the certification itself. Certification in this sense is an assurance that the EHR functions in way that could enable an eligible provider or eligible hospital to meet the CMS requirements of Meaningful Use Stage 1. Or to restate it more directly, CMS is expecting eligible providers or eligible hospitals to use their EHR in “meaningful way” quantified by various quantitative measure metrics and eligible providers or eligible hospitals can only be assured they can do this if they obtain a certified EHR technology.

Please let me know if you have any questions.

Thank you,

Joani Hughes
Client Services Coordinator
Drummond Group Inc.

These are direct and clear statements.

My question was certainly answered. ONC cerification is not a safety validation, such as in a document from NASA on aerospace software safety certification, "Certification Processes for Safety-Critical and Mission-Critical Aerospace Software" (PDF) which specifies at pg. 6-7:

In order to meet most regulatory guidelines, developers must build a safety case as a means of documenting the safety justification of a system. The safety case is a record of all safety activities associated with a system throughout its life. Items contained in a safety case include the following:

• Description of the system/software
• Evidence of competence of personnel involved in development of safety-critical software and any
safety activity
• Specification of safety requirements
• Results of hazard and risk analysis
• Details of risk reduction techniques employed
• Results of design analysis showing that the system design meets all required safety targets
Verification and validation strategy
• Results of all verification and validation activities
• Records of safety reviews
• Records of any incidents which occur throughout the life of the system
• Records of all changes to the system and justification of its continued safety

Health IT testing conspicuously lacks attention to most of the aerospace software safety points above. I note that there appears to be no reasonable excuse for such omissions.

IOM has recently studied the issue of HIT safety. IOM states in a Nov. 2011 report that HIT safety and safety testing is unsatisfactory, and has recommended HHS study it as well. IOM recommends HHS annually re-evaluate whether regulation is needed to improve safety, although IOM favors industry self-policing [2].

Thus, buyers and users of even "ONC certified" health IT are not indemnified from liability due to medical errors or problems caused by the health IT.

Sellers who exaggerate the value of certification or imply its meaning is akin to FDA device approval, likewise, could be faulted for making false representations about their products.

It would appear the sellers could potentially be sued for doing so by purchasers/users who themselves get into legal hot water due to EHR defects or other problems.

-- SS


[1] I believe confusion about EHR "certification" is in part due to the term itself. I raised objections to this term when it was first proposed based on my experience in pharma, suggesting what I felt was the more accurate expression "
features qualification" instead.

[2] "Health IT and Patient Safety: Building Safer Systems for Better Care", Institute of Medicine of the National Academies, Nov. 2011,

1 comment:

Anonymous said...

The HIT industry sniffs out a bit too much like the tobacco industry and its trade group, The Tobacco Institute_see Wikipedia:"The Tobacco Institute, Inc. was a United States tobacco industry trade group, founded in 1958 by the American tobacco industry.[1]:251[2] It was dissolved in 1998 as part of the Tobacco Master Settlement Agreement.[3]:25

The Tobacco Institute was founded in 1958 as a trade organization by cigarette manufacturers, who funded it proportionally to each company's sales. It was initially to supplement the work of the Tobacco Industry Research Committee (which later became the Council for Tobacco Research). The TIRC work had been limited to attacking scientific studies that put tobacco in a bad light, and the Tobacco Institute had a broader mission to put out good news about tobacco, especially economic news. It also attacked scientific studies, although more by casting doubt on them rather than by rebutting them directly. It also lobbied Congress, although initially at a low level.[4]:211,466"

Now for the history lesson: HIMSS, the HIT vendors' trade group, and its CEO, H. Stephen LIEber,needed to legitimize the devices the vendors were producing. Thus, he and HIMSS established CCHIT.

CCHIT's dog and pony show provided the illusion to many, including the Congress of the United States, that its certification of HIT devices meant they were safe and efficacious...

Just as the Tobacco Institute declared tobacco to be safe.

Yup, another sham.