Sunday, October 25, 2009

Washington Post Article: Electronic medical records not seen as a cure-all

Regarding the very well done Sunday Oct. 25, 2009 story in the Washington Post "Electronic medical records not seen as a cure-all" by staff writer Alexi Mostrous - signup may be required for access - I have several observations.

(Not including the observation that Mr. Mostrous probably deserves an award for being the first major newspaper reporter to broach this topic in a serious and balanced manner.)

First, I believe healthcare IT can live up to all the predictions made about its benefits - but only if done well. There is massive complexity behind those two words "done well", and that is HIT's key stumbling block in 2009. I believe we are only in the adolescent stage of knowing how to "do health IT well."

Second, I should point out that the intended consequences of health IT include, among many other things, the following "hiding in plain sight" (i.e., not often verbalized) intended consequences:

  • The improvement of medicine ... in the context of protection of patient rights established over centuries of development of modern medicine.
  • The improvement of IT itself through cross disciplinary collaboration between IT and medicine, of the science of IT (computer science), the social aspects of computerization ("sociotechnical issues"), and improvement of the our understanding of the intersection of medicine and computers.

Instead, we largely have the opposite. Patients' rights are trampled, and hostility and territoriality has arisen between clinicians (including medical informaticists) and IT, groups that rarely if ever interacted in hospitals ten or twenty years ago.

Of concern, when scientific study sections evaluate NIH grant proposals calling for testing of new IT that involves patients, patient protections and informed consent processes are a paramount concern since such activities are considered research. Yet, in implementing large clinical IT system in a hospital with new features, there are no formal regulations, and I'm not sure there's even IRB involvement in most cases. Patients do not get the chance to give informed consent to the use of these IT devices mediating their care. Why the difference?

The unintended results of computerization efforts have also included suppression of research on sociotechnical issues and on informatics, which must include study of the downsides of HIT, and of the failures in addition to the study of the successes. That is scientific fact - there is no room for debate, no room for spin on the need for careful study of the downsides of any mission critical domain. One would think there to be a vibrant literature on these issues, Yet searches on massive biomedical databases such as PubMed on, say, "cerner electronic health record" (or other vendors as well) are disappointing to say the least. Further, my own website on HIT difficulties remains nearly unique (PPT) after ten years online. That is not bragging; it is a disturbing finding to me - symptomatic of an industry that somehow has managed to avoid serious scrutiny.

In a field with downsides, there are:

1) those who know about the problems but fail to speak,
2) those who see the problems but fail to act, and
3) those who see, know and speak/write/research the problems.

That said, now on to the Washington Post article:

... bipartisan enthusiasm has obscured questions about the effectiveness of health information technology products, critics say. Interviews with more than two dozen doctors, academics, patients and computer programmers suggest that computer systems can increase errors, add hours to doctors' workloads and compromise patient care.

I would include the bipartisan enthusiasm under the subject header of "irrational exuberance", which itself falls under the header of "lack of domain knowledge." That itself is a consequence of both failure to study the issues, and suppression of those issues by those with an interest in doing so.

health IT's effectiveness is unclear.

The literature is indeed conflicting, and the need for rigorous scientific study has never been more essential considering the commitment of tens of billions of dollars towards health IT. The time for story telling, marketing based on opinion, name calling, leap-of-faith extrapolations of light year dimensions, and other forms of pseudoscience and non-science are over. The time for objective study is now.

The Senate Finance Committee has amassed a thick file of testimony alleging serious computer flaws from doctors, patients and engineers unhappy with current systems.

Being the ranking member of that committee, Sen. Grassley has a fiduciary responsibility to protect Medicare and Medicaid patients (and one might argue, to protect all patients since those programs often serve as models for private insurers). In that regard, the investigation he has initiated is part of his responsibility as a ranking member of Congress. Politics aside (and there are those who will resort to ad hominem "political witch hunt" arguments), he would have been negligent if he had not initiated an inquiry.

Sen. Grassley has taken on the pharma industry and the government's Food and Drug Administration itself, such as in this recent article "FDA fails to follow up on unproven drugs" where he concluded from a GAO study he ordered that "FDA has fallen far short of where it should be for patient safety." He seems quite serious about medical safety.

If only others in Congress had done their job similarly regarding national finance, we might now not be in the worst economic crisis since '29 with many major industries failing.

David Blumenthal, the head of health technology at the Department of Health and Human Services, acknowledged that the systems had flaws. "But the critical question is whether, on balance, care is better than before," he said. "I think the answer is yes."

This sounds uncomfortably like how a pharmaceutical company might respond to doubts about drug effectiveness and safety. In reality it's really irrelevant what he "thinks." Where's the data? Is this a political statement, a personal belief, or a statement backed up by scientific fact that is not cast into doubt by other research results? Our own National Research Council, Joint Commission, and other international organizations have written about their doubts and concerns about HIT [as that IT is designed and implemented in 2009]. If there is rigorous, systematic research weighing pro's and con's to back this assertion, I wish it would be published.

For his statement is really saying "we don't really know how many systems have flaws, we know some do, and we don't really know the full extent of the impact of those flaws, but because there can be some benefits, let's spend $50+ billion before we know the extent of the problems and fix them." I point out research from Harvard forty years ago, when Harvard informatics pioneer Dr. Octo Barnett published the "Ten Commandments of HIT." Two of those commandments were:

... 8. Thou shall be concerned with realities of the cost and projected benefit of the computer system [i.e., ROI - ed.]

10. Be optimistic about the future, supportive of good work that is being done, passionate in your commitment, but always guided by a fundamental skepticism.

The full set is in this post. Somewhere in the past 40 years, the rigorous ROI evaluations (which also must include systematic evaluations of risk, as any businessperson knows) and the fundamental skepticism seem to have gotten lost.

Over the next two months, Blumenthal will finalize the definition of "meaningful use," the standard that hospitals and physicians will have to reach before qualifying for health IT stimulus funds.

This is an example of putting the cart before the horse, and is a semantically-based, self contained logical fallacy of sorts. If a health IT system is harmful, the term "meaningful use" is itself Orwellian. If we don't know if HIT is beneficial, or have doubts, then such as term presupposes that health IT is inherently beneficial. A better term would have been "good faith use" - use based on the faith or hope that health IT will have an overall positive effect. The term "meaningful use" jumps the gun and is more a political slogan than a "meaningful term."

"If you look at other high-risk industries, like drug regulation or aviation, there's a requirement to report problems," said David C. Classen, an associate professor of medicine at the University of Utah who recently completed a study on health IT installations.

This is obvious, the reasons for it are obvious, and the reasons why health IT needs a requirement for problems reporting (one aspect of post-marketing surveillance, the "Phase IV" study as it is known in pharma) is obvious. Yet in 2009, no such requirement exists (see my post "Our Policy Is To Always Have Unabashed Faith In The Computer" for more on why we need reporting requirements.) Why do these requirements simply not exist in HIT?

"It's been a complete nightmare," said Steve Chabala, an emergency room physician at St. Mary Mercy Hospital in Livonia, Mich., which switched to electronic records three years ago. "I can't see my patients because I'm at a screen entering data." Last year, his department found that physicians spent nearly five of every 10 hours on a computer, he said. "I sit down and log on to a computer 60 times every shift. Physician productivity and satisfaction have fallen off a cliff."

The industry in the past has called such physicians "luddites", "resistant to change", "stubborn" etc. However, argumentum ad hominem is a fallacious mode of argument that has no place in a scientific field such as biomedicine. There also seems to be quite a lot of such concerns expressed by a large number of physicians, nurses, etc., and dismissing their concerns with a wave of the hand is cavalier in the extreme - again, these are first principles, without room for argument or debate. Let's study the issues rigorously and scientifically before resorting to ad hominem.

Other doctors spoke of cluttered screens, unresponsive vendors and illogical displays. "It's a huge safety issue," said Christine Sinsky, an internist in Dubuque, Iowa, whose practice implemented electronic records six years ago.

See my eight part series on mission hostile clinical IT here for examples of what Dr. Sinsky is referring to.

She emphasized that electronic records have improved her practice. "We wouldn't want to go back," she said. "But EHRs are still in need of significant improvement."

Yes, not cancellation, but improvement. And, quite importantly, before tens of billions of dollars are spent. Hospitals and physician offices are not an IT development laboratory, since the users of these facilities (patients) have very special rights and the clinicians, very special obligations and responsibilities.

Legal experts say it is impossible to know how often health IT mishaps occur. Electronic medical records are not classified as medical devices, so hospitals are not required to report problems.

That after decades of HIT development, sales and implementation we cannot know with certainty how often mishaps occur is, quite simply, a scandal of major proportions. Quoting an old House of God law, #10: "if you don't take a temperature, you can't find a fever." Another applicable aphorism seen on another discussion board: "you can only be so negligent or craven before the only remaining rationale is that you intended the result."

"Doctors who report problems can lose their jobs," Hoffman said.

I've taken risks with my own career in criticizing health IT, as have my colleagues. Hoffman is not exaggerating.

"Hospitals don't have any incentive to do so [speak out about problems with HIT] and may be in breach of contract if they do."

Imagine the outcry if the same prevailed regarding drugs or medical devices. The cemeteries would be lined with people whose epitaph could read "we bury our mistakes."

While orange-shirted vendor employees "ran around with no idea how to work their own equipment," the internist said, doctors struggled to keep chronically ill patients alive. "I didn't go through all my training to have my ability to take care of patients destroyed by devices that are an impediment to medical care."

This gets to issues I first raised in my website on HIT difficulties: who are these IT personnel, and what are their qualifications, exactly, to be working in mission critical medical environments? How is their competence evaluated?

I think these are questions that need to be answered.

-- SS

9 comments:

Anonymous said...

Your blizzard of concerns have a tendency to seek perfection at the expense of the good. After having battled to implement a lab system in a large 700 bed consortium in 1980 and succeeding to do so, I have some good "street" feelings about moving ahead. Medical staff and Administration joint committees can protect patient data by fiat. Let's quit dithering and get on with the task. Spend the $$ upfront to train every doc,nurse & ancillary person now. Stop the nonsense research & select Oracle or some known company to deveop a database system acceptable to a team of physicians and IT folks. Roll out the system in 2012 in a couple of large hospital systems connected to physicians offices. After approval, begin a full 50 state rollout into all hospitals & clinics. Begin a nationwide review of success/failure upon implementation. Morrie Foutch

InformaticsMD said...

Morrie Foutch wrote:

"our blizzard of concerns have a tendency to seek perfection at the expense of the good."

Morrie, thanks for the comment. I want to clarify that I do not seek anything near "perfection." I do seek relief from flaws that basic good software and human computer interaction engineering and design practices would prevent from ever reaching the clinical suite.

"Let's quit dithering and get on with the task. Spend the $$ upfront to train every doc,nurse & ancillary person now. Stop the nonsense research & select Oracle or some known company to deveop a database system acceptable to a team of physicians and IT folks. Roll out the system in 2012 in a couple of large hospital systems connected to physicians offices. After approval, begin a full 50 state rollout into all hospitals & clinics. Begin a nationwide review of success/failure upon implementation."

While I share the ideas conceptually, years of hard experience have taught those of us in Medical Informatics, especially those who study the organizational and social issues in health IT, that it's not that straightforward. Computerizing healthcare may be more of a "wicked problem" than a tractable one. (A wicked problem is one that is difficult or impossible to solve because of incomplete, contradictory, and changing requirements that are often difficult to recognize. Moreover, because of complex interdependencies, the effort to solve one aspect of a wicked problem may reveal or create other problems - Wikipedia).

That doesn't mean it can't be done, but 'linear' solutions just don't work as well as might be thought.

Training is not the solution either. See the very wise essay from the Air Force in the 1980's about IT design at http://hcibib.org/sam/. Scroll down to the section on "significance of the user interface."

I still believe our application and even understanding of the practices needed to successfully computerize medicine are in the adolescent stage.

-- SS

InformaticsMD said...

Jeremy wrote:

"How can healthcare data pooling lead to a better system?"

Agree.

Caveat: you need good data, or advanced statistical research on how to account for the wide variations of quality, completeness, interobserver interpretation variances, etc. prevalent in such data. It's as far from RCT data as one can imagine.

Research on statistical methods to deal with such data is underway, as in this paper, but I wouldn't want major medical decisions on my own care to be made just yet based on EHR-originated datasets.

-- SS

Jay Beecham said...

I've been implementing large scale database and business process systems for 12 plus years now. I am also a paramedic, and so somewhat familiar with drug calcs and the basics of medical care, and finally a cancer survivor as well, so well acquainted with disease surveillance and complex care mgmt.

Funny - Here and in the WaPo article, people talk about Med IT systems as though the system itself was to blame for the errors in patient care and drug miscalculations. Often not true. In some cases, the logic of the calculation may be wrong, but it would continue to be wrong each and every time that subroutine was called, so hopefully would be corrected fairly quickly. Most times, it is user error that causes the problem.

Being primarily responsible for legacy "data conversion" during my career, I have found that most systemic problems occur because of poor quality data and resistance from users to the change required by IT systems that generalize processes over an entire industry. Your industry, has 3-4 industries to consider, maybe more!

These systems EMR/EHR/HDE (health data exchange) are all in their adolescent years, and Mostrous' article did nothing, or only provided a sentence here and there, to represent the benefits of these systems, and the successes that have occurred. He also didn't present any data that described errors that are occurring every day using the status quo paper based system. That doesn't seem so balanced to me.

HIT systems are a lot like other business support systems. For instance, accounting software has evolved to a state where almost anyone can do their own taxes, or figure out advance accounting metrics by simply plugging in numbers. Finance software on the other hand, requires a large degree of subject matter expertise in order to derive meaningful data. I think medicine is similar to financial software in that it will require more expertise and "the art of the discipline" in order to work well. Systems will never replace the physician, but may over time allow PAs & NPs to relieve some of the work load.

I for one, really appreciate that my physician has an EMR, and does all of his data entry directly, while he is conducting the exam. He is almost 100% on time, or within 5 minutes of our scheduled appt as there is no dictating or other duties between patients, and he spends the entire appt talking to me about how I am feeling or coaching me on how to take better care of my physical health. All of my labs and procedure results come back thru the data exchange. MRIs, CT Scans, and xrays are all posted to my record, and I still have some degree of privacy in that I still have to sign a release to let other docs see components of my record. The privacy that I had before was better in that my records were hard to locate and so were private in that sense, even from me.

InformaticsMD said...

Jay Beecham wrote...

I have found that most systemic problems occur because of poor quality data and resistance from users to the change required by IT systems that generalize processes over an entire industry.

You assume that the clinician "resistance" to HIT is without significant merit. This is at best an ill-informed position.

You also may believe that medicine is like an assembly line where nameless providers provide "processes" to nameless customers. Medicine does not work that way, and "generalization of processes" over such a sociotechnically complex domain with its wicked problems is deleterious to that system due to false assumptions from the management information systems world.

HIT systems are a lot like other business support systems.

Both run on computers, but beyond that, they are as different as psychiatry and neurosurgery. They require different approaches and expertise, even though both are medicine. Clinical computing is a subspecialty of computing very different than MIS or business computing.

The assumption that they are one and the same is largely responsible for HIT failures, amply documented on this site and others (e.g., here).

Mostrous' article did nothing, or only provided a sentence here and there, to represent the benefits of these systems, and the successes that have occurred.

On the contrary, it did quite a lot, finally breaking the 'code of silence' on HIT adverse unintended consequences in a major national publication.

Aside from the issue that some of the "benefits" of HIT may be in fact related to other variables that occur over time surrounding an installation, not to the IT itself, here you appear to be demonstrating an ethical lapse.

I have long written that health IT can achieve the benefits claimed for it, with the caveat that only when done well.

However, if benefits come at the cost of avoidable (and worse, unquantified) harm to some patients, then that is a devil's bargain, ethically speaking. The evidence that this occurs is substantial but far more work is needed.

I can also add that absence of evidence in not evidence of absence regarding unintended consequences (UC’s) of health IT, events none of us ever want to see. UC’s need to be eliminated.

We need to know the extent of UC's before we roll out a national system, ranging from, on one extreme, the possibility that unintended consequences never occur, to the opposite possibility that they occur in hair raising numbers but are not reported on, due to what really are social reasons, or somewhere in between. I do not believe we know the true rates of UC’s with any certainty in 2009.

I think the HIT industry can learn from the pharma industry, though. Per observations of my blog colleague Roy Poses on a new study in the Archives of Internal Medicine that focused on how articles report adverse effects found by clinical trials. The authors concluded, “the reporting of harm remains inadequate.”

See the whole piece, and an accompanying editorial commentary by J.P. Ioannidis at this link.

I think there are lessons to be learned from this regarding IT.

I for one, really appreciate that my physician has an EMR, and does all of his data entry directly, while he is conducting the exam.

The "I, for one" is a major problem in your logic. Many others others resent it, and some researchers find the distractions deleterious to good medicine.

A person should not base their opinions on a very large social re-engineering project based on their own personal experience with one physician. See, for example, this article (PDF) on the unintended deleterious effects clinical IT is having on medical education.

See my major site on these issues here.

InformaticsMD said...

I can also note that most of the points I raise in my comment above are in my blog posting to which these comments are attached.

I would ask commenters to read my posts first, and then respond to specific points I raise.

Allscripts said...

but my point that EMR improve clinicians´ ability to help patients manage treatment of complex diseases, including MDR-TB. Due to long treatment and complex drug regimens, MDR-TB is a difficult disease to monitor. With an EMR, treatment adherence and follow-up, as well as changes in medications and drug forecasting are made possible.

InformaticsMD said...

but my point that EMR improve clinicians´ ability to help patients manage treatment of complex diseases

I'm not sure who the "my" refers to.

I still don't think my post has been read.

Jonathan Paul said...

You may be surprised at what an EMR can do for reducing liability. Improving patient documentation, audit trails, and accuracy would not only reduce incidents of medical errors, but also improve your chances of receiving discounts from liability insurers. To participate in a P4P program, you will need to track and measure your care, and monitor your efficiency of delivering quality care at the best cost. You must also document the patients’ experiences using post-exam surveys. Most EMRs are capable of meeting these requirements while simplifying the process. They provide context-sensitive information during the examination and alert physicians to next-step treatment options or additional preventative care diagnosis.
For more information, visit at: Medical negligence.