Thursday, March 15, 2012

EHRs and test ordering: Health Affairs authors reply to ONC

At my March 9, 2012 post "Increased Lab Ordering with EHR's?" I refuted ONC's response to a Harvard-based research study "Giving Office-Based Physicians Electronic Access To Patients’ Prior Imaging And Lab Results Did Not Deter Ordering Of Tests" that may contradict the notion that HIT reduces medical costs.

Now the authors themselves have responded to ONC. There are many similar themes in their response. I've bolded them below:

The Effect Of Physicians’ Electronic Access To Tests: A Response To Farzad Mostashari

March 12th, 2012

by Danny McCormick, David Bor, Stephanie Woolhandler, and David Himmelstein

Our recent Health Affairs article linking increased test ordering to electronic access to results has elicited heated responses, including a blog post by Farzad Mostashari, National Coordinator for Health IT. Some of the assertions in his blog post are mistaken. Some take us to task for claims we never made, or for studying only some of the myriad issues relevant to medical computing. And many reflect wishful thinking regarding health IT; an acceptance of deeply flawed evidence of its benefit, and skepticism about solid data that leads to unwelcome conclusions.
Dr. Mostashari’s critique of our paper, will, we hope, open a fruitful dialogue. We trust that in the interest of fairness he will direct readers to our response on his agency’s site. [If not, HC Renewal posts get highly ranked by Google - ed.]
Our study analyzed government survey data on a nationally representative sample of 28,741 patient visits to 1187 office-based physicians. We found that electronic access to computerized imaging results (either the report or the actual image) was associated with a 40% -70% increase in imaging tests, including sharp increases in expensive tests like MRIs and CT scans; the findings for blood tests were similar. Although the survey did not collect data on payments for the tests, it’s hard to imagine how a 40% to 70% increase in testing could fail to increase imaging costs.
Dr. Mostashari’s statement that “reducing test orders is not the way that health IT is meant to reduce costs” is surprising, and contradicts statements by his predecessor as National Coordinator that electronic access to a previous CT scan helped him to avoid ordering a duplicate and “saved a whole bunch of money.” A Rand study, widely cited by health IT advocates including President Obama, estimated that health IT would save $6.6 billion annually on outpatient imaging and lab testing. Another frequently quoted estimate of HIT-based savings projected annual cost reductions of $8.3 billion on imaging and $8.1 billion on lab testing.
We focused on electronic access to results because the common understanding of how health IT might decrease test ordering is that it would facilitate retrieval of previous results, avoiding duplicate tests. Indeed, it’s clear from the extensive press coverage that our study was seen as contravening this “conventional wisdom”.
Nonetheless, Dr. Mostashari criticizes us for analyzing the impact of physicians’ electronic access to imaging and test results, but not other aspects of electronic health record (EHR) use. We did, however, analyze the relationship of EHRs to test ordering in a subsidiary analysis. While physicians use of a full EHR was associated with a 19% increase in image ordering, as we noted in the paper this finding was not statistically significant. While we cautiously (and properly) interpreted this as a “null” finding, these data are inconsistent with Mostashari’s optimistic view that use of a full EHR reduces costs.
He asserts that our 2008 data are passe, and that health IT meeting today’s “meaningful use” criteria definitely saves money. The data we analyzed were the latest available data when we initiated the study. While the proportion of outpatient physicians utilizing health IT has grown since 2008, we are unaware of any “game changing” health IT developments in the past four years that are would produce substantially different results if the study were repeated today. The EHR vendors that dominated the market in 2008 remain, by and large, today’s market leaders, and their products have undergone mostly modest tweaks. Mostashari’s contention that 2012 EHRs – incorporating decision support and electronic information exchange – save money in ways not possible in 2008 should be tested through additional research but remains merely a hypothesis. We hope that some day his predicted savings can be achieved.
Dr. Mostashari offers his own explanation for our findings, suggesting that doctors who are inclined to order more tests are also inclined to purchase health IT for viewing test results electronically rather than on paper. He offers no evidence for this assertion and ignores the fact that we explored (and rejected) this explanation by analyzing subgroups of doctors who are unlikely to be the decision maker for IT purchases – e.g. employed physicians, those working in an HMO setting etc. In other words, electronic access to results predicted more test ordering whether or not the ordering physician was responsible for health IT purchases.
He incorrectly states that our analysis did not take into account patients’ severity of illness, physicians’ level of training, and the nature of physicians’ financial arrangements. In fact, we reported subsidiary multivariate analyses that included several serious diagnoses; all of our models included physician specialty (which we specified in several different ways); and all models included adjustment for an extensive list of indicators of financial arrangements (e.g. whether the physician owned the practice or was an employee; the type of office; whether the practice was owned by a hospital; whether the physician was a solo practitioner; whether the physician’s compensation was based, in part or whole on “profiling”; and whether the practice was predominantly prepaid). We also performed a series of subsidiary analyses that explored whether physicians with a proclivity to “self refer” patients for imaging tests accounted for our finding; they didn’t.
Dr. Mostashari criticizes us for failing to assess whether health IT improved the quality or appropriateness of care. Of course, these were not the topic of our research. Those are different studies for a different time. However, we would note that other large-scale studies have found no, or trivial quality improvements associated with HIT outside of a few flagship institutions4-6.
Dr. Mostashari’s strongest claim is that observational studies like ours (and most other health policy studies, including some by Dr. Mostashari himself) cannot prove causation. This is surely true. As long time teachers of evidence based medicine we took care to couch our conclusions in cautious terms, stating only that “Computerization, whatever its other benefits, remains unproven as a cost control strategy.”
But Dr. Motashari is less cautious, asserting that the case for HIT is closed. The paper he cites to buttress this claim (authored by members of his own agency) culled studies reporting any impact of HIT on virtually any aspect of care, and accepted authors’ claims of benefit without regard to study quality or statistical niceties. Thus, a focus group’s impressions of benefit are accorded the same weight as nationwide studies of Medicare data showing virtually no impact of computerization on quality measures. Reports of a reduction from 70% to 38 % in “missed billing opportunities” or a $7,000 reduction in office supply costs are among the 92% of studies judged “positive”. While the literature review he cites is interesting, nothing in it contradicts our findings.
Dr. Mostashari is also correct in reiterating that randomized trials are the best way to assess health IT. In fact, no randomized trial has ever been published that examines patients’ outcomes or costs associated with off-the-shelf health IT systems that dominate the U.S. market. No drug or new medical device could pass FDA review based on such thin evidence as we have on health IT. Yet his agency is disbursing $19 billion in federal funds to stimulate the adoption of this inadequately evaluated technology. Dr. Mostashari is perhaps the only person in our nation who commands the resources needed to mount a well done randomized controlled trial to fairly assess the impact of health IT, and the comparative efficacy of the various EHR options.
Finally, Dr. Mostashari’s unbridled faith in technology is mirrored by his belief that ACOs are the next panacea for health costs and quality. That health policy flavor-of-the-month also remains wholly unproven.

I think readers comparing my response to the authors', and who are familiar with the many posts at HC Renewal going back to 2004, will recognize the themes I have bolded in the author's response to HHS.

-- SS


Afraid said...

Um, their response is a masterpiece.

Not that I don't like yours as well Scot, but um, a masterpiece theirs is none the less.

InformaticsMD said...

their response is a masterpiece

Agreed. More academics need to take a stand for rigorous science in health IT.

We need true evidence-based computing before computing can "transform" medicine.

Anonymous said...

The evidence supporting HIT devices is indeed thin. That this experiment has been allowed to continue is not acceptable.

The HHS has shirked its responsibility. The FDA has not enforced the Federal Food Drug and Cosmetic Act.

The propaganda from the HIT industry and financially conflicted leaders enable the scandal to continue.

Anonymous said...

Healthcare Business News

Mostashari is right on

Posted: March 16, 2012

Regarding "EHR study authors slam Mostashari response":

I completely agree with Dr. Mostashari. The only "deeply flawed evidence" is what the researchers used to come to their study conclusions. To say that electronic health-record systems vendors' "products have undergone mostly modest tweaks" since 2008 is incredibly naive and shortsighted.

EHRs have changed dramatically in the past four years, and many still have much to do to achieve meaningful use. If people cannot see the positive and deep changes in EHRs over the past four years as a result of the development of the patient-centered medical home and achieving meaningful use, then they are not paying attention to what is happening in the real world. And there's more to come! When hospitals, primary-care physicians and specialists start communicating with each other through their EHRs, the savings will be huge. And that movement is well on its way. Keep up the good fight, Farzad. Your leadership is changing health IT in a strongly positive way.

Dr. Al Puerini
Rhode Island Primary Care Physicians Corp

Read more: Mostashari is right on - Healthcare business news and research | Modern Healthcare

InformaticsMD said...

Dear Dr. Puerini,

Thank you for your comments.

You state:

EHRs have changed dramatically in the past four years...When hospitals, primary-care physicians and specialists start communicating with each other through their EHRs, the savings will be huge."

"Changed dramatically" is a strong assertion. "Will be" is even stronger, a definitive prediction of the future.

I would be quite interested in references to back up these assertions. Significant literature seems to conflict with your assessments.

See for example the articles and documents aggregated at this link.

Also see my essays on the Ddulite disposition at this query link: if you have not already.


S. Silverstein, MD

Anonymous said...

I took interest in Dr Purini's unsubstantiated statements. I see many changes caused by EHRs and few if any have improved patient care.

They waste a lot of professional time. Conversations are vapid. All I hear at nurses' stations now is stuff like how do I enter the stool volume in the EHR and the like. It used to be, Dr. X, please help me with this arrhythmia.

Delays are routine and are widespread, outages are common causing much scurrying of the staff like chickens with heads cut off, and the deaths go unreported as being due to such infrastructure failures.

Additionally, the systems are not secured, they violate patient privacy, and there are no proven benefits it outcomes.

Anonymous said...

To the PCP CEO,

I am not aware of any material improvements in EHRs and CPOE in the past five years. The devices remain as user unfriendly as ever. The fact remains that these devices have not been subjected to safety and efficacy evaluations and the sea chit certification ignores safety.

I have always thought that PCPs would like these devices since they seem to be more manageable when the patient only has a few simple problems. The PCPs, being as busy as they are also like the fact that paraprofessionals can use them and sign the attending to maximize payments, and then again, there are the four page progress notes that are meaningfully useless to the clinician, but bring in big bucks from the payees.

InformaticsMD said...

Anonymous March 16, 2012 10:40:00 PM EDT writes:

To the PCP CEO, I am not aware of any material improvements in EHRs and CPOE in the past five years. The devices remain as user unfriendly as ever. The fact remains that these devices have not been subjected to safety and efficacy evaluations

The PCP CEO has had adequate time to respond to my questions.

No reply has been received to date.

-- SS

Kavinace said...

Though our nation's economy has recently lost millions of jobs, the health care industry has continued to add them. Not surprisingly, unions are eager to sign up health care workers. In the last 10 years, the rate of union wins in the health care industry has grown faster than the national average. Unions are uniting to lobby for labor-friendly legislation to promote increased union membership in the health care sector.