"Why Most Published Research Findings Are False", John P. A. Ioannidis, PLoS medicine, 2005 August; 2(8): e124
He wrote:
There is increasing concern that most current published research findings are false. The probability that a research claim is true may depend on study power and bias, the number of other studies on the same question, and, importantly, the ratio of true to no relationships among the relationships probed in each scientific field. In this framework, a research finding is less likely to be true when the studies conducted in a field are smaller; when effect sizes are smaller; when there is a greater number and lesser preselection of tested relationships; where there is greater flexibility in designs, definitions, outcomes, and analytical modes; when there is greater financial and other interest and prejudice; and when more teams are involved in a scientific field in chase of statistical significance. Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true. Moreover, for many current scientific fields, claimed research findings may often be simply accurate measures of the prevailing bias.
In other words, in the all-too-common insufficiently powered studies, and even seemingly robust studies in domains with small effect sizes, financial interests, prejudices and other factors more often than not produce false results.
Ioannidis and co-authors recently took their sword to "underperforming Big Ideas in research" (including the "miracles" touted by hyper-enthusiasts such as in genomics and in cybernetics), via a new JAMA viewpoint piece:
What Happens When Underperforming Big Ideas in Research Become Entrenched?
Michael J. Joyner, MD1; Nigel Paneth, MD, MPH2; John P. A. Ioannidis, MD, DSc3
JAMA. Published online July 28, 2016. doi:10.1001/jama.2016.11076
http://jama.jamanetwork.com/article.aspx?articleid=2541515
For several decades now the biomedical research community has pursued a narrative positing that a combination of ever-deeper knowledge of subcellular biology, especially genetics, coupled with information technology will lead to transformative improvements in health care and human health. In this Viewpoint, we provide evidence for the extraordinary dominance of this narrative in biomedical funding and journal publications; discuss several prominent themes embedded in the narrative to show that this approach has largely failed; and propose a wholesale reevaluation of the way forward in biomedical research.
The key word is "narrative." As per Hayek, those with little real-world operational experience, i.e., intellectuals and academics, often the uncritical cheerleaders for electronic records despite considerable downsides, have only the "narrative" upon which they base their beliefs in healthcare IT exceptionalism:
It is perhaps the most characteristic feature of the intellectual that he judges new ideas not by their specific merits but by the readiness with which they fit into his general conceptions, into the picture of the world which he regards as modern or advanced. . . . As he knows little about particular issues, his criterion must be consistency with his other views and suitability for combining into a coherent picture of the world. . . . It is the intellectuals in this sense who decide what views and opinions are to reach us, which facts are important enough to be told to us, and in what form and from what angle they are to be presented. Whether we shall ever learn of the results of the work of the expert and the original thinker depends mainly on their decision.
(I can add that blogs have to some small degree ameloriated "whether we shall ever learn of the results of the work of the expert and the original thinker", but only to a small degree.)
The "general conception" in cybernetics is that computers are a silver bullet in any domain, and can only result in massive improvements.
My experience for the past twenty+ years in the Electronic Medical Records/clinical information technology domain, where quality, safety, usability, confidentiality, and other critical real-world issues have been ignored in favor of EHR hyper-enthusiasm, supports Hayek's observations regarding prevalent unfettered beliefs in healthcare IT exceptionalism.
Ioannidis et al. state the factual situation with EHR technology unapologetically, clearly and succinctly:
... The financial and clinical benefits predicted from shifting to EHRs have also largely failed to materialize because of difficulties in interoperability, poor quality, and accuracy of the collected information; cost overruns associated with installation and operation of EHRs at many institutions; and ongoing privacy and security concerns that further increase operational costs.
I would change "interoperability" to "operability." Otherwise, they're quite correct. For example, the "Big Data" hyper-enthusiasts quite irrationally believe data from these systems - as they are today - will somehow "revolutionize" medicine, while at the very same time the IT industry itself and its pundits ignore fundamental precepts of computer science, information science, biomedical informatics, biomedicine and biomedical research itself.
Some of the hyper-enthusiasts have made predictions that are astonishingly naive, delusionally grandiose and just plain perverse, e.g., see for instance my Jan. 2014 post "Computers + a few docs can manage 'an entire city', and other cybernetic miracles" at http://hcrenewal.blogspot.com/2014/01/computers-few-docs-can-manage-entire.html .
The new JAMA paper continues:
... These features make the use of EHRs for research into the origins of disease, as proposed in the Precision Medicine Initiative, highly problematic.No clearly specified targets for either improved outcomes or reduced costs have been developed to assess the performance efficiency of EHRs.
Those targets were never specified, but The Market seems to have corrected for that, e.g., via this Jan. 2015 letter from ~40 different medical societies:
 |
| Full letter to HHS available at http://mb.cision.com/Public/373/9710840/9053557230dbb768.pdf |
The authors continue:
... Although it is difficult to argue for a return to paper records, any claim of future transformation of the medical record should include well-defined accountability and review mechanisms. Otherwise, the health care system may become hostage, wasting increasing resources to continuously upgrade electronic technology without really helping patients.
It is clear to me that the health care system and its clinicians are already hostage to the cybernetic hyper-enthusiasts, as evidenced by letters such as the above and many other sources about the mayhem being caused, e.g., a small sampling on this blog at query links http://hcrenewal.blogspot.com/search/label/glitch, http://hcrenewal.blogspot.com/search/label/healthcare%20IT%20difficulties and http://hcrenewal.blogspot.com/search/label/medical%20record%20confidentiality.
Finally, I disagree with the authors that "it is difficult to argue for a return to paper records."
Paper has its proper place, and "paperless" is a utopian dream of the hyper-enthusiasts that causes significant damage to the primary role of clinicians - to take care of patients. I make this argument (with a real-world, highly successful example of my own creation) at my Aug. 9, 2016 post "More on uncoupling clinicians from EHR clerical oppression" at http://hcrenewal.blogspot.com/2016/08/more-on-uncoupling-clinicians-from-ehr_91.html.
In summary, the authors of this JAMA piece clearly and succinctly break through the "narrative" about hyper-enthusiast dominated fields, including clinical information technology and the belief in
healthcare IT exceptionalism.
Sadly, theirs is almost a single voice in a wilderness dominated by the hyper-enthusiasts - and the profiteers.
-- SS
No comments:
Post a Comment