I occasionally elevate comments and replies to the level of full posts if I feel they better illustrate and clarify significant points I raise.
In my Jan. 9, 2010 post "Does the CEO of Google Use Google? - And: Platform, Platform, Who's Got The Platform?" I lamented the intrusion of ill-informed, reductionistic, "database platform"-centric views of non-clinical IT personnel into healthcare.
I received the following feedback from an anonymous commenter:
Anonymous apparently meant these article references to exemplify the coming Age of Cybernetic Miracles in medicine.
My response to this anonymous feedback was placed in the comment thread of the "Does the CEO use Google" post and covered a number of bases.
I've reproduced my response, and added additional explanatory notes not in my original response in bold red italics below:
I should also add that the second article referenced by "Anonymous" refers to "obscure conditions such as Cushing's syndrome." I'm not exactly sure to whom Cushing's disease is an obscure disease. A non-medical IT person, perhaps?
Now, back to reality. Computers serve as aids to clinicians, when the IT is "done well." However, there is no substitute -- except in sci fi -- for expertise.
Medical expertise can only come from ~4 years of premed and then 4+ years of hard medical study in a wide variety of preclinical and clinical sciences leading to the M.D. degree, 3-4 years of postgraduate residency, and often 2+ additional years of postdoctoral fellowship training beyond residency for specialists (it should be noted, unfortunately, that there are no additional medical degrees beyond M.D., although residency and fellowship training often makes pursuing a Ph.D. in most fields seem like a cakewalk). Add to all that years of additional clinical experience in actual practice.
In my Jan. 9, 2010 post "Does the CEO of Google Use Google? - And: Platform, Platform, Who's Got The Platform?" I lamented the intrusion of ill-informed, reductionistic, "database platform"-centric views of non-clinical IT personnel into healthcare.
I received the following feedback from an anonymous commenter:
Anonymous wrote:
Or then again, there's this:
"Non-physicians may reach correct diagnoses by using Google: a pilot study."
http://www.ncbi.nlm.nih.gov/pubmed/19130327
"Researchers found that almost six-in-10 difficult cases can be solved by using the world wide web as a diagnostic aid."
http://www.dailymail.co.uk/news/article-415562/Doctors-using-Google-diagnose-illnesses.html
I think this post [i.e., my HC Renewal post about Google's CEO and "platforms" - ed.] misses the larger point that Gawande and Schmidt were addressing (and to some extent agree on) - the "art" of the practice of medicine must be transformed to the science of the practice of medicine. And today, science and IT are necessarily intertwined in practical terms.
The future will no doubt involve ever more sophisticated and useful clinical DSS. Historically speaking, invoking the complexity argument is fraught with issue as technological advances provide better and better ways to represent and manage it.
January 16, 2010 10:03:00 AM EST
Anonymous apparently meant these article references to exemplify the coming Age of Cybernetic Miracles in medicine.
My response to this anonymous feedback was placed in the comment thread of the "Does the CEO use Google" post and covered a number of bases.
I've reproduced my response, and added additional explanatory notes not in my original response in bold red italics below:
MedInformaticsMD wrote:
Re: Anonymous Jan 16 @ 10:03 AM-
The article [the first, at nlm- ed.] you cite reports the following:
Non-physicians may reach correct diagnoses by using Google: a pilot study.
OBJECTIVE: We endeavoured to determine whether individuals who are not physicians are likely to arrive at correct diagnoses [note the stated objective carefully -- "are likely to arrive at correct diagnoses" - ed.] by using Internet resources.
METHODS: In this prospective study four non-physicians used Google to search for diagnoses. They reviewed the 26 diagnostic cases presented in the case records of the New England Journal of Medicine during 2005; they were blind to the correct diagnoses. The main measurement was the percentage of correct diagnoses arrived at by non-physicians by using Google. The diagnostic success of the four non-physicians was compared to that of four young physicians.
RESULTS: The average diagnostic success of non-physicians was 22.1% (95% confidence interval [CI] 4.5-39.7%). There was no statistically significant difference between the non-physicians regarding this outcome (p = 0.11). They took 8.9 +/- 6.7 (mean +/- standard deviation) minutes for case record reading and 17.4 +/- 7.9 minutes for Google searching per case. Non-physicians performed worse than physicians (50.9% [95% CI 37.4-64.5%]) in regard to diagnostic success (p <0.001).
CONCLUSION: Non-physicians, at least those who have similar characteristics to the participants in the present study, may occasionally reach correct diagnoses by performing a brief web-based search. Doctors should realise that patients may assume a more active role in their health decision-making process and take this development into consideration in physician-patient interaction.
--------------------------
This article suffers from what can be termed "the syndrome of useless information."
[It is, in fact, a failure or negative finding with regard to its stated objective - ed.]
Let's see:
The average diagnostic success of non-physicians was 22.1%
They omit the opposite semantic: that the google-armed nonphysicians got more than three quarters of the diagnoses WRONG. Will you trust your grandmother to them?
Non-physicians performed worse than [young, a.k.a. inexperienced] physicians (50.9% [95% CI 37.4-64.5%]) in regard to diagnostic success (p <0.001).
Young, inexperienced physicians (trainees or residents?) arriving at 51% of the diagnoses correctly using ONLY google (not, for example, Harrison's, and other standard medical texts that would not be highly usable to most non-physicians lacking background to fully understand it) is not unexpected.
What is surprising is that the article omitted an essential control group: *experienced* physicians. It's not that they're hard to find.
The article concludes:
Non-physicians, at least those who have similar characteristics to the participants in the present study, may occasionally reach correct diagnoses by performing a brief web-based search.
Occasionally reach correct diagnoses? How about "most often reach incorrect diagnoses?" Cf. a broken clock is still occasionally correct.
[I repeat, the article is, in fact, a failure or negative finding with regard to its stated objective of "determining whether individuals who are not physicians are likely to arrive at correct diagnoses." They are in fact unlikely to get it right, with only a 22% hit rate, although the authors appear to have de-emphasized that fact. This will only make physicians' work harder as they "take this development into consideration", i.e., deal with patients armed with search engine-gleaned misdiagnoses - ed.]
I do not view this article as revealing anything of practical value other than perhaps the dangers of allowing non-physicians armed with search engines to think they can perform medical diagnosis to any meaningful extent.
Anonymous wrote: the "art" of the practice of medicine must be transformed to the science of the practice of medicine. And today, science and IT are necessarily intertwined in practical terms.
Reductionist views of non-medical IT personnel spouting off about 'database platforms' will not advance the science or the art of medicine.
Also, yes, IT is a tool of science; however, it and its designers and implementers facilitate science; the enablers of science are: scientists, using their insight, creativity, ingenuity, experience and expertise.
Concerning the second article [at dailymail.co.uk - ed.], I note:
"But they [the authors] stress the efficiency of the search and the usefulness of the retrieved information depend on the searchers' knowledge base." [The 'searchers' in this case were apparently - experienced physicians - ed.]
In other words, search engines can facilitate experts. That is not exactly new knowledge.
The 'art' of medicine, i.e., judgment, likewise is both indispensible, and irreproducible via "database platforms." Perhaps one day with advances in cognitive computing we will get to that point, but at present we can't even do as well as a cat. Note the statement from the IBM P.I. that "there are no computers that can even remotely approach the remarkable feats the mind perform."
Until we get there, I think it's not unreasonable to hold off on non-clinicians touting reductionist information retrieval-centric views.
Anonymous wrote: Historically speaking, invoking the complexity argument is fraught with issue as technological advances provide better and better ways to represent and manage it.
You must not have read my original post. It is advances in information science (i.e., in informatics) that will provide those advances, not advances in [information] technology. IT is a tool; information science is an activity of the mind.
Your statements clearly demonstrate a conflation of information technology and information science.
Computers facilitate information science research, but they are certainly not its sine qua non.
I should also add that the second article referenced by "Anonymous" refers to "obscure conditions such as Cushing's syndrome." I'm not exactly sure to whom Cushing's disease is an obscure disease. A non-medical IT person, perhaps?
Now, back to reality. Computers serve as aids to clinicians, when the IT is "done well." However, there is no substitute -- except in sci fi -- for expertise.
Medical expertise can only come from ~4 years of premed and then 4+ years of hard medical study in a wide variety of preclinical and clinical sciences leading to the M.D. degree, 3-4 years of postgraduate residency, and often 2+ additional years of postdoctoral fellowship training beyond residency for specialists (it should be noted, unfortunately, that there are no additional medical degrees beyond M.D., although residency and fellowship training often makes pursuing a Ph.D. in most fields seem like a cakewalk). Add to all that years of additional clinical experience in actual practice.
I am quite disturbed that this study and experience has become so casually regarded outside of medicine, for example in the IT sector. I think it reflects poorly on the IT culture specifically, and on our current culture as a whole.
Medical judgment is borne of that study and experience, not of healthcare computer dabbling typical of the business IT/MIS world in hospitals, and the all-too-often technical bachelor's degrees held by leaders in that domain. Google CEO Schmidt at least is a true computer scientist with a doctorate in CS; that does not, however, qualify him to comment on medical and medical informatics-related issues as in my prior post.
Stated frankly: he and many others with similar views are venturing far outside their competencies and as a result are talking nonsensically.
Incidentally, I believe it's time to move on from continually responding to the non-medical, IT-personnel proffered platformorrhea on how IT databases, decision support, artificial intelligence etc. will transform medicine from an art to a science, revolutionize medicine, and similar positions trafficked for at least several decades now.
They are largely manifestations of the Syndrome of Inappropriate Overconfidence in Computing (SICC syndrome), a term I coined in the 1990's and described is some detail here in a post entitled "A 21st Century Plague? The Syndrome of Inappropriate Over-Confidence in Computing." At that post I stated that:
In the future, similar comments as those from "Anonymous" will simply get referred back to the post you are now reading and the bolded one on SICC above.
Stated frankly: he and many others with similar views are venturing far outside their competencies and as a result are talking nonsensically.
Incidentally, I believe it's time to move on from continually responding to the non-medical, IT-personnel proffered platformorrhea on how IT databases, decision support, artificial intelligence etc. will transform medicine from an art to a science, revolutionize medicine, and similar positions trafficked for at least several decades now.
They are largely manifestations of the Syndrome of Inappropriate Overconfidence in Computing (SICC syndrome), a term I coined in the 1990's and described is some detail here in a post entitled "A 21st Century Plague? The Syndrome of Inappropriate Over-Confidence in Computing." At that post I stated that:
I, for one, would welcome a cessation of claims that IT will "revolutionize" any field that depends primarily on cognition, such as biomedicine, and a return to more temperate attitudes instead of the almost bellicose grandiosity about HIT we see today. That is to say, that HIT - with proper contributions from the aforementioned specialties [e.g., social science, social informatics, biomedical informatics, HCI, etc.] will facilitate better health care, not "revolutionize" it.
In the future, similar comments as those from "Anonymous" will simply get referred back to the post you are now reading and the bolded one on SICC above.
Finally, I will not hold my breath for Emergency Medical Hologram Mark I, or R2D2 and C3PO to appear anytime soon.
-- SS
Every single one has an ideal attached to it, in which every single individual gets accessible health care whenever they need it at an affordable rate.
ReplyDeleteEvery single one has an ideal attached to it
ReplyDeleteThe subject of your sentence in unclear. Every single one of ... what?
As the original "Anonymous" poster of the comment now elevated to its very own illustrious (and presumably illustrative) post, I suppose I should be flattered to have participated in demonstrating the textbook case of the Syndrome of Inappropriate Over-Confidence in Computing.
ReplyDeleteI'll submit that perhaps my crude examples of physician and non-physician interaction with Google may have been taken a bit more literally than I intended. No one I know is suggesting a substrate neutral mathematical modeling technology like Pagerank is a substitute for cognition and training. It IS interesting that the evidence I cited suggests physicians produce better diagnoses with the help of a tool like Google (which to me has the same tenor as the point Schmidt was making in the PCAST meeting).
In practice, I don't disagree with much of your assessment of historical HIT and I understand the perspective you bring. I have lived through many an "IT led" HIT experiment that inevitably ends as expensive disaster to any honest assessor.
That said, from my vantage point, the HIT industry and buyers of HIT products recognized the shortcomings of IT driven projects after a few rounds of expensive industry failures in the late 90s and early 2000s. Since then, you've seen the emergence of the predominantly physician CMIO / Clinical Transformation role which purportedly elevates the role of informatics and the reality of clinical practice in the design and development of HIT products. This, in my opinion, has changed the class but not the rate of failure in HIT software.
And this brings me to the exception I take to your perspective. Across the breadth of your writing, I perceive a tendency to blame IT for all HIT product failures of every kind and preach informatics as its salvation. I have yet to see the magic wand of medical informatics produce the kind of results you suggest are possible. In practice, informaticists haven't solved basic problems with core elements of the profession - controlled medical vocabulary usefulness, meaningful extra-organizational data interoperability, practical use of evidence-based order sets, elimination of unstructured data in medical records, just to name a few. Worse yet, informaticists generally have little understanding of good software architecture practices - resulting in incoherently jumbled inflexible amalgams of "platforms" (a term I agree should be used with derision).
In my view, deploying HIT to advance the practice of medicine requires cooperative involvement by both technologists and informaticists. I agree that informatics should provide the leadership in the organization and use of medical information. However, maximizing value of information requires organized frameworks that anticipate change without compromising structural integrity. I would submit that computer science is best capable of designing this foundational soundness.
There will not simply be an information scientist painting the canvas of the future with the brush of technology. Designing the right brush (or more correctly, arsenal of brushes) to execute the vision requires a partnership, communication, and mutual respect. That is the message I feel often gets lost in perpetually overheated rhetoric in this discussion.
Anonymous @ Feb 4 3:19:00 PM EST wrote:
ReplyDeleteAs the original "Anonymous" poster of the comment now elevated to its very own illustrious (and presumably illustrative) post
Since this and your current comment under the moniker "n8ideas" is also anonymous, that is not verifiable.
That said, from my vantage point, the HIT industry and buyers of HIT products recognized the shortcomings of IT driven projects after a few rounds of expensive industry failures in the late 90s and early 2000s. Since then, you've seen the emergence of the predominantly physician CMIO / Clinical Transformation role
No and no. You've missed my many comments about Director of Nothing and Chief of Nothing roles.
And this brings me to the exception I take to your perspective. Across the breadth of your writing, I perceive a tendency to blame IT for all HIT product failures of every kind and preach informatics as its salvation.
IT personnel without healthcare backgrounds have little role in clinical settings and in clinical affairs, except with very strong supervision by clinicians. IT pesonnel's lack of experience dooms HIT if their lack of experience and the wrong decisions that result (which can be decribed precisely under the term 'mismanagement') are not tempered and controlled. Medicine is not something you pick up in a "Learn Diagnosis in 24 Hours" book. Worst of all, IT personnel lacking medical backgrounds should not have leadership authority over HIT projects.
HHS now accepts this concept as per the recent announcements from ONC regarding the need for healthcare IT education in the IT workforce.
I have yet to see the magic wand of medical informatics produce the kind of results you suggest are possible
You are not the world. I *have* seen it.
Worse yet, informaticists generally have little understanding of good software architecture practices
I'm not sure which informaticists you refer to, and why you feel an informatics specialist leading a clinical IT project should not delegate this issue to IT personnel reporting to them, who have that experience if the informatics specialist lacks it. Further, your comment itself seem to reflect the very problem I am writing about - a technology/designer-centric rather than a user/work-centric view of HIT. The problem with HIT currently has little to do with software architecture and platforms. It has to do with the enduser experience, which is all too often mission hostile. The clinician doesn't care what software architecure is used to put that 20 diagonal inches in front of his or her nose.
n my view, deploying HIT to advance the practice of medicine requires cooperative involvement by both technologists and informaticists. I agree that informatics should provide the leadership in the organization and use of medical information.
Perhaps all the above back-and-forth was unnecessary, since I wholeheartedly agree with the solution.
The devil, however, is in the details with regards to organizational structures that allow true leadership to occur (e.g., control of resources and budget, direct reporting relationships, hire and dismissal authority, etc.) "Internal consultant" is not leadership, and IT personnel of all people know this well. As a "CMIO" I controlled a budget of zero, not even able to buy lunch for my meetings; had but one direct report, and was ignored repeatedly. Many of my colleagues in both hospitals and vendor shops report similar arrangements even in 2010. (As opposed to pharma. Soon after my CMIO role in an R&D IT and information science support role I was put in charge of ~12 direct reports, total ~55 reports, GOA of $100K, a budget of $13M and hiring authority. The contrast with the IT backwaters of hospitals was as stark as could be.)
-- continued below due to length limits --
-- continued from above --
ReplyDeleteAlso, unfortunately, the culture gap between clinicians and IT (e.g, IT's culture of control), and the wildly asymmetric responsibilities, obligations and liabilities of clinicians compared to IT personnel make finding the appropriate balance quite difficult.
That is the type of issue that can only be settled by admitting there is a territorial problem, and the need for that admission needs to come primarily from the IT side. Physicians have a job to do - taking care of patients - and shouldn't have to defend their territory to non-clinician IT facilitators. Period.
I would submit that computer science is best capable of designing this foundational soundness.
Again, agree, but show me a hospital with IT personnel bearing MS-CS or PhD-CS degrees in leadership roles (as opposed to MIS, an entirely different profession), and I'll show you a hen's many teeth.
There will not simply be an information scientist painting the canvas of the future with the brush of technology
No, but that's where the solutions to medicine's vexing informational problems must originate - from those who understand biomedical information science, Bayesian and other forms of medical reasoning, optimal HCI for clinical settings, and a host of other primary issues.
Finally, I remind readers of a paragraph from, of all places, a 1981 Radio Shack TRS-80 catalog (page 2 at this link):
"What is a Computer?"
... A computer is an extension of the human brain. It is to the mind what the lever is to the arm - a machine capable of multiplying effectiveness. It can free you from tedious, repetitive work which does not require judgment. IT can provide facts and figures with lightning speed, giving you more time to exercise your judgment thoughtfully.
These Texans understood computing well. These words are just as valid today as three decades ago.
I believe the understanding of the issue that computers free professionals from tedium, but only so that they may more freely exercise judgment, has been lost in the omnipotent marketing hype that's taken over the IT world.
That is what I refer to as the "syndrome of inappropriate overconfidence in computing" - a belief that expert judgment, arrived at after many years of grueling training and experience, can be duplicated or improved upon under real world conditions at our current understanding of information science (computers, of course, being simply a convenient vessel or vehicle for bringing forth information science to the applied setting).
-- SS