Saturday, January 09, 2010

Does the CEO of Google Use Google? - And: Platform, Platform, Who's Got The Platform?

Over at the WSJ health blog, reporter Jacob Goldstein's Jan. 8, 2010 post "Google CEO & Harvard Surgeon Talk Health IT" quotes Google's CEO:

"Google’s CEO Eric Schmidt doesn’t know why docs haven’t embraced databases to help them sort through medical information."

[Schmidt said] ... So when you show up at the doctor with some set of symptoms, in my ideal world what would happen is that the doctor would type in the symptoms he or she also observes, and it would be matched against the data in this repository

[such as
DXplain? -- which we learn from a - er, um, Google search - was developed starting in the mid 1980's by medical informatics researchers who actually know this domain, and which offers this explanation and warning: "DXplain uses an interactive format to collect clinical information and makes use of a modified form of Bayesian logic to derive clinical interpretations ... DXplain does not offer definitive medical consultation and should not be used as a substitute for physician diagnostic decision making"? - ed.]

... Then this knowledge engine would use best practices, and all the knowledge in the world to give physicians some sort of standardized guidance. This is a generalized form of the checklists that you’re talking about …

As computer scientists, this is a platform database problem, and we do these very, very well, as a general rule. And it befuddles me why medicine hasn’t organized itself around these platform opportunities. [No - the successful practice of medicine is not a 'platform database' or any other reductionistic information retrieval problem - ed.]

I note that IT personnel like to refer to "platforms", "solutions" - a rather presumptuous term, "paradigms", and other buzzwords to mask the fact that what they're referring to are more commonly known as "hardware" and "software" and arrangements thereof.

Does this "platform opportunity" view reflect naïveté about the complexities of medicine and medical decision making, or does it reflect something else? Could this "befuddlement" be construed as calling physicians obtuse? Is this yet another example of what I referred to in my post "Healthcare IT Failure and The Arrogance of the IT Industry" and other writings as a cross-occupational invasion of medicine by IT?

IT personnel seem to have a propensity to offer healthcare-related opinions far outside their own areas of expertise -- or if in healthcare organizations, edicts - based upon the narrow view of their own relatively linear and deterministic fields. The risk is, especially when coming from high perches, that such opinions and edicts can result in deleterious actions (e.g., government initiatives).

In an absurdist reductio ad absurdum, deliberately made absurdist due to many years of exposure to equally absurd (to those with actual domain experience) "who needs medical school/residency/patient care experience to profess on medical matters?" attitudes:

Why don't physicians offer the advice that Google could improve its search algorithms, or Intel and AMD their microprocessors, by utilizing intelligent psittacine platforms as in this British Broadcasting Company (BBC) video narrated by a true technology expert, Captain James T. Kirk?


Kirk did have a (computerized) physician son: Nomad!

----------------

Quite seriously, physician reluctance to "embracing databases" and health IT in general is not about database platforms. I only wish it were so simple.

The WSJ seems to understand this. In a Jan. 12, 2009 article by reporter Bret Stephens entitled "Can Intelligence be Intelligent?", the observation is made that technology is a mere facilitator, and intelligent, well trained, experienced, critical-thinking people are the enablers of any complex field that requires human judgment. They must be unfettered by machine and bureaucrat:

... Terrifying as the thought may be to many of its current practitioners, the true art of intelligence requires, well, intelligence. That is a function neither of technology nor of "systems" [a.k.a. "platforms" - ed.], which begin as efforts to supplement and enhance the work of intelligence and typically wind up as substitutes for it. It is, instead, a matter of experience, intellect, initiative and judgment, nurtured within institutions that welcome gadflies in their midst.


I've left the following comment at the WSJ health blog:

Mr Schmidt,

If you’d like to learn more about why many physicians are reluctant to embrace clinical IT, you might also do a Google search on “healthcare IT failure and similar terms.


Need I say anything more about the irony of that advice?

I'd also noted a fixation on "platforms" as solutions to biomedical problems (best when they come in shrinkwrapped, off the shelf, "on the IT roadmap" packages!) in my June 2008 post "An Open Letter to Merck CEO Richard Clark on Merck's Mission to Rediscover the Wheel."

A nonmedical research IT leader, who'd found a move from basic research to clinical IT "quite an eye opening experience" (i.e., a domain in which she had little or no experience but was paradoxically appointed to lead) talked all about "platforms" in Bio-IT World:

... We've invested a lot in some core platforms; we need to start translating that into results in the clinic at some point. And so having people who have an understanding of what does that really take to help inform the earlier research directions, the platform directions [i.e., research direction = platform direction - ed.], is a key theme...We already have siloed platforms to show that data, we need to integrate it more than it is... combining the results data from clinical samples with the associated patient data, what's that platform?

Platform, platform, platform. Who's got the platform?

My comments to that CEO in my Open Letter were that this was the wrong mindset and question, based upon an IT person's focus on information technology. This is as opposed to a focus on information science and on facilitating people in interacting with data and information in order to gain actionable knowledge, i.e., an information science and human-computer interaction-based approach that those in medical informatics thought about long ago.

In line with the conclusions of Greenhalgh et al. [1] who called for "eschewing sanitized accounts of successful projects" and instead recommending studies of clinical IT in organizations that “tell it like it is” using the de-identified critical fiction technique, I'd written on how conflation of information technology and information science impaired R&D in pharma at my essay "Sure path to R&D failure: Conflation of IT with information science in the pharmaceutical industry."

That piece and the aforementioned Open Letter were written before Merck sold itself to Schering-Plough in a "reverse merger" due to the unsustainability of doing business from an empty wagon of new products, a sign of just how well this IT-centric "platformania" has been working out for R&D.

In the information science-centered view and approach, the "platform," a.k.a. computer technology, is merely a canvas and facilitator, the artist (clinician or scientist) and the brush wielded by them being the primary enabler of and contributor to the masterpiece.

Unfortunately, I don't think anyone is "home" in pharma or in the HIT sector anymore to parse these ideas; in fact I've only recently learned that the people I did work with who could parse these ideas into creative reality were laid off by the very IT people making such statements and asking such questions.

IT personnel perhaps need to move away from their reductionist platformania. (Perhaps they are confusing "platforms" with "pixie dust.") Rather, they need to start thinking in terms of facilitating clinicians and scientists through domain specific and individualized-to-need information science and HCI innovation that arises of true cross-disciplinary expertise.

They need to leave creation of cybernetic miracles to people such as Irwin Allen and George Lucas. And platforms to carpenters.

-- SS

[1] Tensions and Paradoxes in Electronic Patient Record Research: A Systematic Literature Review Using the Meta-narrative Method. Greenhalgh, Potts, Wong, Bark, Swinglehurst, University College London. Milbank Quarterly, Dec. 2009. Available at: http://eprints.ucl.ac.uk/18821/

10 comments:

Eric Milgram, Ph.D. said...

Very well said!

Jeff said...

This seems a bit harsh. I agree medicine is far too complex for current search algorithms to be of the kind of use Mr. Schmidt imagines. But that is not to say computers could not help the information overload problems that physicians are facing as medical knowledge rapidly expands. DXplain hasn't been very successful because it replaces the clinician, rather than providing him/her cognitive supports such as Dr. Gawande's checklists.

Anonymous said...

Google is going to search medical databases??? Look at the search results from Google. You get 10^8 results and most of those are irrelevant. The ones that show up on the first page are usually just ads.

If Eric Schmidt wants to talk about doing medicine perhaps he could create a search engine that gives relevant results first.

MedInformaticsMD said...

Jeff writes,

"This seems a bit harsh."

Reality is a hard master, as I'd written in a prior post.

I agree medicine is far too complex for current search algorithms to be of the kind of use Mr. Schmidt imagines.

That is my point, although an additional point is that the issue is not just developing the proper "search algorithms." That is a reductionist view.

Medical decisionmaking is not done through a mental search of a gray matter database alone. There is much cognitive processing going on that computers and expert systems cannot emulate well (in part because the processes are not well understood). Bayesian approaches to decision making are at best a crude approximation.

Medicine is indeed complex at many levels, not just at the level of the information model and knowledgebase size. This is why computers cannot replace physicians at present. They can play great chess, a very well defined domain of near limitless but constrained possibilities, but cannot write music at the level of a Beethoven or make deductions at the level of a Sherlock Holmes or good street cop. To assert that they could if only we had the right "database platform" is absurd, wouldn't you agree?

Statements such as "so when you show up at the doctor with some set of symptoms, in my ideal world what would happen is that the doctor would type in the symptoms he or she also observes, and it would be matched against the data in this repository ... Then this knowledge engine would use best practices, and all the knowledge in the world to give physicians some sort of standardized guidance" seem equally as absurd to clinicians.

But that is not to say computers could not help the information overload problems that physicians are facing as medical knowledge rapidly expands.

I agree with you. It's simply that those who conflate IT and IS will not succeed in building such tools. They're too involved with the IT issues, not with the IS, HCI and other real world issues that really matter. See the linked case study in pharma and this series on HIT user experience hostility for examples of how that instantiates itself in the lab and clinic.

By the way, try a big-city municipal ED at midnight on a Saturday night, after you've worked since 7 AM that morning, and will not be off until 5 PM the next day perhaps with no sleep.

Now, that's harsh.

-- SS

MedInformaticsMD said...

Anonymous @ Jan 10 3:48 PM wrote:

Google is going to search medical databases??? Look at the search results from Google.

It's pretty good at finding reasonably relevant "hits" that appear in the first few pages of search results to many queries, minus the noise; however, years ago I showed domain specific, human-based indexing to be superior to some very fancy indexing and search algorithms.

Unfortunately, there are simply not enough humans to index every web page and site from the perspective of multiple domains, with respect to the query types and subtypes (e.g., modifiers) that commonly occur in those domains.

Hence, one should not expect a cybernetic search esp. by a domain novice to approach the quality of a human expert-led search that is facilitated by computing. That is why in pharma I had a troupe of domain experts to perform comprehensive literature searches for R&D, and why the searches were often quite lengthy and subject to iterative, back and forth refinement.

This is also why my college has, in addition to information technology and information systems-related curricula, a good old fashioned Library Information Science (LIS) tract.

-- SS

MedInformaticsMD said...

Another thought occurs ... why is it that physicians are increasingly needing to explain the complexities of medicine to non medical IT personnel? There is a pathology in this.

-- SS

Anonymous said...

Or then again, there's this:

"Non-physicians may reach correct diagnoses by using Google: a pilot study."

http://www.ncbi.nlm.nih.gov/pubmed/19130327

"Researchers found that almost six-in-10 difficult cases can be solved by using the world wide web as a diagnostic aid."

http://www.dailymail.co.uk/news/article-415562/Doctors-using-Google-diagnose-illnesses.html

I think this post misses the larger point that Gawande and Schmidt were addressing (and to some extent agree on) - the "art" of the practice of medicine must be transformed to the science of the practice of medicine. And today, science and IT are necessarily intertwined in practical terms.

The future will no doubt involve ever more sophisticated and useful clinical DSS. Historically speaking, invoking the complexity argument is fraught with issue as technological advances provide better and better ways to represent and manage it.

MedInformaticsMD said...

Re: Anonymous Jan 16 @ 10:03 AM-

The article you cite reports the following:

OBJECTIVE: We endeavoured to determine whether individuals who are not physicians are likely to arrive at correct diagnoses by using Internet resources. METHODS: In this prospective study four non-physicians used Google to search for diagnoses. They reviewed the 26 diagnostic cases presented in the case records of the New England Journal of Medicine during 2005; they were blind to the correct diagnoses. The main measurement was the percentage of correct diagnoses arrived at by non-physicians by using Google. The diagnostic success of the four non-physicians was compared to that of four young physicians. RESULTS: The average diagnostic success of non-physicians was 22.1% (95% confidence interval [CI] 4.5-39.7%). There was no statistically significant difference between the non-physicians regarding this outcome (p = 0.11). They took 8.9 +/- 6.7 (mean +/- standard deviation) minutes for case record reading and 17.4 +/- 7.9 minutes for Google searching per case. Non-physicians performed worse than physicians (50.9% [95% CI 37.4-64.5%]) in regard to diagnostic success (p <0.001). CONCLUSION: Non-physicians, at least those who have similar characteristics to the participants in the present study, may occasionally reach correct diagnoses by performing a brief web-based search. Doctors should realise that patients may assume a more active role in their health decision-making process and take this development into consideration in physician-patient interaction.

This article suffers from what can be termed "the syndrome of useless information."

Let's see:

The average diagnostic success of non-physicians was 22.1%

They omit the opposite semantic: that the google-armed nonphysicians got more than three quarters of the diagnoses WRONG. Will you trust your grandmother to them?

Non-physicians performed worse than [young, a.k.a. inexperienced] physicians (50.9% [95% CI 37.4-64.5%]) in regard to diagnostic success (p <0.001).

Young, inexperienced physicians (trainees or residents?) arriving at 51% of the diagnoses correctly using ONLY google (not, for example, Harrison's, and other standard medical texts that would not be highly usable to most non-physicians lacking background to fully understand it) is not unexpected.

What is surprising is that the article omitted an essential control group: *experienced* physicians. It's not that they're hard to find.

The article concludes:

Non-physicians, at least those who have similar characteristics to the participants in the present study, may occasionally reach correct diagnoses by performing a brief web-based search.

Occasionally reach correct diagnoses? How about "most often reach incorrect diagnoses?" cf. a broken clock is still occasionally correct.

I do not view this article as revealing anything of practical value other than perhaps the dangers of allowing non-physicians armed with search engines to think they can perform medical diagnosis to any meaningful extent.

the "art" of the practice of medicine must be transformed to the science of the practice of medicine. And today, science and IT are necessarily intertwined in practical terms.

Reductionist views of non-medical IT personnel spouting off about 'database platforms' will not advance the science or the art of medicine.

Also, yes, IT is a tool of science; however, it and its designers and implementers facilitate science; the enablers of science are: scientists, using their insight, creativity, ingenuity, experience and expertise.

(continued)

MedInformaticsMD said...

(continued from previous)

Concerning the second article, I note:

"But they [the authors] stress the efficiency of the search and the usefulness of the retrieved information depend on the searchers' knowledge base."

In other words, search engines can facilitate experts. That is not exactly new knowledge.

The 'art' of medicine, i.e., judgment, likewise is both indispensible, and irreproducible via "database platforms." Perhaps one day with advances in cognitive computing we will get to that point, but at present we can't even do as well as a cat. Note the statement from the IBM P.I. that "there are no computers that can even remotely approach the remarkable feats the mind perform."

Until we get there, I think it's not unreasonable to hold off on non-clinicians touting reductionist information retrieval-centric views.

Historically speaking, invoking the complexity argument is fraught with issue as technological advances provide better and better ways to represent and manage it.

You must not have read my original post. It is advances in information science (i.e., in informatics) that will provide those advances, not advances in [information] technology. IT is a tool; information science is an activity of the mind.

Your statements clearly demonstrates a conflation of IT and information science.

Computers facilitate information science research, but they are certainly not its sine qua non.

-- SS

MedInformaticsMD said...

Note: I've elevated the last comment and my response to a full blog post here. I think it worthy of such elevation.