Monday, December 09, 2013

Sickness in the Information Technology Sector: Technical problems, discord plagued Maryland health care site

Eye-opening, ground-level, no-holds-barred accounts of IT mismanagement and mayhem are too rare, considering the stakes in 2013.  The behind-the-scenes crap that goes on needs a great deal of sunlight. Below is such an account of great merit, in the Baltimore Sun.

First, I note physicians and nurses are generally able to collaborate to take care of sick patients.  They are natural "team players"; taking care of patients is their professional calling.  I did it all the time in my patient care years, especially in critical care settings, where lack of collaboration with colleagues could lead to dead patients.

IT personnel are another species entirely.  It's all about territory, competition, self-promotion, empire-building, drinking from the ever-flowing fountain of money, etc.; process matters more than results; and to hell with the end users, let alone patients.

Here is, in macro form, the Fifty Shades of Mass Dysfunction I've often encountered when my professional work intersected with IT personnel, whether in hospitals or industry.  And I was supposed to simply "shut up" about this crap - i.e., go along to get along:

Technical problems, discord plagued Maryland health care site,0,6559272.story 

By Meredith Cohn and Andrea K. Walker, The Baltimore Sun 10:50 p.m. EST, December 7, 2013

Although state officials have provided the public scant detail about the troubled launch of Maryland's version of Obamacare, emails and documents show that the project was beset behind the scenes for months by an array of technical issues, warring contractors and other problems.

Since Maryland's online health exchange opened Oct. 1 for people to buy insurance under the Affordable Care Act — and immediately crashed — the two main companies in charge of the website have taken their fight to court, a corporate project manager was replaced and a high-powered consulting firm was quietly brought in to restore order. Though state officials initially said the crash of the online exchange was an unexpected and fixable problem, emails and documents obtained by The Baltimore Sun through state open-records laws outline serious issues before and after the launch.

The revelations came just days before Rebecca Pearce, the head of the exchange, resigned. State officials announced that move Friday night and pulled Carolyn Quattrocki from the governor's health reform office to serve as an interim replacement

I am going to reproduce some paragraphs that show just how chaotic are the processes - in any health IT endeavor, whether they be for the exchanges that are essential to getting insured, or to the hospital IT systems that are essential to getting out of the hospital in something other than a pine box.

Emphases mine:

Just two weeks before the launch, Pearce visited the prime contractor's Linthicum headquarters and found a room of empty seats. She fired off an email questioning the company's commitment to resolve problems and reminding the contractors of what was at stake: "Tonight, I am begging. I don't know how else to say it: we have got to make this a reality."

Despite her proddings, in-fighting between contractor Noridian Healthcare Solutions and a key subcontractor, EngagePoint Inc., disintegrated amid finger-pointing and accusations in court papers. At one point, after Noridian severed contractual ties between the companies but continued to ask for help, EngagePoint CEO Pradeep Goel emailed Noridian officials: "Are you people on crack cocaine?"

Contractors, subcontractors, all offering "solutions" in healthcare and all likely to have not a single soul on board with anything beyond a merchant-computing background.

And nice, friendly questions amongst the friendly, collaborative finger-pointing and accusations typical of the IT world when mass mayhem occurs.

And a begging boss?  How lovely.

... Pearce, who was hired in 2011 at a salary of $175,000, declined to comment on problems surrounding the exchange or her resignation.  [If I were her I would have resigned out of disgust with these idiots.  Oh wait - I did leave IT positions for reasons, in part, like that  - ed.]

The emails provided by the state covered the two weeks before and after the website launch. They give only a limited behind-the-scenes view of creating and launching the exchange. Officials withheld an unknown number of emails, saying state law exempts them from disclosure because they involved the decision-making process of high-ranking executive officials.

The troubles in Maryland mirror problems faced by other state exchanges, as well as the federal portal providing insurance options to consumers in 36 states.

 Emails withheld under claims of legal privilege means we'll never know all the dirt.

... In Maryland, Sharfstein said the complexity was compounded because of an aging state Medicaid computer system that needed to be integrated into the exchange. Officials also chose to customize existing technology that proved tougher to retrofit than expected, he said.

"Unlike buying a book online from Amazon, this process is more akin to applying for a passport, buying a home, and receiving an individually calculated tax credit all through a single web portal," O'Malley said Friday night. "We had more user glitches and user problems than we had hoped.

"A longer testing period might have allowed us to prioritize and address more of these problems before the launch date. Time and ultimate success will tell whether the decision to purchase off-the-shelf software and employ multiple contractor entities were good or bad decisions."

"Hope" is not a proper project management technique of which I am aware, especially in a project of the admitted complexity as this one.

"Tougher to retrofit than expected?" Expected by whom?  IT "experts" who believe in unicorns and the tooth fairy?  I point out that in medicine, such mistakes are called "malpractice."

A longer testing period might have allowed us to address more problems?  But they went live anyway?

... In early 2012, the state gave a $71 million contract to develop the website to a Noridian-led team that included Curam Software, IBM and Connecture. To save time in creating the exchange, the Maryland legislature exempted the contract from the normal procurement process, and North Dakota-based Noridian outscored three other bidders.

Sharfstein said Noridian will likely remain at work in its Linthicum offices beyond its contract's year-end expiration. The company has already been paid about $57 million but the state contract allows penalties for delays. State officials declined to comment on whether any penalties will be sought.

Noridian is ultimately responsible for delivering the system, Sharfstein said. EngagePoint, which is based in Calverton, was not included in the original contract and appeared to have been hired without the exchange's knowledge, officials said.

The state first learned of the companies' "deep strains" in the three months before the website launched, according to documents in U.S. District Court in Baltimore. The issues disputed included accounting, project management, intellectual property and payment.

Emails offer a glimpse at how their differences affected efforts to build the site and then fix post-launch problems. Pearce repeatedly questioned the contractors' commitment to the project after Gov. Martin O'Malley announced on national TV that Maryland's site would go live on time.

On Sept. 22, after Sen. Barbara Mikulski echoed the governor in publicly applauding Maryland's readiness, Pearce wrote the contractors: "It's time to get this right. Now. Period."

Noridian was also criticizing the subcontractor it hired. On Sept. 25, Noridian's project manager wrote to Goel, complaining that EngagePoint refused to perform critical work: "EngagePoint is responsible for 'designing and implementing [an exchange] system,'" the project manager wrote.

The 8 a.m. launch was supposed to allow the estimated 800,000 uninsured Marylanders to sign in and browse 45 plans from six insurers. Officials had warned of "bumps in the road," but the site crashed in minutes.

... "As the executives in charge of this program, I would like to understand from you exactly what is happening with the project and what you are doing to address the issues," she wrote to the contractors at 7:56 a.m. on Oct. 2. By 4:10 p.m., she questioned why 85,000 people had hit the "get started" button, but there fewer than 500 accounts had been created.

About a half-hour later, she wrote to the contractors, "Can you please provide an update on what is going on right now? Who is on site? What has anyone learned?"

Some of the companies' emails focused on achievements rather than dwelling on worsening problems.

Noridian CEO Tom McGraw wrote to state officials on Oct. 4, "We have seen increases in all aspects of the system performance over the last several hours and anticipate that these will start showing in the next report."

But four days later McGraw notified state officials that the project manager was being replaced.

Conflict, favoritism, fighting, "it's not my job", lies, spin, stonewalling, strife ... chaos.

"Efforts to build the site and then fix post-launch problems" is also typical of the IT world.  In health IT, especially clinical IT, those harmed or killed during the "fixing" process are considered a necessary sacrifice, a "bump in the road."

... Paul von Ebers, CEO of Noridian Mutual Insurance Co., Noridian's parent company, wrote on Oct. 10 that the consultants "expressed concern with ongoing coordination issues between the Noridian and EngagePoint teams." He requested a meeting to resolve "working differences" between the companies.

This was days before Noridian fired EngagePoint, sparking the angry email exchanges and dueling lawsuits between the companies. Noridian then sought to hire EngagePoint workers; EngagePoint sued and was met with a counter-suit.

"We are expected to do piecemeal work for Noridian after contract termination because you just woke up and decided you don't know what you are doing?" Goel wrote Oct. 26. "We are not going to respond to ridiculous emails from Noridian demanding our team members show up for work after being escorted out of the office."

That's the path to progress:  firing subcontractors and attempting to steal their employees, and lawsuits and countersuits.

I could go on, but read the entire postmortem account (itself rare in large IT projects) at the Baltimore Sun link above.

Some of the de-identified cases at my academic site on health IT difficulties reflect this type of discord; some are mine personally such as at , and

I was expected to "go along to get along" with this BS, and was deemed "not a team player" when I spoke up for enduser (clinician) and ultimate customer (patient) rights.

In summary, the IT world is demonstrably dysfunctional, with the types of conflict as in the Baltimore Sun story more the rule than the exception, especially where healthcare is concerned.  Good, relatively brief resources for better understanding these issues are the following papers:

Social Informatics.  An introductory essay entitled “Learning from Social Informatics” by R. Kling at the University of Indiana can be found at this link (PDF).  The book “Understanding And Communicating Social Informatics” by Kling, Rosenbaum & Sawyer, Information Today, 2005 ( link here) was based on this essay.

Pessimism, Computer Failure, and Information Systems Development in the Public Sector.  (Public Administration Review 67;5:917-929, Sept/Oct. 2007, Shaun Goldfinch, University of Otago, New Zealand).  Cautionary article on IT that should be read by every healthcare executive documenting the widespread nature of IT difficulties and failure, the lack of attention to the issues responsible, and recommending much more critical attitudes towards IT.  link to pdf

Defensive climate in the computer science classroom” by Barker et al., Univ. of Denver.  Link here (subscription required).  May help explain the control-seeking culture of IT personnel.   As part of an NSF-funded IT workforce grant, the authors conducted ethnographic research to provide deep understanding of the learning environment of computer science classrooms. Categories emerging from data analysis included 1) impersonal environment and guarded behavior; and 2) the creation and maintenance of informal hierarchy resulting in competitive behaviors. These communication patterns lead to a defensive climate, characterized by competitiveness rather cooperation, judgments about others, superiority, and neutrality rather than empathy.

If a person thinks this technology will "revolutionize" healthcare anytime soon, considering the "people issues" involved let alone the technical ones, then that person is either hopelessly naïve, or needs a mental health evaluation (colloquially, "needs their head examined").

-- SS


Anonymous said...

Similar to the HIt farces in hospitals, where patients are dying from the chaos.

Trevor3130 said...

Scot, perhaps those government agencies responsible for spending large $$$ on IT projects should spend more time evaluating the Chief Executives (MDs) of companies, as well as details of contacts.
To wit, I am impressed by a (growing) list of essential reading.
1) 'Thinking, Fast and Slow' by Daniel Kahneman.
2) 'The Honest Broker' by Roger A. Pielke, JR.
3) 'Being Wrong' by Kathryn Schulz.
Also impressive is Garry Kasparov, making his thoughts known in the Twitter-sphere. For example,
"Putting hysterical propagandist Kiselyov in charge of new Kremlin PR initiative sends clear message of hard line, paranoia & bigotry."

Shane Irving said...

Now you can't say that this would not have made a great reality show. I have seen this type of dysfunction too many times. It takes someone who can span both sides (IT and User (or Clinician)) to drive these kinds of projects down the right path. Getting 3 IT vendors together and heading the same direction is about as likely as herding cats on crack. It looks like Oregon may have had the same experience with no end in sight right now. Just keep writing those checks.....