Ross Koppel, PhD is on the faculty of the Sociology Department, the School of Medicine, and the Graduate School of Medicine of the University of Pennsylvania.
You’re a sociologist. What would you say is the major sociology at work among vendors, doctors, and CIOs?
A bit of context may help understand my research on the sociology of HIT. My first love was sociology of work. The use of technology in the workplace came next. About 10 years ago, I was working with some physicians to try to understand medication errors by young docs due to workplace issues like fatigue, inconsistent supervision, and dealing with life and death issues for the first time in their lives. And, remember, hospitals are workplaces.
With colleagues at the Penn Med school, we got a grant to study stressors in the hospital workplace. In response to my questions, the residents always insisted on showing me the remarkably lousy CPOE system. It was generating errors — some horrible, most easily fixed, but not fixed.
If you think about it, my looking at these issues is not all that odd. I’d been studying work, workplaces, technology, and sociology of medicine for 34 years. Also, the 2005 JAMA article combined sociological skills in ways that not many others had at their disposal: focus groups, expert interviews, shadowing docs and nurses, an extensive survey of 90% of the residents, intensive interviews, and observations on the floor. I’ve been teaching research methods at Penn for 17 years. I should have learned something.
Now, to your question directly. The physicians want to get their work done, ideally with greater safety and in less time. The vendors want to sell their wares and capture market share. The hospital CMIOs don’t want the software to crash catastrophically on their watch. None of them want to hurt patients, but the combination of forces is often counter-productive.
Vendors seek market penetration ASAP because user implementation costs prevent reconsideration of other options once a hospital or even medical practice is committed. But vendor product cycles do not allow the ongoing feedback and adjustments that allow rapid improvements. The vendors are eager to roll out new iterations while the industry structure does not encourage patient safety or the actual needs of hospitals and clinicians.
Also, hospitals want to show how wise are their investments, so few benefit from discussing errors. And many contracts prohibit open discussion of problems. Added to this is the reality HIT only works when it is embedded in complex organizations with other HIT systems. And that’s darn near impossible to test a priori. So even if the software had viable user interfaces and transparent coding, we could never be sure it is safe and functional until we can examine it in situ, which is both difficult and expensive.
Last, we must remember that most of these systems are built on back office programs from the 1980s with interfaces that are state of the art from the 1990s. There are structural issues that are difficult to surmount.
You got a lot of reactions to the 2005 article you wrote about CPOE. Which of them made you really mad?
The ones that misquoted or never read. The University of Pennsylvania wrote a reasonable press release. JAMA wrote more of an incendiary press release. A lot of people never actually read the article, so the things that pissed me off the most were those that were saying, “He only did the three focus groups,” when in fact I had a 90% sample of all of the residents and I had complete logs of every order put in.
There were all kinds of mythologies that were developed that bore no relationship to what I did. That was irritating as hell. I think there were some fair criticisms that the system that I studied was, in fact, an old one, and in fact I have a new study based on a brand new system that’s not been yet published.
It was the misquoting and the misrepresentation that most pissed me off. I mean, I’m an academic. I’m used to people commenting on my studies. I’m not used to vendors creating a whole mythology from what I said when I never said it.
Was it surprising that HIMSS jumped out and wrote a disclaiming memo of their own when they were not even really directly involved?
Well, HIMSS is a sales agent of the industry, so no, it’s not surprising at all.
The vendors would say, at least based on some of your writings, that you will always find the anti-IT angle to every issue. Is that fair?
No. They should read some of my other writings. I have an article published in JAMIA that speaks very much in favor of CPOE as a patient safety device and develops a whole algorithm to use that. A lot of the work that I did with Michael Harrison — there are two publications there — talk about unintended consequences that focus mainly on the role of hospitals and healthcare organizations in creating errors. In the most recent JAMA article, half of the article, or certainly 40%, is devoted to protecting vendors from unfair attacks by incompetent or misguided or maltrained or otherwise inadvertent errors introduced by hospitals and other healthcare organizations.
I view myself as deeply in favor of HIT. The problem is, some of the HIT vendors and some supporters have some sort of a siege mentality. They see my critiquing some of the problems of the HIT as an attack on the HIT. But that’s like saying that the guys who wanted American car makers to build more fuel-efficient cars are trying to destroy the industry, when in fact, they were trying to save it, maybe not just to save the industry but also for more noble social purposes.
But I see the vendors who misread or mischaracterize my work as being incredibly myopic on their part. I am a lover of HIT. I think it’s going to eventually produce some of the stuff that we all want it to. I think prematurely putting out HIT that’s primitive, that the user interfaces are barbaric, doesn’t do themselves any good.
Ever since that 2005 article — actually, I presented examples of it earlier — I’ve received about 10 or 20 e-mails a day from physicians who say, “Stop me before I kill again” and they send me illegal screenshots and the like. I have a whole battery of material that would scare the hell out of HIT vendors that I’ve never ever, ever, ever shown to anyone … which they know about, because it gets reported to them, but physicians and others are not allowed to discuss it among themselves because of the non-disclosure clause.
So what responsibility does the hospital and the CIO have? They bought the stuff and they’re the ones who signed on to use it.
What a great question. Yes; first of all, they are responsible for some of the errors. If they insist on a blue background because it’s a hospital color or something, and the warning notices come in blue, then how can the vendor know that a priori? They can’t know that in advance. They should be doing more due diligence.
But let’s talk about this. I talked to a CMIO the other day in the New York City area and they wanted to see examples of XYZ vendor software. They were given a hospital in Texas and a hospital somewhere else to go to, and the guys — you know, it’s not like the CMIOs don’t know who they are in a city, and we’re talking major hospitals — they said, “What about X, or Y, or Z two subway stops away, or M fifteen minutes away, or whatever?” And the vendor said, “No, we don’t want you to go there.”
So they shipped them off to their – you know what a Potemkin village is? [note – they were fake village facades constructed to fool Russian Empress Catherine II in 1787] Potemkin hospital 2,000 miles away, and even then, when they asked to see something, they started to say, “No, no, no, you can’t see that screen!” and they covered it with a sheet, claiming HIPAA protective law, which, by the way, is psychotic — I mean, one of the people in the team was a lawyer for the hospital who said, “No, all he has to do is sign a release. We all have HIPAA certification. Three of us are CMIO types or CIO types with medical training.”
I think vendors go out of their way to sell — I’m trying to avoid the word vaporware — I think vendors go out of the way to put a best foot forward in ways that really are more of a marketing effort than an information effort.
But it works, the free market. People are buying their stuff. They’re delivering their promise to the free market to provide what the customers want.
No, it’s different. If you buy a bad toaster and you realize you screwed up, you’ll eventually throw away the toaster. If you’ve spent a hundred million bucks on a whole system-wide software, and then 600 million installing the thing and training all of your staff — you know, hospitals cannot say to people in — pick a neighborhood: northwestern Wisconsin — “Nobody’s allowed to get sick for the next six months while we bring this thing up live.” You can’t do that.
The training and cross-cover for hospitals is an awesome responsibility. I’m not making those numbers up, right? And you know that. The installation, implementation, and then you’ve got to get it working with other systems already in place. That’s a non-trivial task. You just can’t say, “We screwed up; we got Sunrise but we should have gotten Epic,” or “We got Epic and we should have gotten Sunrise.” You’re married. It’s worse than really expensive divorce.
So do CMIOs and CIOs have a responsibility? Yes. But there are all kinds of pressures on them, including the most recent ARRA thing, to buy this stuff. I think the pressure should be removed until they can have software that’s really worthy of the promise that we’re getting.
That does not mean that I don’t think they should buy software. I think that what it does is wonderful. I would much rather be in a hospital with an EMR and a CPOE and an eMAR than one that’s based on paper, but from the reports that I get and my own research, I can tell you some of those interfaces are nightmarishly bad. I mean, if it takes four screens and seven scrolls to find both systolic and diastolic on the same patient, we have to open trap doors to find the lab report you’re seeking that should be right there. This is primitive. You’re in this business. You know the reality of some of those bad interfaces.
But the worst reality is that none are clearly so much better that the ones that aren’t suffer from it.
From what I’ve seen of eClinical Works, it’s significantly better, but that’s just an impression.
Yes, it’s probably more dramatic on the physician practice side. I’m more of a hospital guy, but …
Most of my research is hospital. Although hospitals are moving into — I don’t know what to call it, “eClinical Works Turbocharged” or something — but yeah, you’re right. But on the other hand, they have different faults. One has a reasonable lab reporting system, the other has a reasonable medication list display or whatever. But yeah, there’s an awful lot of mediocre software out there.
Why would the industry take critiques about it as an attack when I desperately love this stuff? I want to see it better so that they can make even more sales with it. This is like selling a really mediocre, unsafe car and insisting that if you point out that it’s unsafe, that somehow you’re doing the car industry a failure. It’s ridiculous.
Nobody seems to correlate the fact that the utilization isn’t very good among doctors and nurses, which may be directly related to poor usability in software design.
No, they blame doctors for being troglodytic, for being technophobes, none of which is true. Doctors may have an unfortunate taste in golf attire, but they’re not morons and they really want to do what’s right.
I’ve been through the training sessions. I’ve watched the software. I’ve watched allergy indications almost impossible to update. I’ve watched patient information disappear or appear, require seven clicks and twelve scrolls to see two pieces of information that should be contiguous. A 12-year-old programmer would say, “So you want these together, right?”
So why are people still buying? Why doesn’t that move some vendor to say, “Well, I’ll be the usability king and I’ll make a ton of money”?
That’s a great question, and there are a couple of reasons. One, remember back when we were young, they used to say something like “God could create the world in six days and then rest,” and then “Why can’t we have a software program?” And the line was, “God didn’t have to deal with an installed base.”
Some of that is that the software is incredibly complex. When they fiddle with one thing, it screws up the finance department linkage or something like that. Some of it is that they prefer the mythology of blaming doctors for being idiots. Some of it is that when they hire doctors to work with them, the doctors go native.
You know that reference to anthropology? You send out an anthropologist to study some tribe somewhere and he’s supposed to write a perfectly accurate and scientific ethnography of this tribe. Finally you go search for him because you haven’t gotten any reports, and there he is with a bone through his nose and war paint and strutting around. You say, “Doctor Whatever, you’ve gone native! You were supposed to write … ”
When I work with doctors who have been working on these things and I say, “I know we’re deeply concerned about the inability of the physician to see the current medication on one screen,” they go, “Yeah, but look, we’re dealing with pixel X, and with bandwidth problems, and with…” And I’m going, “Wait a second, you did not go to medical school to give me lectures about bandwidth and utilization rates and the time…” There’s got to be a way in which somebody is protecting patient safety and clinician safety. That goes by the wayside all too frequently.
There’s another issue. Given what we’ve said before about trying to capture the market, because once they’ve got you, they’ve got you, right? They are rushing versions to market. They want to grab market share, and grabbing market share often gets in the way of making the product what we really want and know we can achieve. So it’s sort of a structural problem within the industry.
You’ve written about vendors who know about software defects and yet prohibit, in one way or another, customers from saying anything about it to other customers.
The non-disclosure clause. It’s a massive problem because it perpetuates exactly the kind of thing that you were asking about. It perpetuates the slow responsiveness.
So the vendor gets 2,000 complaints. Let’s say — I’m making this number up — 500 are ridiculous, it’s just the guy didn’t know how to plug something in. But of the remaining 1,500, the vendor then picks and chooses on the basis of what has to be immediately fixed to avoid being shown lethal versus what can come in the next upgrade, and what can come in the upgrade again, and then the upgrade later, and how many complaints did they get about this.
And so the vendor picks and chooses on the basis of a market model and a marketing strategy, not on the basis of what is greatest for the greatest number of patients and clinicians. Now, if that were transparent and we could see that there are, of the 1,500 complaints, there have been 10,000 dealing with — those are categories of complaints — I don’t know, the impossibility of entering allergies, or when you enter an allergy, it wipes out the previous allergy. So if the first allergy was anaphylactic shock and the second was a mild rash to latex, anaphylaxis dies, disappears, and you get the mild rash to latex coming up.
If you saw there were 10,000 complaints about that bug, there would be no way in hell a vendor could ignore that for the next iteration. But with the vendor having complete control over the listing, and clinicians being unable to see it and to talk with each other about this except sub rosa, not being able to send screenshots, it just perpetuates the economic self-serving model which does not serve patient safety or clinicians.
Now, why do clinicians accept this? It’s because they didn’t go to law school. And by the way, I’m speaking very soon to a group of healthcare lawyers and the like. The CMIOs come to me and say, “Look at this, we bought this and now we can’t address this,” and the lawyers for the hospitals say, “Schmuck. People come to me with a $5,000 contract to make sure it’s passing muster. You signed a $100-million contract, and now you come to me now that you’re stuck.”
But if we were talking about any other industry that didn’t involve patients, what vendors do would just be considered admirable business — they do what it takes to make a profit and keep shareholders happy. Is it inherently impossible to have a for-profit, publicly traded vendor model and expect them to really care beyond what is in their obligation to deliver? How do you make their interests align with those of their customers?
Exellent question. I think you can make their interests align because, ultimately, they have to be concerned about patient safety, because some of this stuff is going to come out.
In the most recent JAMA article, March 25th, we did a lot of research. We found that the vendors, when they settle with the hospital and with clinicians and with patients, that’s settled with a closed case, right? And nobody’s allowed to disclose the terms of settlement. It’s not in the hospital’s interest, because the hospital doesn’t want to say, “We screwed up, Mrs. Jones, even though it wasn’t our fault, it was XYZ software’s.” The doctor doesn’t want to say, “I screwed up the patient, but it wasn’t my fault; it was XYZ software’s.” And the patient or the patient’s widow is told that if they want the $1.2 million, they have to stay quiet. So nobody exposes that.
So we know that there are a lot of subverts. If there was openness about the problems because of the patient safety issue, then that avenue of hiding stuff would not be available.
In terms of your attack on capitalism, no, I don’t agree. I think that it is possible to be vigilant about patient safety to produce the best imaginable software and nevertheless to produce profit. I think that the fact that there are well-made, very safe cars on the road is proof that there can be pressures equated to safety, and nevertheless profitability. This is the wrong time of the economic cycle to make this argument, but nevertheless, I still defend capitalism that way.
I think what the vendors have been so frightened of in the 13 years since they held that meeting where everyone — except the AMA, for some reason — agreed that they must do everything in their power to avoid federal regulation. I think that they have put themselves in the position where increasingly, regulation is a possibility.
And by the way, in my article, I argued that there were about 12 steps that they could do before regulation and I laid them out. There could be professional oversight — in other words, the physicians in the hospital associations would say, “We will not accept contracts with non-disclosure, period,” it would end the problem. The lawyers’ association for the hospital could simply say that. The IT industry itself could say that. “We, from this day forward, will not include that.” AMIA could say that.
That said, let’s see what happened two days ago, or four days ago, in JAMA. There was a letter — including my letter, then a rebuttal — that said, “Guess what? The Joint Commission rules state specifically that a hospital and its clinicians are prohibited from not talking about patient safety-related errors in the HIT.” So every single hospital in America, theoretically, that’s involved with HIT would be in non-compliance, would lose accreditation if they follow the rules of the current HIT vendor contract.
Has anyone contacted the Joint Commission or has the Joint Commission read the letter in JAMA? And the answer is yes. Not me — I didn’t contact them. But it turns out that the Joint Commission reads JAMA, and I have been contacted by them, and they said, wow!
So the Joint Commission may simply say, “There can be no non-disclosure anymore if you want to stay accredited.”
Last question. If you were healthcare IT king for a day, what would you do?
I would say to the vendors, “The solution to your marketing problem is to create really good software, not to abuse people who study it and find problems, not to place the blame on ‘moron doctors’ or ‘idiot CIOs’, but to make really good software.”
And good software includes superb usability. I would say that there are excellent models out there of good usability. For God’s sake, look at Google, look at Google Maps, look at some of the people who are doing really innovative work out there. I would not shunt them away, but rather I would choose them as models.
I would develop a system of openness wherein we could really compare and test, not these Potemkin hospitals where you send potential purchasers, and really seek to improve on a real basis what is best. I mean, do the manufacturers of Epic really think that nobody from Sunrise ever sees their software or the like? There should be a real, open — call it a bake-off, wherein people can see what’s happening, and they’re not selling vaporware.
I went to one hospital the other day where they gave the top vendors a scenario then said, “I want you to do this, and then you get interrupted, and then you have to have a lab report to comment on, and then you’re going to wait for another drug information, and you have to do this live.” They did a real test. Most of the vendors either refused or failed. One vendor who I don’t particularly love passed.
Why did it have to be this one hospital CMIO type to call this test? This should be standard. There shouldn’t be hiding about this stuff. Patients’ lives are on the line and physicians’ reputations are on the line. Hospital bottom lines are on the line. Nobody benefits from having mediocre software, except maybe the salespeople of mediocre software. We can do so much better.
It is contained in the same Forbes article. Google “paywall remover” to find the same webpage I used to read…