Home » Dr. Jayne » Currently Reading:

Curbside Consult with Dr. Jayne 8/1/11

August 1, 2011 Dr. Jayne 6 Comments

I always know I’m in for a treat when Inga sends an article my way. She didn’t disappoint with Industry jeers peer-nominated Top Doctors list

Earlier this month, I shared my thoughts regarding websites where patients can rate their physicians. Now it seems the intrepid staff at US News & World Report has gotten into the game.

Most people are familiar with the “Best Hospital” list they put out every year, with the same academic medical systems filling out the top of the list year after year, but with slight reordering. Having trained in some of these institutions, I’m not sure what it really means, but the hospitals sure do like to brag about it.

The physician list is the result of a peer nomination process. It reflects no data on training, experience, board certification status, or disciplinary action. I looked up physicians in my specialty within 25 miles of my ZIP code and found a couple of docs I know. One of then I deeply respect and would trust with a member of my own family.

The other I can only describe as seriously out of date, with a reckless disregard for evidence-based medicine. He’s one of those “great guy” types, but as someone who used to work with him very closely, I couldn’t believe it.

There’s a link in the article to the methodology used in the rankings. The comments section was truly enlightening. They include:

Very disappointed with this list. I have been chief of my department for many years now and know of at least one MD on your list who has had substance abuse problems and has been put on limited restrictions. This is clearly an imperfect and potentially dangerous system that needs some review of its rating system.

While many of the physicians you recognize in your list that practice in the same subspecialty as myself, there is one who is recognized that I have personally worked with and know lacks certain ethical standards in the operating room.

US News isn’t the only news outlet to get into the physician rankings game. One of our local magazines has been doing it for years, to the great amusement of many docs in the area.

One of our colleagues who hasn’t practiced in the area for almost a decade continues to make the list year after year. When we are polled for nominations, we take great pleasure in continuing to nominate her just so we can send her a copy when she makes it again. She hates being on that list — it makes her a magnet for patients unhappy with their current physicians or those expecting miracles.

While I was looking at the rankings, I couldn’t help but think about the recent EHR usability ratings I covered last week and about ratings of systems in general. KLAS is often cited when discussing EHR ratings.

My first experience with KLAS was when I was solicited by a vendor’s project manager for a newly-implemented system. It reminded me of the annoying service rep at the car dealer who always tells me, “If they call, give me all high-fives!” as he hands over my keys. The project manager asked me if I could give the vendor eight or higher on a 0-10 scale. If so, she would see that I received a KLAS survey. She didn’t specify what would happen if I couldn’t give it that kind of a rating.

Luckily, this was one of our stronger vendors who legitimately deserved high scores, so I agreed to participate. But I found the idea that vendors were able to choose who rated their products to be unsavory. (I don’t think KLAS does it that way any more, at least not exclusively, since I found a ‘rate your vendor’ button on their website. Some of the KLAS questions are still somewhat subjective, though.)

Regardless, I’m not sure any of the more objective analyses are able to differentiate products any better. ONC-ATCB lists 164 certified “Complete EHR” systems for Eligible Providers, of which 53 are also CCHIT certified for 2011. This proves that a system contains certain functionality, but doesn’t say much about its ability to improve the patient or physician experience, let alone deliver higher quality care or lower healthcare costs, the reasons most often cited for making the leap to EHR in the first place.

I’m not sure what the answer is. As a clinician, it’s hard to rate clinical systems unless you’ve used more than one. The grass always seems greener on the other side until you actually have to use another system.

For large health systems or multispecialty groups, the functionality expected of EHRs grows every day. There’s no way a single vendor can be good in every specialty and every size practice. But they definitely try and it’s certainly entertaining to watch.

Have a foolproof methodology for ranking clinicians or vendors? E-mail me.

E-mail Dr. Jayne.

HIStalk Featured Sponsors


Currently there are "6 comments" on this Article:

  1. Dr. Jayne,

    It’s just like US News and World Report’s evaluation of top Colleges… you’re thrilled when your college (or Doctor or Hospital) is on top, and you question the methodology when they’re not. I don’t believe there is a foolproof methodology. You do all the research you can, talk to all the customer who will talk, have a great group of people to implement, and pray for the best. I still bleieve that a mediocre system well-implemented beats a great piece of software that “automates the cow path” any day!

  2. It’s easy to differentiate products with competitive analysis. Other products (cameras, motorcycles, et al.) have “Shoot outs” and “Head to head” comparisons by uninfluenced evaluators. These shoot outs are wonderful exhibits of what works well in direct contrast with similar products.

    The closest thing to an unmolested evaluator would be RECs; they’d be great candidates for performing competitive comparisons, but ultimately: gov. funding can’t be used to favor one product over another. So even if RECs were good evaluators, they couldn’t mention it. (IE: vendors don’t pay taxes so the gov. can direct buyers to someone else’s product…)

    Like Columbus without a Nina, Pinta, or Santa Maria, the prospect of a new world (for EHR information) is as unfunded as it is ludicrous. However, that doesn’t mean there aren’t trade winds already blowing right into it. It’s just that trying to provision an exploratory project is ridiculous when medical service providers are convinced that the earth (and the best medical record) is as flat as a piece of paper.

  3. The most objective rating I can think of for a product is whether people are buying it. Economics 101: Consumers will seek out value (=quality/cost). A well selling product is a reflection of the collective wisdom of the consumers.

  4. 1) RECs can be HORRIBLE product evaluators in my experience. It’s been pointed out here on histalk that they can often recommend vendors whose installations bring them return consulting business. Nearly all of the REC employees I’ve met are former employees of these very companies…how do you avoid conflict?

    Some RECs are notoriously bad. Some are pretty good. YMMV.

    2) KLAS interviews folks who submit surveys and they don’t accept unverifiable results. They also identify which vendors provide complete access to their customer list and which don’t. And, they contact as many practices as possible, regardless of who is on the list.

    They contacted plenty of our clients long before we gave them a list.

    At our users’ conference, KLAS would only accept surveys from our clients if a) they were provided to all attendees b) the messaging about the surveys were controlled by KLAS and c) the customers handed the surveys straight to the KLAS representative and NOT have them come through an employee. They _have_ had problems with this, clearly, but they also do an incredible amount to limit that problem.

    The bottom line: check their scores. What I hear on the phone every day jives with their results. We vendors know who sucks – and they suck at KLAS. I know who my competition is, and they score well.

    KLAS isn’t perfect, but they’re also the ONLY one of the reviewers who hasn’t offered a pay-to-play deal to me.

  5. Beyond ONC Meaningful Use certification, physician usability will play a dramatic role in the adoption of EMR technology. KLAS recently asked practices already live with an EMR to rate the physician experience and levels of adoption of key MU components such as physician order entry, e-prescribe, progress notes, rules/alerts, data interoperability, patient electronic access to records, etc. We also asked about the biggest challenges, if any, practices will face in getting to MU. Data for the 25 most considered EMR solutions are included in the study.

    As meaningful use continues to evolve, the physician experience will be a primary focus of future KLAS reports.

    Providers who would like to get access to the study overview can do so by visiting the KLAS website.

    Best regards,
    Chase Titensor

Founding Sponsors


Platinum Sponsors



















































Gold Sponsors














Reader Comments

  • Pay to play: Can we all agree that HCI and Santa Rosa paying a firm to inflate their KLAS numbers is just taking it one step further ...
  • Elizabeth H. H. Holmes: If protests were a significant vector to spread COVID, I would expect Minnesota to be much worse than where they are....
  • Eric: With the current pandemic healthcare need to adapt tele health, remote patient monitoring is very critical and digital h...
  • Art_Vandelay: Is it just me or are others tired of receiving HIMSS Digital Health Indicator and Digital Health Advancement ads? They h...
  • Dan: https://www.nber.org/papers/w27408 This found that the protests caused a net reduction in the transmission of COVID, ...

Sponsor Quick Links