Curbside Consult with Dr. Jayne 2/20/23

ChatGPT and similar tools continue to be some of the hottest topics around the virtual physician lounge. Plenty of clinicians are experimenting with using the tools to help respond to patient messages, and the bravest souls are even looking at using it to create visit documentation.

Although it’s tempting to think that we might be on the cusp of having reliable tools to help us with some of the most time-consuming parts of our jobs, the reality is that the technology is not yet ready for prime time as far as using it in clinical scenarios. Unfortunately, many frontline physicians may not understand the limitations of the system and are wading into some pretty deep water where it comes to patient care.

Some of my non-medical friends have been using it as well and have a lot to say about the fact that its output can sound completely convincing, but is factually incorrect. There are some examples going around, such as where it lists the peregrine falcon as the fastest marine mammal. The computer science folks note that in order for models like ChatGPT to be useful in healthcare, constraints need to be placed on their predictive capabilities.

For example, if you were using the tool to summarize a patient’s chart, you don’t want to allow it to predict procedures or treatments that didn’t happen. My friends seem to think that the easy answer in healthcare is to just have the physician review everything to make sure it’s accurate. However, those of us who practiced back in the days of heavy use of medical transcription know that’s easier said than done. The number of transcriptions that went out the door without proofreading or corrections was staggering, and led to outcomes running the spectrum from laugh-provoking to malpractice.

There’s also the not so small matter of HIPAA and the risks of feeding large quantities of patient information into the dataset used by the tool. Additionally, trying to leverage AI-based technologies for healthcare isn’t cheap. I’ve seen several startups that try to pass their solutions off as “AI-enabled” when all they really have is a bunch of sophisticated decision trees. There’s a certain threshold of money that has to be raised in order to be able to afford the work needed to truly move into the AI space, and understanding whether a company even has the resources to realistically do AI work should be one of the first steps in determining if they’re blowing smoke.

In related topics, some of my colleagues were discussing a recent editorial in JAMA Health Forum titled “Garbage in, Garbage out – Words of Caution on Big Data and Machine Learning in Medical Practice.” The piece opens with a quote from Alan Turing: “A computer would deserve to be called intelligent of it could deceive a human into believing that it was human.” It goes on to talk about machine learning and the use of data to predict clinical outcomes, such as adverse events related to medications. We know all too well the risks of using data sets that aren’t representative of the population in question or that don’t have all the information needed to generate a reliable prediction. The article uses the example of an opioid prediction rule that didn’t included data on cancer diagnoses or enrollment of hospice as a rule that isn’t ready for prime time.

Especially in the primary care trenches, physicians are often so busy just trying to get the daily work done that they may not be digging in to understand exactly how predictive rules are generated or how valid they are. They have to rely on regulatory agencies and the editorial staff of medical journals to vet proposals. Although this can delay the time for new tools to get to the point of care, it can be a valuable step for protecting patient safety. The article notes that it’s also important to reevaluate rules on a periodic basis, since medical knowledge continues to evolve. It gives the evolution of an HIV diagnosis “from a death sentence to a manageable chronic illness” as an example. It’s good food for thought.

Around the administrative / non-clinical physician water cooler, one of hottest topics over the last couple of weeks was that of annual performance reviews. Making the jump from clinical practice to management requires more than just an interest in administrative topics. It also involves understanding how corporations work and some of the tactics that they use to manage their human capital.

A physician who is new to administrative work recently learned that he would have to perform stack ranking when analyzing his team’s performance. For those who may not have run across this, it requires managers to score workers against their peers rather than against goals and objectives. The first time I ran into this was when I worked for a large hospital system, and a management consultant that had been engaged to “trim the fat” forced our department to implement it.

To make matters even worse, annual merit raises were tied to the stack rankings. For managers with exceptionally talented teams who were all working at or beyond their potential and who were achieving great results, it’s agonizing to have to allocate more of a raise to some and less to others when they were all working extremely hard and crushing their goals. As a relatively new physician leader at the time, I hadn’t been exposed to anything like that. It’s not something you learn about in medical school and it certainly wasn’t covered in the couple of physician leadership intensives that I was sent to as the health system prepared me for greater administrative roles. Fortunately, I’ve spent the better part of the last decade working in environments where this methodology isn’t used, and I felt more than a little disbelief at the fact that it seems to be becoming popular again.

I’m a firm believer that if an employee isn’t meeting expectations, that needs to be addressed early and often through individual conversations with their manager and potentially a performance improvement plan if needed. It shouldn’t be left until the annual performance review. On high-performing teams, members should be able to work without fear that they’re going to be unfairly compared to co-workers just because of a methodology. Stack ranking is hard on managers as well as employees, and contributes to an overall toxic workplace culture. The fact that it’s still out there despite the literature about its consequences says a lot about companies that continue to use it.

The last hot topic of the week was a recent study that looked at whether the board members at the nation’s top hospitals have healthcare backgrounds. Published earlier this month in the Journal of General Internal Medicine, it found that less than 15% of board members had a healthcare background versus finance or business services. Other interesting findings: of those with a finance background, 80% had experience with private equity funds, wealth management, or banking. The rest were in real estate or insurance. Of those with healthcare experience, 13% were physicians and less than 1% were nurses. The authors only looked at top hospitals and there were challenges in finding publicly available information about boards. This could be even more challenging when looking at smaller institutions.

These topics are just a sampling of those that are on the collective minds of physicians who are often just trying to put one foot in front of the other as they slog through caring for patients.

What do you hear when you’re working with clinicians? Are there any particularly hot topics? Leave a comment or email me.

Email Dr. Jayne.