Readers Write: Innovate Responsibly – Cutting Through the Hype of Generative AI in Healthcare

Innovate Responsibly – Cutting Through the Hype of Generative AI in Healthcare
By Holly Urban, MD

Holly Urban, MD, MBA is VP of business development for Wolters Kluwer Health.

In the fast-moving world of generative AI (GenAI), it’s easy to get caught up in the allure of shiny new technologies in healthcare. But we can’t let hype alone outpace responsibility. GenAI’s strengths quickly turn into weaknesses if we deploy GenAI in clinical care without carefully vetting it first.

The Shiny Object Dilemma

The healthcare technology market has become flooded with flashy new tools and solutions. According to Deloitte, 75% of leading healthcare companies are already experimenting with GenAI, and our research shows that nearly three-quarters of healthcare professionals recognize the potential of technology like GenAI in aiding professional development, clinical training, and efficiency.

Still, experimentation doesn’t always equate to readiness. What we should be looking at — and answering — is whether GenAI is capable of solving today’s most pressing challenges.

The key to healthcare innovation starts with creating impactful technology and fostering an environment for clinicians and their patients to thrive. That’s only possible by aligning technology with the real needs of healthcare professionals, the patients they’re serving, and demonstrating the return on investment (ROI) in clinical and financial outcomes.

Rolling out new GenAI should be about matching the problems with the right technology. For example, 60% of healthcare professionals believe that GenAI can improve the patient experience, and 41% think that ambient listening capabilities will enrich patient-provider relationships.

Ambient documentation is a prime example of where GenAI is making a significant impact by alleviating one of healthcare’s biggest challenges in a low-risk domain. It can save clinicians hours each week by creating clear and actionable patient summaries, and there’s an incredible opportunity to integrate clinical decision support and revenue cycle into these workflows.

Balancing Hype with Safety

As GenAI gains traction throughout healthcare, risks persist, particularly as GenAI approaches the actual patient and directly impacts their care. One area of concern among healthcare professionals is the overreliance on GenAI. In fact, a preliminary study from MIT explored how GenAI alters the brain’s ability to process information, leading to impaired learning and retention.  

As great as GenAI is at generating content and creating patient summaries in seconds, it’s also capable of hallucinating with complete confidence in the same amount of time. What’s more problematic is the inability to distinguish hallucinations from reality. One study found that up to 45% of residents do not detect hallucinations accurately.

The likes of ChatGPT may perform well on a medical exam or when diagnosing textbook clinical vignettes, but real-world patient care can be far more complex and unpredictable. Patients expect their clinicians to make error-free decisions using trustworthy evidence, not guesswork, to ensure the best possible outcomes.

It’s easy for LLMs to be unaware of clinical context and fail to ask important questions before delivering diagnostic and treatment recommendations when they aren’t held to a gold standard of evidence. LLMs can fail to admit they’re wrong and may lead a clinician down the wrong path if it’s not caught early on.

For example, if you’re treating a patient with a urinary tract infection who is allergic to penicillin, an LLM will likely recommend prescribing fluoroquinolones, which is typically the right course of action. However, if it is not trained to ask if the patient is pregnant, fluoroquinolones could cause a harmful drug reaction in the patient and the fetus.

Real-world concerns can come with severe consequences. GenAI must be fully ready for every clinical application and grounded in rigorously reviewed evidence-based content before doctors rely on it to aid in clinical decision-making.

Making GenAI Responsible for Healthcare

Organizations are beginning to take the lead in building robust AI governance to ensure the safe and responsible use of GenAI at their institutions, as the technology is currently advancing faster than the oversight.

It’s important to learn to walk before you sprint. We’re seeing benefits from gradual rollouts, pilot programs, and industry consortiums offering quality assurance resources for clinical AI. Collaborations are crucial to working towards the same goal of seamless integration and avoiding disruptions or costly errors.

Ultimately, the most effective GenAI tools in healthcare will remove, not add, another layer of complexity to practicing medicine. Our efforts should be grounded in restoring joy to healthcare through the simplification of processes. Patient encounters should focus on care, not on clinicians spending valuable time searching for information.

GenAI offers an incredible opportunity to eliminate friction and accelerate access to the right information at the right time, when clinicians need it. At the end of the day, technology should be an enabler, not a barrier, to delivering the best possible care.