Health systems assessing AI tend to measure what the tools can do. A panel at Cleveland Clinic's A.I. Summit for Healthcare Professionals argued that the more consequential questions concern what the tools do to the people using them—to clinical judgment, to training pipelines, and to how work is distributed across a care team.

The discussion was moderated by Sarah Hatchett, MBA, Senior Vice President and Chief Information Officer at Cleveland Clinic. She asked panelists about the clinical value of AI, how to prepare the workforce to use it, and lessons learned from putting AI into practice.

Efficiencies Achieved With AI

Ms. Hatchett asked Scott Nelson, PharmD, MS, of Vanderbilt University Medical Center in Nashville, about where he sees the greatest potential for AI to improve medication safety and efficiency. Dr. Nelson, of the university's Department of Biomedical Informatics, said AI could help in areas ranging from drug discovery to clinical decision-making. It could use more patient-specific information to guide medication decisions, improve safety, and reduce unnecessary alerts. AI could also help educate patients and improve communication among clinical teams.

“How do we improve the efficiency of our systems [while getting medications to patients] in a safe and effective way?” Dr. Nelson asked. AI-supported clinical decision tools could “bring an additional patient context that can promote safety for the patient.”

Discussing where AI is already improving care, Benjamin Kann, MD, PhD, a radiation oncologist at Dana-Farber Cancer Institute and Brigham and Women's Hospital in Boston, explained that within radiation oncology, as one example, AI can automate segmentation in radiation planning, reducing the time clinicians spend doing this work manually. “We have tools that we are using on a daily basis that are really helping with our bandwidth,” he said.

AI may also help physicians assess tumor response in greater detail. Traditionally, he noted, response has often been measured in a single imaging plane because more detailed analysis was too time-consuming. AI can now identify tumor boundaries, calculate tumor volume, and track changes in size and location over time.

“We can look at granular changes and where they're happening spatially, too, and learn much more about how patients are responding to treatments,” Dr. Kann said. He added that this could have implications for clinical trials as well as treatment decisions in practice.

But automation may come with a trade-off. Radiation oncology trainees once spent hours learning to outline normal structures by hand. AI can now do much of that work, leaving clinicians to review the results. Dr. Kann raised the fear of “de-skilling” as clinicians get less hands-on practice and said training may need to change as AI becomes more common.

Building AI Literacy Before Tool Training

Ms. Hatchett followed up on that issue by asking Travis Zack, MD, PhD, Chief Medical Officer at OpenEvidence and member of the ASCO AI in Oncology editorial advisory board, how health systems should begin preparing their workforce to use AI.

Clinicians need “fundamental understandings of what an AI model is” before focusing on individual tools, Dr. Zack said. That includes understanding how models are built and learning to recognize “failure modes,” accuracy problems, and data set shifts. Such knowledge can help clinicians understand effective uses of AI and “the potential pitfalls that you have to be very careful of when you start.”

Lisa Christensen, global head of learning design and innovation at McKinsey & Company, said AI education should include hands-on experience. “We need to get people experimenting and using AI,” she said. Some health-care workers are eager to use AI, she added, while others remain hesitant. Encouraging staff to start with small steps can help them understand how AI may change their work and which human skills will remain essential.

Avoiding Unintended Consequences

Ms. Hatchett asked about the mistakes seen when teaching people about AI or putting the technology into practice. Erin Losey, RN, a nursing informatics specialist at Cleveland Clinic, emphasized the importance of starting on the right foot. Nurses need clear information about how AI tools are developed and used, she said, especially because some nurses already have concerns about AI use in health care. “We only get one shot with technology to make a first impression,” Ms. Losey said. “We also have to build trust.”

Dr. Zack said another lesson is to create ways for users to provide feedback from the start. “No system is perfect,” including AI, he said. Organizations should learn from errors and negative feedback to improve AI systems over time. Without that feedback, the systems can become “stagnant and stuck.”

Dr. Nelson described another unintended effect: AI may shift work from one person or team to another rather than reduce the overall workload. He cited a radiology practice where AI identified normal chest X-rays and sent abnormal studies to radiologists for review. The system improved triage, but it also meant radiologists were seeing one difficult case after another instead of a mix of routine and complex cases. As a result, they read fewer studies per hour, prompting questions from leadership about why productivity had fallen despite AI assistance.

“Are we actually easing the burden on our clinicians or are we just expecting them to do more with more complicated edge cases?” he asked.

The approach could also affect training, Dr. Nelson said. Residents who rarely see normal studies may have a harder time learning to recognize what is abnormal. Similar concerns could arise if AI handles routine medication orders, leaving pharmacists to review mostly complex or unusual ones.

Dr. Kann highlighted another risk: placing too much confidence in an AI result. Even highly accurate AI models encounter cases where experts disagree. Yet clinicians may see an AI result and think, “It knows what it's talking about,” even when the case itself is uncertain.

Regular use of AI could also make clinicians less careful when reviewing its work, he said. “The need to be extra vigilant when you're reviewing something vs doing it from scratch can be a real problem,” Dr. Kann said.

ASCO AI in Oncology is published by Conexiant under a license arrangement with the American Society of Clinical Oncology, Inc. (ASCO^®). The ideas and opinions expressed in ASCO AI in Oncology do not necessarily reflect those of Conexiant or ASCO. For more information, see Policies.