AI-Driven Multiagent System for Guiding First-Line Immunotherapy for NSCLC

A multi-agent system demonstrated correct and complete reasoning in determining the use of immunotherapy for patients with non–small cell lung cancer (NSCLC) in the first-line setting, according to findings presented during the first European Society for Medical Oncology (ESMO) AI & Digital Oncology Congress (Abstract 71MO).

“[Our] immune-specialized AI agent [system] is able to integrate the multimodal data of our patients to support the decision-making of first-line immunotherapy for NSCLC with good performance,” said presenting author Federica Corso, PhD, Fondazione IRCCS Istituto Nazionale dei Tumori, Milan.

Background

Currently, one biomarker exists for predicting response to immunotherapy—PD-L1 expression—but the researchers noted that it offers limited guidance in identifying potential responders because of various unresolved issues. They aimed to improve predictive tools to guide first-line immunotherapy decisions for patients with previously untreated NSCLC.

Although multimodal AI has demonstrated predictive abilities, existing models perform a single specialized task at a time and do not replicate the broader decision-making process. However, a multiagent system can assess and complete multiple tasks at the same time for a comprehensive treatment recommendation.

Study Design

Researchers conducted a study of 58 patients with stage IV NSCLC, all of whom were included in the APOLLO 11 observational study (ClinicalTrials.gov identifier NCT05550961) of patients with advanced lung cancer receiving innovative therapies, aimed at creating a virtual biobank. They were all treated at Istituto Nazionale dei Tumori in Milan with immunotherapy alone (n = 21) or in combination with chemotherapy (n = 37).

The researchers created a multiagent system that was trained with medical knowledge and access to web searches and gathered patient data, consisting of electronic health records, CT imaging and reports, histology slides and reports, lab work, and molecular reports. The agents were evaluated by four specialized oncologists to assess: the correctness of the immunotherapy treatment recommendation; the helpfulness and completeness of the recommendation; the meaningfulness of the retrieval-augmented generation (RAG) or the generated natural language response based on the information retrieved from databases and documents; and AI tool usage.

Overall, the multiagent architecture included a React agent with access to AI tool outputs and a RAG agent for querying and retrieving documents. Available AI tools for the agent to choose from included the LORIS CLI-Lab model for immunotherapy response prediction and progression-free and overall survival, the MUSK histology vision-language model for predicting immunotherapy response and histology type, the MedGemma radiology vision-language model for report generation, as well as web search and application programming interfaces (APIs).

Dr. Corso explained that the multiagent model reviewed the patient’s clinical content to determine the best treatment option based on the available data and tools. It then outputs a set of key findings, an analysis and rationale for its recommendation, a treatment plan, and possible reasons for ambiguities and contradictions.

Results

The system produced correct statements 72% of the time, as evaluated by the four specialized oncologists, and the recommendations were found to be helpful 72% of the time and complete 91% of the time. A total of 6% of statements were found to be harmful.

Retrieved information appeared to be meaningful in 98% of queries. Tool usage was correct in 56% of cases, while 25% involved failed or missing data, 11% involved incorrect tool usage, and 8% involved useless tools.

Going forward, the researchers are working to validate their system with added tools and evaluation metrics in a cohort of more than 700 patients. Additionally, Dr. Corso noted, “We will enhance the trustworthiness of this system by introducing human-in-the-loop approaches.”

DISCLOSURE: Dr. Corso has served on advisory boards for EVENTs and Merck.

Expert Insight

Danielle S. Bitterman, MD, Assistant Professor of Radiation Oncology, Harvard Medical School, Boston, and Clinical Lead for Data Science/Artificial Intelligence (AI), Mass General Brigham Digital, shared the following commentary regarding the potential of an agentic AI approach in treatment decision-making:

“Agentic AI, where AI models such as large language models have ‘agency’ to interact with each other and use digital tools within multistep workflows, can enable AI to conduct more complex, high-order clinical tasks.

This study, although small and limited to a single institution, is an exciting and innovative proof-of-concept for agentic AI for clinical decision support for immunotherapy. Cancer treatment decision-making often requires contextualizing multimodal patient data within the large and rapidly evolving medical knowledge base, and agentic AI is a promising approach for tackling these complex challenges.

A critical next step for understanding and realizing the value of agentic AI is the development of standardized evaluation and post-deployment monitoring frameworks, especially as agents become more autonomous and used for high-risk, high-reward tasks such as treatment decision support. It is important to keep in mind that while some large language models output reasoning as well as a final decision, these are not always reliable indicators of how the system arrived at its decision.”

ASCO AI in Oncology is published by Conexiant under a license arrangement with the American Society of Clinical Oncology, Inc. (ASCO^®). The ideas and opinions expressed in ASCO AI in Oncology do not necessarily reflect those of Conexiant or ASCO. For more information, see Policies.