An explainable AI–based decision support tool using real-world clinical and blood data improved physicians’ ability to predict outcomes with immunotherapy in patients with advanced non–small cell lung cancer (NSCLC), according to findings from the international I³LUNG study published in Nature Medicine. Only models using such data outperformed traditional single biomarkers and clinical scores.

The study also evaluated models incorporating imaging, digital pathology, and genomic data, although the added benefit of multimodal integration remained uncertain. 

“We need smarter tools,” said senior study author Marina Chiara Garassino, MD, Professor of Medicine at The University of Chicago Medicine, in an institutional press release, which noted that current immunotherapy treatment decisions rely heavily on PD-L1 expression, but its ability to predict benefit is limited.

The investigators, including corresponding and lead study author Arsela Prelaj, MD, PhD, of Fondazione IRCCS Istituto Nazionale dei Tumori Milano, wrote, “The I³LUNG project is a pioneering framework showing the clinical usefulness of AI tools.” 

Study and Model Details

I³LUNG used a two-phase approach to develop a medical device to predict immunotherapy efficacy in patients with NSCLC. The current analysis focused on the retrospective phase and included 2,396 patients with stage IIIC to IVB disease who were treated with immunotherapy alone or in combination with chemotherapy across six centers in Italy, Germany, Greece, Israel, Spain, and the United States.

The investigators assembled clinical and blood data (ie, sex, Eastern Cooperative Oncology Group [ECOG] performance status, smoking status, PD-L1 expression, metastatic sites [bone, liver, brain], neutrophil-to-lymphocyte ratio, and lactate dehydrogenase), as well as CT, digital pathology, and genomic data. Clinical and blood data were available for all 2,396 patients, whereas the other data types were available for subsets of the population; 339 patients had complete data across all modalities.

The investigators evaluated machine learning and deep learning models using clinical and blood data alone as well as multimodal models incorporating the other available data types. The multimodal approaches used machine learning with early fusion (MLEF) and deep learning with intermediate fusion (DLIF), with fusion referring to the stage at which different data types are combined. Early fusion merges the features from each modality up front, whereas intermediate fusion processes each modality separately first and then combines them.

For MLEF, features were concatenated across modalities, and least absolute shrinkage and selection operator (LASSO) regression was applied to select the most relevant predictors. Unlike MLEF, which could not handle patients with missing modalities, DLIF was designed to address this limitation across patient cohorts; it used an attention-based multiple instance learning framework to integrate the data, with modality-specific encoders projecting each modality into a shared embedding space and a crossmodal reconstruction loss to promote consistent and interpretable latent representations across modalities.

Clinical endpoints included real-world overall survival, survival status at 6 and 24 months, and disease control rate. The main predictive cohort comprised 2,075 patients with stage IV NSCLC who received immunotherapy in the first-line or later-line metastatic setting. Of this population, 1,824 were divided into training (n = 1,550) and test (n = 274) sets; a cohort of 251 patients from The University of Chicago was used for external validation.

The investigators also conducted a clinical usability study, in which 20 physicians (10 lung cancer experts and 10 nonexperts) evaluated 100 real-world patient cases for disease control and overall survival, first using clinical and blood data and images when available, and then with the addition of clinical and blood–only machine learning model output and SHapley Additive exPlanations (SHAP)-based values.

Key Findings

Clinical and blood–only machine learning and deep learning models demonstrated consistent performance across outcomes, the investigators wrote, with area under the curve values of up to 0.77 in the test set. Performance declined in external validation, with areas under the curve ranging from 0.55 to 0.72, which they attributed to population differences.

In the test set, the AI models were found to significantly outperform PD-L1 expression, ECOG performance status, neutrophil-to-lymphocyte ratio, lactate dehydrogenase, and the Lung Immune Prognostic Index.

Multimodal integration with MLEF, incorporating clinical and blood, CT, and digital pathology data, appeared to be associated with higher performance, with an area under the curve reaching 0.88 in cross-validation. However, the investigators noted that its incremental benefit remained uncertain, as the improvement did not translate to the test and external validation cohorts. DLIF similarly did not seem to demonstrate an added benefit from multimodal integration.

Based on the clinical usability study, both lung cancer experts and nonexperts improved their predictions when using the explainable AI clinical and blood–only machine learning tool. With AI support, physician sensitivity for predicting disease control increased from 0.72 to 0.87 (P = .0011), and the probability of a correct overall survival prediction increased by 36%, comprising 14% for experts and 61% for nonexperts.

“This alignment between machine and clinical logic is essential for building trust in AI-assisted decision-making,” Dr. Garassino said.

She concluded, “I³LUNG establishes a new benchmark for AI in thoracic oncology. Decision support tools built even from routinely available clinical data can outperform the biomarkers we rely on today.”

Prospective validation of both the clinical and blood–only and multimodal decision support systems is currently underway in more than 2,000 patients.

DISCLOSURES: The I³LUNG project was funded by the European Union’s Horizon 2020 research and innovation program. For full disclosures of the study authors as well as data and code availability, visit nature.com.

ASCO AI in Oncology is published by Conexiant under a license arrangement with the American Society of Clinical Oncology, Inc. (ASCO^®). The ideas and opinions expressed in ASCO AI in Oncology do not necessarily reflect those of Conexiant or ASCO. For more information, see Policies.