Multimodal Model Uses Pathology Data to Predict Immunotherapy Response in NSCLC

Researchers developed a deep learning pathomics platform, Pathology-driven Immunotherapy Optimization (Path-IO), that integrates multimodal data to predict immunotherapy response in patients with non–small cell lung cancer (NSCLC).

“Routine pathology slides can be transformed into powerful, interpretable biomarkers,” said lead author Rukhmini Bandyopadhyay, PhD, a postdoctoral fellow at The University of Texas MD Anderson Cancer Center, during a press conference at the American Association for Cancer Research (AACR) Annual Meeting 2026.

Current biomarkers for stratifying immunotherapy regimens in patients with NSCLC have been inconsistent in identifying those most likely to respond. Dr. Bandyopadhyay noted that AI could help personalize immunotherapy due to its scalability, cost-effectiveness, and global deployability. However, she added that AI-based precision oncology models often lack multicenter data validation and biological interpretability.

The Path-IO model, trained on data from four sources, demonstrated improved performance over existing biomarkers, such as PD-L1 expression, in guiding immunotherapy decision-making. “[Path-IO] can provide independent value beyond the current clinical tools,” Dr. Bandyopadhyay said.

Study and Model Methods

Researchers evaluated the model using data from 797 patients with NSCLC treated with immune checkpoint inhibitors at MD Anderson. External validation included a cohort of 280 patients treated with immunotherapy from Mayo Clinic and Gustave Roussy, as well as participants in the phase III Lung-MAP S1400I clinical trial. In total, Path-IO was tested on more than 1,000 patients with available H&E whole-slide images and associated clinical and outcome data.

The model was initially trained to predict patient outcomes from pathology slides and then benchmarked against other foundational models. The researchers also correlated Path-IO outputs with multiplex immunofluorescence and spatial transcriptomics data from the clinical trial. To further enhance performance, multimodal inputs—including CT images, clinical characteristics, and other data—were integrated into the model. Pathomics data from Path-IO were combined with clinicopathological factors, such as age, sex, disease stage, PD-L1 expression, histology, and metastasis, as well as radiomics data from the previously developed Deep-CT model.

An in-house pathologist classified tissue regions into categories—tumor, stroma, immune, bronchi, vessel, necrosis, normal, and background—to generate a tissue habitat map. More than 71,000 annotated regions of interest were used to train the AI model to predict patient outcomes.

Findings

In internal validation using MD Anderson datasets, the model achieved an accuracy of 86.6% in predicting response to immunotherapy. External validation on The Cancer Genome Atlas and Clinical Proteomic Tumor Analysis Consortium datasets showed accuracies of 82.8% and 88.9%, respectively.

The model stratified patients into low- and high-risk groups, predicting better or worse survival with immunotherapy.

In the MD Anderson internal validation cohort, the hazard ratio (HR) for overall survival based on risk group was 2.51 (95% confidence interval [CI] = 1.51–4.16; P < .001). In the S1400I trial, the HR for overall survival by Path-IO risk group was 2.76 (95% CI = 2.26–3.36; P = .006).

Dr. Bandyopadhyay explained that the high-risk group was characterized by tumor-dominant regions, low immune infiltration, and limited tumor-immune interaction, whereas the low-risk group showed high immune infiltration and active tumor-immune interaction.

Multiplex immunofluorescence and spatial transcriptomics data confirmed the presence of greater immune cell infiltration in lower-risk patients and higher tumor cell abundance in higher-risk patients.

Path-IO demonstrated that biologically derived signals have prognostic value for survival independent of existing clinical factors—such as age, sex, and PD-L1—thereby enhancing predictive performance. The model also showed consistent survival prediction across clinically relevant subgroups, including by whole-slide image type and PD-L1 expression level.

Compared with existing foundation models, Path-IO outperformed or matched their performance across multiple cohorts. Dr. Bandyopadhyay also noted that Path-IO's feature weights are more easily interpretable than those of other models due to the tissue habitat map and its characterization of the tumor microenvironment.

Path-IO also stratified treatment among patients with high PD-L1 expression in the first-line setting, demonstrating potential clinical utility. In the low-risk group, the HR was 0.47 (95% CI = 0.23–0.97; P = .042), suggesting a benefit with immune checkpoint inhibition monotherapy without added chemotherapy. In the high-risk group, the model favored immune checkpoint inhibition combined with chemotherapy (HR = 1.58; 95% CI = 0.99–2.53 ; P = .055). The P value for interaction between groups was .034.

“So, we can say that [Path-IO] can potentially guide immunotherapy selection between IO monotherapy vs IO chemotherapy,” Dr. Bandyopadhyay said.

The composite, multimodal model—integrating clinicopathological, radiomics, and pathomics data—demonstrated superior predictive performance compared with PD-L1 expression, Path-IO alone (P < .001), and Deep-CT plus Path-IO (P = .01).

Looking ahead, Dr. Bandyopadhyay said the team aims to refine the model's predictive utility for treatment planning and to incorporate additional multimodal data to support broader precision oncology applications. The researchers also plan to conduct a prospective study to validate Path-IO in clinical settings.

Expert Perspective

Commenting on the study's context following the presentation, the AACR press conference moderator Ecaterina E. Dumbrava, MD, Co-chair of the AACR Annual Meeting Clinical Trials Committee, and Assistant Professor, Department of Investigational Therapeutics, The University of Texas MD Anderson Cancer Center, said, “Immunotherapy really changed how we treat non–small cell lung cancer, but I am still puzzled how, after 10 years of approvals of anti–PD-1 antibodies, we still are not able to personalize treatment for these patients. Yes, immunotherapy works, but it doesn't work for all patients, and it comes with certain side effects and toxicities. So we need better markers to predict responses for patients and to select patients to personalize their treatments. Integrating AI and using H&E slides to analyze them and integrate that in this composite model could really help to select patients who are more likely to benefit or patients who are unlikely to benefit from immunotherapy.”

“Now, as a clinician, I can tell you that we are always reserved about AI because we don't know what goes in it. It's like a black box, you don't know how it was done. But seeing all the models used and validation and input from clinicians, and having this model validated in that way really helps with trust and being able to generalize the findings of this. Hopefully this will be used in the clinic,” Dr. Dumbrava added.

DISCLOSURES: The study was supported by the National Institutes of Health, The UT MD Anderson Lung Moon Shot Program, and philanthropic contributions from Mrs. Andrea Mugnaini and Dr. Edward L.C. Smith, Rexanna’s Foundation for Fighting Lung Cancer, QIAC Partnership in Research (QPR) funding, and Permanent Health Funds. Scientific and financial support for the Cancer Immune Monitoring and Analysis Centers and Cancer Immunologic Data Center (CIMAC-CIDC) Network was provided by the National Cancer Institute. Dr. Bandyopadhyay reported no conflicts of interest. For full disclosures of the other study authors, visit abstractsonline.com. 

ASCO AI in Oncology is published by Conexiant under a license arrangement with the American Society of Clinical Oncology, Inc. (ASCO^®). The ideas and opinions expressed in ASCO AI in Oncology do not necessarily reflect those of Conexiant or ASCO. For more information, see Policies.