Comparing Accuracy of Open-Source AI Models for EGFR Mutation Prediction in Lung Adenocarcinoma

Open-source AI pathology models can help to predict the presence of EGFR mutations in samples from patients with lung adenocarcinoma. However, differences are found between models, especially when accounting for ancestral subgroups.

“This cohort study found that AI-based pathology tools may serve as preliminary adjuncts for EGFR prediction in lung cancer, though performance differences by ancestry warrant careful interpretation,” Rakaee et al wrote in their report published in JAMA Oncology.

Study Details

The study included patients with lung cancer from two cohorts: Dana-Farber Cancer Institute (DFCI) from June 2013 to November 2023, and a European-based trial (TNM-I) from August 2016 to February 2022. Paired next-generation sequencing data and hematoxylin and eosin (H&E)–stained whole-slide images were available for all patients. In the DFCI cohort, genetic ancestry was assigned on the basis of germline genotype data.

Two open-source AI pathology models were evaluated for prediction of EGFR mutation status: EAGLE and DeepGEM.

Key Findings

A total of 2,098 patients were included in the analyses.

Among 1,759 patients in the DFCI cohort, EGFR mutations were detected in 432 (25%). areas under the receiver operating characteristic curve for the two AI pathology models were 0.83 (95% confidence interval [CI] = 0.81–0.85) for EAGLE and 0.68 (95% CI = 0.65–0.70) for DeepGEM. Further, the EAGLE probability scores did not change significantly according to EGFR subtypes.

Among 339 patients in the TNM-I cohort, EGFR mutations were detected in 50 (15%). Areas under the receiver operating characteristic curve were 0.81 (95% CI = 0.74–0.88) for EAGLE and 0.75 (95% CI = 0.68–0.83) for DeepGEM.

In terms of ancestry, in the DCFI cohort, 54 patients were identified as African, 101 as American, 95 as Asian, and 1,465 as European. Areas under the receiver operating characteristic curve for predicting EGFR mutation for the better-performing AI model, EAGLE, included 0.85 (95% CI = 0.72–0.94) for African ancestry, 0.68 (95% CI = 0.55–0.78) for Asian ancestry, and 0.84 (95% CI = 0.81–0.86) for European ancestry.

In analysis by tissue sample type, the areas under the receiver operating characteristic curve for the better-performing model were 0.66 (95% CI = 0.56–0.76) for pleural specimens and 0.86 (95% CI = 0.83–0.88) for lung specimens.

Use of AI-guided triage analyses was estimated to result in a 57% reduction in rapid EGFR testing, with a sensitivity of 0.84 and a specificity of 0.99 in identifying EGFR mutations. “Such triage could accelerate treatment initiation and reduce molecular assay burden, particularly in resource-limited settings,” the study authors concluded.

Mehrdad Rakaee, PhD, of the Department of Cancer Genetics, Institute for Cancer Research, Oslo University Hospital, Oslo, Norway, is the corresponding author for the JAMA Oncology article.

DISCLOSURE: The study was supported by the Norwegian Cancer Society. For disclosures of the study authors, visit jamanetwork.com.

ASCO AI in Oncology is published by Conexiant under a license arrangement with the American Society of Clinical Oncology, Inc. (ASCO^®). The ideas and opinions expressed in ASCO AI in Oncology do not necessarily reflect those of Conexiant or ASCO. For more information, see Policies.