Comparing Accuracy of Open-Source AI Models for EGFR Mutation Prediction in Lung Adenocarcinoma

Objective:

To evaluate the accuracy of open-source AI pathology models in predicting EGFR mutations in lung adenocarcinoma, emphasizing the significance of performance differences by ancestry.

Key Findings:

• 2,098 patients were included; 25% of DFCI cohort had EGFR mutations, with 1,759 patients in DFCI and 339 in TNM-I.
• EAGLE achieved an AUC of 0.83 in DFCI and 0.81 in TNM-I; DeepGEM had AUCs of 0.68 and 0.75 respectively.
• EAGLE's performance varied by ancestry: 0.85 for African, 0.68 for Asian, and 0.84 for European.
• AI-guided triage could reduce rapid EGFR testing by 57%, with a sensitivity of 0.84 and specificity of 0.99.

Interpretation:

AI-based pathology tools may serve as preliminary adjuncts for EGFR mutation prediction, but performance varies significantly by genetic ancestry, necessitating careful interpretation and consideration of these differences in clinical settings.

Limitations:

• Study limited to specific cohorts which may not represent broader populations, and potential biases in cohort selection should be acknowledged.
• Performance differences by ancestry could affect generalizability.

Conclusion:

AI models like EAGLE show promise in predicting EGFR mutations, potentially accelerating treatment initiation in resource-limited settings, but further validation in diverse populations is essential.