Enhancing medical education with AI: A comparative study of various large language models in radiology

Noha Bakhsh, Ghofran A. Ageely

Abstract

Objective

To compare how various Large Language Models (LLMs) perform when solving text- and image-based medical student-level radiology multiple-choice questions (MCQs).

Introduction

In recent years, artificial intelligence (AI) has transformed numerous domains, including medicine and education. Large Language Models (LLMs), such as ChatGPT and DeepSeek, are advanced AI systems that utilize complex neural networks to process huge amounts of data for text, images, and even video generation [1]. In the context of medical education, LLMs have been able to answer questions related to learning, case simulations, decision-making, and research [2–5]. 

Methods

Study design

This prospective, cross-sectional comparative study evaluated the performance of large language models on a set of author-constructed multiple-choice radiology questions. The study did not involve human participants, identifiable patient data, or animal subjects. Accordingly, institutional review board approval was not required.

Results

Overall model performance

The mean accuracy across the seven evaluated LLMs was 79.9% (671/840 correct responses). Claude outperformed the other models numerically, with an accuracy of 87% (104/120). DeepSeek-V3.1-Terminus and ChatGPT-4o followed closely, with an accuracy rate of 82% (98/120) for both models.

Discussion

Evaluating LLMs on undergraduate radiology MCQs is increasingly important because these models are being used to support exam-style practice and generate radiology educational and assessment content [17–19]. The performance of LLMs can differ by domain and task format, and radiology-focused evaluations have shown that model strengths coexist with clear limitations, which supports the notion that such models should be reviewed carefully when used in a teaching and assessment-preparation context [17,20,21]. 

Conclusion

Advanced AI models are highly effective for text-based radiology reasoning but remain less reliable for image-based interpretation, and this multimodal performance gap, rather than the relative ranking of individual models, is the principal finding of this study. The strong performance of free models relative to paid versions highlights an opportunity to expand equitable access to AI-supported learning.

Acknowledgments

We sincerely thank Prof. Nadeem Butt for assisting with the data analysis and interpretation of the results.

Citation: Bakhsh N, Ageely GA (2026) Enhancing medical education with AI: A comparative study of various large language models in radiology. PLoS One 21(11): e0360928. https://doi.org/10.1371/journal.pone.0360928

Editor: Tarun Kumar Suvvari, Squad Medicine and Research (SMR), INDIA

Received: June 3, 2026; Accepted: October 2, 2026; Published: November 6, 2026

Copyright: © 2026 Bakhsh, Ageely. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All relevant data are within the paper and its Supporting information file (S1 Data), which contains the complete item-level response accuracy dataset for all seven large language models across all 120 study questions.

Funding: The project was funded by KAU Endowment (WAQF) at king Abdulaziz University, Jeddah, Saudi Arabia. The authors, therefore, acknowledge with thanks WAQF and the Deanship of Scientific Research (DSR) for technical and financial support.

Competing interests: The authors have declared that no competing interests exist.