A doctor and an AI system review a chest CT scan together, reflecting how GPT-5’s medical reasoning capabilities are being studied as potential clinical support tools. Image Source: ChatGPT-5
GPT-5 Surpasses Doctors in Medical Reasoning Benchmarks
Key Takeaways:
GPT-5 scored 95.84% on the MedQA clinical exam benchmark, +4.80% higher than GPT-4o.
On multimodal reasoning tasks, GPT-5 reached 70%, improving by nearly +30 points over GPT-4o.
The model outperformed pre-licensed doctors, scoring +24.23% higher in reasoning and +29.40% in understanding.
GPT-5 correctly diagnosed rare conditions like Boerhaave syndrome, recommending appropriate clinical tests.
Researchers found GPT-5 gave clearer explanations, fewer hallucinations, and more consistent results, but still made errors in rare image-heavy cases.
GPT-5 Raises Benchmark Standards
The Emory University team tested GPT-5 against both AI predecessors and medical professionals across a series of reasoning benchmarks.
On MedQA, a dataset modeled on U.S. medical licensing exam questions, GPT-5 achieved 95.84% accuracy. In effect, this test serves as the closest AI equivalent to a doctor’s board exam, making the result especially significant. GPT-4o, the previous best model, scored 91.04%.
When tested on multimodal reasoning tasks, which combine written histories with medical imaging and lab data, GPT-5 scored 70%. That represents a nearly 30-point increase over GPT-4o and shows the model’s growing ability to handle the types of information physicians rely on every day.
Outperforming Human Baselines
In expert-level tests, GPT-5 also outpaced pre-licensed medical professionals. It performed +24.23% better in reasoning and +29.40% better in understanding.
One case study highlighted in the paper showed GPT-5 diagnosing Boerhaave syndrome, a rare and dangerous esophageal rupture. The system not only identified the condition from CT scans and lab results but also recommended a Gastrografin swallow test, consistent with clinical practice.
More Reliable, Fewer Errors
The study went beyond accuracy scores to look at how GPT-5 reasons. Physicians reviewing its responses noted that GPT-5 produced clearer and more interpretable explanations than GPT-4o, making its thought process easier to follow.
Other improvements included:
Fewer hallucinations: GPT-5 fabricated less information than GPT-4o, an important factor for medical trust and AI safety.
Greater consistency: Its performance remained steady across repeated test runs, reducing concerns about variability.
Remaining challenges: GPT-5 still made mistakes on some rare or complex image-based cases, especially when subtle radiology details were involved.
These findings suggest that GPT-5 is not just more accurate but also more stable and trustworthy, qualities essential for clinical decision-making.
Q&A: GPT-5 in Medical Reasoning
Q: How did GPT-5 perform on medical exams?
A: It scored 95.84% on MedQA, a +4.80% gain over GPT-4o.
Q: How did GPT-5 handle multimodal reasoning?
A: It reached 70%, nearly +30 points higher than GPT-4o.
Q: Did GPT-5 outperform human doctors?
A: Yes. GPT-5 scored +24.23% higher in reasoning and +29.40% in understanding than pre-licensed doctors.
Q: What complex case did GPT-5 solve?
A: It diagnosed Boerhaave syndrome and recommended a Gastrografin swallow test.
Q: What are GPT-5’s main limitations?
A: It still struggled with rare, image-heavy cases, though it showed fewer hallucinations, clearer explanations, and more consistent results than GPT-4o.
What This Means
The Emory study shows that GPT-5 is not only more accurate than earlier models but also more dependable in how it reaches its answers. For medicine, this makes AI a step closer to being a tool that clinicians can examine, trust, and integrate into their practice.
At the same time, the researchers stress that benchmarks are not the same as real-world hospital settings. GPT-5 still missed certain image-based diagnoses, and its strengths so far apply only in controlled test environments.
The results nonetheless mark an important shift. With fewer hallucinations and steadier performance, GPT-5 begins to address one of the central questions of AI in medicine: not just can it get the right answer, but can it be relied on to do so consistently.
Editor’s Note: This article was created by Alicia Shapiro, CMO of AiNews.com, with writing, image, and idea-generation support from ChatGPT, an AI assistant. However, the final perspective and editorial choices are solely Alicia Shapiro’s. Special thanks to ChatGPT for assistance with research and editorial support in crafting this article.