Last updated October 2026.
Every month, there’s a new headline about how close AI is to matching the intelligence of doctors. AI has beaten doctors at the medical board exam. AI passed the USMLE. AI will soon be prescribing on its own.
But passing a medical exam only tells you whether a large language model (LLM) knows the textbook answer. It doesn’t tell you whether the model knows when to question its first impression, which is one of the most critical components of delivering safe healthcare.
As AI in healthcare increasingly moves beyond administrative workflows into clinical recommendations, Cortico, one of Canada’s leading electronic medical record automation providers, tested the safety of 20 frontier large language models. The results are sobering.
Our MedSafe-Dx benchmarking found that the safest AI model unnecessarily escalated 28% of benign cases, while still missing 2% of those needing prompt attention. The misses are obviously the most imminent risk to patient safety. However, over-escalation also carries downstream risk through increased alert fatigue and added cognitive load for physicians sorting through inaccurate AI recommendations.
Why AI safety in healthcare matters now
AI tools are rapidly expanding into clinical decision-making. Ambient scribes often now provide clinical recommendations during patient visits. Software drafts insurance approval requests. Automated systems triage messages and symptoms before the patient even reaches a clinician.
This shift is being marketed as a way to reduce pressure on clinical teams and improve access to care. But operational efficiency in care delivery only counts when safety remains intact — and flaws are already beginning to show.
In a May 2026 report by Ontario’s Auditor General, all 20 government-approved ambient AI scribes evaluated generated inaccurate clinical notes, and 60% captured a different medication from the one the doctor prescribed. 45% of these approved scribes hallucinated treatment steps that were never discussed, while 85% missed critical details of patients’ mental health issues in at least one of two tests.
“But clinicians review every AI output and catch the errors” is the common refrain to this criticism. However, there’s growing evidence that even physicians with formal AI-literacy training underperform on clinical reasoning when exposed to flawed AI output.
Overall, it appears that there are safety implications from sorting through AI-generated inaccuracies, which are largely going unrecognized.
Current state of safety benchmarking
The safety benchmarks healthcare AI vendors point to are almost always knowledge tests — MedQA, USMLE-style multiple choice, and structured exams with one correct answer.
Such benchmarks are useful. But they only answer whether the model retrieves the right medical information in a textbook scenario. They do not tell us how safe AI is in real-world clinical scenarios.
That gap between knowing textbook medicine and exercising clinical judgment is where patient safety truly lives, and that’s what MedSafe-Dx was built to measure.
An overview of MedSafe-Dx
In response to growing concern about the lack of behavioral safety testing in healthcare, we’ve built our own AI safety benchmark as an open initiative in corporate social responsibility.
MedSafe-Dx moves past academic recall to evaluate two specific safety behaviors regarding AI in healthcare:
- Escalation Sensitivity: Does the model escalate care when a case needs prompt attention?
- Avoidance of Over-Escalation: Does it avoid escalating cases that are benign?
The benchmark evaluates 20 frontier models from OpenAI, Anthropic, Google, DeepSeek, Meta, xAI, Z.ai, and Moonshot AI across 900 simulated patient cases, with symptoms statistically correlated to diagnoses, drawn from the DDXPlus dataset.
Scoring is completely deterministic and rules-based with no “LLM-as-judge” subjectivity. To ensure independent verification, our full test set, scoring logic, and audit trails are fully open-source on GitHub.
What we found
Four findings clearly stood out from our research:
- Better safety comes at the cost of over-escalation. Models with stronger safety performance achieved it by over-escalating large volumes of benign cases. GPT-5.2 illustrates this most starkly. It missed only 2% of serious cases, but over-escalated on 40% of benign cases. This is the same dynamic that has eroded clinician trust in EHR alerts and clinical decision support for years. Newer AI models over-escalate less. The six highest-scoring models over-escalated on 27% to 29% of benign cases, while missing a similar share of serious cases (1.9% to 2.5%).
- No single model came out ahead. Models from Anthropic, OpenAI and Google took the top six places. Claude Opus 5.5 scored highest, but its lead over the next four models (Claude Fable 5.1, GPT-6.1 Sol, GPT-6 Astra and Gemini 3.1 Pro Preview) was too small to separate them. It’s plausible that on a different set of cases, any of the five could have come out on top.
- Each model missed serious cases. The model with the most misses, Meta’s small Llama 3.1 8B Instruct, missed 316 of the 640 serious cases. Even the highest scorer, Claude Opus 5.5, missed 12, and GPT-6.1 Sol and GPT-6 Astra each missed 11 of 25 pancreatic cancer cases.
- More expensive models aren’t safer. GPT-6 Luna answered all 900 cases for $0.46. GPT-6 Astra cost $21.97, about 48 times as much. Both models missed 14 of the 640 serious cases, and their overall scores were too close to separate.
In our opinion, the biggest danger isn’t just the model being wrong. It’s the model being wrong in nuanced, highly confident, medically correct language.
The leaderboard
The core value of MedSafe-Dx extends beyond the research paper, into the live leaderboard at msdx.cortico.health.
The leaderboard ranks 20 models by a score out of 100, where 100 is a perfect result on every case. The score penalizes unnecessary escalations as well as missed cases. Claude Opus 5.5 currently has the highest score, 73.5 out of 100, with the next four models too close to separate.
A scatter plot shows the safety vs. over-escalation tradeoff.
Anyone can verify the numbers. The frozen test set, scoring logic, and audit trail are open on GitHub.
Limitations
MedSafe-Dx relies on synthetic cases and proxy labels from disease-severity metadata rather than on clinician-adjudicated triage urgency. Therefore, it functions mainly as a stress test, not a clinical trial.
Real-world performance, where patient presentations are far messier and less structured, is more likely to be worse rather than better.
Cortico is independent of the AI developers it tests, but its own AI-assisted intake software performs the kind of triage MedSafe-Dx tests, and its AI Chat assistant for clinicians runs on OpenAI’s GPT-5.6 Terra. Three of the four highest-scoring models (Claude Opus 5.5, Claude Fable 5.1 and GPT-6 Astra) were also used to help build this version of the benchmark.
Implications for clinical practice
The MedSafe-Dx findings raise several important implications for using AI in clinical settings.
Scrutinize any AI-generated plans or triage advice. AI is effective for many tasks, from literature lookup to chart review to documentation drafting. But wherever the model is recommending a clinical course of action, that recommendation needs to be checked. The benchmark suggests this is true even for the safest models available.
Ask your AI vendor for peer-reviewed safety testing. If your ambient scribe, triage tool, or any other AI-enabled product makes safety claims, ask for the published research. If the only evidence is accuracy on MedQA or similar knowledge tests, that’s a gap worth pressing on.
Don’t treat “human in the loop” as a guarantee. This phrase gets used as reassurance, but research suggests that even AI-literate clinicians can be misled by confident-sounding wrong answers. Reviewing AI output is necessary, yet it isn’t a guarantee against missed errors.
Cortico’s next steps
MedSafe-Dx is an open benchmark we aim to continue expanding alongside future iterations of frontier LLMs.
Cortico’s approach to AI is shaped by our experience building clinical workflow automation software. We will continue incorporating AI where we can be confident it safely improves our products and the overall healthcare experience.
A preprint of our paper, MedSafe-Dx (v0): A Safety-Focused Benchmark for Evaluating LLMs in Clinical Diagnostic Decision Support is available for public access on medRxiv, including open-source code, datasets, and a live public leaderboard at msdx.cortico.health. The preprint describes the benchmark’s first version. The current method is set out on the methodology page.
We welcome independent evaluation and scrutiny.
Clark Van Oyen is the CEO and co-founder of Cortico, a healthcare technology company based in Richmond, BC. MedSafe-Dx was co-authored with Namrah Mirza-Haq.
