Medical Superintelligence
How it works
The demonstrated realization relies on an orchestrator (MAI-DxO) that simulates a panel of physicians: it proposes differential diagnoses, strategically selects high-value, cost-effective tests and iteratively queries a gatekeeper model that reveals findings only on request. Performance is measured not only by diagnostic accuracy but also by the cost of visits and tests. The approach is model-agnostic — it works with models from OpenAI, Google Gemini, Anthropic Claude, xAI Grok, DeepSeek and Meta Llama.
Problem solved
Most evaluations of language models in medicine rely on static vignettes and multiple-choice questions that fail to capture the iterative, cost-sensitive nature of real-world diagnosis. The concept addresses whether and how AI can match experts in a realistic, sequential diagnostic process.
Components
A model-agnostic orchestrator that proposes differential diagnoses and strategically selects high-value, cost-effective tests, running a sequential diagnostic process.
A benchmark that turns 304 challenging NEJM-CPC cases into stepwise diagnostic encounters with a gatekeeper model that reveals findings only on request; it scores both accuracy and cost.
Implementation
Results were obtained on 304 challenging NEJM-CPC cases and with a simulated gatekeeper model; they are not representative of routine clinical practice and are not clinically validated.
Evolution
MAI-DxO paired with OpenAI's o3 reaches 80% diagnostic accuracy (vs about 20% for generalist physicians) and lowers test costs; 85.5% in the maximum-accuracy mode.
Hyperparameters (configurable axes)
The language model powering the orchestrator; the approach is model-agnostic (e.g. o3, Gemini, Claude, Grok, DeepSeek, Llama).
The trade-off between accuracy and test cost; the configuration tuned for maximum accuracy reaches 85.5%.