Microsoft AI system outpaces doctors in diagnostic accuracy trials

The Microsoft AI Diagnostic Orchestrator (MAI-DxO) functions by simulating a multi-disciplinary medical team, maintaining a list of working diagnoses while iteratively selecting investigations. Unlike static models that process data in one pass, this system functions like a clinician, requesting specific information and tests as the diagnostic path unfolds. To evaluate this, researchers developed the Sequential Diagnosis Benchmark (SDBench), which feeds patient data incrementally from cases sourced from The New England Journal of Medicine and the MIMIC-IV-ED database.

In emergency department simulations, the AI reached an 80.3% accuracy rate, significantly higher than the 46.8% achieved by the participating physicians. While the system demonstrated superior diagnostic precision, it initially incurred higher costs—averaging $5,431 per complex case versus the physicians' $4,725. However, when researchers introduced spending limits into the model, the AI maintained high accuracy rates while drastically reducing expenditure, hitting 79.6% accuracy at a cost of $1,303.

Despite these results, Harsha Nori, head of AI at Microsoft AI Health, emphasizes that the study focuses on the methodology of the diagnostic work-up rather than clinical replacement. The research has notable constraints: it relies on text-based records, omits direct interpretation of medical imagery, and operates on cases with established diagnoses. Because the study was conducted by Microsoft employees with equity in the firm and pending patents on the evaluation framework, the findings serve as a technical benchmark rather than a validated clinical procedure for routine practice.