Microsoft's latest medical AI research raises some fascinating questions about clinical reasoning, diagnostic accuracy, and how artificial intelligence could change healthcare.
TL;DR
Microsoft's MAI DxO correctly diagnosed 83% of 304 complex cases drawn from The New England Journal of Medicine, against 36% for physicians working with textbooks and online references, and 22% for physicians working without them.
On a separate set representing more typical emergency department presentations, the system reached 80% against 47% for physicians.
Those numbers are measured on written case records. A real consultation starts with what a patient is able to describe, which is where diagnosis most often goes wrong.
The harder questions are organisational. Who owns a diagnosis across five care teams, whether ignoring an accurate recommendation becomes unethical, and what happens to clinical skill once a machine proposes the answer first.
Deployment is an integration problem before it is an accuracy problem. The system needs records, imaging, laboratory results, identity controls and an audit trail, and most hospital estates do not hand those over easily.
A doctor spends years studying medicine, examining patients, interpreting symptoms, and making decisions that can determine whether someone lives or dies. Microsoft has developed an AI system that outperformed physicians at diagnosing medical cases in controlled evaluations.
In recent Microsoft research, the AI Diagnostic Orchestrator, known as MAI DxO, correctly diagnosed 83% of complex medical cases drawn from The New England Journal of Medicine. Physicians working with textbooks and online references achieved 36% accuracy. Without those resources, their accuracy dropped to 22%.
On another set of cases representing more typical emergency department presentations, the AI achieved 80% accuracy compared with 47% for physicians.
The gaps are wide. They deserve attention from doctors, hospital administrators, healthcare technology companies, and anyone who has ever depended on a medical diagnosis. They also introduce a fascinating question.
If an AI system can diagnose diseases more accurately than a doctor, does that make the AI system a better doctor?
A pocket calculator offers a surprisingly useful place to start answering that question.
A Pocket Calculator Is Better at Maths Than a Doctor. So What?
Imagine a paediatric intensive care physician calculating the correct dose of a medication for a critically ill child.
The calculation might involve the child's weight, drug concentration, infusion rate, and several conversion factors. A mistake could have serious consequences. Give the same arithmetic problem to a basic calculator, and the machine will probably produce the correct answer faster and more consistently than the physician calculating mentally.
Nobody would conclude that the calculator should take charge of the intensive care unit.
The tool and the practitioner
The physician must evaluate the child's condition, select the appropriate medication, assess contraindications, monitor the response, identify complications, and make decisions as the clinical situation changes.
The calculator performs one important task exceptionally well. The physician carries responsibility for a much larger process. This distinction becomes useful when interpreting Microsoft's research.
Diagnosing a disease is an important part of medicine. Physicians must also gather reliable information, perform examinations, determine treatments, explain risks, and consider how different medical conditions interact.
A diagnostic benchmark measures a defined set of abilities. A doctor's professional responsibilities extend across an entire episode of care. Yet the calculator analogy has a limitation worth examining.
Microsoft's AI is doing something far more sophisticated than arithmetic.
The system can consider possible diagnoses, decide which medical information to request, evaluate test results, and revise its conclusions as new evidence becomes available. Those abilities resemble a substantial part of clinical reasoning. And that makes Microsoft's findings especially interesting.
What Microsoft Built and Why the Results Matter
Many earlier medical AI evaluations resembled examinations.
A system received a medical question containing symptoms, laboratory findings, and other relevant details. The task was to select the correct answer. Medical practice rarely works so conveniently.
When a patient arrives at a hospital, the physician typically begins with incomplete information. The doctor must decide which questions to ask, whether a physical examination suggests a particular condition, which tests are necessary, and when enough evidence exists to make a decision. Microsoft's research attempts to evaluate that process more realistically.
The company introduced 2 important components.
1. Sequential Diagnosis Benchmark (SDBench)
This evaluation framework presents medical cases progressively. Participants begin with limited information and request additional evidence as they investigate. The framework assesses diagnostic accuracy, the information requested, and the estimated costs of investigations.
2. Microsoft AI Diagnostic Orchestrator (MAI DxO)
This system coordinates multiple AI reasoning processes that function somewhat like a panel of virtual medical specialists. Different reasoning processes propose possible diagnoses, identify useful questions, recommend tests, and challenge emerging conclusions. As new information becomes available, the system updates its assessment.
Consider a patient arriving with fever, abdominal pain, and nausea.
Several conditions could explain those symptoms. The next useful step depends on what the physician already knows.
Where exactly is the pain? When did symptoms begin? Does the patient have relevant medical conditions? What does the examination suggest?
MAI DxO is designed to reason through such decisions progressively.
Microsoft evaluated the system using 304 complex cases from The New England Journal of Medicine and a separate collection of emergency department cases derived from the MIMIC IV ED dataset.
Fig 1 - MAI DxO versus physician diagnostic accuracy
The physician comparison deserves some context. The study evaluated generalist physicians working individually, while particularly complex cases often benefit from specialist expertise and collaboration between clinical teams. The benchmark also relied on structured case information. Direct patient encounters stayed outside the test design.
Microsoft's findings demonstrate stronger diagnostic performance under the tested conditions. Whether that advantage translates into improved patient outcomes, fewer missed diagnoses, or safer clinical decisions requires further evaluation in real healthcare settings.
Microsoft also evaluated the relationship between diagnostic accuracy and investigation costs. Its highest performing configuration achieved approximately 83% accuracy, with estimated investigation costs of $5,431 per case. A configuration optimized for cost achieved approximately 81% accuracy with estimated costs of $2,241.
The results suggest that AI diagnostic systems may be able to balance accuracy against the expense of additional investigations. These figures represent estimated costs within the evaluation framework. They do not include the full expense of deploying, maintaining, and supervising AI systems in hospitals.
That cost finding requires context. The study estimated investigation costs within a simulated diagnostic process. The findings do not establish equivalent savings across real hospital operations.
The physician comparisons also deserve careful interpretation. As explored in our analysis of healthcare AI governance and benchmark evaluation, the conditions under which AI systems are tested can shape how their performance should be interpreted.
Strong benchmark scores provide valuable evidence. Healthcare organizations still need to establish whether those capabilities perform reliably with real patients, diverse clinical conditions, and existing care processes.
Nevertheless, the findings represent meaningful progress in AI assisted medical diagnosis, extending the gains already visible in AI in medical imaging and radiology. Microsoft's results suggest that AI systems can perform sophisticated diagnostic reasoning across multiple steps while weighing the usefulness of additional tests. The next question concerns the information those systems receive.
The Patient Says Their Child Is Wheezing. The Doctor Hears Something Else.
Picture a parent arriving at a clinic with a young child. "My child has been wheezing all night," the parent explains.
Wheezing has a particular medical meaning. Clinicians generally associate the sound with airflow through narrowed airways.
Yet parents understandably describe unfamiliar sounds using everyday language. A blocked nose, noisy breathing, a rattling chest, or upper airway congestion might all be described as wheezing.
A doctor can listen to the child's breathing, observe respiratory effort, ask additional questions, and examine the patient. Those steps may reveal a different clinical picture from the parent's original description. Now consider an AI system receiving the statement, "My child has been wheezing all night."
The quality of its reasoning depends partly on whether that initial description accurately represents the clinical finding. A system may ask excellent questions while still receiving incomplete, ambiguous, or misleading answers. This becomes particularly important with patients who struggle to communicate.
Imagine an 89 year old patient with hearing difficulties, impaired vision, several existing medical conditions, and limited recollection of when symptoms began. The patient may describe dizziness without being able to distinguish between vertigo, lightheadedness, weakness, or unsteadiness. Those distinctions can affect diagnostic decisions.
A physician may need to observe the patient's movements, speak with a family member, review medication records, perform an examination, and reconcile conflicting accounts. Clinical diagnosis depends on identifying what the available information actually means.
The same concern applies to human clinicians, of course. Doctors can misunderstand symptoms, overlook details, and make assumptions. AI may eventually help reduce some of those mistakes through structured questioning and consistent review. That possibility leads to another interesting argument in Microsoft's favour.
What If AI Is Better at Asking Questions Than Doctors?
Medical students learn a familiar principle, when you hear hoofbeats, think horses before zebras. The idea encourages clinicians to consider common explanations before jumping to rare diseases. A patient with a headache is more likely to have a common condition than an exceptionally rare neurological disorder. Yet doctors must also recognize when an apparently ordinary symptom points toward something dangerous.
That balance requires experience, judgment, and a willingness to reconsider an initial assumption. The difference becomes more interesting when we examine how each approach gathers information. In the Microsoft study, the AI system worked through about 25.5 questions on average before ordering its first test, and the physicians in the comparison group asked about 5.6.
That does not automatically make every additional question useful. In urgent clinical situations, delaying a necessary investigation can have serious consequences. The important consideration is whether each question helps narrow the diagnosis and improves the next decision.
A good detective does not solve a case by questioning everyone in the city. The detective identifies which questions are worth asking.
The detective could request every forensic examination available. Another detective might ask a single question that changes the direction of the investigation. The quality of the investigation depends on how intelligently evidence is gathered and interpreted.
Microsoft's sequential diagnostic approach addresses this problem directly.
Its AI system evaluates which additional information could help distinguish between competing diagnoses. The research also considers the estimated cost of obtaining that information. In principle, such capabilities could help clinicians identify missed questions, consider overlooked diagnoses, and avoid some unnecessary investigations.
Those applications require further validation in real clinical environments. Still, the findings suggest that AI could contribute to the process of gathering clinical evidence, as well as interpreting the answers.
A Patient Sees Five Medical Teams. Who Owns the Diagnosis?
Another interesting opportunity for medical AI emerges when we follow a patient through a hospital. Imagine someone arriving at an emergency department with severe abdominal pain. An emergency physician examines the patient, orders blood tests, and requests imaging.
The patient subsequently moves to an acute medical team. A specialist reviews the case and requests additional investigations. Later, another department assumes responsibility.
Each team contributes expertise. Each makes decisions using the information available at that particular moment. As care moves between teams, maintaining a complete and current understanding of the patient's condition becomes increasingly important.
An earlier test result may be difficult to locate. A relevant symptom may receive insufficient attention during a handover. A recommendation recorded in one clinical system may not appear prominently in another.
Hospitals use established handover procedures and clinical records to manage these challenges. Fragmented information remains an important issue for healthcare technology. Now imagine an AI system maintaining an organized view of the patient's diagnostic journey.
The system could potentially track previous findings, record competing diagnostic explanations, identify unanswered questions, and alert clinical teams when new evidence challenges an earlier conclusion.
For example, an initial diagnosis might explain abdominal pain and fever. A later laboratory result could make that explanation less likely. An appropriately designed clinical support system could help ensure that the new finding receives attention.
Such capabilities would require access to reliable information across electronic health records, laboratory systems, imaging platforms such as PACS and RIS, and clinical documentation. The system would also need clear rules governing clinical review, permissions, privacy, and responsibility.
Microsoft has already identified this broader direction as part of its medical AI ambitions. The company is collaborating with Mayo Clinic on research intended to bring advanced AI capabilities together with clinical expertise and patient information collected over time.
The potential extends beyond diagnosing individual cases. Future systems could help clinicians maintain continuity across complex care journeys. That would address a practical healthcare challenge that exists independently of AI.
If AI Is More Accurate, Could Ignoring Its Advice Become Unethical?
Imagine a future medical consultation. A physician reviews a patient's symptoms, examination findings, and laboratory results before concluding that a particular disease is the most likely explanation. An AI diagnostic system reviews the same case and recommends a different diagnosis.
The doctor disagrees.
Now introduce one additional assumption. Extensive clinical studies have established that the AI system performs more accurately than physicians for this particular condition and patient population.
Who should make the final decision?
The doctor may recognize contextual information that the AI failed to consider. The AI may have identified evidence that the physician overlooked. The disagreement creates a professional responsibility to examine both conclusions carefully.
Medical ethics already requires clinicians to make decisions using appropriate evidence, professional judgment, and the patient's interests. As AI systems undergo rigorous clinical validation, recommendations from such systems could increasingly become relevant to that decision process. This introduces 2 difficult scenarios.
Scenario A. The doctor follows the AI recommendation.
The diagnosis turns out to be wrong, and the patient suffers harm.
Who should bear responsibility? The treating physician, the healthcare organization, the system developer, or some combination determined by the circumstances and applicable law?
Scenario B. The doctor rejects the AI recommendation.
The doctor's diagnosis proves incorrect. The AI recommendation would have identified the condition earlier.
Could the decision to disregard a validated diagnostic tool become relevant when evaluating the quality of care?
There is another danger in this relationship. The more accurate an AI system appears to be, the easier accepting its recommendations may become.
This is known as automation bias, the tendency to place excessive trust in automated decisions. A physician who repeatedly sees an AI system reach the correct conclusion might become less inclined to question an answer that conflicts with other clinical evidence. Healthcare organizations will need processes that make disagreements visible, encourage independent clinical judgment, and allow clinicians to escalate uncertain decisions for additional review.
These are questions about a possible future standard of care. Microsoft's present results do not establish that physicians have a legal or ethical obligation to follow MAI DxO recommendations. Clinical responsibility also involves more than selecting whichever option appears most accurate statistically.
A physician must understand whether a recommendation applies to the specific patient, how reliable the supporting evidence is, and what consequences follow from a mistake. AI systems introduce additional concerns.
They may produce confident explanations for incorrect conclusions. Their performance may change across patient populations. Problems with source data can affect recommendations, and clinicians may struggle to recognize errors when a system presents a persuasive rationale.
A doctor who relies excessively on AI could gradually become less practiced at evaluating complex cases independently. That creates another possibility worth considering. If the physician's role evolves toward supervising AI recommendations, maintaining the ability to challenge those recommendations becomes essential.
Are Doctors Heading Toward the Supermarket Self Checkout Moment?
Consider how supermarket shopping changed with self checkout machines. For customers purchasing a few items, scanning products and paying without waiting for a cashier can be convenient.
Some people prefer human assistance. Others value completing a routine transaction independently. Healthcare could eventually experience a similar shift in certain services.
A patient looking for guidance about a straightforward issue might appreciate an accessible digital service that collects symptoms, identifies concerning information, and directs them toward appropriate care. Someone managing a chronic condition may welcome technology that helps organize observations and prepares useful information for a consultation. A person facing a serious diagnosis may place greater value on an established relationship with a clinician who understands their medical history and personal circumstances.
The self checkout comparison also reaches a clear limit. An incorrectly scanned supermarket product creates a very different level of risk from an incorrectly assessed medical condition. Healthcare systems must consider clinical safety, patient autonomy, access to treatment, and professional accountability when introducing automation.
Still, the economic implications deserve attention. If AI systems eventually perform certain diagnostic tasks with validated accuracy, healthcare organizations may reconsider how they allocate clinical work.
Some tasks could require fewer manual steps, as AI prior authorization automation already demonstrates. Others might move toward supervised digital services. Clinicians could spend more time managing complex cases, treatment decisions, and patient communication.
There are potential risks as well. An organization might become tempted to reduce experienced clinical staffing before proving that AI supported services can maintain appropriate standards.
Medical training could also change. If young clinicians routinely receive diagnostic recommendations from AI, hospitals and educational institutions will need ways to preserve independent reasoning skills. The profession may eventually need to reconsider which abilities clinicians must master personally, which tasks can safely involve automation, and how those responsibilities should be assessed.
For patients, the central concern will remain the quality and safety of the care they receive.
Medical Superintelligence Meets the Hospital's Outdated Computer
There is something almost comical about discussing medical superintelligence while imagining a hospital workstation struggling to open an application designed 15 years ago.
The Windows 7 computer is an illustrative example of outdated infrastructure and not a claim about a particular hospital. The wider technical challenge is serious. Hospitals often depend on established software systems that support electronic health records, appointments, prescriptions, laboratory requests, imaging, billing, and administrative operations.
Introducing advanced AI into those environments involves substantial engineering and operational work.
Fig 2 - AI diagnostic system meeting the hospital IT stack
A diagnostic AI system needs timely access to relevant clinical information. That information may exist across several applications with different data formats, permissions, and integration capabilities.
Patient histories must remain accurate. Test results need appropriate context. Records must identify which patient and clinical encounter they belong to.
AI recommendations must reach the right clinician at the appropriate time. Healthcare organizations must also decide how to document recommendations, monitor system performance, manage failures, and protect sensitive patient information.
Clinical documentation tools provide a useful example of this integration challenge. Hospitals adopting structured medical history taking through ambient AI scribes must connect generated notes to electronic health records, preserve clinician review, and make clear how information enters the patient's record. Diagnostic AI would need the same attention to workflow, oversight, and data quality.
An impressive diagnostic accuracy score cannot resolve those requirements by itself. This is where custom healthcare software development and integration become practically relevant. Healthcare technology teams may need to connect existing records with clinical support systems, design review workflows, implement secure data exchange, and establish auditable processes for clinical decisions.
Organizations considering AI assisted diagnosis should evaluate the complete workflow before choosing a product or technical architecture. A useful starting point is to examine a narrowly defined clinical task, identify the available data, establish a reliable performance baseline, and determine how qualified clinicians will review the output. The next stage involves validating the system with representative clinical cases, addressing privacy and security requirements, and measuring performance during supervised use.
Microsoft recognizes the importance of clinical evaluation. The company has announced a partnership with Beth Israel Deaconess Medical Center in Boston to evaluate diagnostic reasoning capabilities in hospital environments. The planned prospective research will involve patients interacting directly with the system, with performance assessed against physicians' independent evaluations.
That research should provide important evidence about how sequential diagnostic AI performs when real patients supply information and clinicians work within normal hospital processes. For healthcare organizations, those findings may prove more relevant to adoption decisions than benchmark accuracy alone.
So, Will Microsoft's AI Replace Doctors?
The numbers from Microsoft's research are difficult to ignore. A system achieving 83% accuracy where physicians with reference resources achieved 36% represents a meaningful result under the study's conditions. Medical AI is progressing from answering examination questions toward more complex tasks involving evidence gathering, competing diagnoses, and investigation decisions.
That progress could change how physicians work.
AI systems may help clinicians consider diagnoses they would otherwise miss. They may recommend useful investigations and make relevant clinical evidence easier to review. With appropriate validation, some narrowly defined diagnostic tasks could eventually support greater automation.
The broader responsibilities of medicine create additional challenges. A doctor examines the patient, interprets findings in context, chooses treatment, monitors the response, communicates with families, and remains accountable for clinical decisions. Patients also bring personal circumstances, preferences, communication difficulties, and practical limitations that influence appropriate care.
Over time, the medical profession may devote increasing attention to supervising AI recommendations, managing complex decisions, and coordinating treatment across healthcare services.
Some existing tasks may require less human effort. New responsibilities could emerge around clinical AI oversight, safety evaluation, and interpreting automated recommendations.
Fig 3 - The supervising clinician
The extent and pace of these changes will depend on clinical evidence, regulation, economics, patient acceptance, and the reliability of healthcare technology. And this brings us back to the pocket calculator.
Calculators changed how professionals perform arithmetic. Physicians now routinely depend on sophisticated medical devices, diagnostic equipment, imaging systems, and clinical software. Each technological advance changes the work that humans need to perform.
Microsoft's research suggests that diagnostic reasoning itself may increasingly become an area where AI systems contribute substantial clinical value. The longer term question concerns how medicine will organize human expertise around those capabilities.
When a machine becomes exceptionally good at diagnosis, what should the doctor of the future spend more time doing?
Answering that question may become one of the most important challenges in healthcare AI.
Frequently Asked Questions
