Hybrid Rule-Based and Agentic Workflows in Healthcare: A Reliability Argument

Consider an autonomous agent managing a patient's transfer from the intensive care unit to a general ward. The system reads the chart, synthesizes the notes, orders medications, schedules follow-up laboratory studies, assigns nursing tasks, and drafts the handoff communication. Each individual step may be performed competently. Each step is also probabilistic. By the completion of the seventh step, the probability that every step has been performed correctly is substantially lower than the per-step accuracy would suggest.

This constitutes the central design problem in current clinical AI. Autonomous agents, defined as systems that read, reason, plan, and act across multiple sequential steps, are capable of handling complexity and novelty that rule-based systems cannot address. However, their reliability characteristics differ from those of deterministic systems in a manner that is consequential for clinical use: error compounds multiplicatively. In a seven-step workflow in which each step is 95% accurate, the probability that the entire workflow completes without error is approximately 70%. At ten steps the corresponding value is approximately 60%, and at twenty steps approximately 36%.

In most applications an incorrect intermediate result is recoverable. In clinical care it may correspond to an omitted medication, an incorrect diagnosis, or an adverse event. Healthcare therefore tolerates neither the reliability arithmetic nor the consequences of purely probabilistic multi-step workflows.

The appropriate response is neither to abandon agent-based systems, which offer genuine capabilities, nor to revert to rule-based systems whose brittleness is well documented. We argue for a deliberate hybrid architecture in which rule-based guardrails govern high-stakes deterministic decisions and agents perform the flexible, reasoning-dependent tasks for which rules do not generalize. The relevant design question is therefore not whether to use an agent or a rule, but which class of decision warrants which approach and how the two are combined safely.

Rule-Based Clinical Decision Support

The history of clinical decision support is substantially a history of rule-based systems. The MYCIN system, developed by Shortliffe at Stanford in the early 1970s, is the canonical example: a rule-based expert system encoding approximately 600 if-then rules for antibiotic selection in bacterial infection. In blinded evaluations, MYCIN's performance matched or exceeded that of infectious disease specialists, and the system established computational decision support as a viable approach.¹

Rule-based systems have persisted for identifiable reasons. They are deterministic, in that identical input produces identical output on every execution. They are auditable, in that every decision traces to an explicit logical chain that clinicians, regulators, and counsel can inspect. They are predictable, in that a drug-drug interaction check that is correctly specified fires on every applicable case.

Rule-based systems also have well-documented limitations. They are brittle: clinical scenarios not anticipated during rule authoring are not handled. They require substantial expert effort to author and maintain. They do not generalize, in that each new drug, care pathway, or variant presentation requires additional rules. Over time, large rule libraries become internally inconsistent, generating contradictory recommendations and, ultimately, the phenomenon that has arguably caused more harm to clinical decision support than any algorithmic limitation: alert fatigue.

Override rates for clinical decision support alerts exceeding 90% have been documented in some systems.² Where rules fire frequently, indiscriminately, or for low-consequence situations, clinicians learn to dismiss them reflexively, and a genuinely critical alert is then presented in the same cognitive context as the numerous non-informative alerts preceding it. Rules deployed without governance may therefore reduce patient safety by conditioning clinicians to disregard warnings.

Agent-Based Workflows: Capability and Risk

Large language models and agent architectures provide a capability that rule-based systems do not: reasoning about novel situations, integration of heterogeneous information, and adaptation to context without explicit programming for each scenario. An LLM-based agent can read an unstructured clinical note, extract the relevant history, compare it against treatment guidelines represented in its training data, and produce a contextualized recommendation without any hard-coded conditional logic.

This flexibility is valuable, since clinical data are unstructured, ambiguous, and frequently inconsistent with the categorical logic that rules require. The relevant question concerns the associated cost.

That cost is probabilism. Every inference produced by an LLM is a sample from a distribution over possible outputs rather than a deterministic computation. Models may hallucinate, generating plausible but factually incorrect outputs with high apparent confidence. Ji and colleagues surveyed hallucination in neural language generation and documented its occurrence across major model architectures and task types, including domains requiring factual precision such as medical question answering.³ Current models produce factual errors at measurable rates, and those rates do not approach zero with increasing scale.

In a single-step application, in which a model generates a clinical summary that a clinician reviews before acting, probabilistic error is manageable, since the human reviewer constitutes the verification layer. In multi-step agentic workflows, in which the output of one step becomes the input to the next without intervening review, errors do not remain isolated. They propagate and compound.

Compounding of Error in Sequential Systems

The reliability arithmetic of sequential probabilistic systems is straightforward. If a single agent step succeeds with probability p, a workflow of n independent steps succeeds with probability pⁿ. The table below reports the resulting values.

A care coordination agent performing ten steps, comprising chart review, identification of follow-up requirements, formulary checking, medication ordering, laboratory scheduling, problem list updating, handoff note drafting, notification of the receiving team, flagging of unresolved issues, and encounter documentation, requires per-step accuracy of 99.9% to achieve 99% end-to-end reliability. Current LLM-based systems do not provide that guarantee for clinical tasks. Moreover, unlike software defects, which are deterministic and can be corrected once identified, probabilistic errors in language models are stochastic: an identical workflow may succeed on one execution and fail on the next without any change to the system.

Two additional factors make the practical situation less favorable than the independence calculation indicates. First, errors in agentic workflows compound semantically: a misinterpreted medication name at step 2 does not produce a null result or an exception, but a plausible incorrect value that becomes the input to step 3, which acts on it without indication of error. Second, interpretability decreases with chain length, and attributing a final error to its origin in a ten-step workflow is substantially more difficult than in a single-step system, precisely when accurate attribution is required for patient safety investigation.

Reported Failures of Single-Paradigm Deployments

IBM Watson for Oncology

IBM Watson for Oncology illustrates the risk of deploying agent-based systems into high-stakes clinical workflows without deterministic constraints. The system was introduced as capable of recommending cancer treatment plans and was deployed across multiple health systems internationally. Internal documents subsequently indicated that the system had generated treatment recommendations characterized by oncologists as unsafe and incorrect, in some cases recommending chemotherapy regimens inconsistent with institutional protocols.⁴

The failure was architectural. The system had been trained substantially on synthetic clinical vignettes authored at a single institution rather than on the distribution of actual patient cases. More consequentially, it operated without rule-based constraints capable of enforcing hard limits such as formulary restrictions, contraindication checks, and allergy alerts prior to presentation of a recommendation. The system was permitted to generate outputs that deterministic rules would have blocked before they reached a clinician. Several institutions subsequently discontinued the program.

The Epic Sepsis Model

At the opposite end of the complexity spectrum, the Epic Deterioration Index, deployed widely as a sepsis early warning system, illustrates how a single-step predictive model may underperform simple rule-based criteria outside its validation environment. Wong and colleagues conducted an external validation of the Epic sepsis model at a large academic medical center and reported an area under the receiver operating characteristic curve of 0.63 at the alert threshold used in practice.⁵ Simpler rule-based screening criteria, specifically SIRS and qSOFA, performed comparably while remaining fully transparent and auditable.

The implication is not that machine learning cannot outperform rule-based criteria, since in many contexts it does. The implication is that probabilistic models degrade unpredictably outside their training distribution, and that such degradation is not observable until external validation is performed. Rule-based systems degrade visibly and predictably, since their behavior and firing conditions are specified. An opaque probabilistic model may fail over an extended period while appearing to function normally.

The Continuing Role of Deterministic Logic

The case for retaining rule-based logic in clinical AI rests on properties of deterministic systems that probabilistic models do not replicate.

Determinism and medication safety. A rule that checks for penicillin allergy before permitting an amoxicillin order either fires or does not. Its behavior is identical across patients and executions, irrespective of when the model was last updated, what data were in the training set, or how the clinical note was phrased. An LLM performing the same check may formulate the query differently, interpret a note ambiguously, or, within a long multi-step workflow, omit it. This is a documented failure mode of systems in which language-based reasoning substitutes for deterministic data retrieval.

Auditability, liability, and learning. Following an adverse event, the clinical and legal systems require an account of why a recommendation was produced. A rule-based decision support system can produce a complete logical trace. A multi-step agentic workflow involving LLM reasoning frequently cannot. Seshia, Sadigh, and Sangiovanni-Vincentelli have argued that verifiable AI requires the capacity to formally specify and certify system behavior, which is tractable for rule-based components and remains largely unresolved for neural models at the reasoning layer.⁶

Checklists as minimal rule systems. Pronovost and colleagues reported that a five-item checklist for central venous catheter insertion, a purely rule-based intervention with no computational component, was associated with a reduction in catheter-related bloodstream infection in Michigan intensive care units from a baseline of 2.7 infections per 1,000 catheter-days to a median of zero over 18 months.⁷ The property that made this possible was determinism: the checklist was either completed or not, without probabilistic variation. For procedural safety, in which failure is severe and the required actions are specified in advance, deterministic rules outperform probabilistic systems not because they encode more knowledge, but because they cannot fail in the specific manner that matters.

Architecture of Hybrid Workflows

The productive formulation is not rules versus agents but rules as constraints and agents as reasoners. The two approaches are complementary when applied to appropriate layers of a clinical workflow.

Topol's account of human-AI partnership in medicine provides the relevant framing: neither unassisted human judgment nor autonomous algorithmic decision-making is optimal, and the combination outperforms either alone.⁸ The same principle applies within AI systems. Rules supply the structural constraints, comprising non-negotiable safety checks, hard eligibility criteria, and deterministic data retrieval. Agents supply the reasoning layer, comprising interpretation, synthesis, and generation of context-dependent recommendations that rules cannot anticipate.

Allocation by Risk and Reversibility

Maddox, Rumsfeld, and Payne identified the appropriate level of human oversight for a given decision as a central design question for healthcare AI.⁹ We extend this into an explicit allocation framework for hybrid workflow design.

High-risk, irreversible decisions should be governed by deterministic rules with mandatory human review. This category includes medication orders, critical value alerts, allergy checking, clinical trial eligibility, and contraindication screening. These must execute correctly on every applicable case. An agent that is 99% reliable leaves 1% of patients exposed, which is not acceptable where the consequence is a fatal drug interaction.

Medium-risk decisions with contextual complexity should be handled by agent reasoning bounded by rules. This category includes differential diagnosis generation, discharge planning recommendations, prior authorization narratives, and treatment plan summaries. These benefit from agent flexibility, since clinical context varies and rigid rules do not generalize. However, they should operate within a rule-based envelope in which the system verifies that no hard constraint has been violated before the output is presented.

Low-risk, reversible tasks may be agent-led with lightweight review. This category includes administrative scheduling, clinical note summarization, and generation of draft patient education materials. Here the consequence of an occasional error is low, the output is reviewed before action, and agent flexibility provides benefit at acceptable risk.

An Analogy from Aviation

Commercial aviation provides an instructive analogy. Modern aircraft operate on autopilot for the majority of each flight, using a system that adjusts trajectory, altitude, and speed on the basis of sensor data and routing logic. Those same aircraft also incorporate deterministic safety systems, including terrain proximity warning, traffic collision avoidance, and stall protection, which cannot be disabled by the autopilot and which activate reliably irrespective of the state of the autonomous systems.

The resulting safety record derives not from infallibility of the autopilot but from reliability of the deterministic envelope. When the autopilot errs, the deterministic systems constrain the consequence. Healthcare AI may reasonably be designed on the same principle: autonomous components for adaptive, context-dependent functions, and deterministic rules for the safety envelope. Neither is sufficient in isolation.

Char, Shah, and Magnus characterized the principal ethical challenge in implementing machine learning in healthcare as one of accountability: when an algorithmic system contributes to harm, responsibility must be attributable, and systems should be designed to preserve that attribution.¹⁰ Hybrid systems, in which rule-based logic is explicitly documented and agent reasoning is logged and bounded, preserve accountability in a manner that pure agent workflows, in which a chain of probabilistic inferences produces an outcome not attributable to any single decision, do not.

---

Conclusion

The comparison between rule-based and agent-based systems is frequently framed as a succession, in which rigid expert systems are superseded by flexible neural reasoners. This framing obscures the design question, which concerns not which paradigm is superior but which is appropriate to which layer of a clinical workflow.

Rule-based systems are brittle, require substantial maintenance, and do not accommodate the full complexity of clinical practice, and alert fatigue is a documented cost of rules deployed without governance. Agent-based systems are probabilistic, are difficult to audit, and exhibit failure probabilities that increase with workflow length. The compounding of error is not a defect that improved models will eliminate; it is a structural property of any system composed of uncertain sequential steps. No model achieves complete accuracy, and no multi-step pipeline constructed from uncertain components achieves end-to-end certainty.

Healthcare does not tolerate frequent error. The appropriate response is a deliberate hybrid architecture in which rules constitute the deterministic structure and agents operate as a reasoning layer within the constraints that structure imposes. Clinical AI systems likely to achieve sustained adoption are those whose designers asked not how much of the workflow can be automated, but for which decisions uncertainty is tolerable and for which correctness must be guaranteed. Answering that question explicitly, and constructing the architecture accordingly, is what makes clinical AI safe.

---

References

1. Shortliffe EH. Computer-Based Medical Consultations: MYCIN. New York: Elsevier/North Holland; 1976.
2. Sutton RT, Pincock D, Baumgart DC, Sadowski DC, Fedorak RN, Kroeker KI. An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ Digit Med. 2020;3:17. doi:10.1038/s41746-020-0221-y
3. Ji Z, Lee N, Frieske R, et al. Survey of Hallucination in Natural Language Generation. ACM Comput Surv. 2023;55(12):248. doi:10.1145/3571730
4. Strickland E. How IBM Watson Overpromised and Underdelivered on AI Health Care. IEEE Spectrum. 2019;56(4):24-31. doi:10.1109/MSPEC.2019.8678513
5. Wong A, Otles E, Donnelly JP, et al. External Validation of a Widely Implemented Proprietary Sepsis Prediction Model in Hospitalized Patients. JAMA Intern Med. 2021;181(8):1065-1070. doi:10.1001/jamainternmed.2021.2626
6. Seshia SA, Sadigh D, Sangiovanni-Vincentelli A. Toward Verified Artificial Intelligence. Commun ACM. 2022;65(7):46-55. doi:10.1145/3503914
7. Pronovost P, Needham D, Berenholtz S, et al. An Intervention to Decrease Catheter-Related Bloodstream Infections in the ICU. N Engl J Med. 2006;355(26):2725-2732. doi:10.1056/NEJMoa061115
8. Topol EJ. High-Performance Medicine: The Convergence of Human and Artificial Intelligence. Nat Med. 2019;25(1):44-56. doi:10.1038/s41591-018-0300-7
9. Maddox TM, Rumsfeld JS, Payne PRO. Questions for Artificial Intelligence in Health Care. JAMA. 2019;321(1):31-32. doi:10.1001/jama.2018.18932
10. Char DS, Shah NH, Magnus D. Implementing Machine Learning in Health Care — Addressing Ethical Challenges. N Engl J Med. 2018;378(11):981-983. doi:10.1056/NEJMp1714229