Automation projects converge on a common difficulty: the intended execution path is comparatively straightforward to implement, whereas the cases that deviate from it determine whether the deployment succeeds. Large language models (LLMs) have altered the range of tasks that can be placed on the intended path, but they have not removed the deviation problem. In this article, we argue that a deterministic automation backbone, coupled with an explicit exception-handling loop, is a prerequisite for the safe use of probabilistic components in clinical and operational workflows.
Probabilistic Components Are Not Deterministic Components
Current enthusiasm for LLMs in clinical and operational workflows frequently treats the model as a form of retrieval: a question is submitted and a correct answer is returned. This framing is understandable, because model output is fluent and confidently formatted, and therefore superficially resembles the output of a rules engine. The underlying mechanism, however, differs substantially. An LLM is a probabilistic next-token predictor that samples from a distribution over plausible continuations conditioned on training data and a prompt. It does not execute a fixed, auditable procedure that returns an identical input-to-output mapping on repeated invocation.
This distinction has direct clinical consequences. A model that summarizes a discharge note incorrectly, misinterprets a laboratory trend, or fabricates a citation is not malfunctioning relative to its own objective; it is generating a highly plausible continuation that is, in this instance, incorrect. Notably, the failure does not present as a crash or an error code. It presents as a fluent, well-formatted, incorrect answer that is structurally indistinguishable from a correct one. Deterministic software tends to fail conspicuously, through null references, stack traces, or rejected input. Probabilistic software fails inconspicuously, in prose, and the resulting output may not prompt user verification.
This is not an argument against the use of LLMs in automation. It is an argument against deploying them as drop-in replacements for deterministic logic. They constitute a different class of component with a different failure signature, and the surrounding system must be designed with that signature in mind.
Exception Handling as the Determinant of Automation Success
Setting the AI component aside, a longstanding principle of automation remains applicable: an automation is not characterized by its performance on the cases for which it was designed, but by its behavior on the cases for which it was not. Operational workflows, including scheduling, prior authorization, laboratory result routing, claims adjudication, and sepsis alerting, exhibit a long tail of edge cases, malformed inputs, ambiguous states, and genuinely novel situations. If the system cannot detect and respond to these exceptions, the automation should be considered unsuccessful irrespective of the quality of its core logic. A workflow that is 95% automated but silently mishandles the remaining 5% may be less safe than a manual process, because routine human review of those cases has been withdrawn.
Effective exception handling comprises three distinct capabilities. Omission of any one of them interrupts the loop.
1. Detection
The system must reliably identify when it has operated outside its zone of competence. This requires explicit boundary conditions, confidence thresholds, and schema and range validation. For an LLM component specifically, it requires that low-confidence or out-of-distribution outputs be treated as exceptions rather than as answers. A system that cannot distinguish between having handled a case and having merely produced output for it will detect nothing.
2. Efficient handling
Detection without a defined response relocates the bottleneck rather than removing it. Efficient handling requires an explicit escalation path, whether to a human reviewer, a fallback rule, or a narrower and more constrained model call, with sufficient context attached that the receiving process does not begin without information. The objective is a handoff measured in seconds to minutes rather than an unresolved item identified after an extended interval.
3. Recording of inputs, decision criteria, and response
This step converts exception handling from a recurring cost into an accumulating asset. Each resolved exception constitutes a labeled example comprising the input, the feature that made it exceptional, the criteria applied by the resolver, and the correct response. Recorded systematically, these examples are precisely what is required either to tighten the deterministic rules or to adjust the prompt or fine-tuning data of the AI component so that equivalent cases are handled automatically thereafter. In the absence of this step, the exception rate remains approximately constant over time, because each case is adjudicated independently regardless of how frequently equivalent cases have occurred.
It should also be noted that healthcare facilities are not equivalent. The correct response to a given set of exception conditions at one institution may not be correct at another. This is a further reason that an LLM cannot constitute a complete solution: each facility requires the ability to specify its preferred response, and this is not a task that a general-purpose language model performs efficiently.
Complementarity of Deterministic Automation and AI
Considered together, these observations suggest a specific architecture. Deterministic automation and AI are complementary rather than competing, and each addresses the principal weakness of the other. Deterministic automation is precise, auditable, and fast, but brittle: it operates only within its specified scope, and anything outside that scope must be treated as an exception. AI methods, and LLMs in particular, generalize to situations that were not anticipated at design time, but they are probabilistic and cannot reliably determine, unaided, when their own output is incorrect.
The complementarity is realized by assigning each component the function it performs well. A deterministic backbone should own orchestration, business rules, state management, and the audit trail, that is, the elements of the system that must behave identically on every execution and must be able to demonstrate that they did. The AI component should be scoped to judgments that benefit from flexibility: interpretation of free text, weighing of ambiguous evidence, and generation of a first draft. The interface between the two, the exception-handling layer, constrains the probabilistic component by capturing its low-confidence and out-of-bounds outputs and returning them as structured training signal rather than as silent errors.
Figure 1 illustrates this arrangement: a deterministic backbone along the central axis, exception handling as the mechanism that detects and manages cases falling outside the rules, and AI-based decision-making as the mechanism that extends the system into ambiguous territory. Without the backbone, the remaining components lack an auditable substrate. Without exception handling and the AI component, the system cannot operate beyond what was explicitly specified. Both are required.
Under this architecture, the reliability of the language model becomes a less critical parameter, because the model is not the final safeguard. The deterministic backbone constrains what the model can affect, the exception-handling loop accumulates a record of each failure, and system performance improves as a function of that record rather than as a function of anticipated improvements in the underlying model. We suggest that this property, improvement that does not depend on the model itself improving, is the principal argument for the design.