Model Drift in Clinical AI: Mechanisms, Consequences, and Monitoring

A pneumonia detection system was placed into clinical service at a hospital in 2019 with 87% sensitivity on internal validation. It was integrated into the radiology workflow and was used routinely by radiologists. By 2025 the same system identified 68% of pneumonia cases, and the decline was not detected. No alert was generated, no monitoring threshold was exceeded, and no report was issued. The software was unchanged and technical monitoring indicated normal operation; what changed was the environment in which the model operated. This phenomenon is termed model drift: the progressive loss of accuracy that occurs when operational data diverge from the distribution on which a model was trained.¹

Definition

Model drift is defined as the degradation of an artificial intelligence system's accuracy over time, occurring not because the implementation has changed or failed, but because the data encountered in deployment have diverged from the training distribution. It is an intrinsic hazard of clinical machine learning, in which both patient populations and clinical practice change continuously. Consider a pneumonia detector trained on chest radiographs acquired in 2018. The model learned from a specific joint distribution comprising the characteristics of the imaged population (age, comorbidity, disease severity), the acquisition parameters (equipment, exposure settings, positioning), and the indications for imaging (referral patterns, pre-test probability, local practice). Performance was optimized for that distribution and validated on held-out data drawn from the same population.

By 2025 the operating environment has changed. The population has aged. The department has replaced scanners, altering detector characteristics, resolution, and reconstruction algorithms. The COVID-19 pandemic changed which patients presented for imaging and their severity at presentation. Reporting standards have been revised. The model performs the same computation on incoming studies, but those studies no longer resemble the training data, and accuracy declines as the divergence increases. The decline is progressive rather than abrupt: the model does not fail, it degrades, identifying fewer cases and missing more atypical presentations until an outcome review identifies the change. By that point, the associated harm, in the form of missed diagnoses and incorrect recommendations, has generally already occurred.

Data Drift and Concept Drift

Two forms of drift are conventionally distinguished, with different mechanisms and different remedies. Data drift, also termed covariate shift, occurs when the input distribution changes while the underlying relationship between input and output is preserved. The model encounters patients or scenarios that differ systematically from its training data although the underlying pathophysiology is unchanged. For example, a sepsis model trained exclusively on intensive care unit patients, with a mean age of 65 and the vital sign, laboratory, and comorbidity profile characteristic of critical illness, may be deployed hospital-wide to include emergency department patients. The model then encounters younger patients with different baseline physiology and comorbidity profiles. Sepsis is unchanged, but the model was calibrated to a different input distribution and may not generalize to the new one.

Concept drift is distinct: the relationship between inputs and outcomes itself changes, such that a previously predictive pattern loses predictive value or reverses in direction. Consider a readmission risk model that learned an association between Friday discharge and elevated risk, plausibly because outpatient support was limited over the weekend. The model flags Friday discharges accordingly. If the institution subsequently implements a weekend discharge planning team, improved handoffs to outpatient providers, and proactive follow-up, Friday discharges may become safer than midweek discharges. The relationship has reversed, but the model continues to apply the prior association and therefore identifies the wrong patients. This illustrates a general property of clinical prediction: healthcare is non-stationary, and interventions designed to address a problem may invalidate the patterns on which a model depends.

Why Drift Is Expected in Healthcare

Because clinical environments change continuously, drift should be regarded as a certainty rather than a possibility for any deployed model. Demographics, practice guidelines, and diagnostic technology all change over time, and each may degrade performance. The principal drivers are described below.

Changing Populations

The composition of the population presenting for care changes as a function of demography, migration, economic conditions, and policy. Communities age, altering the age distribution and the prevalence of age-associated comorbidity. Migration introduces populations with different exposures and disease patterns from those represented in training data. Changes in coverage, including Medicaid expansion and modifications to Medicare eligibility, alter access to care and therefore the severity and socioeconomic composition of the presenting population. Local economic events, such as the closure of a major employer, may alter the occupational health profile of a catchment area. A model trained on a 2018 population generates 2025 predictions under assumptions that may no longer hold, and performance may degrade on that basis alone, even when the disease process is unchanged.

Revised Clinical Guidelines

Practice changes as evidence accumulates and guidelines are revised, whereas a model does not change unless it is retrained. Consider a model predicting which patients with diabetes require retinal screening, trained on practice patterns reflecting an earlier set of American Diabetes Association HbA1c thresholds and visit intervals. When those recommendations are revised, the model continues to apply the standard encoded in its training data. Clinicians following the current guideline will disagree with the model systematically, not because it fails on its own terms, but because its decision criteria have not been updated.

New Diagnostic Tests and Technology

Diagnostic technology and information systems change continuously, and each change may introduce a gap between training and deployment data. Laboratories adopt new assays with different measurement characteristics and reference ranges. Imaging departments install scanners with higher resolution, different noise characteristics, and new reconstruction algorithms, producing images that are statistically distinguishable from those used in training. Migration to a different electronic health record vendor may change data structures, coding systems, and the availability of specific fields as model inputs. Wearable devices and continuous monitors introduce data streams not represented in training. Any of these may degrade performance even when the target condition is unchanged.

Pandemics, Seasonality, and External Shocks

The COVID-19 pandemic constituted an unplanned natural experiment in drift. Most models trained on pre-pandemic data degraded once the pandemic altered patient populations, disease presentations, and patterns of care. Sepsis models calibrated to detect bacterial sepsis and its characteristic inflammatory signature had no representation of the presentations associated with SARS-CoV-2 infection, including inflammatory syndromes that resembled sepsis on laboratory and vital sign data but required different management. Deferred routine care increased the severity of baseline illness at eventual presentation. Intensive care triage protocols were revised repeatedly during surges. Personal protective equipment requirements delayed vital sign measurement and altered the timing of data capture. Models continued to operate under pre-pandemic assumptions while accuracy declined, and most institutions did not assess this, continuing to rely on predictions that may have become unreliable.

A pandemic is not required to produce this effect. Healthcare demand varies seasonally, and seasonality alone may produce periodic drift. A useful assessment is to compare model performance across seasonal extremes, for example January, when influenza activity increases emergency department volume and case severity, against June, when elective activity predominates and baseline health status is higher. A difference of ten percentage points or more in accuracy between these periods indicates a recurring drift problem that will not be detected unless monitoring is stratified by calendar period. Such a difference also suggests that the model has fitted characteristics of its training season rather than relationships that generalize.

Clinical Consequences: Unobserved Failure

The principal hazard associated with drift is not conspicuous failure but progressive decline over months to years that does not trigger any alert. Drift is generally undetected until missed diagnoses and incorrect recommendations have accumulated. Recognition therefore requires familiarity with its characteristic failure modes.

Progressive Loss of Accuracy

Performance rarely collapses abruptly in a manner that would be self-evident. It erodes incrementally: 89% sensitivity in the first year, 84% in the second, which is consistent with normal variation and generates no alert, 78% in the third, at which point cases previously detected are being missed, and 71% in the fourth, which is clinically significant if performance is being measured. No individual quarter appears anomalous, because quarter-to-quarter variation obscures the trend. Over four years, however, the model fails to identify one case in five that it previously identified. The condition is difficult to detect because nothing prompts investigation: there is no fault, no error condition, and no observable event to prompt reassessment of the tool.

Absence of Performance Monitoring

Most deployed systems are monitored for availability rather than for correctness. Technical monitoring reports failures such as server downtime, failed API calls, and timeouts, that is, conditions indicating that the software has stopped functioning. It does not report a decline in sensitivity from 89% to 71%, a doubling of false negatives, or a fall in positive predictive value. The system appears healthy by these criteria: it returns predictions, within the expected interval, without error. That those predictions are progressively less accurate is not detected, because accuracy monitoring is rarely implemented. This asymmetry, in which system function is monitored extensively and clinical performance is not monitored at all, constitutes the principal blind spot in current deployments.

Regulatory Status as a Source of False Reassurance

The designation "FDA cleared" may create an impression of ongoing reliability that delays recognition of drift. A model cleared in 2019 was validated on data from 2019 or earlier, which was appropriate for that period and population. By 2025 that validation is six years old, and the patient population, clinical practice, and acquisition technology have all changed. The clearance, however, does not expire and the regulatory status is unchanged. A clinician who observes an FDA clearance may reasonably interpret it as a statement about current accuracy rather than as a historical determination made under specific conditions. This interpretation is potentially hazardous, because a clinician who relies on the designation may not question whether the tool's recommendations remain consistent with clinical observation.

Reported Examples

Failure of Pre-Pandemic Prediction Models

In March 2020, hospitals were operating models trained on pre-pandemic data to predict:

• Intensive care unit length of stay
• Requirement for mechanical ventilation
• Mortality risk
• Probability of sepsis

Performance of these models degraded substantially. Patients with COVID-19 did not follow the trajectories characteristic of influenza, bacterial pneumonia, or acute respiratory distress syndrome from other causes. The models had no representation of:

• The inflammatory profiles associated with SARS-CoV-2 infection
• Silent hypoxemia, characterized by low oxygen saturation with minimal dyspnea
• Prolonged intensive care stays relative to those typical of bacterial pneumonia

Observed effect: models overestimated mortality in some patient groups and underestimated it in others. Clinical teams progressively disregarded their output.

Degradation of Sepsis Prediction Models

The Epic Sepsis Model, which has been deployed across a large number of health systems, has demonstrated variable performance between institutions. Some sites have reported declining sensitivity over time, attributed to factors including:

• Aging of the patient population
• Improvement in sepsis treatment protocols, which alters baseline outcome rates
• Changes in electronic health record documentation practice, which affect the availability of the features on which the model depends

Few institutions monitor this systematically. Most rely on vendor assurance regarding continued model performance.

Imaging Models and Scanner Replacement

Consider a hospital operating a lung nodule detection system trained on images acquired with a CT scanner of 2015 vintage. In 2023 the scanner is replaced with a system offering:

• Higher spatial resolution
• Different noise characteristics
• Different reconstruction algorithms

The system begins to identify artifacts as nodules, and the false positive rate increases. Radiologists progressively disregard the alerts. The cause is not investigated until a missed malignancy is identified in the course of a malpractice claim.

Limitations of Current Oversight

Regulatory Clearance Does Not Establish Ongoing Safety

FDA clearance for AI and machine learning devices is based on a locked model validated on a specific dataset at a specific point in time. The regulatory process addresses initial safety and effectiveness rather than longitudinal performance monitoring.⁵

There is at present no requirement to:

• Monitor performance following deployment
• Report performance degradation
• Revalidate at defined intervals
• Notify users when accuracy declines

The FDA has begun to address these gaps through the AI/ML-Based Software as a Medical Device (SaMD) Action Plan and related concepts such as predetermined change control plans. However, implementation has been gradual, and most currently deployed models predate these frameworks.⁵

Institutional Monitoring Is Uncommon

Most hospitals lack the infrastructure required to track AI performance longitudinally. Specifically, they lack:

• Ground truth labels, that is, the observed outcome against which the prediction can be compared
• Performance dashboards reporting sensitivity, specificity, and positive predictive value over time
• Governance processes assigning responsibility for monitoring
• Remediation plans specifying the response when drift is identified

Information technology functions manage availability. Responsibility for accuracy is frequently unassigned.

Clinician Expectations Regarding Model Stability

Most physicians assume that a deployed model is static, analogous to a drug dose or an item of equipment. It is generally not appreciated that:

• Models may degrade without any change to the implementation
• Performance depends on the distribution of the input data
• Validation performed three years previously may not describe current performance

Consequently, when a model begins to miss cases, the observation is frequently attributed to general unreliability of AI rather than to drift, and the underlying cause is not investigated.

Components of Responsible Monitoring

Drift constitutes inevitable risk rather than inevitable harm. Monitoring determines which of the two occurs.

Performance Dashboards

Institutions deploying AI responsibly track performance metrics longitudinally. The choice of metric is consequential, as different clinical applications require different evaluation approaches for classification, segmentation, and detection tasks, and metrics should be stratified by patient subgroup in order to detect differential drift.⁶

• Sensitivity and specificity: is the target condition still being identified at the expected rate?
• Positive predictive value: when the model generates an alert, how frequently is it correct?
• Alert volume: is the model flagging more or fewer cases than at baseline?
• Override rate: how frequently do clinicians disagree with the model?

These metrics should be tracked on a rolling weekly or monthly basis so that a downward trend becomes apparent before it manifests as a missed diagnosis. The objective is to monitor clinical output over time in the same manner that system availability is already monitored.

Periodic Recalibration

Some models can be recalibrated without complete retraining. Available approaches include:

• Adjustment of decision thresholds to reflect current prevalence
• Updating of feature scaling to reflect new acquisition equipment
• Retraining on recent data at quarterly or annual intervals

Each of these requires supporting infrastructure:

• Labeled ground truth outcome data
• Retraining pipelines
• Validation sets drawn from recent data
• Governance procedures for approving model updates

Most vendors do not currently provide these capabilities, and most institutions are not able to implement them independently.⁴

Clinician Feedback

Clinicians are frequently the first to detect degradation. Reports that model output has become less accurate should be treated as data rather than as resistance to the technology. Effective monitoring includes:

• A low-friction mechanism for clinicians to flag incorrect predictions
• Scheduled review of flagged cases
• Periodic structured review of whether the model continues to perform acceptably in practice

An increase in override rate warrants investigation and should not be attributed by default to clinician reluctance to use AI.

Recommendations for Clinicians and Institutions

Drift cannot be prevented. It can, however, be detected, and transparency and accountability regarding detection can be required as a condition of adoption.

Questions to Ask Before Adoption

1. When was the model last validated?
   A validation date of 2019 warrants scrutiny in 2025.
2. How is performance monitored following deployment?
   The absence of any post-deployment monitoring should be regarded as disqualifying.
3. What is the response to detected performance degradation?
   Specifically, is there a defined threshold at which the model is withdrawn, and is there a mechanism for notifying users?
4. At what interval is the model retrained or recalibrated?
   An interval of never is not acceptable for a tool used in high-stakes clinical decisions.
5. On what population was the model trained?
   Where the training population is not representative of the deployment population, drift should be expected from the outset.

Requirements Following Deployment

• Performance dashboards should be required. Institutions able to track server availability are able to track model accuracy.
• Apparent degradation should be reported. A clinical impression that model performance has declined is informative and should be documented and escalated.
• Governance should be explicit. Responsibility for monitoring AI performance must be assigned to an identified role.
• Vendor transparency should be required. A vendor unable to provide recent validation data is not managing drift.

Institutional Capacity

Institutions deploying AI require roles and processes that are not present in most current organizational structures:

• AI performance analyst: tracks model metrics, investigates suspected drift, and coordinates retraining.
• Clinical AI committee: reviews performance reports and determines when a model requires recalibration or withdrawal.
• Ground truth collection: systems for capturing outcomes to support ongoing validation, for example whether a flagged patient met criteria for sepsis.

Clinical AI requires active lifecycle management in the same manner as medications, devices, and clinical protocols.

---

Conclusion

Drift is occurring in currently deployed systems, including electronic health record decision-support modules, radiology pipelines, and intensive care early-warning systems. It is ongoing and largely unmonitored, and it constitutes a patient safety concern that institutions, vendors, and regulators have been slow to address systematically.

Most institutions cannot state how far their deployed models have diverged from their validated baselines. Most vendors do not monitor post-deployment performance or supply revalidation data. Most physicians interpret regulatory clearance or prior validation as a statement about current performance rather than as a determination made at an earlier point in time. Consequently, a system that performed acceptably in 2019 may perform substantially worse in 2025, not because any component failed, but because the clinical environment changed while the model did not.

Drift is tractable, but only if clinical AI is treated as a system whose accuracy depends on continued correspondence between operational and training data, rather than as a fixed device that continues to function unchanged. This requires continuous performance monitoring, periodic revalidation on current data, and governance that assigns responsibility for surveillance and defines the threshold at which a drifting model is withdrawn or retrained. These should be regarded as requirements for safe deployment rather than as optional additions.

The relevant question is not whether a deployed model will drift, since it will, but whether the drift will be detected before a patient is harmed, and whether the field will come to treat drift as an expected feature of the AI system lifecycle rather than as an unanticipated event.

---

References & Further Reading

1. Finlayson SG, Subbaswamy A, Singh K, et al. The Clinician and Dataset Shift in Artificial Intelligence. N Engl J Med. 2021;385(3):283-286. doi:10.1056/NEJMc2104626
2. FDA. Artificial Intelligence and Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD) Action Plan. January 2021. FDA.gov
3. World Health Organization. Ethics and Governance of Artificial Intelligence for Health. WHO Guidance. 2021. WHO.int
4. Davis SE, Lasko TA, Chen G, Siew ED, Matheny ME. Calibration Drift in Regression and Machine Learning Models for Acute Kidney Injury. J Am Med Inform Assoc. 2017;24(6):1052-1061.
5. Zhang Y, Saini N, Janus S, Swenson DW, Cheng T, Erickson BJ. United States Food and Drug Administration Review Process and Key Challenges for Radiologic Artificial Intelligence. J Am Coll Radiol. 2024;21(6):920-929. doi:10.1016/j.jacr.2024.02.018
6. Faghani S, Khosravi B, Moassefi M, Rouzrokh P, Erickson BJ. Part 3: Mitigating Bias in Machine Learning—Performance Metrics, Healthcare Applications, and Fairness in Machine Learning. J Arthroplasty. 2022;37(6S):S421-S428. doi:10.1016/j.arth.2022.02.087
7. Erickson BJ, Kitamura F. Artificial Intelligence in Radiology: a Primer for Radiologists. Radiol Clin North Am. 2021;59(6):991-1003. doi:10.1016/j.rcl.2021.07.004