Why AI/ML demands a different validation — within GAMP 5
Classical software is predictable: same input, same output — what is validated once behaves the same tomorrow. An AI model gives no such guarantee. It learns from data, answers probabilistically, and its behaviour can shift the moment the data underneath it changes. The reproducibility classical validation leans on is not a given for models — it has to be actively secured.
The frame stays GAMP 5 nonetheless: risk-based, lifecycle-oriented, critical thinking. How you concretely validate an AI/ML system within it is the applied path of this article. (What changed in the Second Edition in general — critical thinking, agile lifecycles, cloud — is in the GAMP 5 overview.)
Since July 2025, the stand-alone ISPE GAMP Guide: Artificial Intelligence has provided the specialist frame. ISPE describes it as a stand-alone guide to be used in parallel with GAMP 5 Second Edition. It does not replace the familiar lifecycle. The steps below apply the CSV lifecycle to AI and add model- and data-specific activities.
Step 1: state the intended use precisely and classify risk
As with any CSV, it all starts with the intended use — but for AI with more precision: which task does the model perform, on which input data, with which limits, and which decision depends on the output?
From that follows the risk classification. The more directly a model output carries a GxP decision (release, specification, assessment), the stricter the requirements for testing, explainability and oversight. A supporting model (structuring, research assistance) carries less risk than one feeding into a quality decision. This classification — deliberate and justified, not mechanical — is the critical-thinking basis on which every later step sizes its effort.
What evidence does AI/ML system validation need?
A defensible evidence package connects the released system state with the question of whether the model remains fit for its intended use. Its depth is scaled to risk; the following matrix is a review frame, not a universal mandatory checklist.
| Evidence | Documented core | Review question |
|---|---|---|
| Intended use & scope | Task, users, system boundary, output and decision impact | Is it clear what the model is used for — and what it is not? |
| Risk assessment | Impact on patient safety, product quality and data integrity | Do controls and test depth follow from the risk? |
| Data | Origin, suitability, lineage and independent test data | Is performance assessment free from training and tuning influence? |
| Performance | Metrics, acceptance limits, tests and deviations | Are pass, stop and escalation defined in advance? |
| System state | Model, software and configuration version with provenance | Can the released state be reproduced unambiguously? |
| Human oversight | Role, review depth, decision and rationale | Does the subject-matter decision remain attributed to a named role? |
| Operation & change | Monitoring, drift, change triggers, deviation and escalation path | Will a relevant change be detected and assessed under control? |
| Supplier | Responsibilities, change information and available evidence | Can the regulated company carry its assessment despite external components? |
Step 2: data quality and independent test data
The biggest difference from classical CSV lies in the data. Training, validation and test data must be controlled, representative and traceably managed — quality and origin are themselves validation-relevant.
At the centre stand independent test data with three rules:
- Strict data split: the data pool is demonstrably split into separate sets for training, validation (tuning during development) and test.
- Full exclusion: the final test set must be an unbiased, independent check — it must never have been used in training or hyperparameter tuning at any point.
- No data leakage: a test set is not reused for later model iterations — otherwise bias arises (data leakage). Deviating requires a strong, risk-based justification.
Put differently: the performance assessment is evidence only if the model was tested on data it has never seen.
With classical software you test the program. With AI you also test the data — and its clean separation.
Step 3: a model lifecycle, not a point-in-time validation
Classical CSV often thinks in a single event: validated, released, done. For models that does not hold. GAMP 5 explicitly recognises agile and iterative development — provided the controls are maintained (traceability requirement→test, assessed risks, controlled changes).
For critical applications this means in practice: the model is trained, frozen, tested and operated in a defined, demonstrable state. Every retraining or model update is a change with change control and an assessment of the change risk — not silent continued learning in operation. That keeps the validated state demonstrable at all times.
Step 4: monitoring, drift and human oversight
Because real conditions change, the operational phase requires continuous monitoring of model metrics — the keyword being drift: when input data diverges from the training distribution, performance can quietly decline. Defined metrics, thresholds and an escalation path therefore belong in the operating plan, not in an after-the-fact write-up.
Central to all of this is human oversight. For every GxP-relevant model output, the professional review and named release remain mandatory — the AI prepares, the human decides. Source binding, flagged unsupported content and traceable human review are also what the traqx control architecture rests on — Ground, Generate, Verify, Human Review, not a conformity promise.
Guardrails instead of trust
AI work becomes dependable through technical guardrails: source binding of the output and automated content checks that flag the unsupported — combined with documented human review.
Step 5: secure model and data provenance
One point classical CSV does not know, but which is central for AI: provenance — the traceable origin of data and model.
- Data provenance: gap-free capture of data origin (data lineage), rights agreements and the curation and anonymisation steps.
- Model inventory & traceability: a model inventory links the version of the model artifact to the exact training parameters, the training code used and the underlying training data.
- Extended supplier assessment: if an external supplier provides the model, they must show GxP understanding and be assessed for AI-specific quality capabilities — model integrity protection, bias mitigation and defence against attacks such as data poisoning.
Provenance is the AI translation of an old GxP principle — the same evidence discipline behind ALCOA+ and data integrity: claim nothing you cannot trace back.
The honest limits
Three clarifications:
- GAMP 5 is a guide, not a law. The binding requirements are GMP, EU Annex 11 and 21 CFR Part 11 — GAMP 5 is an established industry guide for justifying their fulfilment in a risk-based way.
- Generative AI is a special case. For critical applications, regulators expect controllable, demonstrable model behaviour; generative/probabilistic systems need particular caution and human review there (more in the Annex 22 overview and in Annex 22 vs Annex 11).
- This is orientation, not advice. The original documents and your context-validated assessment, aligned with your QA, are what govern.
No conformity promise
No tool and no model is inherently GAMP 5 compliant. Conformity is always established by your validated process in your context. GAMP 5 is a guide; the binding requirements remain GMP, EU Annex 11 and 21 CFR Part 11.
Frequently asked questions
Does AI/ML software fall under GAMP 5?
Yes. AI/ML systems are validated in the same risk-based GAMP 5 lifecycle as classical software — only with additional activities a one-off validation run does not cover: independent test data, drift monitoring, model provenance and continuous human oversight. GAMP 5 Second Edition remains the lifecycle foundation; the stand-alone GAMP AI Guide details the AI-specific activities.
How do you validate an AI model to GAMP 5?
In five steps: capture the intended use precisely and classify risk; secure data quality with a strict train/validation/test split and independent test data; run a model lifecycle instead of a point-in-time test; monitor for drift in operation with documented human review and decision; and secure model and data provenance (model inventory, data lineage, supplier assessment).
What evidence does AI/ML system validation need?
The risk-based evidence package connects intended use and scope, risk assessment, data origin and independent test data, performance metrics and acceptance criteria, model and configuration state, human oversight, monitoring and change control, the deviation and escalation path, and supplier assessment. Its depth depends on the specific use and risk.
How do GAMP 5 Second Edition and the GAMP AI Guide relate?
The ISPE GAMP Guide: Artificial Intelligence has been a stand-alone guide for AI-enabled computerised systems since July 2025. ISPE states that it is used in parallel with GAMP 5 Second Edition: GAMP 5 remains the risk-based lifecycle foundation, while the AI Guide details AI-specific activities and responsibilities.
Does an AI model need re-assessment after every re-training?
For critical applications, yes. The model is trained, frozen, tested and operated in a defined state. Every re-training or model update is a change with change control and an assessment of the change risk — not silent ongoing learning in operation.
What is different about AI validation versus classical software?
Classical software is reproducible: same input, same output. An AI model answers probabilistically and can shift as soon as the data underneath changes. So three things are added that classical CSV does not know: the data itself becomes validation-relevant, operation needs drift monitoring, and the provenance of data and model must be traceable.