Validating AI/ML systems under GAMP 5 — the practical path for models

Why AI/ML demands a different validation — within GAMP 5

Classical software is predictable: same input, same output — what is validated once behaves the same tomorrow. An AI model gives no such guarantee. It learns from data, answers probabilistically, and its behaviour can shift the moment the data underneath it changes. The reproducibility classical validation leans on is not a given for models — it has to be actively secured.

The frame stays GAMP 5 nonetheless: risk-based, lifecycle-oriented, critical thinking. How you concretely validate an AI/ML system within it is the applied path of this article. (What changed in the Second Edition in general — critical thinking, agile lifecycles, cloud — is in the GAMP 5 overview.)

GAMP 5 (2nd Edition) and the accompanying AI materials introduce no new rulebook for this — they extend the familiar lifecycle with additional activities. The steps below are the ones that, in practice, go beyond classical CSV scope.

Step 1: state the intended use precisely and classify risk

As with any CSV, it all starts with the intended use — but for AI with more precision: which task does the model perform, on which input data, with which limits, and which decision depends on the output?

From that follows the risk classification. The more directly a model output carries a GxP decision (release, specification, assessment), the stricter the requirements for testing, explainability and oversight. A supporting model (structuring, research assistance) carries less risk than one feeding into a quality decision. This classification — deliberate and justified, not mechanical — is the critical-thinking basis on which every later step sizes its effort.

Step 2: data quality and independent test data

The biggest difference from classical CSV lies in the data. Training, validation and test data must be controlled, representative and traceably managed — quality and origin are themselves validation-relevant.

At the centre stand independent test data with three rules:

• Strict data split: the data pool is demonstrably split into separate sets for training, validation (tuning during development) and test.
• Full exclusion: the final test set must be an unbiased, independent check — it must never have been used in training or hyperparameter tuning at any point.
• No data leakage: a test set is not reused for later model iterations — otherwise bias arises (data leakage). Deviating requires a strong, risk-based justification.

Put differently: the performance assessment is evidence only if the model was tested on data it has never seen.

With classical software you test the program. With AI you also test the data — and its clean separation.

Step 3: a model lifecycle, not a point-in-time validation

Classical CSV often thinks in a single event: validated, released, done. For models that does not hold. GAMP 5 explicitly recognises agile and iterative development — provided the controls are maintained (traceability requirement→test, assessed risks, controlled changes).

For critical applications this means in practice: the model is trained, frozen, tested and operated in a defined, demonstrable state. Every retraining or model update is a change with change control and an assessment of the change risk — not silent continued learning in operation. That keeps the validated state demonstrable at all times.

Step 4: monitoring, drift and human oversight

Because real conditions change, the operational phase requires continuous monitoring of model metrics — the keyword being drift: when input data diverges from the training distribution, performance can quietly decline. Defined metrics, thresholds and an escalation path therefore belong in the operating plan, not in an after-the-fact write-up.

Central to all of this is human oversight. For every GxP-relevant model output, the professional review and named release remain mandatory — the AI prepares, the human decides. Source binding, flagged unsupported content and attributed release are also what the traqx control architecture rests on — Ingest, Generate, Verify, Release, not a conformity promise.

Step 5: secure model and data provenance

One point classical CSV does not know, but which is central for AI: provenance — the traceable origin of data and model.

• Data provenance: gap-free capture of data origin (data lineage), rights agreements and the curation and anonymisation steps.
• Model inventory & traceability: a model inventory links the version of the model artifact to the exact training parameters, the training code used and the underlying training data.
• Extended supplier assessment: if an external supplier provides the model, they must show GxP understanding and be assessed for AI-specific quality capabilities — model integrity protection, bias mitigation and defence against attacks such as data poisoning.

Provenance is the AI translation of an old GxP principle — the same evidence discipline behind ALCOA+ and data integrity: claim nothing you cannot trace back.

The honest limits

Three clarifications:

• GAMP 5 is a guide, not a law. The binding requirements are GMP, EU Annex 11 and 21 CFR Part 11 — GAMP 5 is an established industry guide for justifying their fulfilment in a risk-based way.
• Generative AI is a special case. For critical applications, regulators expect controllable, demonstrable model behaviour; generative/probabilistic systems need particular caution and human review there (more in the Annex 22 overview and in Annex 22 vs Annex 11).
• This is orientation, not advice. The original documents and your context-validated assessment, aligned with your QA, are what govern.