The revolution related to the use of Artificial Intelligence (AI) is becoming a reality. Models such as ChatGPT and other AI systems are increasingly supporting the daily work of many professionals. At the same time, regulatory frameworks are being developed – at the European Union level, the Artificial Intelligence Act is taking shape¹, and in the pharmaceutical domain, new GMP regulations are emerging.
In January 2025, the U.S. FDA published a draft guidance on the use of AI in the pharmaceutical industry², while in July 2025, the European Commission launched public consultations on the draft Annex 22 to EU GMP³.
This article discusses the main aspects of the proposed EU regulations concerning AI.
Scope of Annex 22
The draft Annex 22 covers computerized systems with AI functionality used in the manufacture of medicinal products and active substances that have a direct impact on patient safety, product quality, or data integrity.
This document provides additional guidance to Annex 11 for computerized systems incorporating AI models.
The guidance focuses on static AI models, i.e. those that do not change their behavior after the training phase.
It is explicitly stated that dynamic models, which learn continuously and autonomously, should not be used in critical GMP applications – likely due to their lack of repeatability and challenges in validation.
Importantly, the draft does not cover generative AI or large language models (LLMs) such as ChatGPT. These may only be used for non-critical activities, provided that a Human-In-The-Loop (HITL) mechanism is ensured — meaning that results are verified by a competent and appropriately trained individual.
General Principles
- Personnel such as SMEs, QA, IT, and consultants should have a thorough understanding of the intended use of the AI model within the GMP environment and the associated risks. Accordingly, such personnel must have appropriate qualifications, clearly defined responsibilities, and close collaboration during the selection of the algorithm, model training, testing, validation, and operation stages.
- Documentation related to AI activities must be available and reviewed by the manufacturer.
- Quality Risk Management related to patient safety, product quality, and data integrity must be applied when implementing AI.
Key Requirements
1. Description of Intended Use
The tasks for which the model has been designed must be described in detail, based on a deep understanding of the process. The description should include:
- Characteristics of input and output data,
- Acceptable level of variability, limits, and potential errors,
- Division of data into groups according to defined features (e.g. defect types).
If model outputs are used to support human decisions (HITL), the description must also define the scope of human responsibility.
The document must be approved by a Subject Matter Expert (SME) prior to the start of acceptance testing.
2. Definition of Acceptance Criteria
Test scenarios must be developed to verify that the model performs as intended.
For example, for a model classifying a product as “accepted” or “rejected”, the scenarios may include a library of typical defects and parameters such as sensitivity, specificity, accuracy, and precision.
Acceptance criteria must be at least equivalent to those of the currently used process and approved by the SME.
3. Documentation of Test Data
Test data should:
- Be representative of the entire data space,
- Include all subgroups,
- Reflect process limitations, complexity, and variability,
- Enable statistically significant calculations,
- Be properly described and verified.
Any pre-processing (e.g. normalization, standardization, data cleaning) must be described and justified. The generation of test data using AI is not recommended; if it occurs, it must be documented and justified.
4. Ensuring Independence of Test Data
Measures must be implemented to guarantee that test data were not used during model development, training, or validation. Data should be protected through access control and audit trail mechanisms.
Similar rules apply to personnel — individuals involved in model development should not participate in testing.
5. AI Model Testing
Testing aims to confirm that the model performs correctly with new data and is suitable for its intended use.
The test plan must be approved by the SME and include, among others:
- Description of intended use,
- Acceptance criteria,
- Reference to test data,
- Test scenarios with methods for result calculation.
Any deviations from the plan must be documented and justified.
Test documentation must be archived in compliance with GMP principles.
For critical applications, the model should record data features on which its decision (e.g. product rejection) was based, and their review should form part of the result approval process.
6. Recording Confidence Levels
Models used for prediction or classification must record the confidence level of their results. Thresholds should be configurable to limit model actions only to cases where confidence is sufficient.
7. Operational Principles
Key operational principles include:
- Change control for model, system, and process configurations,
- Regular performance monitoring,
- Supervision to ensure that output data remain within the defined space.
For models supporting human decisions, a mandatory review of results may be required according to established procedures.
Summary
Annex 22 to EU GMP introduces new regulations governing the use of AI in the pharmaceutical sector. The objective of these regulations is to enable the use of innovative AI solutions in a controlled, transparent, and GMP-compliant manner.
The key aspects of Annex 22 focus on the following areas:
- Only static AI models are permitted in critical applications.
- Dynamic models may be used only in non-critical activities and always with HITL involvement.
- General AI principles concern the need for qualified personnel, GMP-compliant documentation, and effective Quality Risk Management.
- Key requirements include defining intended use, acceptance criteria, selection and independence of test data, and operational principles for AI models.
References
- The EU Artificial Intelligence Act
- FDA, Considerations for the Use of Artificial Intelligence To Support Regulatory Decision-Making for Drug and Biological Products, Draft Guidance for Industry and Other Interested Parties, January 2025
- European Commission, Annex 22 – Public Consultation Draft, July 2025
Andrzej Szarmanski
Pharma GMP
www.pharma-gmp.com