The new article addresses the matters related to specific aspects to be addressed in the submission – re-training the software function will undergo, and also performance evaluation to be carried out to assess the impact of the modifications implemented on the performance and effectiveness of the product.
Table of Contents
Contents
The Food and Drug Administration (FDA or the Agency), the US regulating authority in the sphere of healthcare products, has published a draft guidance document dedicated to a predetermined change control plan (PCCP) to be included in marketing submissions for medical devices utilising artificial intelligence/machine learning (AI/ML) technologies and respective ML-enabled device software functions (ML-DSFs).
The document provides an overview of the existing regulatory requirements, as well as additional clarifications and recommendations to be considered by medical device manufacturers and other parties involved in order to ensure compliance thereto. At the same time, provisions of the guidance are non-binding in their legal nature, nor are intended to introduce new rules or impose new obligations. Moreover, an alternative approach could be applied, provided such an approach is in line with the existing legislation, and also agreed with the authority in advance. The authority also reserves the right to make changes to the guidance and recommendations provided therein, should such changes be necessary based on the new information becoming available to the authority or to reflect corresponding amendments to the underlying legislation.
The approach described in the guidance is intended to reduce the regulatory burden for medical device manufacturers and also facilitate and streamline the regulatory procedures relates to changes to ML-based products already placed on the market. By submitting an application with a PCCP being a part of it, the applicant provides the authority with all the information related to changes that will be implemented to the device in the future. Once the said plan has been reviewed by the FDA (authorized PCCP), the changes described therein would not require a new submission before implementation.
The scope of the guidance covers, inter alia, the content of the Modification Protocol section – the one where the proposed modifications are to be described.
Re-training Practices
Usually, numerous processing steps take place before the input data would be processed by the software with the output provided. According to the guidance, the re-training practices component of a Modification Protocol should identify the processing steps that are subject to change for each modification and the methods that will be used by the manufacturer to implement modifications to the ML-DSF. As further explained by the FDA, in addition, re-training involves ML architecture modifications (e.g., in a neural network, modifications to training hyperparameters or the number of nodes, layers, etc.), the re-training practices component of a Modification Protocol should also describe the rationale or the justification for each specific architecture modification.
The information regarding the aspects outlined herein above is vitally important for the authority to be able to conduct its review and understand the way the modifications proposed will be achieved by the device of re-training in order to assess whether such changes are in line with the applicable best practices, and also to determine whether an additional performance evaluation will be required. Under the general rule, information on re-training practices employed by the manufacturer is included in the section describing the device in question. The particular scope of the information to be included would depend on the nature of a specific device subject to review and modifications proposed by the manufacturer.
In order to assist medical device manufacturers with complying with the above requirements, the document also contains an annex describing examples of the information to be included in the respective section of the submission. As explained by the FDA, in general, this information should identify the objective of the re-training process, provide a description of the ML model, identify the device components that may be modified, outline the practices that will be followed (e.g., data sequestration strategies during re-training), and identify any triggers for re-training (e.g., when new data reaches a certain size or when a drift in data is observed over time).
Performance Evaluation
Another important aspect addressed in the guidance is related to performance evaluation. The appropriate requirements could be introduced by the authority with respect to the changes included in a PCCP. As explained by the authority, performance evaluation methods should describe the process that will be followed to validate that the modified ML-DSF will meet the specifications identified as part of a specific modification, in addition to maintaining the specifications that are not part of the modification but may be impacted by the modification. It is important to mention that this section should describe the plans related to validation and verification applicable to the entire medical device after the changes took place. For instance, in case the function in question is used to operate a hardware device, the scope of assessment should cover not only the software function itself but also the impact the changes will have on the actual performance of the hardware device as well. In this section, the authority expects the applicant to provide information on the study design, performance metrics, pre-defined acceptance criteria, and statistical tests for each planned modification. Information outlined hereabove will reasonably be necessary for the authority to assess the methods applied by the party responsible for a medical device in order to determine whether they are appropriate in terms of modifications in question, as well as the design of the device and its characteristics.
Thus, medical device manufacturers are expected to submit a Modification Protocol addressing the aspects related to performance evaluation and the tests to be conducted.
In summary, the present FDA guidance provides an overview of certain matters to be covered by the scope of the Modification Protocol section, namely, the re-training process and performance evaluation. The document highlights the key points to be taken into consideration and also explains the approach to be applied.
How Can RegDesk Help?
RegDesk is a holistic Regulatory Information Management System that provides medical device and pharma companies with regulatory intelligence for over 120 markets worldwide. It can help you prepare and publish global applications, manage standards, run change assessments, and obtain real-time alerts on regulatory changes through a centralized platform. Our clients also have access to our network of over 4000 compliance experts worldwide to obtain verification on critical questions. Global expansion has never been this simple.
