The new article provides an overview of the regulatory requirements for the clinical evaluation of medical devices based on novel technologies. In particular, the document describes the three main components of clinical evaluation and highlights the most important aspects associated thereto.

The Saudi Food & Drug Authority (SFDA), a country’s regulatory agency in the sphere of healthcare products, has published a guidance document dedicated to medical devices based on Artificial Intelligence (AI) and Machine Learning (ML) technologies. The document highlights specific issues associated with the application of the said technologies in medical devices and also provides additional clarifications regarding the applicable regulatory requirements, as well as recommendations to be taken into consideration by medical device manufacturers (software developers) in order to ensure compliance thereto. At the same time, provisions of the guidance are non-binding in their nature, nor are intended to introduce new rules or impose new obligations. The authority also reserves the right to make changes thereto, provided such changes are reasonably necessary to reflect corresponding amendments to the underlying regulations. 

The scope of the guidance covers, inter alia, the aspects related to the clinical evaluation of medical devices employing AI and ML technologies. 

Clinical Evaluation: Key Points 

First of all, the authority mentions that there is no internationally aligned framework for the clinical evaluation of AI/ML-based medical devices. Hence, a manufacturer of AI/ML-based medical devices is expected to provide clinical evidence of the device’s safety, effectiveness, and performance before it can be placed on the market. 

When describing a clinical evaluation as a process, the authority refers to the position of the IMDRF according to which in the course of a clinical evaluation, a party responsible for a medical device should generate evidence sufficient to demonstrate compliance with the applicable safety- and performance-related requirements. In particular, it is necessary to demonstrate a valid clinical association, analytical/technical validation, and clinical validation of the product in question. Furthermore, it is stated that the said process should be continuous and iterative. The authority also mentions that the requirements related to clinical evaluation described in the guidance are applicable to all medical devices utilizing AI/ML technologies irrespective of their class under the existing risk-based classification for medical devices. 

 

Scientific Validity 

According to the guidance, in order to demonstrate a valid clinical association between the clinical condition the device in question is intended to address and the output the device provides, an interested party should provide evidence that the device output is clinically accepted based on existing evidence in published scientific literature, original clinical research, and/or clinical guidelines. Moreover, a medical device manufacturer will have to demonstrate that the clinical data used as a reference is relevant and admissible in the context of the general clinical practice, as well as the intended use of the device in question. Should it be identified that the scientific validity of the device cannot be confirmed using the existing data, new evidence should be generated – for instance, by conducting an additional clinical investigation. In this respect, the authority additionally emphasizes the importance of taking into consideration the lack of information related to AI/ML-based medical devices due to the novelty of these technologies. 

 

Analytical/Technical Validation

Apart from scientific validity, medical device manufacturers should also demonstrate analytical/technical validation of the products they are going to place on the market. As explained by the SFDA, the analytical validation evaluates the correctness of input data processing by the AI/ML-based medical devices to create reliable output data. In this respect, a party responsible for a medical device should provide sufficient evidence demonstrating that the device in question complies with the respective specifications based on the intended use of the device. The process of generation of the said evidence is usually covered by the quality management system and constitutes a part thereof. 

Clinical Validation

The third important element addressed in the guidance is clinical validation to be demonstrated by the medical device manufacturer. As explained by the authority, clinical validation is a necessary component of clinical evaluation for all AI/ML-based medical devices and it measures the ability of AI/ML-based medical devices to yield a clinically meaningful outcome associated with the intended use of the device output in the target population in the context of clinical care. The SFDA further emphasizes that the clinical evaluation could be carried out only upon successful completion of the analytical/technical validation described hereabove. According to the document, the evaluation of clinical validity could take place in both pre-market and post-market stages. For this purpose, a party responsible for a medical device can provide data collected in the course of clinical investigations carried out with respect to the same intended use, or other studies the data from which is appropriate and can be used in the context of the medical device in question. Should it be identified that the manufacturer cannot provide such data, a new investigation should be carried out. According to the guidance, a party responsible for the clinical validation should duly provide the list of data sources used, including both the ones supporting the claims by the medical device manufacturer with respect to the safety and effectiveness of the device and also the ones contradicting such claims. The particular scope of data to be provided will depend on the medical device in question, its functions, and features, and also it is intended use and risks associated with the device when used as intended by the manufacturer. As described in the guidance, the key metrics related to clinical validation include, inter alia, the following ones:

  • Specificity;
  • Sensitivity;
  • Positive predictive value (PPV);
  • Negative predictive value (NPV);
  • Likelihood ratio negative (LR-);
  • Likelihood ratio positive (LR+); and 
  • Clinical usability. 

 

In summary, the present SFDA guidance provides an overview of the key elements of clinical evaluation of AI/ML-based medical devices. The document describes the approach to be applied by medical device manufacturers when demonstrating compliance with the relevant safety- and performance requirements the products are subject to. 

 

Sources:

https://sfda.gov.sa/sites/default/files/2023-01/MDS-G010ML.pdf

How Can RegDesk Help?

RegDesk is a holistic Regulatory Information Management System that provides medical device and pharma companies with regulatory intelligence for over 120 markets worldwide. It can help you prepare and publish global applications, manage standards, run change assessments, and obtain real-time alerts on regulatory changes through a centralized platform. Our clients also have access to our network of over 4000 compliance experts worldwide to obtain verification on critical questions. Global expansion has never been this simple.