Table Of Contents:
The scope of the guidance covers, inter alia, the aspects related to the clinical evaluation of AI/ML-based medical devices to be conducted in order to ensure their safety, effectiveness, and proper performance.
Clinical Evaluation: Basics
According to the guidance, certain devices covered by its scope would be subject to an independent review of the clinical evaluation results. In the course of such a review, the clinical meaningfulness should be confirmed. As further explained by the SFDA, in this case, the clinical evaluation of the AI/ML-based medical device should, where possible or as far as possible, be reviewed by someone who has not been significantly involved in the development of the AI/ML-based medical device, and who does not have anything to gain from the device, and who can objectively assess the device’s intended purpose and the conformity with the overall clinical evaluation evidence. According to the guidance, the level of review scrutiny to be applied will depend on the risks associated with the device in question.
Medical device manufacturers may also rely on the information related to a similar device already placed on the market – a comparator device. However, in such cases, sufficient clinical and technical equivalence should be duly demonstrated in order for the clinical evidence to be admissible. Should the manufacturer fail to demonstrate equivalence, an additional investigation could be required in order to collect the evidence needed.
General Principles
The authority acknowledges the absence of international standards prescribing the way the clinical evaluation of AI/ML-based medical devices should be carried out. Under the said conditions, in order to assist medical device manufacturers, the authority outlines the following general principles to be applied:
- The manufacturer should assess whether the promised medical benefit is achieved and is consistent with the state of the art.
- The manufacturer should list alternative methods, technologies, and/or procedures and compare these alternatives with respect to clinical benefits, safety/risks, and performance.
- The manufacturer should assess whether the promised medical benefit is achieved with the quality parameters. In this respect, the authority mentions that the pre-defined outcomes and metrics should be used to evaluate safety and effectiveness.
- User and system elements are duly evaluated.
- Analytical validation should be done using large independent reference dataset reflecting the intended purpose and the diversity of the intended population and setting. The document also outlines the scope of elements such a dataset should include.
- The manufacturer should generate evidence on device performance that can be generalized to the entire intended population, demonstrating that performance will not deteriorate across populations and sites.
- The manufacturer of AI/ML-based medical devices should test performance by comparing it to gold standard, i.e. the reference standard that is being used to evaluate the model has to be evidence-based, demonstrating that the results are repeatable and reproducible in different settings.
- The effects of AI/ML-based medical devices should be evaluated in clinically relevant conditions. According to the guidance, in order to ensure the accuracy and reliability of the results of the evaluation conducted, proper integration is needed so the device in question would actually be used in the intended use environment.
- The scope of evaluation should include a comparison of healthcare performance with and without an AI/ML-based medical device subject to review, but not the performance of healthcare professionals and a medical device separately. The authority explains that AI/ML-based devices medical devices are intended to improve the performance of healthcare professionals, but not be used separately.
- Manufacturers in their study design should consider proactively the effects that their studies may have on healthcare organizations and potentially explore the possibility of prospective real-world studies in order to minimize selection bias, have more control over variables and data collection, and examine multiple outcomes. In this respect, the authority mentions that the published evidence available nowadays derives mostly from the studies conducted in silico – in the course of a computer simulation.
- Report the results of the clinical investigation using AI-specific reporting guidelines and standards.
- The manufacturer should locally validate the AI/ML-based medical devices that were developed and approved in other jurisdictions. Hence, the mere fact that the medical device in question has already been assessed and allowed for marketing and use in another country, does not create a basis for the device to be allowed to be supplied in Saudi Arabia.
- AI/ML-based medical device is unique in its ability for continuous learning, hence, manufacturers are required to use post-market continuous monitoring of safety, effectiveness, and performance to gather and validate relevant performance parameters and metrics for the AI/ML-based medical device in a real-world setting in order to understand and modify software based on real-world performance.
In summary, the present SFDA guidance describes the most important aspects to be taken into consideration with respect to the clinical evaluation of AI/ML-based medical devices. The document also outlines the general principles to be followed by medical device manufacturers.
Sources:
https://sfda.gov.sa/sites/default/files/2023-01/MDS-G010ML.pdf
How Can RegDesk Help?
RegDesk is a holistic Regulatory Information Management System that provides medical device and pharma companies with regulatory intelligence for over 120 markets worldwide. It can help you prepare and publish global applications, manage standards, run change assessments, and obtain real-time alerts on regulatory changes through a centralized platform. Our clients also have access to our network of over 4000 compliance experts worldwide to obtain verification on critical questions. Global expansion has never been this simple.
