The Health Sciences Authority (HSA), Singapore’s regulating authority in the sphere of healthcare products, has published a guidance document dedicated to a life cycle approach for software medical devices. The document covers various aspects related to the main types of software-based products and highlights the most important aspects to be taken into consideration by medical device manufacturers (software developers) and other parties involved to ensure compliance with the applicable regulatory requirements. In particular, the scope of the guidance covers software medical devices based on Artificial Intelligence (AI) technology.
Table of Contents
Continuous Learning Capabilities
When describing the software-based medical devices, the HSA distinguishes AI-based products with continuous learning capabilities as a separate category, as they need special regulatory treatment due to the specific nature of such products. In particular, the actual performance of such products changes in time through their use, hence, general pre-market assessment is no longer sufficient to ensure continuous safety and effectiveness of these software products. In this regard, the authority states that medical device manufacturers should duly develop and implement controls that are necessary to manage the learning process to ensure it would not adversely impact the safety and performance. Moreover, an appropriate validation procedure should be employed to ensure continuous monitoring of the learning process and performance evaluation. The main purpose of these measures is to ensure that AI-based software with continuous learning capabilities would not deviate from the initial specifications. The controls to be introduced by the manufacturer (developer) should control the learning process and respective changes the software is subject to.
According to the guidance, a medical device manufacturer intended to place such a product on the market should duly submit, among other documents, a detailed description of the learning process, which should also cover the appropriate controls and monitoring measures in place. The information to be provided should include, inter alia, the following details:
-
Description of the process of continuous learning of the AI-MD during deployment.
-
Safety mechanism (can be built into the system) to detect anomalies and any inconsistencies in the output result and how these are mitigated. In particular, build-in controls should be able to identify the deviation that occurred and restore the algorithms to the condition existing before the changes took place to recover the normal performance of the product.
-
Information about the source, data type collected, data pre-processing steps and parameters extracted, and also the inclusion and exclusion criteria. The authority emphasizes the importance of all data-related matters since data used by the continuous learning algorithm impacts its future performance.
-
The process to ensure data integrity, reliability, and validity of the new data set used for learning.
-
Software version controls in place.
-
The process to ensure traceability between real-world data for training, learning process, software version number, and the AI-MD’s output during clinical use. As further explained by the HSA, the software developer should provide a method to identify and remove the data which adversely impacts the overall performance of the software and learning process in particular.
-
Validation strategy and verification activities for continuous learning to ensure the performance is within the pre-defined boundaries/envelope.
Post-market Monitoring
Apart from the general points related to the initial submissions for marketing approval, the HSA guidance also covers the aspects to be considered after placing AI-based software medical devices on the market. In this regard, the authority mentions that medical device manufacturers (software developers) should closely collaborate with the actual users of the software. For this purpose, the appropriate post-market monitoring mechanisms should be duly developed and implemented. In certain cases, this could be implemented as a built-in autonomous monitoring system. The matters related to surveillance and ongoing performance monitoring are especially important in the case of AI-based software medical devices with continuous learning algorithms. According to the guidance, the medical device manufacturers should continuously analyze the performance of their products and issues arising to be able to develop new controls addressing the issues.
As in the case with other medical devices, software-based products are subject to periodic reporting requirements. The HSA will communicate additional information regarding the applicable reporting requirements in the process of the initial Product Registration.
Changes to Registered Software
The present HSA guidance also provides additional clarifications and recommendations regarding the regulatory requirements and procedures to be followed in case of changes to AI-based software products already placed on the market. Under the general rule, should any changes to a registered software take place, the appropriate Change Notification should be duly submitted by its manufacturer (developer). To assist with complying with this requirement, the document further provides a flowchart describing in detail the most important aspects to be considered when notifying the regulating authority about the changes implemented. In particular, the flowchart provides the following criteria:
- Is there a change to the AI model?
- Is there a change that involves the addition or reduction of input data type to generate the same output?
- Is there a change to the output results presented which are based on the approved input parameters? This includes changes to how the user should interpret the output results.
- Is there a change to the approved workflow such that the patient result/therapy will no longer be required to be reviewed/supervised by the health care provider/trained professional/user (i.e. no human intervention is required)?
Depending on the answers provided, the changes would be assigned to a specific category: Technical, Review, or Notification, and the respective regulatory requirements should be applied accordingly.
In case of the changes to a continuous learning algorithm, specific criteria should be applied, namely:
- Is there a change in exclusion/inclusion criteria for input data used for continuous learning?
- Is there a change to the defined boundaries for allowance changes in its performance specification?
- Is there a change to the baseline performance specifications used to compare with the evolving performance specification?
All the criteria described hereinabove should be taken into consideration by the medical device manufacturer (software developer) when determining the regulatory status of the changes to the product it is responsible for, while further steps should be based on the applicable regulatory requirements.
In summary, the present HSA guidance addresses the most important aspects associated with AI-based software medical devices (including the ones with continuous learning capabilities). The document covers the key points related to the regulatory requirements for continuous learning algorithms, most-market monitoring, and surveillance, and also change notification rules.
Sources:
How Can RegDesk Help?
RegDesk is a next-generation web-based software for medical device and IVD companies. Our cutting-edge platform uses machine learning to provide regulatory intelligence, application preparation, submission, and approvals management globally. Our clients also have access to our network of over 4000 compliance experts worldwide to obtain verification on critical questions. Applications that normally take 6 months to prepare can now be prepared within 6 days using RegDesk Dash(TM). Global expansion has never been this simple.
