FDA NESTcc Data Quality Framework

General Approach to Data Quality

Describing the approach the NESTcc uses regarding the quality of data, first of all, it is important to outline key definitions and concepts used by the organization. These include:

• Real-World Evidence (RWE),
• Total Product Lifecycle (TPLC),
• Real-World Data (RWD) – information related to the health of the patient usually obtained from the electronic health records (EHRs) and other sources of patient-generated statistics.

Thus, the organization is focused on establishing the process of collection of reliable real-world evidence during the whole total product lifecycle based on high-quality real-world data.

 The particular guidance has been developed by the Data Quality Subcommittee of the NESTcc to improve the quality of data used by all parties involved in the ecosystem. The draft version of the guidance was available for public comments earlier in 2019, and the feedback received from the industry representatives has been taken into consideration during the development of the final version of the framework. The initial version of the document was dedicated to the information collected for the purpose of post-market evaluation of medical devices, but later the scope had been extended, and in the final version the guidance also covers the electronic health records and not only the information collected specially only for evaluation purposes.

The document contains five core sections, namely:

1. Governance – measures and approaches implemented to reduce the potential impact of bias.
2. Characteristics of Data – requirements regarding the sources and limitations related to imperfections and defects.
3. Capture and Transformation – the way secondary data should be used or transformed into another type.
4. Curation – the process applied to turn data into information admissible for further processing and analysis.
5. NESTcc Data Quality Maturity Model – the concept describing the way the data should be collected, validated and used for the purpose of assessment and identification of new opportunities. In particular, the model allows data processors to identify potential weaknesses and ensure the relevance, accuracy, and completeness of information that has been collected.

All the sections listed above represent the overall approach to the way the parties involved should operate with the information to improve the effectiveness of data-related processes associated with medical devices. 

Data Governance

The NESTcc emphasizes the importance of the special measures to be taken in order to prevent the impact of bias or conflict of interest. For this purpose, a wide range of stakeholders including patients, healthcare professionals, representatives of regulating authorities and industry should be involved in all processes related to operations with the information. Besides the application of the highest scientific standards on each step of the process of using real-world data to generate real-world evidence, it is also important to ensure the protection of patients’ information due to its sensitive nature. For this purpose, the appropriate national and international data protection regulations should be applied.

According to the position described in the framework, the internal policies governing the way the particular entity operates with the data should be based on the following principles:

1. Transparency and Integrity: the organization should be responsible for all operations with the data it performs, patients should be actively engaged in all processes and their consent should be duly obtained when it is necessary, and key stakeholders should be engaged in the development and execution of the projects related to real-world information and evidence. It is also required to make any information related to the potential and actual conflicts of interest publicly available. In certain cases it could also be necessary to establish an independent advisory board that would be also responsible for potential legal issues.
2. Data Access, Management, Linkage and Aggregation, and Use. The principles related to this sphere should ensure the accuracy and completeness of the data collected. The way the information should be collected should prevent any unauthorized access while the appropriate agreements should be concluded with all the parties involved in processing the data.
3. Requests Submission, Management, Review, and Acceptance. Any and all requests related to the real-world information and evidence should be publicly disclosed, and all scientific summaries should be made publicly available too. It is also required to notify the FDA about any identified issues related to the safety and effectiveness of medical devices.

According to the document, real-world data and real-world evidence play a significant role in the assessment and evaluation of medical devices. Thus, it is necessary to establish effective procedures governing all operations performed with the information by any party engaged in such operations to ensure patients’ safety and protection of public health.

Data Characteristics and Requirements

All evidence used for the decision-making process is generated from the data duly collected and processed. To be suitable for this purpose, it should meet two important criteria:

• Reliability – the quality and accuracy of data should be high,
• Relevance – information should fit the intended purpose.

To be able to choose the proper information and to use it in the most effective way, it is necessary to take into account all potential shortcomings and identified shortcomings since the use of erroneous or irrelevant data may lead to the significant mistakes in the decision-making process.

All these aspects should also be considered during the development of the design of the study as the first step of the data collection process. In particular, the entity intending to commence a study in order to obtain data that would be used to generate the evidence should consider the following:

• The populations covered by the study should be relevant.
• Performance and outcomes should be duly measured and assessed.
• Any and all adverse events should be properly recorded for further investigation.
• The sources of data used should be assessed from a point of the potential bias that could impact the accuracy of the data collected.
• It is advised to use experimental methods and approaches in case they allow improving the quality of the information. 

Summarizing the requirements, the NESTcc provides the list of characteristics the data should meet in order to be suitable, namely:

1. High quality,
2. Relevance to the intended purpose,
3. Convertibility to evidence through applying the appropriate methods, and
4. Interpretability.

Thus, the framework issued by the NESTcc emphasizes the importance of data used to generate clinical evidence and provides the description of principles and approaches to be used to improve the quality of information collected.

How Can RegDesk Help?

RegDesk is a next-generation web-based software for medical device and IVD companies. Our cutting-edge platform uses machine learning to provide regulatory intelligence, application preparation, submission and approvals management globally. Our clients also have access to our network of over 4000 compliance experts worldwide to obtain verification on critical questions. Applications that normally take 6 months to prepare can now be prepared within 6 days using RegDesk Dash(TM). Global expansion has never been this simple. 

Sources:

https://nestcc.org/data-quality-and-methods/

https://www.fda.gov/about-fda/cdrh-reports/national-evaluation-system-health-technology-nest