Improving Consistency in Audits with Human Research Participants: Determining the Optimal Percent of Research Records to Review
Matthew Gooden1, Gina Norato1, Sandra Martin1, Avindra Nath2, Lauren Reoma3
1Clinical Trials Unit, NINDS, NIH, 2Section of Infections of the Nervous System, NINDS, NIH, 3Clinical Trials Unit and Section of Infections of the Nervous System, NINDS, NIH
Objective:
The objective of this study was to gain additional understanding about the optimal percentage of subject records needed to review to confidently identify events of non-compliance during an audit.
Background:
Applying current quality assurance guidelines to research studies has a risk of either overestimating the incidence of non-compliance events and potentially wasting study resources or underestimating and missing events which may cause unnecessary harm to patients. This study evaluates the variability found while auditing protocols according to current practices in NINDS.
Design/Methods:
A 100% audit was performed for a NINDS natural history protocol. Events identified were categorized by nature and severity. Simulations were performed to visualize how varying the size of the audit impacts the accuracy of identifying non-compliance: 1) Data were randomly sampled from 5% of total participants and the number of deviations per person was calculated; 2) Resampling, with replacement, was performed 99 more times; and 3) The above procedures were repeated for 10% of participants, 15%, 20%, etc., until 100%, where 100% is the true deviation event rate.
Results:
This protocol enrolled 52 participants who experienced 371 total events of non-compliance, 249 (67%) of which were procedural non-compliance and 30 (8%) of which were major non-compliance. Results indicated that the current guidelines of reviewing 10% of charts for large studies may result in excessive variability in gaining an understanding of the study team’s compliance. Quality assurance teams should review more than 10% of charts to provide additional accuracy to protect patient safety and data integrity.
Conclusions:
This study demonstrates the importance of understanding the variability that may result from reviewing a standard percentage of charts during an audit and its subsequent effects on potentially impacting patient safety or data integrity. Future studies are needed to determine if the results are consistent with larger studies and with other trial types.