Veranex Risk Based Monitoring Business Delivery Model – A Case Study

Veranex has successfully designed and implemented a “5D Framework” to support its Risk Based Monitoring service delivery model.  The 5 components are:

• Define: Asking the right questions
• Distill: Use of statistical models to extract information
• Discover: Data trends & patterns are identified using Predictive Modelling & Machine Learning and Time Series
• Deliver: Findings are then linked to the business insights for RBM Recommendations
• Drive: Recommendations provided are linked into actions & risk mitigation for the KRI

Within this framework, several different analytic approaches (“D2P Analytic Value Chain”) are used to answer the questions:

• Descriptive Analytics: What is happening?
• Diagnostic Analytics: Why is it happening?
• Predictive Analytics: What will/can potentially happen?
• Prescriptive Analytics: What can we do about it?

This article is a follow-up to illustrate the RBM approach, and is based on a prior article which laid out the framework in more detail.

A CASE STUDY ON “SCREEN FAILURE ANALYSIS” USING D2P ANALYTICS VALUE CHAIN

Dr. Sakthivel Sivam, PhD, Senior Director – Biostatistics has worked collaboratively with Biswadhip Pai, MRes, Senior Manager – Risk Based Monitoring, at Veranex to successfully implement the “D2P Analytics” framework.

Introduction

With an increasing regulatory focus on risk-based approaches, quality management has become a core area in clinical development. RBM has an emphasis on improving both subjects’ safety and data quality. In the current clinical trial landscape, there are many new data driven technologies being implemented. However, there remains a pressing need for efficient, simple, and easily interpretable Risk Based Monitoring (RBM) solutions with a high accuracy rate and reliable anomalies detection mechanism.

This case study focuses on an important KRI, enrolment and screen failures, in an ongoing clinical trial. Screen Failure (SF) rate is an important metric for clinical operations. Understanding the reasons for SFs will help to achieve the targeted numbers for enrolment without increasing the enrolment accrual period. The goal is to help a client make informed decisions about some of the operational milestones/challenges by providing reliable analyses.

Methodology

The design and implementation for each step of D2P is flexible and can vary from one project to another. However, D2P usually starts with analysing the current situation of a KRI by using both descriptive and diagnostic approaches, which follow:

Descriptive– To understand the current recruitment status of a trial, an analysis of the enrolment trends needs to be assessed. This helps in determining the strategies to detect anomalies. Two graphic displays follow to illustrate this approach.

Diagram 1: The anomalies with high Screen Failure Percentage Vs Screened subjects are detected and placed in a sequential approach. As % of Screen Failures increase, all/some of these sites (identified by red circles) would be selected for review.

Diagram 2: Each line represents % of screen failures from a site. The top red horizontal line does not cross the reference line (black dotted vertical line is line of ‘no effect’) and is identified as an outlier site with a high % of screen failures.

Diagnostic– While descriptive statistics are useful for outliers/ inliers identification, the diagnostic approach essentially involves “Root Cause Analysis” of the detected anomalies. In our current Screen failure analysis, a simple “Tree map” (Diagram 3) can be obtained to identify reasons for screen failure across all sites and/ or identified anomalies. The screen failure subjects from the detected anomalies can be analysed to prioritize sites for follow up. (Diagram 4)

Diagram 3: The reasons for Screen Failure presented through a simple tree map indicate priority reason(s) for follow up. Site level trainings can be designed by considering this information across all sites for the detected anomalies.

Diagram 4: This graph shows the number of screen failure subjects by site by given reason(s).  It illustrates the top reasons impacted sites are assessed to understand the screening capabilities of such sites. This information is useful to determine the risk mitigation steps design.

Predictive Modelling– To enhance decision making for our screen failure data, we turn to Predictive Analytics using appropriate Bayesian and Classical Statistical Modeling techniques.  Following are some questions that can be answered through Predictive Analytics, based on an observed SF rate and Site activation rate from an ongoing study.

1. What is the probability of completing the targeted number of enrolments in a certain time period (t)? (Diagram 5)
2. How long (t) do we need to wait to complete the targeted enrollment? (Diagram 6)
3. How many new sites may we need to initiate to complete the remaining subject’s enrollment in a certain time period? (Diagram 6)

Prescriptive Approach– The Veranex RBM team collects Screen failure data and analyses the patterns, and trends with the analytic approaches used. The prescriptive approach takes the information and integrates it into actionable recommendations for changes in process, with study team and Sponsor collaborative input.

Actionable changes may be:

• Retraining site staff on protocol processes for subject screening
  • Attention to be focused on sites with high percentages of SFs
  • Attention to be focused on high percentages of inclusion/exclusion criteria contributing to the SFs
  • If needed, a CAPA would be issued to initiate preventive actions with respect to screening procedures
• Data quality improvement to ensure that all the required CRF data fields are completed without discrepancies
• Communication improvements
  • Periodic and ongoing Screen failure trend reports to be circulated to CRAs to check the enrolment metrics for respective sites and report, in real-time, to study team/Sponsor.

Conclusion

Identification of data related problems and provision of actionable insights to the study team and Sponsors frequently lack a structured approach. As a result of this, the clinical trial operations may not go as smoothly as intended. Our RBM strategies are designed to detect anomalies and increase the reliability of detected anomalies so that efforts can be focused where they need to be. Meaningful information for KRIs is provided, and translated into recommended actions for an effective risk mitigation process.