Using AI to Provide New Insights into Intentional Dose Non-Adherence

By Ken Getz, MBA

Inadvertent and intentional dose non-adherence in clinical trials has been a major drug development challenge for decades.  Studies in the literature have demonstrated that a high percentage of study volunteers — as high as 75% — conceal or fabricate information and provide inaccurate self-reports of adherence.

Participant non-adherence reduces statistical power and leads sponsors to underestimate the true dose response and toxicity from investigational treatments and to overestimate the true dose level required.  The anticipation of study participant non-adherence compels sponsor companies to expend resources and personnel to recruit an inflated number of study volunteers and to unethically expose larger numbers of patients to an investigational treatment.  Partial and full non-adherence may also cause sponsor companies to miss detecting treatment effects when they do occur, resulting in failure to identify a potentially effective drug.

Although there is strong and broad consensus that the accurate estimation of non-adherence rates in clinical trials is essential to determining the dose-response relationship and to allowing for a valid analysis of treatment safety and efficacy, accurate estimates have been elusive. Many approaches cannot distinguish between a patient simply missing a dose or deliberately concealing one.  Pill counting —one of the most commonly used approaches —is unreliable and tends to underestimate non-adherence rates. Patient questionnaires and diaries rely on subjective assessments that can easily be misstated. Ingestible sensors are more precise, but many study volunteers are uncomfortable with this approach.  Medication packages with imbedded memory chips (e.g., MEMS) have serious limitations since this approach cannot detect what the patient is doing once the cap has been removed.  Evaluations of drug levels in biologic fluids and biologic markers are impractical because they must be performed at distinct time periods when blood samples are collected and within a finite timeframe before the study drug has been metabolized.  These approaches are also subject to patient-specific pharmacokinetic variation.  

The most reliable and accurate method is direct patient observation. However, this approach has historically been cost prohibitive and difficult to carry out in outpatient settings. In recent years, artificial intelligence has improved the feasibility of direct patient observation and presented a unique opportunity for the Tufts Center for the Study of Drug Development (Tufts CSDD) to conduct an academic study in late 2019 to assess the magnitude of intentional dose non-adherence and inform statistical analysis planning.

Tufts CSDD conducted this study using an artificial intelligence platform developed by AiCure.  The platform relies on computer vision and machine learning technologies using a front-facing camera to capture dosing data followed by human review if deliberate non-adherence cannot be determined.  Study volunteers used the AiCure software on their smartphone during all dosing administrations.  In all, the adherence behaviors of 2,796 study volunteers and 257,672 doses were analyzed. This data was taken from 23 recently conducted, IRB-approved clinical trials primarily for CNS and neuromuscular diseases.  

Of the total 257,672 doses observed, 4% were confirmed as intentionally non-adherent. 

Half (48%) of all study volunteers had at least one intentionally non-adherent dose. Nearly one-in-ten (9%) study volunteers had more than 10% of their total doses deliberately non-adherent over the course of their clinical trial. One-out-of-twenty (5%) study volunteers were intentionally non-adherent for more than one-third of their total doses required by the protocol.  

A number of factors were associated with intentional dose non-adherence. For example, less experienced, lower volume investigative sites had a significantly higher incidence of study participants who were intentionally dose non-adherent. Further, as the duration of the clinical trial increased, the likelihood of intentional non-adherence increased significantly. 

Study volunteers whose first dose was intentionally non-adherent had a significantly higher mean intentional non-adherence rate throughout the clinical trial.  The mean intentional non-adherence rate was 26% for study volunteers whose first dose was intentionally non-adherent — nearly six times the mean for participants whose first dose in the clinical trial was adherent (4%).  Study volunteers with at least one intentionally non-adherent dose in the first week of the clinical trial had a significantly higher rate of non-adherence throughout the clinical trial. 

The results of this study offer insights into potential new practices and policies. Raising awareness, training, and additional support focusing on patient adherence specifically for less experienced, lower volume investigative sites may be one way to help address the higher incidence of intentional non-adherence. 

Implementing practices and solutions designed to detect and act on intentional non-adherence within the first week of a clinical trial will likely have a major impact. Remedial patient education, for example, targeting non-adherent study volunteers at the outset of a clinical trial may help increase overall participant adherence.  Clinical trial enrichment strategies designed to identify non-adherence during a placebo run-in period might be strengthened by including a pre-specified threshold of intentional non-adherence as part of the eligibility criteria for randomization. 

Tufts CSDD hopes to conduct future research that will look to gather adherence benchmarks across a larger and broader mix of disease conditions — particularly the most active including oncology, infectious diseases, and endocrinology.  This study also only included US clinical trials. Future research will expand our scope to include international clinical trials to derive global baseline measures.