The Clinical Trial Technical Challenge: Programmability and Extensibility at Scale

Clinical trials are an extraordinarily complex and vitally important undertaking. These are enormous projects, with the cost of developing a single new drug averaging $2.6 billion. A phase-3 clinical trial — the final stage before approval — will involve thousands, sometimes tens of thousands, of participants. Hundreds, even thousands, of data points for each participant must be tracked over time for years. Ultimately, this means there could be hundreds of millions of data points that must be protected, secured and blinded. And, ultimately, all of that data will need to pass through multiple different applications for analysis.

Don’t be fooled by the extraordinarily fast pace of COVID-19 vaccine development. This is not the norm. These clinical trials were conducted safely and rigorously, but they also had unprecedented resources and the urgent need for a vaccine to combat a deadly, highly infectious disease pushing them forward. Most clinical trials take six to seven years to complete. Given the stakes — these new treatments could save and vastly improve the quality of life for millions of people — if there’s a way to speed up the process without compromising the science or ballooning budgets to the size required to combat an urgent global crisis, shouldn’t we do so?

In fact, there is a way to significantly reduce the length of clinical trials. In many cases, pharmaceutical companies use dozens of software platforms from different vendors, with data stored in multiple, disconnected silos. By unifying the myriad of applications and data stores into an integrated whole that operates efficiently, a clinical trial can shave 20% or more off the normal amount of time it takes to complete it.

Four Primary Capabilities for a Clinical Trial Data Platform

The last statement is much easier said than done. Creating that unified system for managing clinical trial data requires an enormous feat of engineering. Specifically, there are four capabilities that any such unified system must enable.

1. The software platform must be able to change and extend a data model for each clinical trial without having to rebuild a schema every time. The data models and platform have to be flexible yet efficient to manage hundreds of millions of records across multiple trials. The data a trial needs to collect differs a great deal depending on the condition the drug aims to treat. As one can imagine, an oncology trial will be looking at very different data than one for pain management.
2. Next, the data must be secure. Not only must it be protected from external threats such as ransomware and data theft, but it must also be blinded. The gold standard for a clinical trial is “double-blinded,” which means that neither the participants nor their physicians (among other roles) can know who received the pharmaceutical candidate and who received the placebo. For IT and DevOps teams, this means that data must be blinded according to the role at a low level, which is difficult to achieve while continuing to provide strong performance.

3. The platform must be extensible. Over seven years, the data collected may change even within a single trial. The ability to extend a base schema on the fly to adapt it to current needs will save an enormous amount of time, because building a brand new schema and then transferring data to it is a lengthy process, particularly when you’re dealing with hundreds of millions of data points. 

4. Finally, the platform must be as easy as possible to modify by non-programmers. If the study designer can implement rules, such as “if a patient is biologically male, they cannot be pregnant,” without having to employ the skills of a programmer, the trial will move ahead faster, especially since each trial can have hundreds or thousands of rules. 

The Road Ahead

All of these goals become much more challenging to implement at scale. The flow of information within a clinical trial is complex, and a single change can have cascading effects, as the data is blind. 

One major way that pharmaceutical companies are achieving these goals is by leveraging the Cloud and DevOps. The advantages are many: data resides in a unified, central repository that’s accessible from anywhere; there’s no hardware to manually configure, which slows things down; and, modern cloud infrastructures are extraordinarily resilient, secure, and robust. 

Many of the goals above have been largely met through the development of software-as-a-service (SaaS)-based platforms, but the challenges for improving the technology foundation for clinical trials are far from over. For instance, AI can be applied to auto-classify documentation. There’s a lot of opportunities to automate data collection through wearable devices. Also, all of this functionality will require the implementation of programmability and extensibility at scale. 

It’s a massive challenge, but many of the best minds in software engineering are making progress. Given the goal — more powerful treatments becoming available for sick people faster — these are challenges worth tackling.