Randomized Controlled Trials vs Real-World Data in the COVID-19 Era: A False Narrative.

Lyman, Gary H; Kuderer, Nicole M

While randomized controlled trials (RCTs) remain the gold standard for evaluating therapeutic efficacy, the role of studies based on real world data (RWD) has gained increasing relevance and importance. While both approaches have strengths and limitations, we believe that the narrative of RCTs vs RWD is a false narrative. Clearly, both approaches are needed to capture the totality of evidence and answer critical clinical questions in as comprehensive and timely manner as possible. They represent complementary approaches which often appear synergistic with one another. RCTs are generally slow and costly to develop and conduct and limited by the eligible and accrued population included while RWD is subject to confounding by both unknown factors as well as known factors not available in the data at hand. At the same time, not all RWD are created equal with the gold standard for RWD represented by prospective cohort studies designed a priori to capture specific data to answer important pressing questions. A great strength of RWD is the ability to capture information on large numbers of individuals at speed and scale. It is essential to acknowledge, however, that while larger numbers may enhance the precision of the effect estimates provided, in and of themselves, larger numbers do not increase the accuracy of estimates if patient selection and other biases have crept in. While we have become increasingly sophisticated in efforts to adjust for such potential bias, the risk of residual confounding remains including confounding my indication with patients with greater illness more likely to receive certain treatments based on clinical judgment rather than by randomization. Nevertheless, in crises situations, like a pandemic, it is important to gather as much information on risk factors and complications including mortality as rapidly as possible to inform clinical and public health decision making and provide a solid framework for clinical trials that follow. We need both carefully designed, conducted and interpreted RCTs and RWD to definitively answer critical questions rapidly in times of crises like COVID-19 and in a real-world setting.