When Can We Rely on Real‐World Evidence to Evaluate New Medical Treatments?

Simon, Gregory E.; Platt, Richard; Watanabe, Jonathan H.; Bindman, Andrew B.; John London, Alex; Horberg, Michael; Hernandez, Adrian; Califf, Robert M.

Concerns regarding both the limited generalizability and the slow pace of traditional randomized trials have led to calls for greater use of real‐world evidence (RWE) in the evaluation of new treatments or products. The RWE label has been used to refer to a variety of departures from the methods of traditional randomized controlled trials. Recognizing this complexity and potential confusion, the National Academies of Science, Engineering, and Medicine convened a series of workshops to clarify and address questions regarding the use of RWE to evaluate new medical treatments. Those workshops identified three specific dimensions in which RWE studies might differ from traditional clinical trials: use of real‐world data (data extracted from health system records or data captured by mobile devices), delivery of real‐world treatment (open‐label treatments delivered in community settings by community practitioners), and real‐world treatment assignment (including nonrandomized comparisons and variations on random assignment such as before‐after or stepped‐wedge designs). For any RWE study, decisions regarding each of these dimensions depends on the specific research question, characteristics of the potential study settings, and characteristics of the settings where study results would be applied.