Methodological evaluation of bias in observational COVID-19 studies on drug effectiveness

Martinuka, Oksana; von Cube, Maja; Wolkewitz, Martin

BACKGROUND AND OBJECTIVE: Observational studies may provide valuable evidence on real-world causal effects of drug effectiveness in patients with Coronavirus Disease 2019 (COVID-19). Since patients are usually observed from hospital admission to discharge and drug initiation starts during hospitalization, advanced statistical methods are needed to account for time-dependent drug exposure, confounding, and competing events. Our objective is to evaluate the observational studies on the three common methodological pitfalls in time-to-event analyses: immortal time bias, confounding bias, and competing risk bias. METHODS: We performed a systematic literature search on October 23, 2020, in the PubMed database to identify observational cohort studies that evaluated drug effectiveness in hospitalized patients with COVID-19. We included articles published in four journals: The British Medical Journal (The BMJ), the New England Journal of Medicine (NEJM), the Journal of the American Medical Association (JAMA), and The Lancet as well as their sub-journals. RESULTS: Overall, out of 255 articles screened, eleven observational cohort studies on treatment effectiveness with drug exposure-outcome associations were evaluated. All studies were susceptible to one or more types of bias in the primary study analysis. Eight studies had a time-dependent treatment. However, the hazard ratios were not adjusted for immortal time in the primary analysis. Even though confounders presented at baseline have been addressed in nine studies, time-varying confounding caused by time-varying treatment exposure and clinical variables was less recognized. Only one out of eleven studies addressed competing event bias by extending follow-up beyond patient’s discharge. CONCLUSIONS: In the observational cohort studies on drug effectiveness for treatment of COVID-19 published in four high impact journals, the methodological biases were concerningly common. Appropriate statistical tools are essential to avoid misleading conclusion and receive a better understanding of potential treatment effects.