Methodological assessment of systematic reviews and meta‐analyses on COVID‐19: A meta‐epidemiological study

Rosenberger, Kristine J.; Xu, Chang; Lin, Lifeng

RATIONALE, AIMS, AND OBJECTIVES: COVID‐19 has caused an ongoing public health crisis. Many systematic reviews and meta‐analyses have been performed to synthesize evidence for better understanding this new disease. However, some concerns have been raised about rapid COVID‐19 research. This meta‐epidemiological study aims to methodologically assess the current systematic reviews and meta‐analyses on COVID‐19. METHODS: We searched in various databases for systematic reviews with meta‐analyses published between 1 January 2020 and 31 October 2020. We extracted their basic characteristics, data analyses, evidence appraisal, and assessment of publication bias and heterogeneity. RESULTS: We identified 295 systematic reviews on COVID‐19. The median time from submission to acceptance was 33 days. Among these systematic reviews, 73.9% evaluated clinical manifestations or comorbidities of COVID‐19. Stata was the most used software programme (43.39%). The odds ratio was the most used effect measure (34.24%). Moreover, 28.14% of the systematic reviews did not present evidence appraisal. Among those reporting the risk of bias results, 14.64% of studies had a high risk of bias. Egger's test was the most used method for assessing publication bias (38.31%), while 38.66% of the systematic reviews did not assess publication bias. The I(2) statistic was widely used for assessing heterogeneity (92.20%); many meta‐analyses had high values of I(2). Among the meta‐analyses using the random‐effects model, 75.82% did not report the methods for model implementation; among those meta‐analyses reporting implementation methods, the DerSimonian‐Laird method was the most used one. CONCLUSIONS: The current systematic reviews and meta‐analyses on COVID‐19 might suffer from low transparency, high heterogeneity, and suboptimal statistical methods. It is recommended that future systematic reviews on COVID‐19 strictly follow well‐developed guidelines. Sensitivity analyses may be performed to examine how the synthesized evidence might depend on different methods for appraising evidence, assessing publication bias, and implementing meta‐analysis models.