Should low-quality evidence dominate high-level evidence? A systematic review and meta-analysis of systematic reviews of musculoskeletal physical therapy interventions.

Riley, Sean; Swanson, Brian T; Sawyer, Steven F; Brismée, Jean-Michel; Staysniak, Geoffrey

Objectives: To determine the: 1) quality of articles cited in systematic reviews (SRs); 2) methodological quality of the SRs; and 3) impact of quality on level 1A evidence. Methods: SRs related to musculoskeletal physical therapy interventions were identified. The methodological quality of the SRs and articles cited by the SRs were assessed by two blinded reviewers. Data analysis was performed by a third blinded researcher. Additional comparisons were made based on the Journal Impact Factor, spin, financial bias, and conflict of interest. Results: Twenty-four SRs were identified; 21/24 SRs had 'critically low' quality on the AMSTAR 2. Thirty-four percent of included studies were 'low quality,' and 58% of SRs included studies that had unreported external validity. One-half of the SRs represented 'spin,' and one-third of the SRs generated conclusions based on low-quality clinical trials. Discussion: The 'critically low' SRs methodological quality was exacerbated by low-quality research inclusion. Most SRs failed to follow best practices, including prospective registration and integration of professional librarians in the search process. Based on the high proportion of SRs that include low-quality trials and overall low methodological quality, further discussion regarding practice recommendations on level vs. quality of evidence is warranted. Level of Evidence: 1a.