Low reproducibility of randomized clinical trials methodology related to sampling: a systematic methodological review.

Riley, Sean P; Swanson, Brian T; Brismée, Jean-Michel; Sawyer, Steven F; Dyer, Elizabeth J

OBJECTIVES The reporting of sampling methods in Randomized Clinical Trials (RCTs) allows for research quality assessment, determination of sampling bias, and assures the presence of details necessary for reproducibility in future trials. The purpose of this study was to: (1) determine if sampling methodology was reproducible in RCTs related to musculoskeletal physical therapy (MSKPT) interventions to treat non-specific low back pain (NSLBP) and (2) establish if there was a relationship between sample reproducibility and established measures of research quality. METHODS Data were collected through a systematic review by a professional librarian. The identified RCTs were assessed for methodological quality by two blinded individual reviewers. Data analysis was performed by a third, blinded researcher; additional comparisons were made based on Journal Impact Factor and PEDro score. RESULTS Ninety-nine published peer-reviewed RCTs were identified that met inclusion criteria. Only 29% of the articles were judged to be reproducible based on the reported sampling methodology. There were meaningful correlations between two out of ten of the sampling reporting criteria and the judgement made if the sample was reported in significant detail to allow for replication. There was no relationship between sampling reporting criteria, Journal Impact Factors (JIFs), and Physiotherapy Evidence Database (PEDro) scores. DISCUSSION The reporting of sampling methodology needs to be considered to ensure reproducibility and avoid sampling bias. Despite the proliferation of measures of research quality, the overall reporting quality of RCTs continues to be inadequate to allow widespread reproducibility of trials. LEVEL OF EVIDENCE 1a.