Systematic review automation tool use by systematic reviewers, health technology assessors and clinical guideline developers: tools used, abandoned, and desired

Scott, A. M.; Forbes, C.; Clark, J.; Carter, M.; Glasziou, P.; Munn, Z.

Objective We investigated the use of systematic review automation tools by systematic reviewers, health technology assessors and clinical guideline developers. Study design and settings An online, 16-question survey was distributed across several evidence synthesis, health technology assessment and guideline development organisations internationally. We asked the respondents what tools they use and abandon, how often and when they use the tools, their perceived time savings and accuracy, and desired new tools. Descriptive statistics were used to report the results. Results 253 respondents completed the survey; 89% have used systematic review automation tools - most frequently whilst screening (79%). Respondents' Top 3 tools include: Covidence (45%), RevMan (35%), Rayyan and GRADEPro (both 22%); most commonly abandoned were Rayyan (19%), Covidence (15%), DistillerSR (14%) and RevMan (13%). Majority thought tools saved time (80%) and increased accuracy (54%). Respondents taught themselves to how to use the tools (72%), and were most often prevented by lack of knowledge from their adoption (51%). Most new tool development was suggested for the searching and data extraction stages. Conclusion Automation tools are likely to take on an increasingly important role in high quality and timely reviews. Further work is required in training and dissemination of automation tools and ensuring they meet the desirable features of those conducting systematic reviews.