Background: Covidence is a web-based tool that Cochrane recommends for screening articles for inclusion in systematic reviews. Graduate students enrolled in the Systematic Reviews and Meta-Analysis course at the Johns Hopkins Bloomberg School of Public Health had the option of using Covidence in performing a systematic review, the main course requirement.
Objective: To assess how well Covidence meets the needs of systematic reviewers.
Methods: We surveyed 46 enrolled students regarding their use of Covidence, their satisfaction with Covidence features (very satisfied; somewhat satisfied; neither satisfied nor dissatisfied; somewhat dissatisfied; very dissatisfied; or not applicable), and what they would change. We calculated summary statistics and qualitatively assessed text-response questions.
Results: Twenty-nine (63%) students responded to our survey. All of them (29/29) used Covidence to screen titles and abstracts, and they screened an average of 2365 records per person. Most students performed most of their screening on a laptop, and 15/29 (52%) used the mobile web application at least some of the time. Students expressed concern that the mobile application had different response options (Yes, No, Skip) than the version available on their computers (Yes, No, Maybe). Some students used other available features, including full-text screening (19/29; 66%), importing records (6/29; 21%), and exporting records (7/29; 24%). Most students (18/29; 62%) were very satisfied with title/abstract screening, and 9/29 (31%) were somewhat satisfied. Most students found screening and reconciliation tools intuitive. However, some found the tool for exporting records unintuitive. Students had suggestions for changes to Covidence, including ways to undo the last item screened in the event of an error; to save references that are not eligible but would provide helpful background information; and to make screening assignments to co-authors.
Conclusions: Respondents found Covidence to be a satisfactory screening tool, including the mobile app, and made suggestions for updates. Covidence should consider adding functionality that allows for user-defined features.