Evaluating the Consistency and Quality of Search Strategies and Methodology in Cochrane Urology Group Systematic Reviews

Abstract

Introduction: Systematic reviews (SRs) are the foundation of evidence-based medicine. As essential tools for synthesizing and evaluating evidence, they guide informed decision-making for clinicians and other stakeholders. In particular, the SRs produced by The Cochrane Collaboration are considered to be standards of methodological rigor and comprehensiveness. Therefore, it is imperative that Cochrane Systematic Reviews (CSRs) adhere to the highest standards, particularly in terms of the fundamental collection of evidence represented by databases searching and transparency of reporting search methods.

Methods: To assess the quality of searches and reporting in 65 Cochrane Urology Group Systematic Reviews and Protocols covering the past 2 decades, the authors created an evaluation form based on the PRESS Checklist, the Cochrane Handbook, and the Methodological Expectations of Cochrane Intervention Reviews. The search methodology, strategies, and reporting for each was independently reviewed by two librarians; any conflicts were resolved by group discussion.

Results: Comprehensive search methodology reporting, quality and inclusion of the search strategies varied widely over time. Fifteen percent (10/65) did not report a single full search strategy, and 62% (40/65) did not include search strategies for all databases reported. Errors in search strategies included line number mistakes, misspellings, incorrect syntax, and incorrect subject headings.

Conclusion: While CSRs are highly esteemed for methodological exactitude in other areas, they remain in need of improved search quality and reporting. Transparent reporting of search methods and reproducible search strategies is vital to the future of SRs if they are to continue to be a cornerstone of evidence-based medicine.