Overviews of reviews: Unique challenges and opportunities of synthesising syntheses




Poster session 2 Thursday: Evidence synthesis - methods / improving conduct and reporting


Thursday 14 September 2017 - 12:30 to 14:00


All authors in correct order:

Maynard B1, Polanin J2, Dell N3
1 Saint Louis University and Campbell Collaboration, USA
2 Development Services Group, USA
3 Saint Louis University, USA
Presenting author and contact person

Presenting author:

Brandy Maynard

Contact person:

Abstract text
Background: Syntheses of syntheses, otherwise known as overviews, have become an increasingly prevalent approach to synthesising research. Although overviews are becoming prevalent, they are a relatively nascent and undeveloped synthesis method that pose unique methodological challenges. While significant empirical work has been undertaken to inform and improve systematic review methods, limited research and guidance is available for overviews.

Objectives: To examine the methodological quality of overviews and provide guidance for the conduct and reporting of overviews.

Methods: A systematic search for overviews that aimed to synthesise more than one empirical education-related review was conducted in multiple online databases and grey literature repositories. Two authors independently screened and selected studies and extracted data using a standardised codebook. Studies were analysed descriptively; we aimed to elucidate all aspects of the overviews and calculate the proportion of characteristics reported across 17 different methodological characteristics. We also compared methodological quality and reporting of recent overviews to early overviews using t-tests.

Results: Twenty-five overviews met eligibility criteria for this study. Our analysis revealed that many commonly reported aspects of systematic reviews were regularly unreported. Of the 25 overviews included, 11 used a narrative synthesis technique (44%) whereas 14 used a quantitative analytic technique (56%). Overview authors rarely took into account overlap between reviews or up-to-datedness of reviews.

Conclusions: Overviews offer an exciting, yet challenging new method for synthesising and managing the ever-expanding volume of research. Given the issues and limitations identified in this study, care must be taken in interpreting and using extant overviews. Moreover, to ensure the validity and utility of overviews to inform practice and policy, it is important that the conduct and reporting of overviews improve. As a result of this study, we developed conduct and reporting guidelines for overviews which will be presented and discussed.