Background: In Germany, the Federal Joint Committee commissions the Institute for Quality and Efficiency in Health Care (IQWiG) to assess evidence-based guidelines and prepare guideline synopses in order to identify relevant recommendations for disease management programmes (DMPs). The Appraisal of Guidelines for Research & Evaluation (AGREE) II tool is used to assess the (methodological) quality of guidelines. It consists of 23 appraisal criteria with 7-point scales in 6 independent domains with standardised scores between 0 and 100%, higher scores indicating better quality.
Objectives:To apply different cut-offs for guideline quality and evaluate their impact on the guideline pool and on the statements regarding DMP relevance in the guideline synopses, as well as to evaluate whether the use of fixed cut-offs leads to a loss of information on healthcare aspects.
Methods: We considered 4 guideline synopses on chronic heart failure (IQWiG Project V14-01), rheumatoid arthritis (V14-02), osteoporosis (V14-03), and chronic back pain (V14-04). Several publications using cut-offs to distinguish guidelines by means of their quality were identified. These cut-offs were based on standardised domain scores for individual domains or on the assessment of overall guideline quality. We applied these fixed cut-offs to the 4 synopses.
Results: The synopses included 6 to 22 guidelines with 195 to 996 recommendations. Depending on the respective cut-off, between 0 (0%) und 16 (72.7%) guidelines and between 1 (0.1%) und 700 (70.3%) recommendations were omitted. Between 0 (V14-04) and 12 (V14-01) changes to DMP relevance were reported in the conclusions of the guideline synopses. Due to the use of a cut-off, between 0 (V14-02) and 6 (V14-01) healthcare aspects were lost.
Conclusions: When using cut-offs to determine guideline quality, it should be considered that high cut-offs may considerably limit the conclusions of guideline synopses, but this is not reliably predictable. Even though cut-offs can distinguish between high and low-quality guidelines, at the same time statements on certain healthcare aspects may be completely lost.