Univariable meta-regression may be more conservative compared to chi-square in sub-group analyses.
Background: Authors of systematic reviews exploring heterogeneity typically use the chi-square test, the default statistical method for sub-group analysis in most statistical packages. Another analytical method to assess effect modification or heterogeneity of both binary and continuous variables is meta-regression.
Objectives: To explore the extent to which chi-square and meta-regression provide different results for subgroup analysis.
Methods: We present our experience with applying both the chi-square method and a random-effects univariable meta-regression to a recent subgroup analysis in a prognostic review on deterioration of transcatheter aortic valve implants. For this analysis, we used the DerSimonian and Laired random effect model with a Freeman-Tukey transformation. R (version 3.3.2) provided the statistical package for our analyses.
Results: The pooled incidence rate of valve deterioration from 13 observational studies was 28 (95% CI: 2 to 73) per 10 000 patient years. We observed a higher incidence rate in the subgroup of studies with no anti-coagulation at discharge (126, 95% CI: 97 to 160, I2 = 0%) than in the subgroup of studies not reporting on anticoagulation (14, 95% CI: 0.2 to 40, I2 = 87%). The chi-square test showed an interaction p-value of <0.0001 whereas the meta-regression showed a p-value of 0.01. We hypothesise this difference may occur when there is high heterogeneity within the sub-group(s). To test this, we are now identifying a sample of Cochrane reviews published in 2016 that reported a subgroup analysis with a chi-square p<0.1. We will repeat the subgroup analyses using meta-regression. We will present the results at the Summit and compare the results.
Conclusions: Utilisation of meta-regression for test of sub-group effect in one instance provided a more conservative p-value compared to the chi-square test. It would be useful to determine if this is an isolated instance or a generalisable phenomenon and, if so, explain the difference.