Reviewing systematic reviews: Can technology boost children’s learning? A meta-analysis of What Works Clearinghouse (WWC) early childhood education programmes




Poster session 3 Friday: Evidence Tools / Evidence synthesis - creation, publication and updating in the digital age


Friday 15 September 2017 - 12:30 to 14:00


All authors in correct order:

Streke A1, Chan T2
1 Mathematica Policy Research, USA
2 American Institutes for Research, USA
Presenting author and contact person

Presenting author:

Tsze Chan

Contact person:

Abstract text
Background: Computer-assisted interventions (CAIs) have become increasingly popular as an alternative to the traditional intervention (non-CAI) on improving children performance on various topics. CAIs are usually associated with moderate learning gains for school-age children. This research is to investigate if CAI is a useful tool to enhance learning skills in preschool children. We outline findings from the meta-analysis of WWC-reviewed evaluations in the area of Early Childhood Education (ECE). The dataset is drawn from the WWC public database and consists of 48 studies and 275 effect sizes.

Research questions: Does the evidence in WWC reports indicate that CAIs programs improve cognitive, math and literacy outcomes for preschoolers (3- to 6-year-old children)?
Are CAIs more effective than non-CAIs in increasing learning gains of pre-schoolers?

Methods: First, a meta-analysis is performed to synthesise evaluations of ECE programmes. We calculate summary statistics from 48 ECE studies, and separately for 10 CAI studies and 38 non-CAI studies. Then, we compare the effectiveness of CAIs with non-CAIs reviewed within the ECE topic area. The comparison is made in a random-effects model by meta-regression technique. Because several effect sizes (ESs) can be nested within a study, our study uses the robust variance estimation procedure to adjust for the within study dependencies among effect sizes (Hedges et al., 2010).

Results: The study finds that ECE interventions are associated with substantive learning gains (ES=0.28), with CAIs (0.42) seemingly outperforming non-CAIs (0.24). However, results from meta-regression show that the difference in effectiveness between CAIs and non-CAIs is not statistically significant. Among seven moderator variables, the sample size plays a critical role, indicating that studies with smaller sample sizes are associated with larger learning gains.

Conclusions: The findings are discussed in light of our previous work on CAIs for beginning readers (grades K-3).