Employment, Social Affairs & Inclusion

Catch Up® Numeracy

Evidence level:
 
Evidence of effectiveness:
? - 0 + ++
Transferability:
? - 0 + ++
Enduring impact:
? - 0 +

Project overview

Catch Up® Numeracy is a fairly non-intensive intervention (consisting of two 15-minute sessions delivered twice a week by classroom assistants who have undergone three half-days training sessions). The intervention is designed to address relatively mild persistent numeracy difficulties; in different words, the intervention is intended for children who already have some knowledge and understanding of numbers but are below the attainment level expected of their age cohort. The intervention was launched in 2007 by the not-for-profit UK charity,

Catch Up®, and since then it has been implemented in over 4,500 schools across the UK.

The intervention largely builds upon the Numeracy Recovery research while also combining some elements from the Catch Up® Literacy intervention. Namely, from the former, the intervention designers have adopted a simplified version of the componential view of arithmetic - which assumes that mathematical development involves potentially independent and separately developing skills and processes; thus, the intervention divides arithmetical ability into various components, which children are assessed against, and then tailors intervention according to the outcomes of the assessment - while the structure of the intervention has been based on the latter. Similarly to Catch Up Literacy, Catch Up Numeracy consists of four stages and is delivered through three-part 15-minute individual sessions taking place twice a week. It also has the same levels of attainment.

The children’s progress throughout the intervention is scaffolded on two dimensions: 

individual components of numeracy and the levels of attainment.

Practice category

  • Helping Vulnerable Children

Recommendation pillars

  • Improve education systems’ impact on equal opportunities

Countries that have implemented practice

  • United Kingdom
  • Australia

Age groups

  • Middle Childhood (age 6 to 12)
  • Teenagers (age 13 to 19)

Target groups

  • Children,
  • Children/teenagers with Special Educational Needs (SEN)

Years in operation

  • 2007  - still operating

Type of organisation implementing practice

  • Private Education Organization

Evidence of effectiveness

Evaluation 1

The evaluation randomly assigned children, who had been identified by their teachers as struggling with mathematics, to either to the treatment group (in which students participated in the Catch Up Numeracy intervention) or one of the two control groups (one in which students received an alternative matched-time mathematics support and one in which students received no additional support).

All 440 children taking part in the intervention were assessed by their teachers using the Basic Number Screening Test immediately before the intervention took place and as soon as it was completed. The outcome variable measured in the evaluation has been the mathematics age, and more precisely the mean gain in mathematics age and the mean ratio gain for children included in the treatment group compared to those in the control groups.

Wayne Holmes and Ann Dowker (2013) Catch Up Numeracy: a targeted intervention for children who are low-attaining in mathematics, Research in Mathematics Education, 15:3, 249-265, DOI: 10.1080/14794802.2013.803779.

Summary of Results for Evaluation 1

Outcome

Treatment group

Control Group

Matched-time group

No-intervention group

Outcomes improved (statistically significant)

Mean gain in Mathematics
Age (months)

11.27 (9.14)

7.0 (SD=8.25)

6.29 (SD=8.79)

Mean ratio gain

2.51 (SD=1.9)

1.49 (SD=1.78)

1.37 (SD=1.8)

Outcomes with no effect


 

 

 

This evaluation used a sub-sample of the original treatment group, which included 141 children, all belonging to the Vale of Glamorgan local authority. The reason for conducting a second evaluation using this specific sub-set of children was that the aforementioned local authority was the only one that provided detailed information about the number of sessions delivered each week which indicates the extent of Catch Up Numeracy support received by the 141 children studied. As the selected 141 children are a sub-sample of the overall sample used for the overall intervention evaluation, they were subject to the same selection process and study design described above.

Evaluation 2 (using sub-sample of evaluation 1)

Wayne Holmes and Ann Dowker (2013) Catch Up Numeracy: a targeted intervention for children who are low-attaining in mathematics, Research in Mathematics Education, 15:3, 249-265, DOI: 10.1080/14794802.2013.803779.

Summary of Results for Evaluation 2

Outcome

Treatment Group 1*

Treatment Group 2**

Outcomes improved (statistically significant)

Mean gain in Mathematics
Age (months)

12.32 (SD=8.99)

15.96 (SD=6.87)

Mean ratio gain

1.75 (SD=1.30)

3.69 (SD=1.93)

Outcomes with no effect

 

 

* in which 85 children were administered by school-based classroom assistants;
**in which 56 children were administered by members of a local authority-based peripatetic team of classroom assistants.

Evaluation 3

The evaluation was a three-arm randomised controlled trial using intent to treat analysis (i.e. the pupils were compared in the groups to which they were originally assigned randomly regardless of whether they received the program), with the randomization being stratified by school. Each of the schools had 2 teaching assistants and 6 eligible pupils (i.e. pupils identified as struggling with numeracy) participating in the evaluation. While the participating teaching assistants and the pupils were selected by the schools, they were randomly assigned by the evaluator to the treatment or control groups. More specifically, the teaching assistants were assigned either to the Catch Up treatment group or to a time equivalent one-to-one intervention group while the pupils were assigned to either one of those two treatment groups or the control group (i.e. ‘business as usual’).

Overall, 336 pupils, out of 54 schools, took part in the evaluation (with 112 pupils in each of the three groups) and the intervention period amounted to 30 weeks. The numeracy skills of the pupils were measured before and after the intervention took place.

Education Endowment Foundation (2014) Catch Up® Numeracy- Evaluation Report and Executive Summary, February 2014. Independent evaluators: National Foundation for Educational Research (NFER).

Summary of Results for Evaluation 3*  **

Analysis

Catch Up Treatment Group

Control Group

Intention to treat analysis

-0.79 (CI=-3.54 - 1.96)

-3.54 (CI=-6.28- -0.80)

Sub Group analysis

1.42 (-3.21 - 6.05)

-3.30 (CI=-7.95 - 1.35)

Intention to treat analysis for dataset with omissions

-0.90 (-3.86 - 2.06)

-3.62 (-6.52 - -0.72)


** Significant results are presented in bold* The analysis was based on a multi-level, multivariate regression in which belonging to the Catch Up treatment group as well as belonging to the control group are included as independent binary variables (with the ref. category being belonging to the time-equivalent intervention group) and the dependent variable is the score in the Basic Number Screening Test.

Transferability

This intervention has not been evaluated in additional populations, so it is unclear whether it is effective with populations beyond the original study population.

Enduring impact

There are no follow-ups conducted later than five months after the implementation intervention, and therefore it is unclear whether the intervention has an enduring impact.

Issues to consider

One of the co-authors of the first evaluation is the developer of the Numeracy Recovery intervention, upon which Catch Up Numeracy was largely based. Therefore there should to be an independent evaluation for firm conclusions to be drawn.

The control group is substantially smaller than treatment group in the first evaluation, and this difference is due to the way in which the teachers randomly allocated the students to the conditions. This will have reduced the ability to find differences between the treatment and control groups, as the power of the analysis is heavily dependent upon the size of the smallest group used in the comparison.

As mentioned, the intervention builds on the Numeracy Recovery intervention as well as on the Catch Up Literacy intervention, both of which have been evaluated (albeit the former has only been a subject to pilot evaluation) and found to have positive outcomes on children’s mathematics and reading skills (Dowker, A. (2001). Numeracy recovery: a pilot scheme for early intervention with young children with numeracy difficulties. Support for learning 16(1); Holmes, W., Reid, D., and Dowker, A. (2012). Early intervention to prevent long-term literacy difficulties: the case of Catch Up Literacy, Procedia - Social and Behavioural Sciences, 46: 4498 – 4503).

The intervention has also been evaluated qualitatively, and the authors reported that teachers and teacher assistants who have delivered the intervention reported that Catch Up Numeracy helped to raise the children’s self-esteem and confidence as well as helped to improve their numerical knowledge and skills. This evaluation is available on the Catch Up website and has been commissioned by Julie Lawes, the Director of Catch Up.

(Ref.: Alan Evans (2008) EVALUATION OF THE CATCH UP NUMERACY PROJECT SECOND INTERIM REPORT ON THE RESEARCH AND DEVELOPMENT PROJECT. A report for Catch Up. Cardiff School of Social Sciences Cardiff University).

Evaluation details

Evaluation 1

The evaluation used a sample consisting of 440 primary school children belonging to 15 local authorities in England and Wales.

Each of the local authorities has had 2 to 6 schools participating in the evaluation and in each of the schools two to four children were selected to take part in the intervention and two more to be included in the control group.  The selection of students was such that first the teaching staff identified students who are believed to have difficulties with arithmetic and then those were randomly assigned (by the independent evaluator) to either the treatment or one of the control groups. Children belonging to the intervention group (n=348) participated in the Catch Up Numeracy intervention delivered by trained teacher assistants. Those belonging to the ‘matched-time’ control group (n=50) were also given individual mathematics support (the nature/ structure of which were decided upon by the classroom assistants delivering the support but could not be Catch Up Numeracy) for the same amount of time and those belonging to the ‘no-intervention’ control group (n=42) received no additional support (i.e. only received their usual classroom lessons). 

No statistically significant differences between the treatment and two control groups with regard to the initial (i.e. baseline) chronological age, Basic Number Screening Test scores or mathematics age (i.e. the output of the Basic Number Screening Test) were reported.

All 440 children taking part in the intervention were assessed by their teachers (using the Basic Number Screening Test) immediately before the intervention took place and as soon as it was completed; the use of the test enabled the calculation of the children’s mathematics ages. The number of months between the before and after tests are used as an approximation of the duration of the intervention and depending on the group those range, on average, between four and a half and five months (more specifically, the mean duration of intervention in months was 4.94, SD= 1.99, for the intervention group; 4.56, SD=0.8, for the matched-time control group; 4.57, SD= 0.82, for the no- intervention group).

As mentioned above, the progress was measured through the Basic Number Screening Test. (Gillham, B., & Hesse, K. 2001. Basic number screening test: National numeracy strategy edition: Forms A & B, for ages 7 to 12 years (3rd ed.). London: Hodder Education). The choice of the test is justified by the fact that it is a commonly used standardized test that can be administered by the schools themselves and does not require the tester to have psychological training as it is with many others. Furthermore, the test also has the benefit of being a general measure of numerical performance.

The outcome variable measured in the evaluation has been the mathematics age and more precisely the mean gain in mathematics age and the mean ratio gain for children included in the treatment group compared to those in the control groups.

To determine the impact of the Catch Up Numeracy intervention on the aforementioned variable (as opposed to other forms of individual mathematics support and no support) the authors used analysis of variance (ANOVAs), which was conducted with the group (i.e. treatment, matched-time and no- intervention) as the factor (independent variable) and the Basic Number Screening Test score at the end of the intervention as the dependent variable.  The results indicated a statistically significant difference between the three groups. The Tamhane T2 post hoc tests showed that children in the intervention group scored significantly higher than those in both control groups (the difference between the outcomes for both control groups has been statistically insignificant). An ANOVA, with once more the group as the factor but with ratio gain (defined as the mean of individual ratio gains), rather than the test scores, as the dependent variable, showed that the treatment group was subject to higher ratio gains than either of the control groups, which once more did not differ significantly from one another. The effect sizes of the differences in gain in mathematics age (calculated using the test scores) between the treatment group and the matched- time group was 0.47, and between the treatment group and the no- intervention control groups was 0.55.

Evaluation 2 (using sub-sample of evaluation 1)

This evaluation used a sub-sample of the original treatment group which included 141 children all belonging to the Vale of Glamorgan local authority. The reason for conducting a second evaluation using this specific sub-set of children resulted from the fact that the aforementioned local authority was the only one which provided detailed information about the number of sessions delivered each week. This in turn provided certainty regarding the extent of Catch Up Numeracy support received by the 141 children studied. The sub-sample can be divided into two groups based on the teacher assistants delivering the intervention; namely, 85 children were administered by school-based classroom assistants while 56 children were administered by members of a local authority-based peripatetic team of classroom assistants.

This analysis was expected to produce results that indicate how the effect varies with the dosage of the intervention.  In the full sample, the intervention time was often redeployed to other classroom tasks, and student attendance was imperfect.

As the selected 141 children were a sub-sample of the overall sample used for the overall intervention evaluation they were subject to the selection process described above (i.e. they were first identified by their teachers as students struggling with mathematics and then through random assignment they were allocated in the treatment group). Furthermore, their outcomes were also assessed using the aforementioned Basic Number Screening Test which was conducted both right before and right after the intervention took place and the results of which were used to calculate the children’s mathematics ages. 

The study design of this evaluation is the same as the one described above as the evaluation uses a subset of the sample collected through the aforementioned cluster-randomized controlled trial (i.e. no additional study was conducted). Similarly, the outcome variable for this study remains the mathematics age of children and more specifically the relative gain in their mathematics age.

The only difference between the two evaluations is that in the second one authors have also had information regarding the extent of implementation of Catch Up Numeracy intervention, i.e. how many 15 minutes long sessions per week, on average, children have actually received.

In their analysis, the authors first calculated the mean number of (15 minutes long) sessions received by children in the two sub-groups and found that in both cases children received less support than recommended by the intervention developers. Namely, children in the first group (i.e. the 85 children who were administered by school-based classroom assistants) had on average less than one session per week (approximately 13 minutes of intervention per week) while children belonging to the second group (i.e. the 56 children who were administered by members of a local authority-based peripatetic team of classroom assistants) received on average 1.51 sessions per week (approximately 23 minutes of intervention).

However, even though not fully implemented according to the intervention designers’ recommendations, the intervention did prove effective as paired t-test analyses of the mean mathematics ages at the start and end of the intervention indicated that the children in both groups had increased their mathematics age. When comparing the outcomes of the two groups, the authors concluded that the latter group had significantly higher mean mathematics age and ratio gains than the former one. While those differences can most probably, at least partially, be attributed to the differences in the volume of support time delivered per week, only further study, which would also take into account other factors (such as the different types of teacher assistants delivering the intervention), can determine this with greater certainty.

Evaluation 3

The evaluation was a three-arm randomised controlled trial using intent to treat analysis (i.e. the pupils were compared in the groups to which they were originally assigned randomly), with the randomization being stratified by school. It involved 54 schools (which agreed to participate out of 130 approached by
Catch Up) predominantly based in Wales and the southern half of England. Each of the schools had 2 teaching assistants and 6 eligible pupils participating in the evaluation. The eligible pupils were identified by the schools, before the training of the teaching assistants has begun, using the schools’ own assessments of need. The choice of the two teaching assistants has also been done by the schools. 

Having received the lists of the participating teaching assistants and eligible pupils from each of the schools, the evaluator has randomly assigned the teaching assistants to one of the two following intervention groups: one delivering the Catch Up intervention and the other delivering a time equivalent intervention (also delivered on a one-to-one basis). Furthermore, the six eligible pupils within each school were also randomly allocated by to one of the following three groups: Catch Up, time equivalent support and control (‘business as usual’). Overall, 336 pupils participated in the study and they were divided equally amongst the three groups (i.e. each of the groups consisted of 112 pupils).

As described in more detail above, eligible pupils (i.e. identified by their teachers as struggling with numeracy) have been assigned randomly to one of three groups: the Catch Up group, a time- equivalent, one-to-one intervention group and control (‘business as usual’) group. The time- equivalent intervention group has been introduced to account for the general positive effects of one-to-one teaching that are likely to be assigned to the Catch Up Numeracy specifically when only using a control group as a comparison.

The intervention period amounted to 30 weeks and the numeracy skills of the pupils have been measured before and after the intervention took place, i.e. pre-tests were conducted between September and November 2012 and the post-tests between June and July 2013.

The analysis was undertaken on an intention to treat (ITT) basis. Given that the randomization was stratified by school (and hence the school had to be taken into account in the analysis) a multi-level model was used. In the model, the dependent, outcome variable was numeracy ability measured using the Basic Number Screening test. Furthermore, the pre-intervention test scores were used as a covariate to the post-intervention test score and finally, the model also included two dummy variables which were used to identify pupil membership to either the control group or Catch Up intervention group. This allowed identifying any differences between the equivalent time group and the control group as well as between the Catch Up intervention group and the equivalent time group.

Following the intention to treat analysis, a sub-group analysis which introduced variables for eligibility for free school meals (FSM) and gender (as well as a number of interactions between the new variables and the previously described ones) was conducted. This analysis helped to determine whether the effects of the intervention differed based on pupils’ characteristics. This analysis has also allowed for the differences observed at baseline with regards to the gender ratio as well as the proportion of pupils eligible for free school meals between the treatment and control groups.

The results obtained suggest that while there is a significant difference in the numeracy ability of children who obtained Catch Up Numeracy support compared to those in the control group, the scores of the pupils belonging to the Catch Up group are not statistically different from those of pupils belonging to the alternative time equivalent group.

Furthermore, once gender and eligibility for free meals are controlled, the previously reported statistically significant difference between the Catch Up group and the control one becomes insignificant as well. In fact, as can be observed in the table below, the sub-group analysis (which includes all of the controls) does not identify any significant factors impacting  the results of the post-test apart from the score obtained by the pupil in the pre-test.

Note

Catch Up is a not-for-profit UK registered charity (1072425)
Catch Up is a registered trademark
Catch Up Ltd is an endorsed charitable institute ABN: 62154644498

Bibliography

Wayne Holmes and Ann Dowker (2013) Catch Up Numeracy: a targeted intervention for children who are low-attaining in mathematics, Research in Mathematics Education, 15:3, 249-265, DOI: 10.1080/14794802.2013.803779.

Education Endowment Foundation (2014) Catch Up® Numeracy- Evaluation Report and Executive Summary, February 2014. Independent evaluators: National Foundation for Educational Research (NFER).

Contact information

Name

 Mrs Theresa Rogers

Organization

 Catch Up

Address

 Keystone Innovation Centre, Croxton Road, Thetford, Norfolk IP24 1JD, UK

Email

 info@catchup.org

Website

http://www.catchup.org/CatchUpNumeracy/CatchUpNumeracy.aspx

Last updated

December 2018

Find a practice (within the 'Evidence based practices'):

The practices can be also searched along the three policy pillars of the Recommendation for Investing in Children:


Share this page