Many schools make use of educational interventions such as tutoring to support pupil progress and attainment. Ben Dryden and Douglas Fairfield consider how we can evaluate effectively and efficiently whether interventions are having the desired impact
Image: Adobe Stock -

The effectiveness of pupil interventions depends not only on their delivery, but also on developing robust systems for evaluating their impact (Sharples et al, 2018).

Although the term “intervention” might encompass different activities from setting to setting, this article defines an intervention as the dedicated time spent by an individual or small group of pupils with an educator, aiming to improve attainment in a specific area or subject.

 

The challenge of quality assurance

One of the key challenges faced when implementing an intervention is determining whether or not the said intervention is having the desired impact (Ofsted, 2023).

The method of evaluation depends to a great extent on the intervention’s objectives but should ideally be a formal process. For example, where the objective is to improve pupil motivation, the evaluation might require formally recorded feedback on pupil confidence levels from the pupils and their parents/carers.

Conversely, for an objective to improve pupil attainment, self-report from pupils wouldn’t necessarily be an adequate yardstick of impact and would need to be supplemented with a comparison of pre and post-intervention performance, as well as monitoring of the quality of intervention delivery.

Measuring impact purely on a summative basis risks “wasted” time and resources that might otherwise have been deployed more fruitfully in a different intervention or activity.

In its 2023 findings following its review of tutoring in schools, Ofsted observed that while some schools had established processes for monitoring the outcomes of a particular intervention, fewer had implemented systems for quality assurance of the intervention sessions.

That said, if the goal of intervention is to improve attainment, then it is essential to:

  1. Measure, at the end of the programme, its impact against pre-determined goals.
  2. Regularly monitor the quality of the delivery of the intervention.

 

Research-based strategies for impact evaluation of interventions

Developing a structured evaluation framework: Ofsted’s findings suggest that schools should implement a structured framework which includes:

  • Predefined specific and measurable metrics for success.
  • Regular assessment point(s) for measuring progress against those predefined metrics.
  • Clear criteria of what constitutes effective intervention delivery.

This may sound onerous, but it is likely that systems and processes like these are already in place in your school. By mirroring the evaluation processes used for their embedded curriculum, schools can foster a consistent quality standard across all educational interventions.

 

Leveraging standardised assessments as metrics for success: According to findings from a National Foundation for Educational Research evaluation of the National Tutoring Programme, standardised assessments can be used to effectively gauge impact (Lucas et al, 2023).

Schools participating as their “research champions” used standardised English or maths assessments to obtain a quantitative measure of the effectiveness of tutoring.

Schools should consider incorporating similar assessments, ensuring they have baseline and end-point data to measure progress accurately. Where possible, assessments should align with the intervention’s focus in some way, e.g. by testing skills and knowledge from previous assessments as well as the prevailing curriculum.

 

Incorporating pupil and parent feedback: Informal feedback from pupils and parents remains a valuable source of insight into the impact of tutoring. Schools should establish a system to collect feedback, using surveys or focus groups at regular intervals to gather perceptions regarding the effectiveness of the intervention, particularly in enhancing pupil confidence and engagement.

 

The role of curriculum alignment

A recurring theme in the impact evaluation of tutoring is the importance of aligning the intervention with the school's curriculum. Ofsted’s review (2023) found that tutoring worked best where there was a close alignment between the educator leading the intervention and the class teacher – and better still where the class teacher was the one delivering the intervention.

Effective tutors collaborate closely with classroom teachers to ensure that tutoring sessions address specific learning gaps and reinforce classroom instruction. This alignment enhances the coherence of pupils' learning experiences, making it easier to evaluate the impact of tutoring in the context of overall educational outcomes.

 

Final thoughts

As schools strive to maximise the benefits of in-school interventions, the importance of a systematic, evidence-based approach to evaluating impact cannot be overstated.

By implementing structured evaluation frameworks, ensuring curriculum alignment, and supporting the quality of those delivering the intervention, schools give themselves the best chance of enhancing the effectiveness of interventions while also providing clear evidence of their value in supporting pupil achievement.

  • Ben Dryden is school partnerships manager and Douglas Fairfield education advisor at the Education Development Trust. Visit www.edt.org and @EDTvoice.

 

Further information & resources