In its 25 years as an inspection agency one question has been asked, but not answered, by Ofsted.
Just how reliable are its inspection judgements? More specifically, can Ofsted be sure that two inspectors in the same school, on the same day, would come to the same judgement about the school?
Now we know the answer. National director for schools policy at Ofsted, Sean Harford, admitted recently that: “Ofsted has not done enough in the past to test the reliability of inspection; we have concentrated on quality assurance. This provides assurance that the process is carried out consistently as we would wish, but not directly that different inspectors in the school on the same day would give the same judgement.”
Whichever way you look at it, this is a remarkable admission. Schools, and school leaders, live under the constant fear of a poor Ofsted judgement. The consequences for school leaders should their school be deemed in need of improvement or be put in special measures are usually devastating and signal the end of their careers.
And school leaders’ fear of Ofsted has resulted, for teachers, in spiralling workload pressures, where dedicated professionals are forced to record every aspect of their practice so that, when Ofsted calls, everything is documented. Work overload is leading to professional exhaustion, burn-out and a looming recruitment and retention crisis. And all of this has a detrimental impact on children’s learning in schools.
Ofsted judgements matter to the profession, to politicians and to parents. But we now know that Ofsted has no means of knowing whether its judgements are reliable – although it has a plan.
Ofsted has “built in” reliability testing for the pilots for the new short inspections starting this term, and Mr Harford assures us: “If reliability is a problem, we will review the issues to see what we need to do to make the inspections reliable.”
So, that’s okay then! After 25 years of Ofsted inspections, upon which millions of pounds of tax-payers’ money has been spent, teachers have sweated millions of hours over extra workload, heightened levels of stress have led to the loss, to the profession, of thousands of well-qualified and highly conscientious teachers, Ofsted has stirred itself and will test the reliability of its judgements in the pilots of the new short inspections.
It is remarkable that Ofsted has been allowed to evade its responsibility for assuring the reliability of its inspection judgements. Successive secretaries of state for education should have insisted that Ofsted had robust systems in place to secure reliability.
As Professor Robert Coe, director of the Centre for Evaluation and Monitoring at Durham University, has argued: “Establishing the validity of judgements is important enough in research, but much more so if those judgements are going to be used to make crucial decisions about a school or teacher.
“It seems incomprehensible to me that in the 20 years Ofsted has operated no-one has thought this should be required. And, just to be clear, providing evidence of the validity of judgements shouldn’t be done once in a half-baked study conducted by Ofsted, but should be done rigorously, with independent scrutiny, on an on-going basis every time a judgement is made. I know many researchers (including me) would be willing to collaborate with Ofsted to support this work.”
Prof Coe’s rigorous requirements will not be met under Ofsted’s proposals which can only be described as half-baked. They will not be objective – how could they be when Ofsted is inspecting its own ability to generate reliable judgements of school quality? Nor will they be rigorous, because this would require independent evaluation of any findings. Let me assure you, Ofsted has no intention of opening itself up to that level of scrutiny – it has too much to lose.
-
Dr Mary Bousted is general secretary of the Association of Teachers and Lecturers. Visit www.atl.org.uk