Dylan Wiliam, emeritus professor of educational assessment at University College London, told the Association of School and College Leaders’ annual conference in Birmingham that every school leader “needs to be a critical consumer of educational research”, because it can never definitively tell you what to do.
One issue highlighted by Prof Wiliam was “aptitude treatment interaction” – what he called a “posh psychological term for when (an intervention) works in some places, and it doesn't work in other places”.
He explained: “What you find is, you get huge effects in some studies and tiny effects in others, and therefore the average is irrelevant. It's the equivalent of claiming that a man with one foot in boiling water and one foot of freezing water is, on average, comfortable.”
Prof Wiliam is well known for his work on the quality of teaching and assessment practices in schools – including his book Inside the Black Box: Assessment for Learning in the Classroom, which he co-wrote with Paul Black.
During his session, he said that governments had failed to acknowledge these challenges: “There's a big push from policy-makers and others to use this thing called ‘evidence-based approaches to education’,” he said. “What I want to remind people of is that it's a continuum ... so there's no such thing as an evidence-based approach, because the evidence is so complex. What I'm suggesting is that there's a continuum of evidence.”
Prof Wiliam gave his own research as an example, referring to a study of ability grouping in maths conducted in the 1990s. With colleagues, he tracked 1,000 students in six different schools being taught in mixed ability groups from year 7 to 11.
“We saw what results they got on their GCSEs and we drew some conclusions about the effects of setting,” Prof Wiliam explained. “I now regard that work as useless, because we never got a handle on teaching quality. We did see some negative effects of grouping by sets, but we didn't know whether that was caused by kids in the bottom sets being given less effective teaching ... and, in many schools, the highest achieving students getting the best teachers.”
Prof Wiliam concluded: “Even though I have produced quite a lot of educational research in my time, I actually now think that educational research is limited when it comes to education.”