What lessons can we learn from the GCSE marking debacle? Perhaps one thing: so far no-one can really explain what happened. In particular, why relatively small statistical changes can have such a devastating impact on large numbers of people. And it is vital that, in all the abstract debate on statistics, we remember that there are real people who have been dealt a severe blow.
This confusion is in itself disturbing. It suggests that we are not truly in control of our examination system, that the tail wags the dog. The regulator did not have the data, skills, resources or systems to react to changes at the speed required to stay on top of things. The very complexity of the system makes it difficult to predict consequences.
More troubling is the phenomenon of what we might call "exam optimisation" – the complete focus on the C/D boundary in the core subjects measured by the English baccalaureate and the league tables.
This optimisation manifests itself in all sorts of ways:
Close attention to predicted grades and targeting of resources to those on the borderline.
Selection of exam boards.
Coaching in exam technique and "teaching to the test".
The marginalisation of non-examined activities.
Close analysis of marking schemes.
Early entry and 'banking' of grades.
Although far from what we might consider an ideal education experience, it is important to stress that such optimisation is – mostly – an ethical stance. The C/D boundary, and the core academic subjects, matter as much to the student as they do to the school. If you get a student a B in maths and a D in English, when they could have got a C in both, you have done them no favours for their career or further education. Exam optimisation is a right choice in a wrong system. The question is, can we get a better system?
Before we get to some suggestions, there is one more factor we need to take into account: comparable outcomes. This is really no more than norm-referencing with caveats and introduces some dilemmas into the system. Most obviously it attempts to hold the proportions achieving each grade steady while every school is required, on threat of conversion, to increase the proportions achieving the higher grades. It has no objective method for assessing improvements in the quality of teaching other than the very exam results it is supposed to adjust – an entirely circular argument.
Fundamentally, a norm-referenced exam system cannot be used to hold schools accountable in the way we have become accustomed to.
A number of proposals might help, without obscuring or weakening accountability. The trick is to reduce the worst of the perverse incentives. Unless these steps are taken, however, the government's GCSE reforms will have reduced impact. All the old problems will eventually reinfect the new system. Indeed, it might be a law of organisational life that any high stakes performance measurement system becomes quickly obsolete and inaccurate.
Step one is to replace the focus on cliff-edge measures, such as the percentage of students achieving five or more good GCSEs, with a points-based system which values the progress of students at every ability level. You could even weight the points to value the core academic subjects more highly. I think this can be done in a way which is easily digestible by parents and the public.
Step two is to abandon floor standards and connect intervention for under-performance solely to inspection outcomes. Inspection takes account of both attainment and progress, as well as curriculum breadth, but it has some level of judgement involved. This would also result in a single accountability system rather than the competing pressures from the Department for Education and Ofsted. From the government's perspective, their powers of intervention are much more robust in relation to schools in categories than in relation to floor standards; although I suppose I shouldn't be offering tips on this topic.
I should also add a hasty step three here: a reformed inspection system with much tighter quality control, credible independent appeals and arbitration, and a much heavier practitioner involvement.
Step four: if you are going to have a baccalaureate, make it a proper baccalaureate – with evidence of engagement in sport, culture, arts, enterprise and civic responsibility attached to the award of a wrap around certificate. In our high stakes system, I fear this may be the only way to reward breadth right through to sixteen and beyond.
My final point would be to encourage schools and the government to take advantage of the rise in the participation age to design new models of study for level two qualifications. Rather than two years of falling behind and growing alienation, followed by repeated re-sits, what about a three-year course for those who would benefit, which ensures that each concept is fully embedded before moving on? This would be of great benefit to maths study, for example.
Russell Hobby is general secretary of the National Association of Head Teachers. Visit www.naht.org.uk