The usefulness and veracity of the PISA tests divide opinion in the education world. But, in the UK at least, the international comparisons can offer a useful insight into how our 15-year-olds are performing against each other, and the world.
The main focus of the recently published 2015 PISA tests was science, but students also had to complete questions on maths, reading and problem-solving. The exercise did not test knowledge, but rather students’ reasoning and interpretation skills and their ability to solve problems.
The PISA tests, administered by the Organisation for Economic Co-operation and Development (OECD), provide in-depth contextual information about different education systems, schools, teachers, students and how they live, and examine the relationships between these factors and levels of achievement.
This information enables governments to inform their own policy-making. However, caution is needed. How students perform could also be down to a range of other factors that are not accounted for in the tests.
The NFER’s briefing paper, Key Insights from PISA 2015 for the UK Nations, urges caution on how much we can deduce from changes in students’ performance.
It states: “Simply looking at whether the score for science, maths or reading is higher or lower than in a previous PISA cycle does not accurately tell us whether achievement has improved, is stable or is in decline.”
It is crucial to consider whether a score is statistically significantly different; in other words, that differences have not arisen solely by chance. It cites the example of Northern Ireland, where maths scores went up by six points since 2012 and yet the analysis found that performance had remained stable.
Furthermore, while it might be tempting to focus on rankings when trying to compare achievement between countries, this can be misleading as differences in scores might not be statistically significant. So while England is five positions higher in the rankings than Scotland, their scores are not significantly different.
Science
So how did the home countries’ performance compare? The PISA results show that students in England achieved significantly higher scores in science than their peers in the other three nations, with students in Wales scoring significantly lower.
Further analysis of the highest and lowest performers reveals that England had the highest number of top performers at 12 per cent, compared with Scotland at eight per cent, Northern Ireland at seven per cent, and Wales at five per cent.
England also had the lowest percentage of low performers, at 17 per cent, followed by Northern Ireland at 18 per cent, Scotland at 20 per cent, and Wales at 22 per cent. There were no significant gender gaps in performance in any of the four UK countries.
Wales recorded the smallest difference between the highest and lowest achievers, and England the largest – the equivalent of nearly nine years of schooling.
Performance in science had declined in Scotland and Wales since it was last the focus of PISA in 2006, while in England and Northern Ireland there were no significant differences.
Mathematics
In maths, students in England, Northern Ireland and Scotland scored slightly above the OECD average, while in Wales the score was significantly lower.
Once again, England had the highest percentage of top performers at 11 per cent, followed by Scotland at nine per cent, Northern Ireland at seven per cent, and Wales at five per cent.
However, 22 per cent of students in England failed to reach the baseline ability in maths and lower performing students had lower average scores than their peers elsewhere in the UK. The percentage not reaching baseline ability in Northern Ireland was 19 per cent, in Scotland 20 per cent, and in Wales 23 per cent.
Wales had the smallest difference between high and low achievers, while England had the biggest gap, which was equivalent to eight years of schooling. Boys performed better in maths than girls in England and Wales, but this was not a pattern repeated in Northern Ireland or Scotland. Generally, maths scores have remained stable for all UK nations since maths was last the focus for PISA in 2012.
Reading
In reading, no significant differences were recorded in scores in England, Northern Ireland and Scotland, but students in all three scored higher than young people in Wales.
England had the highest average reading score among top performers, followed respectively by Scotland, Northern Ireland and Wales. In England, one in 10 students were deemed as top performers, while in Scotland and Northern Ireland six per cent reached this level and just four per cent in Wales.
At the other end of the scale, 18 per cent of students did not reach the baseline level of ability in England, compared with 15 per cent in Northern Ireland, 18 per cent in Scotland, and 21 per cent in Wales. Once again, Wales had the smallest difference between high and low achievers, and England, the largest, equating to more than eight years of schooling.
Overall, across the home countries, girls were found to be better readers than boys but there has been no major shift in reading performance since 2009, although there has been a decline in Scotland since 2012.
Contextual data
As well as focusing on test scores, PISA seeks to explain how and why students perform as they do in different countries by looking at contextual data. An analysis of socio-economic status (SES), for example, reveals that England has the largest gap in performance of students with high and low SES, and Wales, the lowest.
This means that, in Wales, performance has less to do with affluence than in England, and that other factors will also have affected student success.
The school environment may have an impact on outcomes. Headteachers in England and Scotland were more likely than colleagues elsewhere in the UK to report teacher shortages, while heads in England and Wales were more likely to cite inadequate or poorly qualified teachers as a concern. Welsh heads were also most likely to report that teachers being poorly prepared for class was a barrier to learning.
Maths in England
In its report Is Mathematics Education in England Working for Everyone?, the NFER analysed PISA data to find out how well England was supporting pupils from disadvantaged backgrounds in the teaching and learning of maths.
It found that, while no worse than in many other OECD countries, the gap between the most and least disadvantaged pupils was equivalent to three years of schooling at age 15. International evidence suggests this is a gap that is hard to plug.
Pupils in England were not found to be lacking in any particular aspect of maths but were weaker in the subject across the board.
In considering recommendations, the report highlighted evidence that grouping pupils by ability can have detrimental effects, and can lead to low-ability children being exposed to less rigorous maths and so fewer opportunities to reach their potential.
The report suggests that new methods of measuring deprivation need to be found and that summer-born children, who were found to be less likely to overcome disadvantage than their autumn-born classmates, need specific strategies to ensure they are not left behind.
Furthermore, new research is needed on those children who beat the odds to perform well, and on the sharing of successful, evidence-based strategies that schools are adopting to support disadvantaged pupils.
- Dorothy Lepkowska is a freelance education journalist.
Further information
You can read NFER’s education briefings Key Insights from PISA 2015 in Scotland and Key Insights from PISA 2015 for the UK Nations, via www.nfer.ac.uk/research/pisa-2015/
NFER Research Insights
This article was published as part of SecEd’s NFER Research Insights series. A free pdf of the latest Research Insights best practice and advisory articles can be downloaded from the supplements page of this website: www.sec-ed.co.uk/supplements