The publication of the PISA results in December was a major news story.
Indeed, the BBC dubbed it ‘The world’s most important exam’. But behind the headlines on the continuing strength of the Far Eastern school systems, and the ‘middling’ performance of the UK, lie 4 volumes of extensive data.
One of the most consistent findings of PISA 2012 was the increase in the use of achievement data. Indeed, 35 of the 38 countries who took part in PISA 2003 and PISA 2012 have increased their use of data. PISA has looked consistently at 8 uses of achievement data.
- To inform parents about their child’s progress
- To make decisions about students’ retention or promotion
- To group students for instructional purposes
- To compare the school to district or national performance
- To monitor the school’s progress from year to year
- To make judgments about teachers’ effectiveness
- To identify aspects of instruction or the curriculum that could be improved
- To compare the school with other schools
Comparisons between 2003 and 2012 reveal some interesting findings.
Using average figures, the biggest increases have been in comparing the school to district or national performance (46.2% in 2003 to 61.7% in 2012) and comparing the school to other schools (39.6% in 2003 to 52.5% in 2012).
Looking at individual countries, points of interest include Ireland’s use of data to compare the school to district or national performance which has increased from 17.2% to 77.3% – or from less than 1 in 5 schools to nearly 4 in 5. Denmark has increased its use of data to compare schools with other schools from 2.9% to 55.9%. Both Hong Kong China and Macao China have vastly increased their use of data for comparison purposes.
When one looks at all participating countries in the 2012 survey the average for use of data to compare the school to district or national performance is 64%.
The report also produced data in ‘an index of assessment practices’ which combined the use of data across the 8 areas. Interestingly, no schools reported that they consistently used all 8 practices and in only 4 OECD countries (New Zealand, Israel, UK and USA) did more than 50% of schools report they used 6 of the 8.
In the UK there is a national system, (RAISEonline) that allows schools to maximise their use of data, as well as results analysis services from the major academic awarding bodies. The report demonstrates that this seems to be very well used but that more could be done to release the accumulated and combined power of data to improve learning outcomes.
The use of achievement data beyond school
PISA also asked school principals whether achievement data is posted publicly, or tracked over time by an administrative authority.
Findings include the fact that tracking achievement data over time seems to be a more common practice than posting such data publicly which is often a national policy decision. On average across OECD countries, 72% of students are in schools whose principals reported that achievement data is tracked over time by an administrative authority. In 31 countries and economies, over 80% of students attend schools whose principals reported this, while only in Japan do fewer than 10% of students (7%) attend such schools.
The debate over data is far too often centred on decisions as to what should or shouldn’t be made publicly available with the inevitable resulting tables and rankings.
PISA 2012 has demonstrated that schools around the world are increasingly using achievement data for far more than external accountability. The UK should use its leadership in the provision of data to support and develop the effective use of such data. As the constituent parts of the UK seek to improve their PISA performance the potential for data to be used in a holistic manner is clear.
Martin Milner is Managing Director of White Swan Consulting and is focussed on helping organisations successfully implement and realise the potential of e-assessment.