Confusion in the Ranks

Confusion in the Ranks: how good are England’s schools? Report written by Alan Smithers for The Sutton Trust. From Executive Summary:

The most recent international league tables of pupil performance differ considerably. England languishes well down the  list in PISA 2009, stars in the Pearson Global Index 2012, and lies somewhere in-between in TIMSS 2011. This report seeks to explain the differences and highlight some underlying consistencies.
There are three main reasons for the different rankings:

  • Countries are ranked on scores which may not be different;
  • Different countries are involved;
  • The tests differ and some countries are ahead on one but not the other.[...]

Secondary School Pupils

We can see how these differences play out if we look in detail at the maths performance of secondary school pupils as an example. PISA 2009 has England joint 27 out of 65 countries and TIMSS 2011 tenth out of 42.
If we want to be at least 95% sure that a country has performed above England, then there are 20 above England in PISA and six in TIMSS.
Of those countries, five are above in both: Japan, Hong Kong, Singapore, South Korea and Taiwan.
Eleven countries were above England in PISA, but did not take part in TIMSS: Belgium, Canada1, Denmark, Estonia, Germany, Iceland, Liechtenstein, Macao, the Netherlands, Shanghai, and Switzerland.
Four countries were above in PISA, but not in TIMSS: Australia, Finland, New Zealand and Slovenia.
Russia was above England in TIMSS, but not PISA. [...]

Perhaps the most interesting part of the Report is a detailed analysis of the use of  international league tables in political debates.