Giving Compass' Take:

• International test rankings lack the nuance required to make appropriate comparisons between countries.

• How can these test scores be altered to better represent global education status? How can the dialogue around the tests change to reflect potential shortcomings of the format?

• Find out why, in spite of shortcomings, international test rankings matter.

Renowned statistician and education expert Judith Singer has issued a new warning against overinterpreting the results of international test rankings. In an article published Thursday in the journal Science, she writes that tests like PISA and TIMSS bear important data for education policymakers around the world, but that rankings systems based on scores ultimately distort more than they reveal.

U.S. students’ mediocre performance on tests of math, reading, and science routinely yield glum narratives in the national press, particularly when held up against the relatively higher scores in East Asian countries like Korea, Japan, and Singapore.

The insight gained from large-scale assessments provides a valuable window into academic practices, Singer says, but journalists and politicians are too often transfixed by their countries’ respective spots in the global pecking order.

Worse than the alarmism accompanying news stories, Singer says the rankings themselves are frequently arbitrary and mercurial. Positions change from year to year for reasons having little or nothing to do with student performance in a given country. And the rules of the tests allow for a certain amount of gamesmanship, as when Shanghai earned a top ranking for math in the 2012 PISA exam — only for the world to later discover that it had excluded 27 percent of its 15-year-olds from taking it.

On the 2015 PISA, Japan improved on its fourth-place ranking for scientific literacy three years earlier, moving to second overall. But the jump wasn’t because of improved performance; scores actually went down, though not as much as other countries’.

Read the full article on international test rankings by Kevin Mahnken at The 74