New book promotes informed conversations about international testing

May 30, 2017
Lindsay Yakimyshyn
The Global Education Race: Taking the Measure of PISA and International Testing. Sam Sellar, Greg Thompson and David Rutkowski. With a foreword by David C. Berliner and Pasi Sahlberg. Edmonton, Alta: Brush Education Inc, 2017.

Book Review

The Programme for International Student Assessment (PISA) results have incited education ministers and policy-makers to either laud or denounce their education systems. The results have formed the basis of countless headlines, some of which declare a state of educational crisis and call for reform. But what is PISA? What are its benefits and its limitations? What—if anything—does it tell us about our schools and our children? In The Global Education Race: Taking the Measure of PISA and International Testing, Sam Sellar, Greg Thompson and David Rutkowski tackle these questions as they promote conversations about PISA and international testing more generally.

As David Berliner and Pasi Sahlberg state in the foreword, this book “informs and warns, but it is neither anti-testing nor anti-PISA.” Instead, it promotes informed, critical engagement with the discourse and the politics surrounding international testing. It achieves this by describing and unpacking PISA in a lucid, concise and—most importantly—balanced manner.

The monograph frames the issue of international testing as a race in which countries participate “in the urgency of getting ahead.” In exploring this, Sellar, Thompson and Rutkowski quickly hone in on PISA, a “globally influential assessment” that tests 15-year-old students’ reading, mathematical and scientific literacy. They offer a clear description of PISA, its purpose and its origins to provide the reader with an important access point to the debate on international testing.

The authors provide a roadmap to their book, highlighting how they will address such matters as the role of media in creating stories out of PISA results; how the test’s rankings distract from other PISA findings; the construction of international, large-scale assessments; difficulties in comparing educational systems; and validity issues in standardized testing. Pervading each section of the book is an emphasis on the need for careful interpretation of assessment results, particularly as PISA “shape[s] thinking about education around the world.” 

The authors stress that the “power of PISA” is inextricable from the stories it generates. Specifically, they argue that the media and leaders in education use PISA results to construct different narratives that often encourage policy change. The media’s tendency to focus on rankings becomes particularly problematic as rankings in fact represent a range (because of standard error), are limited due to sampling issues and “reduce thousands of pages of complex analysis into a single idea.” In communicating PISA results, the authors suggest, the participating countries need to “specify clearly [the assessment’s] uses and limitations” and “define the goals of PISA in national contexts.”

Noting the need to contextualize international assessments and their results, Sellar, Thompson and Rutkowski productively consider the effects of culture on student performance and on the interpretation of PISA data. They also, however, strongly caution against constructing causal relationships. For example, performance on standardized tests is connected to the students’ cultural background and socioeconomic status (significantly more than to their schools); yet, this does not mean that these factors cause the students to perform well or poorly. In a similar vein, the authors warn readers to be wary of any connections drawn between PISA results and a jurisdiction’s economic performance. 

Even as they highlight PISA’s limitations, Sellar, Thompson and Rutkowski speak highly of the international assessment. In fact, they suggest that PISA “is perhaps one of the best efforts that has been made to accurately measure educational outcomes.” But this praise is followed by a warning: “The positive potential of educational measurement is eroded when data are gathered and used by one group of people to make potentially high-stakes judgments and decisions about another group of people.” With this in mind, the authors call for technical democracy, with wider and more informed debate on international testing. The book itself can facilitate such debate.

A useful read for academics, this book engages a broader audience and is primarily geared toward parents and educators. It equips readers with talking points to enable them to engage confidently in conversations about international testing. 

The book’s key selling point is that it does not work to sell a particular perspective. Instead, Sellar, Thompson and Rutkowski outline key considerations related to PISA to improve data literacy and to encourage readers to think critically about and contribute to debate on PISA and international testing. 

Dr. Lindsay Yakimyshyn is an administrative officer involved in research at the Alberta Teachers’ Association.