The Alberta Teachers’ Association

New book promotes informed conversations about international testing

A new book commissioned by the Alberta Teachers’ Association is attempting to generate critical engagement around international testing programs such as the Programme for International Student Assessment (PISA).

The Global Education Race: Taking the Measure of PISA and International Testing is co-authored by education scholars Sam Sellar, Greg Thompson and David Rutkowski.

The book is not anti-PISA but is “an effort by three globally recognized scholars to address the mission creep of international assessments, often co-opted by education bureaucracies in the guise of research, in terms of the cost/benefit to students,” said J-C Couture, associate co-ordinator of research for the ATA.

He said the Association commissioned the book’s independent publication this past year in an effort to engage teachers and the public regarding the impacts of PISA and how it relates to “the growing Global Learning Metrics” industry, a phenomenon that’s described in the book’s foreword by researchers Pasi Sahlberg and David Berliner.

“Although the book takes a neutral stance on the current impacts of PISA around the world, here in Alberta the Association is concerned that international benchmarking has now moved to new levels of surveillance and control in classrooms,” Couture said.

The book frames the issue of international testing as a race in which countries participate “in the urgency of getting ahead.” The authors offer a clear description of PISA, which tests the reading, mathematical and scientific literacy of 15-year-old students, describing the program as a “globally influential assessment.”

Among the topics that the authors address are the role of media in creating stories from PISA results; how the test’s rankings distract from other PISA findings; how international, large-scale assessments are constructed; difficulties in comparing educational systems; and validity issues in standardized testing. Pervading each of the book’s sections is an emphasis on the need for careful interpretation of assessment results, particularly as PISA “shape[s] thinking about education around the world.”

The authors stress that the “power of PISA” is inextricable from the stories it generates. Specifically, they argue that the media and education leaders use PISA results to construct different narratives that often encourage policy change. They argue that the media’s tendency to focus on rankings becomes particularly problematic as rankings in fact represent a range (because of standard error), are limited due to sampling issues, and “reduce thousands of pages of complex analysis into a single idea.”

In communicating PISA results, participating countries need to “specify clearly [the assessment’s] uses and limitations” and “define the goals of PISA in national contexts,” the authors suggest.

Even as they highlight PISA’s limitations, Sellar, Thompson and Rutkowski speak highly of the international assessment. In fact, they suggest that PISA “is perhaps one of the best efforts that has been made to accurately measure educational outcomes.” But this praise is followed by a warning.

“The positive potential of educational measurement is eroded when data are gathered and used by one group of people to make potentially high-stakes judgments and decisions about another group of people.”

With this in mind, the authors call for technical democracy, with wider and more informed debate on international testing.

In Alberta, accountability has gone beyond existing as a policy framework to become a “daily habit that has saturated the teaching and learning experience,” Couture said. He added that no other teachers in Canada participate in more large-scale testing than Alberta teachers, who typically spend one-third of their 60-hour workweek gathering and reporting data on student performance.

“Ultimately, the three nested questions the book raises are applicable to the pervasive culture of testing that has become a way of life in Alberta schools: who is counting what and for whom?” Couture said.

“The publication of The Global Education Race represents one more step in reimagining a new way to recount student success.” ❚

The Global Education Race: Taking the Measure of PISA and International Testing. Sam Sellar, Greg Thompson and David Rutkowski. With a foreword by David C. Berliner and Pasi Sahlberg. Edmonton, Alta.: Brush Education Inc 2017. Get a copy The book is available as a trade paperback or ebook through a variety of retailers, including Amazon, Indigo, Kobo and Barnes & Noble.