PISA 2015: how big is the ‘mode effect’ and what has been done about it?

PROGRAMME for International Student Assessment; EDUCATIONAL tests & measurements; EDUCATIONAL quality; EDUCATION policy; GRADING of students; TEENAGERS; HIGHER education

The Programme for International Student Assessment (PISA) is an important cross-national study of 15-year-olds’ academic knowledge and skills. Educationalists and public policymakers eagerly await the tri-annual results, with particular interest in whether their country has moved up or slid down the international rankings, as compared to earlier rounds. In 2015 a major change was implemented in PISA, with the introduction of computer-based assessment. This has the potential to reduce comparability of PISA test scores across countries and over time. We investigate this issue using PISA 2015 field trial data for three countries: Germany, Sweden, and Ireland. We show how, if left unaccounted for, the change to computer-based testing could limit the comparability of PISA test scores. We then describe the methodology the study organisers have used to account for such mode effects. Our key conclusion is that although the adjustment made is unlikely to overcome all the potential challenges of switching to computer-based tests, it represents an improvement over the alternative of making no adjustment at all. [ABSTRACT FROM AUTHOR]

Copyright of Oxford Review of Education is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)