As participation in international assessments (conducted by the OECD and IEA) increases, there is a growing discourse in the media and public opinion that these assessments are 'report cards' of the status and well-being of educational systems in participating countries. However, many participating countries also have national assessments or public examinations system in place to fill this role. National-level assessments typically have specific standards of performance defining acceptable performance, reflecting the desired quality of the education system expected by stakeholders in each country. The context of international assessments provides an opportunity to address the question of how these national-level performance standards relate to international benchmarks by establishing a link between the national and international scales.
One example of such a linking study has been done by Cartwright, Lalancette, Mussio, & Xing (2003). In a feasibility study a technical procedure was developed that allowed to link provincial tests from Canada (British Columbia’s annual Foundation Skills Assessment, FSA) with the Programme for International Student Assessment (PISA) so that standards can be compared and results reported on a common scale. Results of linking the two assessments showed that the FSA standard to recognize reading excellence in British Columbia was set higher than the PISA standard used to identify top readers across 32 countries. The linking procedures were valid for linking test scores for groups of students (greater than 30, for most statistics). However, they were not appropriate for linking and reporting scores for individual students. A second linking study that connects British Columbia English Examination to the PISA 2009 Reading Scale was done by Cartwright (2012).
In Germany, actually two linking studies are running. In grade level 4, a linking between (1) the international Trends in Mathematics and Science Study (TIMSS), the Germany’s Standard-based National Assessment (NA) and the multi-cohort National Educational Panel Study (NEPS) (Nissen, et al., 2012). The second linking study is focusing on LSA in grade level 9 and analysis the linking of the Mathematics and Science scales from PISA, NA, and NEPS.
This discussion group will examine the conceptual, technical and practical issues associated with establishing links between national and international assessments. Specifically, how should content-equivalence be evaluated? What is the impact of differences in psychometric methods? How should the results be communicated or used? In light of the aphorism that 'all models are wrong, but some are useful,' what are the limits to how far the linking exercise deviates from a true 'test equating' scenario while still providing useful information?