TY - JOUR

T1 - Reality-based tasks for competency-based education

T2 - The need for an integrated analysis of subject-specific, linguistic, and contextual task features

AU - Leiss, Dominik

AU - Ehmke, Timo

AU - Heine, Lena

N1 - Publisher Copyright: © 2024 The Authors

PY - 2024/8/1

Y1 - 2024/8/1

N2 - In evaluating competency-based education, effective test instruments must address real-life complexities. The impact of subject-specific, linguistic, and contextual task features, alongside central personal characteristics, on the empirical challenge of such tasks is unclear. We developed mathematics tasks from 30 real-world contexts, each with three questions of varying complexity, administered through a systematically rotated experimental design to 535 German grades 9 and 10 students. Various student variables were collected. Generalized linear mixed models revealed that contextual and mathematical task features significantly contributed to task difficulty variance. Language features had no intermediate-level influence, while students' mathematical self-efficacy moderated low task context familiarity's impact. These findings guide the construction of reality-based mathematics tasks to tailor empirical difficulty. Educational relevance: In the context of worldwide competence-orientated education, it is crucial to reform in-class and national tests. Traditional task formats are limited in representing authentic problems. In most school subjects, a lack of understanding exists in designing reality-based competence-oriented tasks that ensure fair test conditions and meet the subjects' normative demands. This study addresses this gap by empirically investigating the interplay of subject-related, linguistic, and contextual aspects of reality-based tasks in mathematics. Teachers and researchers can use these insights to improve competence-oriented performance situations, sparking further questions. These findings encourage similar studies across subjects for broader applicability.

AB - In evaluating competency-based education, effective test instruments must address real-life complexities. The impact of subject-specific, linguistic, and contextual task features, alongside central personal characteristics, on the empirical challenge of such tasks is unclear. We developed mathematics tasks from 30 real-world contexts, each with three questions of varying complexity, administered through a systematically rotated experimental design to 535 German grades 9 and 10 students. Various student variables were collected. Generalized linear mixed models revealed that contextual and mathematical task features significantly contributed to task difficulty variance. Language features had no intermediate-level influence, while students' mathematical self-efficacy moderated low task context familiarity's impact. These findings guide the construction of reality-based mathematics tasks to tailor empirical difficulty. Educational relevance: In the context of worldwide competence-orientated education, it is crucial to reform in-class and national tests. Traditional task formats are limited in representing authentic problems. In most school subjects, a lack of understanding exists in designing reality-based competence-oriented tasks that ensure fair test conditions and meet the subjects' normative demands. This study addresses this gap by empirically investigating the interplay of subject-related, linguistic, and contextual aspects of reality-based tasks in mathematics. Teachers and researchers can use these insights to improve competence-oriented performance situations, sparking further questions. These findings encourage similar studies across subjects for broader applicability.

KW - Assessment

KW - Competency-based education

KW - Language competence

KW - Mathematical competence

KW - Test items

KW - Mathematics

KW - Didactics of Mathematics

UR - http://www.scopus.com/inward/record.url?scp=85199531360&partnerID=8YFLogxK

U2 - 10.1016/j.lindif.2024.102518

DO - 10.1016/j.lindif.2024.102518

M3 - Journal articles

AN - SCOPUS:85199531360

VL - 114

JO - Learning and Individual Differences

JF - Learning and Individual Differences

SN - 1041-6080

M1 - 102518

ER -