The language characteristics of mathematics tests can influence the validity and the fairness of assessments especially for second language learners (Wolf & Leon, 2009). For example, Abedi and Lord (2001) showed that the students’ language background has an impact on their performance on math word problems and that their performance can be influenced through modifying the linguistic structure of the test. Furthermore, Martinello (2008) showed that the complexity of the linguistic structure of the test items can cause differential item functioning (DIF) for English Language Learners. According to this, it is important for the evaluation and interpretation of assessments to consider a possible influence of language characteristics on the test scores.
The aim of this study was to analyze the impact of different language characteristics of 285 mathematics items from the two mathematics assessments National Educational Panel Study (NEPS) and Standard-based National Assessment in Germany (NA) on the item difficulty. The empirical basis for this analysis was built by a study where N=752 fourth graders were assessed with the NEPS and N=1584 fourth graders were assessed with the NA mathematics test. For the estimation of the item difficulty the one dimensional scaling of the Rasch-Model is used.
Furthermore, the mathematics items were classified according to their language demands on word-level, sentence-level, text-level and presentation formats. In order to explore the relationships between the item difficulty and the language characteristics correlation analyses and regression analyses were carried out. The outcomes reveal an impact from divergent linguistic characteristics on the item difficulty, whereby the extent of the influence differs for distinct subpopulations (e.g. migration).
These results enhance the importance of taking into account the impact certain linguistic characteristics of the mathematics items have on item difficulty for a valid evaluation and interpretation of the test scores and for creating fair test items.