The eTIMSS and TIMSS Measurement Invariance Study: Multigroup Factor Analyses and Differential Item Functioning Analyses with the 2019 Cycle
Research output: Journal contributions › Journal articles › Research › peer-review
Standard
In: Journal of Measurement and Evaluation in Education and Psychology, Vol. 15, No. 2, 30.06.2024, p. 94-119.
Research output: Journal contributions › Journal articles › Research › peer-review
Harvard
APA
Vancouver
Bibtex
}
RIS
TY - JOUR
T1 - The eTIMSS and TIMSS Measurement Invariance Study
T2 - Multigroup Factor Analyses and Differential Item Functioning Analyses with the 2019 Cycle
AU - Yalçınkaya, Murat
AU - Atılgan, Hakan
AU - Daşcıoğlu, Selim
AU - Aydın, Burak
N1 - Publisher Copyright: © 2024 Association of Measurement and Evaluation in Education and Psychology (EPODDER). All rights reserved.
PY - 2024/6/30
Y1 - 2024/6/30
N2 - In this study, measurement invariance and differential item functioning (DIF) studies of the TIMSS 2019 4th and 8th-grade mathematics and science achievement tests were conducted for the country groups participating in both TIMSS and eTIMSS. The study sample consisted of 9560 responders of the first booklet of the 2019 cycle. Multiple Group Confirmatory Factor Analysis (MGCFA) was utilized to test measurement invariance, and Mantel-Haenszel (MH), Logistic Regression (LR), and SIBTEST were used for the DIF analyses. The measurement invariance results indicated strict invariance between groups for all tests which included 111 items in total. In the DIF analyses, for the 4th and 8th-grade mathematics tests, only three items showed moderate DIF with MH, and four items showed DIF with SIBTEST. For the 4th-grade science test, one item showed moderate DIF with both MH and SIBTEST. However, in the 8th-grade science test, no items showed DIF with MH and LR methods, while four items showed moderate DIF with SIBTEST. Overall, MH and SIBTEST techniques were in agreement, whereas LR method produced inconsistent results and showed disagreement with these two methods. The results of the measurement invariance analysis and the LR method were consistent and indicated equivalency of TIMSS and e-TIMSS scores.
AB - In this study, measurement invariance and differential item functioning (DIF) studies of the TIMSS 2019 4th and 8th-grade mathematics and science achievement tests were conducted for the country groups participating in both TIMSS and eTIMSS. The study sample consisted of 9560 responders of the first booklet of the 2019 cycle. Multiple Group Confirmatory Factor Analysis (MGCFA) was utilized to test measurement invariance, and Mantel-Haenszel (MH), Logistic Regression (LR), and SIBTEST were used for the DIF analyses. The measurement invariance results indicated strict invariance between groups for all tests which included 111 items in total. In the DIF analyses, for the 4th and 8th-grade mathematics tests, only three items showed moderate DIF with MH, and four items showed DIF with SIBTEST. For the 4th-grade science test, one item showed moderate DIF with both MH and SIBTEST. However, in the 8th-grade science test, no items showed DIF with MH and LR methods, while four items showed moderate DIF with SIBTEST. Overall, MH and SIBTEST techniques were in agreement, whereas LR method produced inconsistent results and showed disagreement with these two methods. The results of the measurement invariance analysis and the LR method were consistent and indicated equivalency of TIMSS and e-TIMSS scores.
KW - Computer-Based Assessments
KW - DIF
KW - Differential Item Functioning
KW - Multiple Group Confirmatory Factor Analysis
KW - Paper-Pencil Assessments
KW - TIMSS
KW - Educational science
UR - http://www.scopus.com/inward/record.url?scp=85200151611&partnerID=8YFLogxK
U2 - 10.21031/epod.1426739
DO - 10.21031/epod.1426739
M3 - Journal articles
AN - SCOPUS:85200151611
VL - 15
SP - 94
EP - 119
JO - Journal of Measurement and Evaluation in Education and Psychology
JF - Journal of Measurement and Evaluation in Education and Psychology
SN - 1309-6575
IS - 2
ER -