1. //
  2. Start
  3. //
  4. Publications
  5. The eTIMSS and TIMSS Measurement Invaria...
  6. Output formats
  7. //

The eTIMSS and TIMSS Measurement Invariance Study: Multigroup Factor Analyses and Differential Item Functioning Analyses with the 2019 Cycle

Research output: Journal contributions › Journal articles › Research › peer-review

Standard

The eTIMSS and TIMSS Measurement Invariance Study: Multigroup Factor Analyses and Differential Item Functioning Analyses with the 2019 Cycle. / Yalçınkaya, Murat; Atılgan, Hakan; Daşcıoğlu, Selim et al.
In: Journal of Measurement and Evaluation in Education and Psychology, Vol. 15, No. 2, 30.06.2024, p. 94-119.

Research output: Journal contributions › Journal articles › Research › peer-review

Harvard

Yalçınkaya, M, Atılgan, H, Daşcıoğlu, S & Aydın, B 2024, 'The eTIMSS and TIMSS Measurement Invariance Study: Multigroup Factor Analyses and Differential Item Functioning Analyses with the 2019 Cycle', Journal of Measurement and Evaluation in Education and Psychology, vol. 15, no. 2, pp. 94-119. https://doi.org/10.21031/epod.1426739

APA

Yalçınkaya, M., Atılgan, H., Daşcıoğlu, S., & Aydın, B. (2024). The eTIMSS and TIMSS Measurement Invariance Study: Multigroup Factor Analyses and Differential Item Functioning Analyses with the 2019 Cycle. Journal of Measurement and Evaluation in Education and Psychology, 15(2), 94-119. https://doi.org/10.21031/epod.1426739

Vancouver

Yalçınkaya M, Atılgan H, Daşcıoğlu S, Aydın B. The eTIMSS and TIMSS Measurement Invariance Study: Multigroup Factor Analyses and Differential Item Functioning Analyses with the 2019 Cycle. Journal of Measurement and Evaluation in Education and Psychology. 2024 Jun 30;15(2):94-119. doi: 10.21031/epod.1426739

Bibtex

@article{ba829d893ff1448abdb940bb2c07b235,
title = "The eTIMSS and TIMSS Measurement Invariance Study: Multigroup Factor Analyses and Differential Item Functioning Analyses with the 2019 Cycle",
abstract = "In this study, measurement invariance and differential item functioning (DIF) studies of the TIMSS 2019 4th and 8th-grade mathematics and science achievement tests were conducted for the country groups participating in both TIMSS and eTIMSS. The study sample consisted of 9560 responders of the first booklet of the 2019 cycle. Multiple Group Confirmatory Factor Analysis (MGCFA) was utilized to test measurement invariance, and Mantel-Haenszel (MH), Logistic Regression (LR), and SIBTEST were used for the DIF analyses. The measurement invariance results indicated strict invariance between groups for all tests which included 111 items in total. In the DIF analyses, for the 4th and 8th-grade mathematics tests, only three items showed moderate DIF with MH, and four items showed DIF with SIBTEST. For the 4th-grade science test, one item showed moderate DIF with both MH and SIBTEST. However, in the 8th-grade science test, no items showed DIF with MH and LR methods, while four items showed moderate DIF with SIBTEST. Overall, MH and SIBTEST techniques were in agreement, whereas LR method produced inconsistent results and showed disagreement with these two methods. The results of the measurement invariance analysis and the LR method were consistent and indicated equivalency of TIMSS and e-TIMSS scores.",
keywords = "Computer-Based Assessments, DIF, Differential Item Functioning, Multiple Group Confirmatory Factor Analysis, Paper-Pencil Assessments, TIMSS, Educational science",
author = "Murat Yal{\c c}ınkaya and Hakan Atılgan and Selim Da{\c s}cıoğlu and Burak Aydın",
note = "Publisher Copyright: {\textcopyright} 2024 Association of Measurement and Evaluation in Education and Psychology (EPODDER). All rights reserved.",
year = "2024",
month = jun,
day = "30",
doi = "10.21031/epod.1426739",
language = "English",
volume = "15",
pages = "94--119",
journal = "Journal of Measurement and Evaluation in Education and Psychology",
issn = "1309-6575",
publisher = "Association of Measurement and Evaluation in Education and Psychology (EPODDER)",
number = "2",

}

RIS

TY - JOUR

T1 - The eTIMSS and TIMSS Measurement Invariance Study

T2 - Multigroup Factor Analyses and Differential Item Functioning Analyses with the 2019 Cycle

AU - Yalçınkaya, Murat

AU - Atılgan, Hakan

AU - Daşcıoğlu, Selim

AU - Aydın, Burak

N1 - Publisher Copyright: © 2024 Association of Measurement and Evaluation in Education and Psychology (EPODDER). All rights reserved.

PY - 2024/6/30

Y1 - 2024/6/30

N2 - In this study, measurement invariance and differential item functioning (DIF) studies of the TIMSS 2019 4th and 8th-grade mathematics and science achievement tests were conducted for the country groups participating in both TIMSS and eTIMSS. The study sample consisted of 9560 responders of the first booklet of the 2019 cycle. Multiple Group Confirmatory Factor Analysis (MGCFA) was utilized to test measurement invariance, and Mantel-Haenszel (MH), Logistic Regression (LR), and SIBTEST were used for the DIF analyses. The measurement invariance results indicated strict invariance between groups for all tests which included 111 items in total. In the DIF analyses, for the 4th and 8th-grade mathematics tests, only three items showed moderate DIF with MH, and four items showed DIF with SIBTEST. For the 4th-grade science test, one item showed moderate DIF with both MH and SIBTEST. However, in the 8th-grade science test, no items showed DIF with MH and LR methods, while four items showed moderate DIF with SIBTEST. Overall, MH and SIBTEST techniques were in agreement, whereas LR method produced inconsistent results and showed disagreement with these two methods. The results of the measurement invariance analysis and the LR method were consistent and indicated equivalency of TIMSS and e-TIMSS scores.

AB - In this study, measurement invariance and differential item functioning (DIF) studies of the TIMSS 2019 4th and 8th-grade mathematics and science achievement tests were conducted for the country groups participating in both TIMSS and eTIMSS. The study sample consisted of 9560 responders of the first booklet of the 2019 cycle. Multiple Group Confirmatory Factor Analysis (MGCFA) was utilized to test measurement invariance, and Mantel-Haenszel (MH), Logistic Regression (LR), and SIBTEST were used for the DIF analyses. The measurement invariance results indicated strict invariance between groups for all tests which included 111 items in total. In the DIF analyses, for the 4th and 8th-grade mathematics tests, only three items showed moderate DIF with MH, and four items showed DIF with SIBTEST. For the 4th-grade science test, one item showed moderate DIF with both MH and SIBTEST. However, in the 8th-grade science test, no items showed DIF with MH and LR methods, while four items showed moderate DIF with SIBTEST. Overall, MH and SIBTEST techniques were in agreement, whereas LR method produced inconsistent results and showed disagreement with these two methods. The results of the measurement invariance analysis and the LR method were consistent and indicated equivalency of TIMSS and e-TIMSS scores.

KW - Computer-Based Assessments

KW - DIF

KW - Differential Item Functioning

KW - Multiple Group Confirmatory Factor Analysis

KW - Paper-Pencil Assessments

KW - TIMSS

KW - Educational science

UR - http://www.scopus.com/inward/record.url?scp=85200151611&partnerID=8YFLogxK

U2 - 10.21031/epod.1426739

DO - 10.21031/epod.1426739

M3 - Journal articles

AN - SCOPUS:85200151611

VL - 15

SP - 94

EP - 119

JO - Journal of Measurement and Evaluation in Education and Psychology

JF - Journal of Measurement and Evaluation in Education and Psychology

SN - 1309-6575

IS - 2

ER -

Other publications by the same author(s)

Automated scoring in the era of artificial intelligence: An empirical study with Turkish essays

Aydın, B., Kışla, T., Elmas, N. T. & Bulut, O., 10.2025, In: System. 133, 12 p., 103784.

Research output: Journal contributions › Journal articles › Research › peer-review

ICT knowledge absorptive capacity: A critical factor for technology integration in schools

Fischer‐Schöneborn, S., Brown, C., Aydin, B., MacGregor, S. & Pietsch, M., 13.06.2025, (E-pub ahead of print) In: British Journal of Educational Technology. 25 p.

Research output: Journal contributions › Journal articles › Research › peer-review

Investigating the situational impact of academic language demands on university students’ boredom with an instructional video

Wirth, L., Aydin, B., Ehmke, T., Retelsdorf, J. & Kuhl, P., 10.03.2025, In: European Journal of Psychology of Education. 40, 1, 22 p., 50.

Research output: Journal contributions › Journal articles › Research › peer-review

Not invented here, not shared here: How school leaders’ attitudes towards external knowledge affect collaborative innovation and collective teacher innovativeness in Germany

Witthöft, J., Aydin, B. & Pietsch, M., 04.06.2025, (E-pub ahead of print) In: Educational Management Administration and Leadership. 27 p., 17411432251346950.

Research output: Journal contributions › Journal articles › Research › peer-review

Examining long-term impacts of a training programme to improve quality of IEP goals

Rakap, S., Balikci, S., Kalkan, S., Coleman, H., Aydin, B. & Gulboy, E., 2025, In: European Journal of Special Needs Education. 40, 1, p. 35-52 18 p.

Research output: Journal contributions › Journal articles › Research › peer-review

DOI

Recently viewed

Activities

  1. Can a University-wide Freshmen’s Kick-off Week about Starting a Business Enhance Students’ Entrepreneurial Self-efficacy and Attitude Towards Venturing? – A Pilot Study
  2. Ringvorlesung: Die Musik im Zeitalter ihrer technischen (Re)Produzierbarkeit 2012
  3. Vortrag: Weiter auf der Suche nach der richtigen Methode?: Ergebnisse einer Langzeitstudie über Effekte eines Unterrichts mit "Fara und Fu", "Tobi", "Lesen durch Schreiben" und der "Silbenanalytische Methode"
  4. 26. Kongress der Deutschen Gesellschaft für Fremdsprachenforschung - 2015
  5. Leuphana Universität Lüneburg (Organisation)
  6. Arizona State Universität
  7. European Cooperation in Science and Technology Association (Externe Organisation)
  8. Evaluation kultureller Institutionen
  9. Especialización „Educación para el desarrollo sustentable: Criterios de calidad para una educación para el desarrollo sustentable
  10. (Hoch)Schulen als nachhaltige Konsumkulturen: Transdisziplinäre Innovationsforschung am Beispiel des Projektes BINK
  11. Integration through Trust. How Refugee Families Build Trust in Early Childhood Education and Care in Germany when Interacting with Volunteers
  12. Vortrag: Lernen mit eigenen und fremden Unterrichtsvideos in der Lehrerfortbildung.

Publications

  1. Waterfronts im Wandel
  2. Defining the goal and scope for life cycle sustainability assessment
  3. When Catholic and Polish Identity Goals Meet
  4. Der/Die schlaflose Cyborg
  5. Wege ins Studium für nicht-traditionelle Studierende
  6. Foucault e Kant
  7. Ipgrave, Michael, Trinity and Inter Faith Dialogue.Plenitude and Plurality, Oxford u.a. 2003
  8. Minderheitensprachen im deutschen Schulwesen - Eine Analyse der Implementierung allochthoner und autochthoner Sprachen
  9. Konfliktkulturen, Konfliktdefinitionskonflikte und Mediation
  10. Auf dem Weg zu einer kritischen Theorie des Unternehmens
  11. What's legit?
  12. Education for Sustainable Development
  13. Integrierte Sprachbildung im Fach Kunst
  14. Exports, R&D and Productivity: A test of the Bustos-model with German enterprise data
  15. Arbeit am Bild
  16. Epilog - Und wo sind wir?
  17. Kunst im Nationalsozialismus
  18. Aquatic habitat use by amphibians with specific reference to Rana temporaria at high elevations (Retezat Mountains National Park, Romania)
  19. Inkorporierte Normen in der Sprache der Erziehung
  20. Zur UN-Dekade an Hochschulen: Es geht voran, aber nur langsam!
  21. International Workshop (2019): "The LOSC 25 Years After its Entry into Force: Between Comprehensiveness and Exceptionalism”
  22. Internetbasierte Nachhaltigkeitskommunikation
  23. Recent research and developments on wrought magnesium alloys
  24. Emotional states of drivers and the impact on driving behaviour - a simulator study
  25. In situ synchrotron radiation diffraction during solidification of Mg4Y and Mg4YxGd alloys (x - 1, 4 wt.%)
  26. Ecosystem service coproduction across the zones of biosphere reserves in Europe