How to get really smart: Modeling retest and training effects in ability testing using computer-generated figural matrix items

Publikation: Beiträge in ZeitschriftenZeitschriftenaufsätzeForschungbegutachtet

Standard

How to get really smart: Modeling retest and training effects in ability testing using computer-generated figural matrix items. / Freund, Philipp Alexander; Holling, Heinz.
in: Intelligence, Jahrgang 39, Nr. 4, 07.2011, S. 233-243.

Publikation: Beiträge in ZeitschriftenZeitschriftenaufsätzeForschungbegutachtet

Harvard

APA

Vancouver

Bibtex

@article{3f131713a30c4416813a82a38b05ad28,
title = "How to get really smart: Modeling retest and training effects in ability testing using computer-generated figural matrix items",
abstract = "The interpretation of retest scores is problematic because they are potentially affected by measurement and predictive bias, which impact construct validity, and because their size differs as a function of various factors. This paper investigates the construct stability of scores on a figural matrices test and models retest effects at the level of the individual test taker as a function of covariates (simple retest vs. training, use of identical vs. parallel retest forms, and general mental ability). A total of N=189 subjects took two tests of matrix items that were automatically generated according to a strict construction rationale. Between test administrations, participants in the intervention groups received training, while controls did not. The Rasch model fit the data at both time points, but there was a lack of item difficulty parameter invariance across time. Training increased test performance beyond simple retesting, but there was no large difference between the identical and parallel retest forms at the individual level. Individuals varied greatly in how they profited from retest experience, training, and the use of identical vs. parallel retest forms. The results suggest that even with carefully designed tasks, it is problematic to directly compare scores from initial tests and retests. Test administrators should emphasize learning potential instead of state level assessment, and inter-individual differences with regard to test experience should be taken into account when interpreting test results.",
keywords = "Economics, empirical/statistics, Figural matrix items, Individual change, Rational item construction, Retest effects, Training effects",
author = "Freund, {Philipp Alexander} and Heinz Holling",
year = "2011",
month = jul,
doi = "10.1016/j.intell.2011.02.009",
language = "English",
volume = "39",
pages = "233--243",
journal = "Intelligence",
issn = "0160-2896",
publisher = "Elsevier Ltd",
number = "4",

}

RIS

TY - JOUR

T1 - How to get really smart: Modeling retest and training effects in ability testing using computer-generated figural matrix items

AU - Freund, Philipp Alexander

AU - Holling, Heinz

PY - 2011/7

Y1 - 2011/7

N2 - The interpretation of retest scores is problematic because they are potentially affected by measurement and predictive bias, which impact construct validity, and because their size differs as a function of various factors. This paper investigates the construct stability of scores on a figural matrices test and models retest effects at the level of the individual test taker as a function of covariates (simple retest vs. training, use of identical vs. parallel retest forms, and general mental ability). A total of N=189 subjects took two tests of matrix items that were automatically generated according to a strict construction rationale. Between test administrations, participants in the intervention groups received training, while controls did not. The Rasch model fit the data at both time points, but there was a lack of item difficulty parameter invariance across time. Training increased test performance beyond simple retesting, but there was no large difference between the identical and parallel retest forms at the individual level. Individuals varied greatly in how they profited from retest experience, training, and the use of identical vs. parallel retest forms. The results suggest that even with carefully designed tasks, it is problematic to directly compare scores from initial tests and retests. Test administrators should emphasize learning potential instead of state level assessment, and inter-individual differences with regard to test experience should be taken into account when interpreting test results.

AB - The interpretation of retest scores is problematic because they are potentially affected by measurement and predictive bias, which impact construct validity, and because their size differs as a function of various factors. This paper investigates the construct stability of scores on a figural matrices test and models retest effects at the level of the individual test taker as a function of covariates (simple retest vs. training, use of identical vs. parallel retest forms, and general mental ability). A total of N=189 subjects took two tests of matrix items that were automatically generated according to a strict construction rationale. Between test administrations, participants in the intervention groups received training, while controls did not. The Rasch model fit the data at both time points, but there was a lack of item difficulty parameter invariance across time. Training increased test performance beyond simple retesting, but there was no large difference between the identical and parallel retest forms at the individual level. Individuals varied greatly in how they profited from retest experience, training, and the use of identical vs. parallel retest forms. The results suggest that even with carefully designed tasks, it is problematic to directly compare scores from initial tests and retests. Test administrators should emphasize learning potential instead of state level assessment, and inter-individual differences with regard to test experience should be taken into account when interpreting test results.

KW - Economics, empirical/statistics

KW - Figural matrix items

KW - Individual change

KW - Rational item construction

KW - Retest effects

KW - Training effects

UR - http://www.scopus.com/inward/record.url?scp=79957661124&partnerID=8YFLogxK

U2 - 10.1016/j.intell.2011.02.009

DO - 10.1016/j.intell.2011.02.009

M3 - Journal articles

VL - 39

SP - 233

EP - 243

JO - Intelligence

JF - Intelligence

SN - 0160-2896

IS - 4

ER -

DOI

Zuletzt angesehen

Aktivitäten

  1. Negotiators Facing Externalities: An Experimental Investigation Into Negotiators’ Balancing of Multiple Conflicts of Interests
  2. Nordic Seminar on Computational Mechanics - NSCM 2016
  3. SPARQL Querying Benchmarks
  4. Thinking of Time - A Resource which Should be Allocated Equally
  5. Predicting negotiation success with a multitude of negotiators’ inter-individual differences—a latent personality model of the successful negotiator
  6. Vortrag: Assessing and Managing Sustainable Business Models - A Status Update
  7. Understanding Corruption by Means of Experiments
  8. The Ambiguity of In/Activity in John Knights “The Right to be Lazy”
  9. Contracting for Space - 2009
  10. Strange Signs: Invented Languages from Alienation to Zany
  11. Reading Strategy Programs and their Effects on Teachers’ Instruction and Students’ Comprehension in Secondary Schools
  12. Team Entrepreneurial Network(ing)
  13. Crazy, Classified City Life - Hackfeminist Future-Making Practices between Dystopia and Utopia, Predictability and Possibility
  14. Shifting Regimes of Proof: On the Contested Politics of Identification in Border and Migration Management
  15. Building the virtual stage. Insights from co-creating online theatre events
  16. Künstliche Intelligenz in der Hochschullehre
  17. Contrastivity and the pragmatic variable: Responses to thanks across English as a pluricentric language (Language, Nations, Cultures: Pluricentric Languages in Contest (s), Universität Stockholm)
  18. Teaching Game Studies at DiGRA 2014
  19. Mutual Knowledge Conference
  20. Towards more radical forms of governance for business and human rights: From management practice to management education
  21. International Conference on Road Safety and Simulation 2017
  22. CRISIS-DRIVEN TRANSFORMATION IN ORGANIZATIONS
  23. ‘Cultural Ecosystem’ as tool for researching alternative cultures – potentials, limits and the question of ‘another black box’.
  24. EVA 2004
  25. SOLWODI-Fachtagung 2020
  26. Eliane Gilg
  27. Contrastive Pragmatics: A Cross-Disciplinary Journal (Zeitschrift)
  28. From Compliance to Due Diligence: Opportunities and Tensions in the Development of New Sustainable Supply Chain Governance Regimes
  29. Tragic Differing, Discordant Times: Reiner Schürmann’s Ruins
  30. Towards the Design of Active Pharmaceutical ingredients (APIs) mineralizing readily in the Environment
  31. Reviewer Information and Organization
  32. Space and Organizing Practices: Researching Individual Entrepreneurs in the Creative Industries