How to get really smart: Modeling retest and training effects in ability testing using computer-generated figural matrix items

Research output: Journal contributionsJournal articlesResearchpeer-review

Authors

The interpretation of retest scores is problematic because they are potentially affected by measurement and predictive bias, which impact construct validity, and because their size differs as a function of various factors. This paper investigates the construct stability of scores on a figural matrices test and models retest effects at the level of the individual test taker as a function of covariates (simple retest vs. training, use of identical vs. parallel retest forms, and general mental ability). A total of N=189 subjects took two tests of matrix items that were automatically generated according to a strict construction rationale. Between test administrations, participants in the intervention groups received training, while controls did not. The Rasch model fit the data at both time points, but there was a lack of item difficulty parameter invariance across time. Training increased test performance beyond simple retesting, but there was no large difference between the identical and parallel retest forms at the individual level. Individuals varied greatly in how they profited from retest experience, training, and the use of identical vs. parallel retest forms. The results suggest that even with carefully designed tasks, it is problematic to directly compare scores from initial tests and retests. Test administrators should emphasize learning potential instead of state level assessment, and inter-individual differences with regard to test experience should be taken into account when interpreting test results.
Original languageEnglish
JournalIntelligence
Volume39
Issue number4
Pages (from-to)233-243
Number of pages11
ISSN0160-2896
DOIs
Publication statusPublished - 07.2011

Recently viewed

Activities

  1. Building Collective Institutional Infrastructures for Decent Platform Work: The Development of a Crowdwork Agreement in Germany
  2. Expansive Museality: Camila Caris Seguel in Conversation with Sebastián Eduardo Dávila on Museo del Mundo and the Garífuna Nation
  3. „Einführung“ in dem Interdisziplinärer Workshop „Ästhetische und Künstlerische Praktiken als Reorganisation von Wahrnehmungsweisen“
  4. Sustainability-oriented targets in executive compensation – Symbolic measures or significant catalyst for a sustainable transition?
  5. Assessing Key Competencies In Higher Education For Sustainable Devlopment: Insights From the Deployment Of Innovative Instruments
  6. Medienreflexion – Förderung von Medienkompetenz in der Lehrkräftebildung beim Einsatz multiperspektivischer Unterrichtsaufnahmen
  7. Sustainability-oriented targets in executive compensation – Symbolic measures or significant catalyst for a sustainable transition?
  8. Research Seminar at the Australian Centre for Entrepreneurship Research Exchange, Queensland University of Technology - ACERE 2013
  9. Ungleiche Partizipationschancen in heterogenen Lerngruppen. Eine ethngrafische Studie in Klassenräten inklusiver Grundschulklassen
  10. Power poles and tall trees adjacent to sunflower fields increase pest damage caused by the invasive Rose-ringed Parakeet in Israel
  11. Shortening the potential duration of unemployment benefits and labor market outcomes: Evidence from a natural experiment in Germany
  12. “You wanna be on top?” – (Critical) Findings of a Transcultural Television Analysis of America's Next Top Model and Germany's next Topmodel
  13. From Where I Stand meets Pact of Silence – How to break it? Institutional critique & instituting otherwise from a feminist perspective
  14. Gemeinsame Stellungnahme zum Entwurf des Runderlasses „Schulische Förderung von Mehrsprachigkeit und Interkulturalität“, i. d. F. 22.11.23
  15. Umwelt- und klimaverträgliche Entwicklung des Tourismus. Keynote beim Zukunftsdialog zur Entwicklung der nationalen Tourismusstrategie.
  16. Why Do School Teachers Engage with Sustainable Consumption? Exploring the Role of Personal, Group and Organizational Influence Factors
  17. Wirksamkeit eines Online Regenerationstrainings für hoch beanspruchte Berufstätige. Ergebnisse einer randomisiert-kontrollierten Studie.

Publications

  1. Technische, betriebswirtschaftliche und rechtlichte Analyse des Einsatzes regionaler virtueller Kraftwerke in der Region Nord
  2. Schreiben im Beruf und an berufsbildenden Schulen für sprachlichheterogene Zielgruppen – Anforderungen, Tendenzen und Ausblicke
  3. Towards the design of organosilicon compounds for environmental degradation by using structure biodegradability relationships
  4. Identification of ozonation by-products of 4- and 5-methyl-1H-benzotriazole during the treatment of surface water to drinking water
  5. Timing matters: Distinct effects of nitrogen and phosphorus fertilizer application timing on root system architecture responses
  6. REVIEW: Dangerous Mediations: Pop Music in a Philippine Prison Video, Áine Mangaoang, New York, NY: Bloomsbury, 2019, ISBN 9781501331534.
  7. Barley shoot biomass responds strongly to N:P stoichiometry and intraspecific competition, whereas roots only alter their foraging
  8. Using measures of reading time regularity (RTR) to quantify eye movement dynamics, and how they are shaped by linguistic information
  9. Cypionka, Anselm; Ehrentraut-Daut, Hans Peter; Deichmann, Carl; Müller, Eva (Hrsg.): Perspektiven Politischer Kultur in Thüringen. Erfurt, 2007
  10. Payment offers, suggestions to share expenses and payment negotiation sequences on initial dates in Germany and the United Kingdom
  11. Effects of daily static stretch training over 6 weeks on maximal strength, muscle thickness, contraction properties, and flexibility
  12. Colonization and extinction lags drive non-linear responses to warming in mountain plant communities across the Northern Hemisphere
  13. »keep the contemporaryartmuseum groovy – keep the home fires burning«. Zum Verhältnis von feministischer Kritik und Kunstinstitutionen
  14. E-Learning in PE-Teacher Education - Blended learning as a promising alternative to conventional academic teaching for sport pedagogy?
  15. A transdisciplinary evaluation framework for the assessment of integration in boundary-crossing collaborations in teacher education
  16. Daily breath-based mindfulness exercises in a randomized controlled trial improve primary school children’s performance in arithmetic
  17. Virtuelle Realitäten zur Kompetenzentwicklung und Reflexion in der universitären Lehrkräftebildung am Beispiel von Elterngesprächen
  18. Vereinbarungen zur Chancengleichheit von Frauen und Männern - Verbreitung und Determinanten in Niedersachsen, West- und Ostdeutschland
  19. Technology management and collaboration profile: virtual companies and industrial platforms in the high-tech biotechnology industries