How to get really smart: Modeling retest and training effects in ability testing using computer-generated figural matrix items

Research output: Journal contributionsJournal articlesResearchpeer-review

Authors

The interpretation of retest scores is problematic because they are potentially affected by measurement and predictive bias, which impact construct validity, and because their size differs as a function of various factors. This paper investigates the construct stability of scores on a figural matrices test and models retest effects at the level of the individual test taker as a function of covariates (simple retest vs. training, use of identical vs. parallel retest forms, and general mental ability). A total of N=189 subjects took two tests of matrix items that were automatically generated according to a strict construction rationale. Between test administrations, participants in the intervention groups received training, while controls did not. The Rasch model fit the data at both time points, but there was a lack of item difficulty parameter invariance across time. Training increased test performance beyond simple retesting, but there was no large difference between the identical and parallel retest forms at the individual level. Individuals varied greatly in how they profited from retest experience, training, and the use of identical vs. parallel retest forms. The results suggest that even with carefully designed tasks, it is problematic to directly compare scores from initial tests and retests. Test administrators should emphasize learning potential instead of state level assessment, and inter-individual differences with regard to test experience should be taken into account when interpreting test results.
Original languageEnglish
JournalIntelligence
Volume39
Issue number4
Pages (from-to)233-243
Number of pages11
ISSN0160-2896
DOIs
Publication statusPublished - 07.2011

Recently viewed

Publications

  1. Fostering Circularity: Building a Local Community and Implementing Circular Processes
  2. Heuristic approximation and computational algorithms for closed networks
  3. How to get really smart: Modeling retest and training effects in ability testing using computer-generated figural matrix items
  4. Managing Business Process in Distributed Systems: Requirements, Models, and Implementation
  5. Learning Rotation Sensitive Neural Network for Deformed Objects' Detection in Fisheye Images
  6. Analyzing different types of moderated method effects in confirmatory factor models for structurally different methods
  7. On the Power and Performance of a Doubly Latent Residual Approach to Explain Latent Specific Factors in Multilevel-Bifactor-(S-1) Models
  8. Building Assistance Systems using Distributed Knowledge Representations
  9. NH4+ ad-/desorption in sequencing batch reactors
  10. Multidimensional Cross-Recurrence Quantification Analysis (MdCRQA)–A Method for Quantifying Correlation between Multivariate Time-Series
  11. An extended analytical approach to evaluating monotonic functions of fuzzy numbers
  12. Analysis and Implementation of a Resistance Temperature Estimator Based on Bi-Polynomial Least Squares Method and Discrete Kalman Filter
  13. Q-Adaptive Control of the nonlinear dynamics of the cantilever-sample system of an Atomic Force Microscope
  14. A Review of Latent Variable Modeling Using R - A Step-by-Step-Guide
  15. Digging into the roots
  16. A Study on the Performance of Adaptive Neural Networks for Haze Reduction with a Focus on Precision
  17. Restoring Causal Analysis to Structural Equation ModelingReview of Causality: Models, Reasoning, and Inference (2nd Edition), by Judea Pearl
  18. Gain Adaptation in Sliding Mode Control Using Model Predictive Control and Disturbance Compensation with Application to Actuators
  19. Experiences of the Self between Limit, Transgression, and the Explosion of the Dialectical System
  20. Optimized neural networks for modeling of loudspeaker directivity diagrams
  21. Primary Side Circuit Design of a Multi-coil Inductive System for Powering Wireless Sensors
  22. Experiments on the Fehrer-Raab effect and the ‘Weather Station Model’ of visual backward masking
  23. Need Satisfaction and Optimal Functioning at Leisure and Work: A Longitudinal Validation Study of the DRAMMA Model
  24. Taking the pulse of Earth's tropical forests using networks of highly distributed plots
  25. Automatic three-dimensional geometry and mesh generation of periodic representative volume elements for matrix-inclusion composites
  26. Rapid grain refinement and compositional homogenization in a cast binary Cu50Ni alloy achieved by friction stir processing
  27. Managing complexity in automative production