Modeling items for text comprehension assessment using confirmatory factor analysis

Publikation: Beiträge in ZeitschriftenZeitschriftenaufsätzeForschungbegutachtet

Authors

Reading is a complex cognitive task with the ultimate goal of comprehending the written input. For longer, connected text, readers generate a mental representation that serves as its basis. Due to limited cognitive resources, common models of discourse representation assume distinct processing levels, each relying on different processing mechanisms. However, only little research addresses distinct representational levels when text comprehension is assessed, analyzed or modelled. Moreover, current studies that tried to relate process measures of reading (e.g., reading times, eye movements) to comprehension did not consider comprehension as a multi-faceted, but rather a uni-dimensional construct, usually assessed with one-shot items. Thus, the first aim of this paper is to use confirmatory factor analysis (CFA) to test whether comprehension can be modelled as a uni-or multi-dimensional concept. The second aim is to investigate how well widely used one-shot items can be used to capture comprehension. 400 participants read one of three short stories of comparable length, linguistic characteristics, and complexity. Based on the evaluation of three independent raters per story, 16 wh-questions and 60 yes/no-statements were compiled in order to retrieve information at micro and inference level, and 16 main contents were extracted to capture information at the macro level in participants’ summaries. Still, only a fraction of these items showed satisfactory psychometric properties and factor loadings – a blatant result considering the common practice for item selection. For CFA, two models were set up that address text comprehension as either a one-dimensional construct (a uni-factor model with a single comprehension factor), or a three-dimensional construct reflecting the three distinct representational levels (three correlated first-order factors). Across stories and item types, model fit was consistently better for the three-factor model providing evidence for a multi-dimensional construct of text comprehension. Our results provide concrete guidance for the preparation of comprehension measurements in studies investigating the reading process.

OriginalspracheEnglisch
Aufsatznummer966347
ZeitschriftFrontiers in Psychology
Jahrgang13
Anzahl der Seiten10
ISSN1664-1078
DOIs
PublikationsstatusErschienen - 20.10.2022

Bibliographische Notiz

Funding Information:
The study was funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) by grants to SW (project numbers 397523278 and 442405852).

Publisher Copyright:
Copyright © 2022 Tschense and Wallot.

Dokumente

DOI

Zuletzt angesehen

Publikationen

  1. Digging into the roots
  2. Gain Adaptation in Sliding Mode Control Using Model Predictive Control and Disturbance Compensation with Application to Actuators
  3. Challenge-oriented policy making and innovation systems theory: reconsidering systemic instruments
  4. Mostly harmless econometrics? Statistical paradigms in the ‘top five’ from 2000 to 2018
  5. Faulty Process Detection Using Machine Learning Techniques
  6. Development and evaluation of a training program for dialysis nurses - An intervention study
  7. Extending talk on a prescribed discussion topic in a learner-native speaker eTandem learning task
  8. Quantifying diffuse and point inputs of perfluoroalkyl acids in a nonindustrial river catchment
  9. A Multimethod Latent State-Trait Model for Structurally Different and Interchangeable Methods
  10. Enhanced Calculation Procedures for Material and Energy Flow Oriented EMIS
  11. Guest Editors' Introduction
  12. Towards an open question answering architecture
  13. Knowledge transfer during the integration of knowledge-intensive acquisitions
  14. Development and application of a simplified sampling method for volatile polyfluorinated alkyl substances in indoor and environmental air
  15. Earnings Less Risk-Free Interest Charge (ERIC) and Stock Returns—A Value-Based Management Perspective on ERIC’s Relative and Incremental Information Content
  16. Introduction to Automatic Imitation
  17. SoilTemp: A global database of near-surface temperature
  18. Science-Related Outcomes
  19. The complexity of integrated flood management
  20. Navigating (In)Visibility
  21. An empirically grounded ontology for analyzing IT-based interventions in business ecosystems
  22. Influence of Mg content in Al alloys on processing characteristics and dynamically recrystallized microstructure of friction surfacing deposits
  23. The development of an eco-label for software products
  24. Teaching Sustainable Development in a Sensory and Artful Way — Concepts, Methods, and Examples
  25. The Weird and the Eerie
  26. Insights into creep behavior of Mg–14Gd–1Zn–0.4Zr (wt.%) alloy containing β- and γ-type precipitates
  27. On walks in molecular graphs.
  28. Linking concepts of change and ecosystem services research: A systematic review
  29. Concurrently Observed Actions Are Represented Not as Compound Actions but as Independent Actions
  30. Microstructure and mechanical properties of as-cast Mg-Sn-Ca alloys and effect of alloying elements
  31. Utilization of protein-rich residues in biotechnological processes