Modeling items for text comprehension assessment using confirmatory factor analysis

Research output: Journal contributionsJournal articlesResearchpeer-review

Authors

Reading is a complex cognitive task with the ultimate goal of comprehending the written input. For longer, connected text, readers generate a mental representation that serves as its basis. Due to limited cognitive resources, common models of discourse representation assume distinct processing levels, each relying on different processing mechanisms. However, only little research addresses distinct representational levels when text comprehension is assessed, analyzed or modelled. Moreover, current studies that tried to relate process measures of reading (e.g., reading times, eye movements) to comprehension did not consider comprehension as a multi-faceted, but rather a uni-dimensional construct, usually assessed with one-shot items. Thus, the first aim of this paper is to use confirmatory factor analysis (CFA) to test whether comprehension can be modelled as a uni-or multi-dimensional concept. The second aim is to investigate how well widely used one-shot items can be used to capture comprehension. 400 participants read one of three short stories of comparable length, linguistic characteristics, and complexity. Based on the evaluation of three independent raters per story, 16 wh-questions and 60 yes/no-statements were compiled in order to retrieve information at micro and inference level, and 16 main contents were extracted to capture information at the macro level in participants’ summaries. Still, only a fraction of these items showed satisfactory psychometric properties and factor loadings – a blatant result considering the common practice for item selection. For CFA, two models were set up that address text comprehension as either a one-dimensional construct (a uni-factor model with a single comprehension factor), or a three-dimensional construct reflecting the three distinct representational levels (three correlated first-order factors). Across stories and item types, model fit was consistently better for the three-factor model providing evidence for a multi-dimensional construct of text comprehension. Our results provide concrete guidance for the preparation of comprehension measurements in studies investigating the reading process.

Original languageEnglish
Article number966347
JournalFrontiers in Psychology
Volume13
Number of pages10
ISSN1664-1078
DOIs
Publication statusPublished - 20.10.2022

Bibliographical note

Funding Information:
The study was funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) by grants to SW (project numbers 397523278 and 442405852).

Publisher Copyright:
Copyright © 2022 Tschense and Wallot.

    Research areas

  • comprehension assessment, discourse representation, mental model, reading, reading comprehension, text comprehension
  • Psychology

Documents

DOI

Recently viewed

Publications

  1. Effectiveness of a guided multicomponent internet and mobile gratitude training program - A pragmatic randomized controlled trial
  2. A statistical study of the spatial evolution of shock acceleration efficiency for 5 MeV protons and subsequent particle propagation
  3. Four Methods to Distinguish between Fractal Dimensions in Time Series through Recurrence Quantification Analysis
  4. Comparing the performance of computational estimation methods for physicochemical properties of dimethylsiloxanes and selected siloxanols
  5. Tree diversity increases forest temperature buffering via enhancing canopy density and structural diversity
  6. Stepwise-based optimizing approaches for arrangements of loudspeaker in multi-zone sound field reproduction
  7. A Review of the Application of Machine Learning and Data Mining Approaches in Continuum Materials Mechanics
  8. Formative Perspectives on the Relation Between CSR Communication and CSR Practices
  9. Sensor Fusion for Power Line Sensitive Monitoring and Load State Estimation
  10. Experimentally established correlation of friction surfacing process temperature and deposit geometry
  11. Changes in the Complexity of Limb Movements during the First Year of Life across Different Tasks
  12. Neural network-based estimation and compensation of friction for enhanced deep drawing process control
  13. Does thinking-aloud affect learning, visual information processing and cognitive load when learning with seductive details as expected from self-regulation perspective?
  14. Privatizing the commons
  15. Tree diversity and mycorrhizal type co-determine multitrophic ecosystem functions
  16. The temporal and spatial development of MeV proton acceleration at interplanetary shocks
  17. Exploring the limits of graph invariant- and spectrum-based discrimination of (sub)structures.
  18. Effects of diversity versus segregation on automatic approach and avoidance behavior towards own and other ethnic groups