Modeling items for text comprehension assessment using confirmatory factor analysis

Publikation: Beiträge in ZeitschriftenZeitschriftenaufsätzeForschungbegutachtet

Authors

Reading is a complex cognitive task with the ultimate goal of comprehending the written input. For longer, connected text, readers generate a mental representation that serves as its basis. Due to limited cognitive resources, common models of discourse representation assume distinct processing levels, each relying on different processing mechanisms. However, only little research addresses distinct representational levels when text comprehension is assessed, analyzed or modelled. Moreover, current studies that tried to relate process measures of reading (e.g., reading times, eye movements) to comprehension did not consider comprehension as a multi-faceted, but rather a uni-dimensional construct, usually assessed with one-shot items. Thus, the first aim of this paper is to use confirmatory factor analysis (CFA) to test whether comprehension can be modelled as a uni-or multi-dimensional concept. The second aim is to investigate how well widely used one-shot items can be used to capture comprehension. 400 participants read one of three short stories of comparable length, linguistic characteristics, and complexity. Based on the evaluation of three independent raters per story, 16 wh-questions and 60 yes/no-statements were compiled in order to retrieve information at micro and inference level, and 16 main contents were extracted to capture information at the macro level in participants’ summaries. Still, only a fraction of these items showed satisfactory psychometric properties and factor loadings – a blatant result considering the common practice for item selection. For CFA, two models were set up that address text comprehension as either a one-dimensional construct (a uni-factor model with a single comprehension factor), or a three-dimensional construct reflecting the three distinct representational levels (three correlated first-order factors). Across stories and item types, model fit was consistently better for the three-factor model providing evidence for a multi-dimensional construct of text comprehension. Our results provide concrete guidance for the preparation of comprehension measurements in studies investigating the reading process.

OriginalspracheEnglisch
Aufsatznummer966347
ZeitschriftFrontiers in Psychology
Jahrgang13
Anzahl der Seiten10
ISSN1664-1078
DOIs
PublikationsstatusErschienen - 20.10.2022

Bibliographische Notiz

Funding Information:
The study was funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) by grants to SW (project numbers 397523278 and 442405852).

Publisher Copyright:
Copyright © 2022 Tschense and Wallot.

Dokumente

DOI

Zuletzt angesehen

Publikationen

  1. Alternating between Partial and Complete Organization
  2. Robust feedback linearization using an adaptive PD regulator for a sensorless control of a throttle valve
  3. Practice and carryover effects when using small interaction devices
  4. Applied quality assurance methods under the open source development model
  5. Failure to Learn From Failure Is Mitigated by Loss-Framing and Corrective Feedback
  6. Designing a Thrifty Approach for SME Business Continuity: Practices for Transparency of the Design Process
  7. Visualizing the Hidden Activity of Artificial Neural Networks
  8. Grazing, exploring and networking for sustainability-oriented innovations in learning-action networks
  9. Visualization of the Plasma Frequency by means of a Particle Simulation using a Normalized Periodic Model
  10. Learning how to request using textbooks
  11. From Open Access to Open Science
  12. Technical concept and evaluation design of the state subsidized project [Level-Q]
  13. Combining multiple investigative approaches to unravel functional responses to global change in the understorey of temperate forests
  14. Situated multiplying in primary school
  15. A Trajectory Generation Algorithm for Optimal Consumption in Electromagnetic Actuators
  16. Conceptual understanding of complex components and Nyquist-Shannon sampling theorem
  17. A tutorial introduction to adaptive fractal analysis
  18. Early Detection of Faillure in Conveyor Chain Systems by Wireless Sensor Node
  19. Managing complexity in automative production
  20. What Makes for a Good Theory? How to Evaluate a Theory Using the Strength Model of Self-Control as an Example
  21. Executive function and Language Learning
  22. Proxies
  23. An Adaptive and Optimized Switching Observer for Sensorless Control of an Electromagnetic Valve Actuator in Camless Internal Combustion Engines
  24. Parametric finite element model and mechanical characterisation of electrospun materials for biomedical applications
  25. Web-scale extension of RDF knowledge bases from templated websites
  26. Kalman Filter for Predictive Maintenance and Anomaly Detection
  27. Scale-dependent diversity patterns affect spider assemblages of two contrasting forest ecosystems
  28. The Creation of the Concept through the Interaction of Philosophy with Science and Art
  29. Understanding and Supporting Management Decision-Making