Reliability and Validity of Assessing User Satisfaction With Web-Based Health Interventions

Research output: Journal contributionsJournal articlesResearchpeer-review

Authors

Background: The perspective of users should be taken into account in the evaluation of Web-based health interventions. Assessing the users' satisfaction with the intervention they receive could enhance the evidence for the intervention effects. Thus, there is a need for valid and reliable measures to assess satisfaction with Web-based health interventions. Objective: The objective of this study was to analyze the reliability, factorial structure, and construct validity of the Client Satisfaction Questionnaire adapted to Internet-based interventions (CSQ-I). Methods: The psychometric quality of the CSQ-I was analyzed in user samples from 2 separate randomized controlled trials evaluating Web-based health interventions, one from a depression prevention intervention (sample 1, N=174) and the other from a stress management intervention (sample 2, N=111). At first, the underlying measurement model of the CSQ-I was analyzed to determine the internal consistency. The factorial structure of the scale and the measurement invariance across groups were tested by multigroup confirmatory factor analyses. Additionally, the construct validity of the scale was examined by comparing satisfaction scores with the primary clinical outcome. Results: Multigroup confirmatory analyses on the scale yielded a one-factorial structure with a good fit (root-mean-square error of approximation =.09, comparative fit index =.96, standardized root-mean-square residual =.05) that showed partial strong invariance across the 2 samples. The scale showed very good reliability, indicated by McDonald omegas of .95 in sample 1 and .93 in sample 2. Significant correlations with change in depressive symptoms (r=-.35, P<.001) and perceived stress (r=-.48, P<.001) demonstrated the construct validity of the scale. Conclusions: The proven internal consistency, factorial structure, and construct validity of the CSQ-I indicate a good overall psychometric quality of the measure to assess the user's general satisfaction with Web-based interventions for depression and stress management. Multigroup analyses indicate its robustness across different samples. Thus, the CSQ-I seems to be a suitable measure to consider the user's perspective in the overall evaluation of Web-based health interventions.

Original languageEnglish
Article numbere234
JournalJournal of Medical Internet Research
Volume18
Issue number8
Number of pages13
ISSN1439-4456
DOIs
Publication statusPublished - 31.08.2016

    Research areas

  • Psychology - Internet, Mental health treatments, evaluation, clinical effectiveness, personal satisfaction

Documents

DOI

Recently viewed

Researchers

  1. Nele Meyer

Publications

  1. Comparison of modeling approaches based on the microstructure of thermally sprayed coatings
  2. Modeling Efficient Grounding in Chat-based CSCL
  3. Joint Proceedings of Scholarly QALD 2023 and SemREC 2023 co-located with 22nd International Semantic Web Conference ISWC 2023
  4. Traffic Life: Temporal Dynamics and Regulatory Dimensions in Agent-Based Transport Simulations
  5. The use of the online Inverted Classroom Model for digital teaching with gamification in medical studies
  6. Extraterritorial Asylum Processing. The Libya-Niger Emergency Transit Mechanism
  7. Non-native populations of an invasive tree outperform their native conspecifics
  8. Using work values to predict post-retirement work intentions
  9. "If you like something, you want it to develop."
  10. Investigating quality mathematics teaching - the DISUM project
  11. Predator assemblage structure and temporal variability of species richness and abundance in forests of high tree diversity
  12. y-Randomization and its variants in QSPR/QSAR
  13. Modeling Self-Organization
  14. The dependency of the banks’ assets and liabilities
  15. Payments for ecosystem services – for efficiency and for equity?
  16. Understanding role models for change
  17. Computer-Kriegs-Spiele oder: eine Kultur der Gewalt
  18. A new didactic approach in Engineering Education for conceptual understanding of Euler's Formula
  19. Numerical responses of saproxylic beetles to rapid increases in dead wood availability following geometrid moth outbreaks in sub-arctic mountain birch forest
  20. Credit Constraints and Exports
  21. Erosion modelling designed for water quality simulation
  22. Lernende in der Hauptschule
  23. Mentoring in International Assignments
  24. An idea and a person whose time had come
  25. Update wurde nicht ausgeführt
  26. Canopy functional trait variation across Earth’s tropical forests
  27. Learning from the richness of diversity