Reliability and Validity of Assessing User Satisfaction With Web-Based Health Interventions

Research output: Journal contributionsJournal articlesResearchpeer-review

Authors

Background: The perspective of users should be taken into account in the evaluation of Web-based health interventions. Assessing the users' satisfaction with the intervention they receive could enhance the evidence for the intervention effects. Thus, there is a need for valid and reliable measures to assess satisfaction with Web-based health interventions. Objective: The objective of this study was to analyze the reliability, factorial structure, and construct validity of the Client Satisfaction Questionnaire adapted to Internet-based interventions (CSQ-I). Methods: The psychometric quality of the CSQ-I was analyzed in user samples from 2 separate randomized controlled trials evaluating Web-based health interventions, one from a depression prevention intervention (sample 1, N=174) and the other from a stress management intervention (sample 2, N=111). At first, the underlying measurement model of the CSQ-I was analyzed to determine the internal consistency. The factorial structure of the scale and the measurement invariance across groups were tested by multigroup confirmatory factor analyses. Additionally, the construct validity of the scale was examined by comparing satisfaction scores with the primary clinical outcome. Results: Multigroup confirmatory analyses on the scale yielded a one-factorial structure with a good fit (root-mean-square error of approximation =.09, comparative fit index =.96, standardized root-mean-square residual =.05) that showed partial strong invariance across the 2 samples. The scale showed very good reliability, indicated by McDonald omegas of .95 in sample 1 and .93 in sample 2. Significant correlations with change in depressive symptoms (r=-.35, P<.001) and perceived stress (r=-.48, P<.001) demonstrated the construct validity of the scale. Conclusions: The proven internal consistency, factorial structure, and construct validity of the CSQ-I indicate a good overall psychometric quality of the measure to assess the user's general satisfaction with Web-based interventions for depression and stress management. Multigroup analyses indicate its robustness across different samples. Thus, the CSQ-I seems to be a suitable measure to consider the user's perspective in the overall evaluation of Web-based health interventions.

Original languageEnglish
Article numbere234
JournalJournal of Medical Internet Research
Volume18
Issue number8
Number of pages13
ISSN1439-4456
DOIs
Publication statusPublished - 31.08.2016

    Research areas

  • Psychology - Internet, Mental health treatments, evaluation, clinical effectiveness, personal satisfaction

Documents

DOI

Recently viewed

Publications

  1. Developing a Complex Portrait of Content Teaching for Multilingual Learners via Nonlinear Theoretical Understandings
  2. German Utilities and distributed PV
  3. Markups and Concentration in the Context of Digitization
  4. Using conditional inference trees and random forests to predict the bioaccumulation potential of organic chemicals
  5. (Re)productivity
  6. Quantification of amino acids in fermentation media by isocratic HPLC analysis of their
  7. Intraspecific trait variation patterns along a precipitation gradient in Mongolian rangelands
  8. A data-driven methodological routine to identify key indicators for social-ecological system archetype mapping
  9. Analysis of the construction of an autonomous robot to improve its energy efficiency when traveling through irregular terrain
  10. Rapid grain refinement and compositional homogenization in a cast binary Cu50Ni alloy achieved by friction stir processing
  11. Internal forces in robotic manipulation and in general mechanisms using a geometric approach
  12. Learning linear classifiers sensitive to example dependent and noisy costs
  13. Visual Detection of Traffic Incident through Automatic Monitoring of Vehicle Activities
  14. THE PARALLAX OF INDIVIDUATION
  15. Octanol-Water Partition Coefficient Measurement by a Simple 1H NMR Method
  16. Insights into Jatropha Projects Worldwide
  17. Deeper Insights into Different Consumer Perceptions of CSR Communication
  18. Development and comparison of processing maps of Mg-3Sn-1Ca alloy from data obtained in tension versus compression
  19. How Differences in Ratings of Odors and Odor Labels Are Associated with Identification Mechanisms
  20. Survey on challenges of Question Answering in the Semantic Web
  21. Application of friction surfacing for solid state additive manufacturing of cylindrical shell structures
  22. Towards productive functions?
  23. Determining Lot Sizes in Production Areas
  24. Methane yield of biomass from extensive grassland is affected by compositional changes induced by order of arrival
  25. Host functional and phylogenetic composition rather than host diversity structure plant–herbivore networks
  26. An integrative research framework for enabling transformative adaptation
  27. An Optimization Approach for Crew Rostering in Public Bus Transit
  28. Finding Datasets in Publications: The University of Paderborn Approach