Reliability and Validity of Assessing User Satisfaction With Web-Based Health Interventions

Research output: Journal contributionsJournal articlesResearchpeer-review

Authors

Background: The perspective of users should be taken into account in the evaluation of Web-based health interventions. Assessing the users' satisfaction with the intervention they receive could enhance the evidence for the intervention effects. Thus, there is a need for valid and reliable measures to assess satisfaction with Web-based health interventions. Objective: The objective of this study was to analyze the reliability, factorial structure, and construct validity of the Client Satisfaction Questionnaire adapted to Internet-based interventions (CSQ-I). Methods: The psychometric quality of the CSQ-I was analyzed in user samples from 2 separate randomized controlled trials evaluating Web-based health interventions, one from a depression prevention intervention (sample 1, N=174) and the other from a stress management intervention (sample 2, N=111). At first, the underlying measurement model of the CSQ-I was analyzed to determine the internal consistency. The factorial structure of the scale and the measurement invariance across groups were tested by multigroup confirmatory factor analyses. Additionally, the construct validity of the scale was examined by comparing satisfaction scores with the primary clinical outcome. Results: Multigroup confirmatory analyses on the scale yielded a one-factorial structure with a good fit (root-mean-square error of approximation =.09, comparative fit index =.96, standardized root-mean-square residual =.05) that showed partial strong invariance across the 2 samples. The scale showed very good reliability, indicated by McDonald omegas of .95 in sample 1 and .93 in sample 2. Significant correlations with change in depressive symptoms (r=-.35, P<.001) and perceived stress (r=-.48, P<.001) demonstrated the construct validity of the scale. Conclusions: The proven internal consistency, factorial structure, and construct validity of the CSQ-I indicate a good overall psychometric quality of the measure to assess the user's general satisfaction with Web-based interventions for depression and stress management. Multigroup analyses indicate its robustness across different samples. Thus, the CSQ-I seems to be a suitable measure to consider the user's perspective in the overall evaluation of Web-based health interventions.

Original languageEnglish
Article numbere234
JournalJournal of Medical Internet Research
Volume18
Issue number8
Number of pages13
ISSN1439-4456
DOIs
Publication statusPublished - 31.08.2016

    Research areas

  • Psychology - Internet, Mental health treatments, evaluation, clinical effectiveness, personal satisfaction

Documents

DOI

Recently viewed

Publications

  1. Convergence of adaptive learning and expectational stability
  2. Metaheuristics approach for solving personalized crew rostering problem in public bus transit
  3. Unraveling Privacy Concerns in Complex Data Ecosystems with Architectural Thinking
  4. "Introduction," communication +1
  5. The structure of emotions in learning situations
  6. Gaussian trajectories in motion control for camless engines
  7. A high-resolution approach for the spatiotemporal analysis of forest canopy space using terrestrial laser scanning data
  8. The buffering effect of selection, optimization, and compensation strategy use on the relationship between problem solving demands and occupational well-being
  9. Integration of demand forecasts in ABC-XYZ analysis
  10. Integrating business models and enterprise architecture
  11. Operationalizing Network Theory for Ecosystem Service Assessments
  12. The explanatory power of Carnegie Classification in predicting engagement indicators
  13. An automated, modular system for organic waste utilization using Hermetia illucens larvae
  14. The impact of explicit references in computer supported collaborative learning: Evidence from eye movement analyses
  15. Imaginary practices as the nexus between continuity and disruptive change
  16. Complexity of traffic scenes and EEG-measures of processing workload in car driving
  17. Utilizing learning analytics to support study success
  18. Is the market classification of risk always efficient?
  19. Global fern and lycophyte richness explained: How regional and local factors shape plot richness
  20. NNARX networks on didactic level system identification
  21. Document assignment in multi-site search engines
  22. Development and criterion validity of differentiated and elevated vocational interests in adolescence
  23. From simulation to real-world robotic mobile fulfillment systems
  24. Determinants in the online distribution of digital content
  25. Competence-Oriented Teaching
  26. Inventing a secure future