Developing a Process for the Analysis of User Journeys and the Prediction of Dropout in Digital Health Interventions: Machine Learning Approach

Publikation: Beiträge in ZeitschriftenKonferenzaufsätze in FachzeitschriftenForschungbegutachtet

Authors

Background: User dropout is a widespread concern in the delivery and evaluation of digital (ie, web and mobile apps) health interventions. Researchers have yet to fully realize the potential of the large amount of data generated by these technology-based programs. Of particular interest is the ability to predict who will drop out of an intervention. This may be possible through the analysis of user journey data—self-reported as well as system-generated data—produced by the path (or journey) an individual takes to navigate through a digital health intervention.

Objective: The purpose of this study is to provide a step-by-step process for the analysis of user journey data and eventually to predict dropout in the context of digital health interventions. The process is applied to data from an internet-based intervention for insomnia as a way to illustrate its use. The completion of the program is contingent upon completing 7 sequential cores, which include an initial tutorial core. Dropout is defined as not completing the seventh core.

Methods: Steps of user journey analysis, including data transformation, feature engineering, and statistical model analysis and evaluation, are presented. Dropouts were predicted based on data from 151 participants from a fully automated web-based program (Sleep Healthy Using the Internet) that delivers cognitive behavioral therapy for insomnia. Logistic regression with L1 and L2 regularization, support vector machines, and boosted decision trees were used and evaluated based on their predictive performance. Relevant features from the data are reported that predict user dropout.

Results: Accuracy of predicting dropout (area under the curve [AUC] values) varied depending on the program core and the machine learning technique. After model evaluation, boosted decision trees achieved AUC values ranging between 0.6 and 0.9. Additional handcrafted features, including time to complete certain steps of the intervention, time to get out of bed, and days since the last interaction with the system, contributed to the prediction performance.

Conclusions: The results support the feasibility and potential of analyzing user journey data to predict dropout. Theory-driven handcrafted features increased the prediction performance. The ability to predict dropout at an individual level could be used to enhance decision making for researchers and clinicians as well as inform dynamic intervention regimens.
OriginalspracheEnglisch
Aufsatznummere17738
ZeitschriftJournal of Medical Internet Research
Jahrgang22
Ausgabenummer10
Anzahl der Seiten20
ISSN1439-4456
DOIs
PublikationsstatusErschienen - 28.10.2020

Bibliographische Notiz

Funding Information:
This study was supported by grant R01 MH86758 from the National Institute of Mental Health. The funding source had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication. The authors thank Christina Frederick, BS, for her help with the study administration tasks. The authors specially thank Gabe D Heath, BA, and Steve P Johnson, BA, developers of the SHUTi intervention, for extracting and making all the data readily available for analysis.

Publisher Copyright:
©Vincent Bremer, Philip I Chow, Burkhardt Funk, Frances P Thorndike, Lee M Ritterband.

Dokumente

DOI

Zuletzt angesehen

Aktivitäten

  1. Where To Start? Exploring 1-Year-Students’ Preconceptions of Sustainable Development
  2. Extending Working Lives in Organizations: The Later Life Workplace Index for Successful Management of an Aging Workforce
  3. 15th International Conference on Sensors and Measurement Technology - SENSOR 2011
  4. Evaluating the efficacy and cost-effectiveness of web-based prevention of major depression
  5. Professional Development Workshop on “What Were You Thinking: Developing Cognitive Sensibilities for Inductive Coding” with Arne Carlsen, Martha Feldman, Claus Rerup, Heather Vogue, and Kristina Workman
  6. How Education Made Computers Personal
  7. Conference - 2022 the 5th World Conference on Computing and Communication Technologies
  8. Personal care products as source for micropollutants in Greywater-Identification, quantification and on-site treatment
  9. ‘Thinking the Problematic‘
  10. Presentation of new book on "Centrist Anti-Establishment Parties"
  11. Member of the selection committee for the COAL Prize Art & Environment 2011
  12. “Take Things Easy First” or “Get Straight to The Point”? The Order of Issue Packages in Negotiation and Its Effect on Dyadic Economic Outcomes.
  13. The relationship between intragenerational and intergenerational justice in the use of ecosystems and their services. An ecological-economic mode.
  14. Right-wing spaces. The transnational dimensions of right-wing extremism
  15. Working Towards Decent Work on Online Platforms: Building and Maintaining an Institutional Infrastructure Based on the German Social Partnership Tradition
  16. In zwei Sprachen leben

Publikationen

  1. Second-Order Sliding Mode Control with State and Disturbance Estimation for a Permanent Magnet Linear Motor
  2. Feedforward and repetitive control of a servo piezo-mechanical hydraulic actuator
  3. Learning Online: A Comparison of Different Media Types
  4. A transdisciplinary evaluation framework for the assessment of integration in boundary-crossing collaborations in teacher education
  5. A trainable object finder, selector and identifier for pollen, spores and other things
  6. Discourse pragmatics
  7. Learning to rank user intent
  8. Model-based estimation of pesticides and transformation products and their export pathways in a headwater catchment
  9. Managing sustainable development with management control systems
  10. Credit constraints and exports
  11. Empathy as a motivator of dyadic helping across group boundaries
  12. Urban Problem Discourses
  13. Using density surface models to assess the ecological effectiveness of a protected area network in Tanzania
  14. Do consumers prefer pasture-raised dual-purpose cattle when considering meat products? A hypothetical discrete choice experiment for the case of minced beef
  15. Vom „rights-based approach" zum "solution-based approach" in der WTO-Streitbeilegung?
  16. Calibrated Passive Sampling - Multi-plot Field Measurements of NH3 Emissions with a Combination of Dynamic Tube Method and Passive Samplers
  17. The Balanced Scorecard and different Business Models in the textile industry
  18. Learning-related emotions in multimedia learning
  19. Cascade MIMO P-PID Controllers Applied in an Over-actuated Quadrotor Tilt-Rotor
  20. Collaborative modelling for active involvement of stakeholders in urban flood risk management
  21. Mechanical properties and microstructures of nano SiC reinforced ZE10 composites prepared with ultrasonic vibration
  22. A cross-scale assessment of productivity–diversity relationships
  23. On the logic of drawing history from symbols, especially from images
  24. Making sense of sustainability transitions locally
  25. Effects of season and man-made changes on baseflow and flow recession
  26. Formative assessment in mathematics
  27. A web- And mobile-based intervention for comorbid, recurrent depression in patients with chronic back pain on sick leave (get.back)
  28. Developing a die casting magnesium alloy with excellent mechanical performance by controlling intermetallic phase