Developing a Process for the Analysis of User Journeys and the Prediction of Dropout in Digital Health Interventions: Machine Learning Approach

Publikation: Beiträge in ZeitschriftenKonferenzaufsätze in FachzeitschriftenForschungbegutachtet

Authors

Background: User dropout is a widespread concern in the delivery and evaluation of digital (ie, web and mobile apps) health interventions. Researchers have yet to fully realize the potential of the large amount of data generated by these technology-based programs. Of particular interest is the ability to predict who will drop out of an intervention. This may be possible through the analysis of user journey data—self-reported as well as system-generated data—produced by the path (or journey) an individual takes to navigate through a digital health intervention.

Objective: The purpose of this study is to provide a step-by-step process for the analysis of user journey data and eventually to predict dropout in the context of digital health interventions. The process is applied to data from an internet-based intervention for insomnia as a way to illustrate its use. The completion of the program is contingent upon completing 7 sequential cores, which include an initial tutorial core. Dropout is defined as not completing the seventh core.

Methods: Steps of user journey analysis, including data transformation, feature engineering, and statistical model analysis and evaluation, are presented. Dropouts were predicted based on data from 151 participants from a fully automated web-based program (Sleep Healthy Using the Internet) that delivers cognitive behavioral therapy for insomnia. Logistic regression with L1 and L2 regularization, support vector machines, and boosted decision trees were used and evaluated based on their predictive performance. Relevant features from the data are reported that predict user dropout.

Results: Accuracy of predicting dropout (area under the curve [AUC] values) varied depending on the program core and the machine learning technique. After model evaluation, boosted decision trees achieved AUC values ranging between 0.6 and 0.9. Additional handcrafted features, including time to complete certain steps of the intervention, time to get out of bed, and days since the last interaction with the system, contributed to the prediction performance.

Conclusions: The results support the feasibility and potential of analyzing user journey data to predict dropout. Theory-driven handcrafted features increased the prediction performance. The ability to predict dropout at an individual level could be used to enhance decision making for researchers and clinicians as well as inform dynamic intervention regimens.
OriginalspracheEnglisch
Aufsatznummere17738
ZeitschriftJournal of Medical Internet Research
Jahrgang22
Ausgabenummer10
Anzahl der Seiten20
ISSN1439-4456
DOIs
PublikationsstatusErschienen - 28.10.2020

Bibliographische Notiz

Funding Information:
This study was supported by grant R01 MH86758 from the National Institute of Mental Health. The funding source had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication. The authors thank Christina Frederick, BS, for her help with the study administration tasks. The authors specially thank Gabe D Heath, BA, and Steve P Johnson, BA, developers of the SHUTi intervention, for extracting and making all the data readily available for analysis.

Publisher Copyright:
©Vincent Bremer, Philip I Chow, Burkhardt Funk, Frances P Thorndike, Lee M Ritterband.

Dokumente

DOI

Zuletzt angesehen

Publikationen

  1. Article 11 Formal Validity
  2. Microstructural and mechanical aspects of reinforcement welds for lightweight components produced by friction hydro pillar processing
  3. Communicating CCS
  4. Second-Order Sliding Mode Control with State and Disturbance Estimation for a Permanent Magnet Linear Motor
  5. Towards Computer Simulations of Virtue Ethics
  6. A cascade regulator using Lyapunov's PID-PID controllers for an aggregate actuator in automotive applications
  7. Effectiveness of self-generation during learning is dependent on individual differences in need for cognition
  8. Accuracy Improvement of Vision System for Mobile Robot Navigation by Finding the Energetic Center of Laser Signal
  9. Mechanical properties and microstructures of nano SiC reinforced ZE10 composites prepared with ultrasonic vibration
  10. Playing in the Spaces: Anarchism in the Classroom
  11. Combining SMC and MTPA Using an EKF to estimate parameters and states of an interior PMSM
  12. Comparison of Backpropagation and Kalman Filter-based Training for Neural Networks
  13. Linked Accomplishment Of Order Management And Production Planning And Control. An Integrated Model-based Approach
  14. Modellieren in der Sekundarstufe
  15. Evidence for singlet state β cleavage in the photoreaction of α-(2,6-dimethoxyphenoxy)-acetophenone inferred from time-resolved CIDNP spectroscopy
  16. Methods in Writing Process Research
  17. Chip extrusion with integrated equal channel angular pressing
  18. Development of Early Spatial Perspective-Taking - Toward a Three-Level Model
  19. Horizontal, but not vertical canopy structure is related to stand functional diversity in a subtropical slope forest
  20. Landscape modification and habitat fragmentation: a synthesis
  21. Belief in free will affects causal attributions when judging others’ behavior
  22. A switching observer for sensorless control of an electromagnetic valve actuator for camless internal combustion engines
  23. Energy-aware system design for autonomous wireless sensor nodes
  24. Accuracy Improvement by Artificial Neural Networks in Technical Vision System
  25. Model-based Analysis of Reassembly Processes within the Regeneration of Complex Capital Goods
  26. Test of advanced hyperfine structure theory by precision radio-frequency and laser spectroscopy in molybdenum
  27. Strategy maps
  28. Microstructure refinement by a novel friction-based processing on Mg-Zn-Ca alloy