Developing a Process for the Analysis of User Journeys and the Prediction of Dropout in Digital Health Interventions: Machine Learning Approach

Research output: Journal contributionsConference article in journalResearchpeer-review

Authors

Background: User dropout is a widespread concern in the delivery and evaluation of digital (ie, web and mobile apps) health interventions. Researchers have yet to fully realize the potential of the large amount of data generated by these technology-based programs. Of particular interest is the ability to predict who will drop out of an intervention. This may be possible through the analysis of user journey data—self-reported as well as system-generated data—produced by the path (or journey) an individual takes to navigate through a digital health intervention.

Objective: The purpose of this study is to provide a step-by-step process for the analysis of user journey data and eventually to predict dropout in the context of digital health interventions. The process is applied to data from an internet-based intervention for insomnia as a way to illustrate its use. The completion of the program is contingent upon completing 7 sequential cores, which include an initial tutorial core. Dropout is defined as not completing the seventh core.

Methods: Steps of user journey analysis, including data transformation, feature engineering, and statistical model analysis and evaluation, are presented. Dropouts were predicted based on data from 151 participants from a fully automated web-based program (Sleep Healthy Using the Internet) that delivers cognitive behavioral therapy for insomnia. Logistic regression with L1 and L2 regularization, support vector machines, and boosted decision trees were used and evaluated based on their predictive performance. Relevant features from the data are reported that predict user dropout.

Results: Accuracy of predicting dropout (area under the curve [AUC] values) varied depending on the program core and the machine learning technique. After model evaluation, boosted decision trees achieved AUC values ranging between 0.6 and 0.9. Additional handcrafted features, including time to complete certain steps of the intervention, time to get out of bed, and days since the last interaction with the system, contributed to the prediction performance.

Conclusions: The results support the feasibility and potential of analyzing user journey data to predict dropout. Theory-driven handcrafted features increased the prediction performance. The ability to predict dropout at an individual level could be used to enhance decision making for researchers and clinicians as well as inform dynamic intervention regimens.
Original languageEnglish
Article numbere17738
JournalJournal of Medical Internet Research
Volume22
Issue number10
Number of pages20
ISSN1439-4456
DOIs
Publication statusPublished - 28.10.2020

Bibliographical note

Publisher Copyright:
©Vincent Bremer, Philip I Chow, Burkhardt Funk, Frances P Thorndike, Lee M Ritterband.

Documents

DOI

Recently viewed

Publications

  1. Spatially assessing unpleasant places with hard- and soft-GIS methods
  2. Accuracy Improvement of Vision System for Mobile Robot Navigation by Finding the Energetic Center of Laser Signal
  3. A simple control strategy for increasing the soft bending actuator performance by using a pressure boost
  4. Duration of Organizational Decision Processes in Organizations in View of Simulation Calculations
  5. Comparison of three methods of length compensation in a parallel kinematic and their equivalence conditions
  6. Explaining General and Specific Factors in Longitudinal, Multimethod, and Bifactor Models
  7. PD/PID-switching control as a human-machine interface for a semi-autonomous driver in automobiles
  8. Early Edema Detection Based on the Examination of Multidimensional Ultra-Wide band Data
  9. Modeling and simulation of the heterogenous material behavior in thermal-sprayed coatings
  10. Validation of Inspection Frameworks and Methods
  11. Creating spaces for cooperation
  12. Towards combined methods for recording ground beetles
  13. A New Approach for Optimal Solving Cyclic and Non-Cyclic Bus Drvier Rostering Problems
  14. How Differences in Ratings of Odors and Odor Labels Are Associated with Identification Mechanisms
  15. Nonlinear anisotropic boundary value problems – regularity results and multiscale discretizations
  16. The interplay between posture control and memory for spatial locations
  17. A toolkit for robust risk assessment using F-divergences
  18. Direct parameter specification of an attention shift: Evidence from perceptual latency priming
  19. Finding Datasets in Publications: The University of Paderborn Approach
  20. On the role of linguistic features for comprehension and learning from STEM texts. A meta-analysis
  21. Effects of accuracy feedback on fractal characteristics of time estimation
  22. Operational integration of EMIS and ERP systems
  23. The shooter bias: Replicating the classic effect and introducing a novel paradigm
  24. Graph-based Approaches for Analyzing Team Interaction on the Example of Soccer
  25. Learning from Erroneous Examples
  26. Performance of an IMU-Based Sensor Concept for Solving the Direct Kinematics Problem of the Stewart-Gough Platform
  27. Geometric series with randomly increasing exponents
  28. Incorporating ecosystem services into ecosystem-based management to deal with complexity
  29. A piezo servo hydraulic actuator for use in camless combustion engines and its control with MPC
  30. Improve a 3D distance measurement accuracy in stereo vision systems using optimization methods’ approach
  31. Priority Rule-based Planning Approaches for Regeneration Processes
  32. An Equation with many Variables
  33. Toward a gecko-inspired, climbing soft robot
  34. Experimental analysis of measurement process for a QCM using the pulse coincidence method