Developing a Process for the Analysis of User Journeys and the Prediction of Dropout in Digital Health Interventions: Machine Learning Approach

Research output: Journal contributionsConference article in journalResearchpeer-review

Authors

Background: User dropout is a widespread concern in the delivery and evaluation of digital (ie, web and mobile apps) health interventions. Researchers have yet to fully realize the potential of the large amount of data generated by these technology-based programs. Of particular interest is the ability to predict who will drop out of an intervention. This may be possible through the analysis of user journey data—self-reported as well as system-generated data—produced by the path (or journey) an individual takes to navigate through a digital health intervention.

Objective: The purpose of this study is to provide a step-by-step process for the analysis of user journey data and eventually to predict dropout in the context of digital health interventions. The process is applied to data from an internet-based intervention for insomnia as a way to illustrate its use. The completion of the program is contingent upon completing 7 sequential cores, which include an initial tutorial core. Dropout is defined as not completing the seventh core.

Methods: Steps of user journey analysis, including data transformation, feature engineering, and statistical model analysis and evaluation, are presented. Dropouts were predicted based on data from 151 participants from a fully automated web-based program (Sleep Healthy Using the Internet) that delivers cognitive behavioral therapy for insomnia. Logistic regression with L1 and L2 regularization, support vector machines, and boosted decision trees were used and evaluated based on their predictive performance. Relevant features from the data are reported that predict user dropout.

Results: Accuracy of predicting dropout (area under the curve [AUC] values) varied depending on the program core and the machine learning technique. After model evaluation, boosted decision trees achieved AUC values ranging between 0.6 and 0.9. Additional handcrafted features, including time to complete certain steps of the intervention, time to get out of bed, and days since the last interaction with the system, contributed to the prediction performance.

Conclusions: The results support the feasibility and potential of analyzing user journey data to predict dropout. Theory-driven handcrafted features increased the prediction performance. The ability to predict dropout at an individual level could be used to enhance decision making for researchers and clinicians as well as inform dynamic intervention regimens.
Original languageEnglish
Article numbere17738
JournalJournal of Medical Internet Research
Volume22
Issue number10
Number of pages20
ISSN1439-4456
DOIs
Publication statusPublished - 28.10.2020

Bibliographical note

Publisher Copyright:
©Vincent Bremer, Philip I Chow, Burkhardt Funk, Frances P Thorndike, Lee M Ritterband.

Documents

DOI

Recently viewed

Researchers

  1. Kerstin Fedder

Activities

  1. Removal of Methotrexate, 5-Fluorouracil and Cyclophosphamide from water by UV, UV/H2O2 and UV/Fe2+/H2O2 processe
  2. Supply Chain Management – Current Practices and Future Developments
  3. The Advance of Diagnosis Chatbots: Should We First Avoid Distrust Before We Focus on Trust?
  4. Is Transaction Cost Theory a useful Perspective for Make-and-Buy?
  5. Just why, how and when should more participation lead to better environmental policy outcomes? A causal framework for analysis
  6. International Flusser Lectures Day - 2010
  7. Methodological Assemblage - Experiences from an Interdisciplinary Project on Artists and Cities
  8. Networks, Transcultural Entanglements, and the Power of Aesthetic Choices: Artistic Encounters in the Medieval Afro-Eurasian World
  9. On Borders, Boundaries, Clouds, and Globalization. And on China.
  10. Extending Working Lives in Organizations: The Later Life Workplace Index for Successful Management of an Aging Workforce
  11. IEEE International Conference on Control Applications - CCA 2012
  12. The more attractive the more effective? Investigating the association of user experience and efficacy of an online and app-based gratitude intervention to reduce repetitive negative thinking
  13. Experiences on the theme of actions for sustainable development in the field of educational systems, together with Ute Stoltenberg
  14. Work well, rest well, be well: How experience-sampling methods help us understand the new world of work
  15. Fostering language development by content based learning in German secondary schools
  16. PEER Workshop, Aalto University, Helsinki

Publications

  1. Pathways of Data-driven Business Model Design and Realization
  2. Development of Early Spatial Perspective-Taking - Toward a Three-Level Model
  3. Simulation and optimization of material and energy flow systems
  4. Public perceptions of CCS in context
  5. Outperformed by a Computer? - Comparing Human Decisions to Reinforcement Learning Agents, Assigning Lot Sizes in a Learning Factory
  6. An Integrative Framework of Environmental Management Accounting
  7. How to support teachers to give feedback to modelling tasks effectively? Results from a teacher-training-study in the Co²CA project
  8. On New Forms of Science Communication and Communication in Science
  9. German Utilities and Distributed PV
  10. A Developmental Trend in the Structure of Time-Estimation Performance
  11. How to attract visitors with strategic, value-based experience design
  12. Clashing Values
  13. Monitoring of microbially mediated corrosion and scaling processes using redox potential measurements
  14. Visual Detection of Traffic Incident through Automatic Monitoring of Vehicle Activities
  15. Geodesign as a boundary management process
  16. Employing a Novel Metaheuristic Algorithm to Optimize an LSTM Model
  17. A Graphic Language for Business Application Systems to Improve Communication Concerning Requirements Specification with the User
  18. Spatio-Temporal Convolution Kernels
  19. Archives
  20. Trap nests for bees and wasps to analyse trophic interactions in changing environments—A systematic overview and user guide
  21. Digital Seriality as Structure and Process
  22. Evaluating a Bayesian Student Model of Decimal Misconceptions
  23. Contextualizing certification and auditing
  24. On Software, or the Persistence of Visual Knowledge.
  25. Online-scheduling using past and real-time data
  26. Leverage points 2019
  27. Semiparametric one-step estimation of a sample selection model with endogenous covariates
  28. Enhancing EFL classroom instruction via the FeedBook: effects on language development and communicative language use.
  29. Nonlinear anisotropic boundary value problems – regularity results and multiscale discretizations
  30. More than a YouTube Channel