Developing a Process for the Analysis of User Journeys and the Prediction of Dropout in Digital Health Interventions: Machine Learning Approach

Research output: Journal contributionsConference article in journalResearchpeer-review

Authors

Background: User dropout is a widespread concern in the delivery and evaluation of digital (ie, web and mobile apps) health interventions. Researchers have yet to fully realize the potential of the large amount of data generated by these technology-based programs. Of particular interest is the ability to predict who will drop out of an intervention. This may be possible through the analysis of user journey data—self-reported as well as system-generated data—produced by the path (or journey) an individual takes to navigate through a digital health intervention.

Objective: The purpose of this study is to provide a step-by-step process for the analysis of user journey data and eventually to predict dropout in the context of digital health interventions. The process is applied to data from an internet-based intervention for insomnia as a way to illustrate its use. The completion of the program is contingent upon completing 7 sequential cores, which include an initial tutorial core. Dropout is defined as not completing the seventh core.

Methods: Steps of user journey analysis, including data transformation, feature engineering, and statistical model analysis and evaluation, are presented. Dropouts were predicted based on data from 151 participants from a fully automated web-based program (Sleep Healthy Using the Internet) that delivers cognitive behavioral therapy for insomnia. Logistic regression with L1 and L2 regularization, support vector machines, and boosted decision trees were used and evaluated based on their predictive performance. Relevant features from the data are reported that predict user dropout.

Results: Accuracy of predicting dropout (area under the curve [AUC] values) varied depending on the program core and the machine learning technique. After model evaluation, boosted decision trees achieved AUC values ranging between 0.6 and 0.9. Additional handcrafted features, including time to complete certain steps of the intervention, time to get out of bed, and days since the last interaction with the system, contributed to the prediction performance.

Conclusions: The results support the feasibility and potential of analyzing user journey data to predict dropout. Theory-driven handcrafted features increased the prediction performance. The ability to predict dropout at an individual level could be used to enhance decision making for researchers and clinicians as well as inform dynamic intervention regimens.
Original languageEnglish
Article numbere17738
JournalJournal of Medical Internet Research
Volume22
Issue number10
Number of pages20
ISSN1439-4456
DOIs
Publication statusPublished - 28.10.2020

Bibliographical note

Publisher Copyright:
©Vincent Bremer, Philip I Chow, Burkhardt Funk, Frances P Thorndike, Lee M Ritterband.

Documents

DOI

Recently viewed

Publications

  1. A Soft Alignment Model for Bug Deduplication
  2. Predicate‐based model of problem‐solving for robotic actions planning
  3. Trajectory tracking using MPC and a velocity observer for flat actuator systems in automotive applications
  4. Deciphering movement and stasis
  5. Using density surface models to assess the ecological effectiveness of a protected area network in Tanzania
  6. Biomedical Entity Linking with Triple-aware Pre-Training
  7. DISKNET – A Platform for the Systematic Accumulation of Knowledge in IS Research
  8. The frame of the game
  9. Implementing UNESCO's Convention on Cultural Diversity at the regional level
  10. Defining the notion of mining, extraction and collection
  11. Integrating Common Ground and Informativeness in Pragmatic Word Learning
  12. Current issues in competence modeling and assessment
  13. An Approach for Ex-Post-Facto Analysis of Knowledge Graph-Driven Chatbots – The DBpedia Chatbot
  14. Covert and overt automatic imitation are correlated
  15. Back from the Deep
  16. Material flow analysis for the incremental sheet-bulk gearing by rotating tools
  17. Political discourse in the media
  18. "Doing" Sustainability Assessment in Different Consumption and Production Contexts-Lessons from Case Study Comparison
  19. Zapping-Fernbedienung
  20. From Fleeting Enchantment to Embodied Commitment
  21. Pathways and mechanisms for catalyzing social impact through Orchestration: Insights from an open social innovation project
  22. A New, Rapid, Fully Automated Method for Determination of Fluconazole in Serum by Column-Switching Liquid Chromatography
  23. TextCSN
  24. Land use affects dung beetle communities and their ecosystem service in forests and grasslands
  25. New incremental methods for springback compensation by stress superposition
  26. Existenzgründungen junger Handwerksmeister
  27. Same but different? Measurement invariance of the PIAAC motivation-to-learn scale across key socio-demographic groups
  28. Landscape modification and habitat fragmentation: a synthesis
  29. It is not what it is
  30. Newsfeed clutter as an inhibitor of sensemaking
  31. SMARTPHONE APPS FOR TINNITUS: A REVIEW ON INTERVENTION COMPONENTS AND BEHAVIOR CHANGE TECHNIQUES USED IN TINNITUS APPS
  32. y-Randomization and its variants in QSPR/QSAR
  33. Exports and productivity: A survey of the evidence from firm-level data
  34. Mythos
  35. Exports, R&D and Productivity
  36. Sigrid Kopfermann
  37. Effects of oral corrective feedback on the development of complex morphosyntax
  38. Quality and time-related indicators in inceptive plans
  39. Online to offline social networking
  40. Silver Work
  41. Towards a Real-world Laboratory
  42. Sustainable Statehood: Reflections on Critical (Pre-)Conditions, Requirements and Design Options