Developing a Process for the Analysis of User Journeys and the Prediction of Dropout in Digital Health Interventions: Machine Learning Approach

Research output: Journal contributionsConference article in journalResearchpeer-review

Authors

Background: User dropout is a widespread concern in the delivery and evaluation of digital (ie, web and mobile apps) health interventions. Researchers have yet to fully realize the potential of the large amount of data generated by these technology-based programs. Of particular interest is the ability to predict who will drop out of an intervention. This may be possible through the analysis of user journey data—self-reported as well as system-generated data—produced by the path (or journey) an individual takes to navigate through a digital health intervention.

Objective: The purpose of this study is to provide a step-by-step process for the analysis of user journey data and eventually to predict dropout in the context of digital health interventions. The process is applied to data from an internet-based intervention for insomnia as a way to illustrate its use. The completion of the program is contingent upon completing 7 sequential cores, which include an initial tutorial core. Dropout is defined as not completing the seventh core.

Methods: Steps of user journey analysis, including data transformation, feature engineering, and statistical model analysis and evaluation, are presented. Dropouts were predicted based on data from 151 participants from a fully automated web-based program (Sleep Healthy Using the Internet) that delivers cognitive behavioral therapy for insomnia. Logistic regression with L1 and L2 regularization, support vector machines, and boosted decision trees were used and evaluated based on their predictive performance. Relevant features from the data are reported that predict user dropout.

Results: Accuracy of predicting dropout (area under the curve [AUC] values) varied depending on the program core and the machine learning technique. After model evaluation, boosted decision trees achieved AUC values ranging between 0.6 and 0.9. Additional handcrafted features, including time to complete certain steps of the intervention, time to get out of bed, and days since the last interaction with the system, contributed to the prediction performance.

Conclusions: The results support the feasibility and potential of analyzing user journey data to predict dropout. Theory-driven handcrafted features increased the prediction performance. The ability to predict dropout at an individual level could be used to enhance decision making for researchers and clinicians as well as inform dynamic intervention regimens.
Original languageEnglish
Article numbere17738
JournalJournal of Medical Internet Research
Volume22
Issue number10
Number of pages20
ISSN1439-4456
DOIs
Publication statusPublished - 28.10.2020

Bibliographical note

Publisher Copyright:
©Vincent Bremer, Philip I Chow, Burkhardt Funk, Frances P Thorndike, Lee M Ritterband.

Documents

DOI

Recently viewed

Publications

  1. Photodegradation of micropollutants using V-UV/UV-C processes
  2. Integrating methods for ecosystem service assessment
  3. Do consumers prefer pasture-raised dual-purpose cattle when considering meat products? A hypothetical discrete choice experiment for the case of minced beef
  4. Contrasting patterns of intraspecific trait variability in native and non-native plant species along an elevational gradient on Tenerife, Canary Islands
  5. Impact factors and regulatory mechanisms for material flow management
  6. Towards a Comprehensive Framework for Environmental Management Accounting
  7. Governmental activity, integration, and agglomeration
  8. Challenges in political interviews
  9. A switching model predictive control for overcoming a hysteresis effect in a hybrid actuator for camless internal combustion engines
  10. Towards a Comprehensive Framework for Environmental Management Accounting
  11. Actuator- and/or sensor element for sleeve in medical field e.g. limb or joint fracture treatment, has nano-wires comprising nano-fibers, where element deforms and acquires dimensional change of nano-fibers via electrical signal
  12. Article 32 Date of Application
  13. Collaborative modelling for active involvement of stakeholders in urban flood risk management
  14. Glitch(ing)! A refusal and gateway to more caring techno-urban worlds?
  15. Probing turbulent superstructures in Rayleigh-Bénard convection by Lagrangian trajectory clusters
  16. Assessing tree dendrometrics in young regenerating plantations using terrestrial laser scanning
  17. Toxicity testing with luminescent bacteria - Characterization of an automated method for the combined assessment of acute and chronic effects
  18. RelHunter
  19. Swarm Robotics, or: The Smartness of 'a bunch of cheap dumb things'
  20. Perceptions of Organizational Downsizing
  21. Policy implementation through multi-level governance
  22. Pre-service mathematics teachers' modelling processes within model eliciting activity through digital technologies
  23. Advantages and difficulties of conducting thinking-aloud protocols in the school setting
  24. Development of a procedure for forming assisted thermal joining of tubes
  25. The complementarity of single-species and ecosystem-oriented research in conservation research