Developing a Process for the Analysis of User Journeys and the Prediction of Dropout in Digital Health Interventions: Machine Learning Approach

Research output: Journal contributionsConference article in journalResearchpeer-review

Authors

Background: User dropout is a widespread concern in the delivery and evaluation of digital (ie, web and mobile apps) health interventions. Researchers have yet to fully realize the potential of the large amount of data generated by these technology-based programs. Of particular interest is the ability to predict who will drop out of an intervention. This may be possible through the analysis of user journey data—self-reported as well as system-generated data—produced by the path (or journey) an individual takes to navigate through a digital health intervention.

Objective: The purpose of this study is to provide a step-by-step process for the analysis of user journey data and eventually to predict dropout in the context of digital health interventions. The process is applied to data from an internet-based intervention for insomnia as a way to illustrate its use. The completion of the program is contingent upon completing 7 sequential cores, which include an initial tutorial core. Dropout is defined as not completing the seventh core.

Methods: Steps of user journey analysis, including data transformation, feature engineering, and statistical model analysis and evaluation, are presented. Dropouts were predicted based on data from 151 participants from a fully automated web-based program (Sleep Healthy Using the Internet) that delivers cognitive behavioral therapy for insomnia. Logistic regression with L1 and L2 regularization, support vector machines, and boosted decision trees were used and evaluated based on their predictive performance. Relevant features from the data are reported that predict user dropout.

Results: Accuracy of predicting dropout (area under the curve [AUC] values) varied depending on the program core and the machine learning technique. After model evaluation, boosted decision trees achieved AUC values ranging between 0.6 and 0.9. Additional handcrafted features, including time to complete certain steps of the intervention, time to get out of bed, and days since the last interaction with the system, contributed to the prediction performance.

Conclusions: The results support the feasibility and potential of analyzing user journey data to predict dropout. Theory-driven handcrafted features increased the prediction performance. The ability to predict dropout at an individual level could be used to enhance decision making for researchers and clinicians as well as inform dynamic intervention regimens.
Original languageEnglish
Article numbere17738
JournalJournal of Medical Internet Research
Volume22
Issue number10
Number of pages20
ISSN1439-4456
DOIs
Publication statusPublished - 28.10.2020

Bibliographical note

Publisher Copyright:
©Vincent Bremer, Philip I Chow, Burkhardt Funk, Frances P Thorndike, Lee M Ritterband.

Documents

DOI

Recently viewed

Publications

  1. Using measures of reading time regularity (RTR) to quantify eye movement dynamics, and how they are shaped by linguistic information
  2. Early Edema Detection Based on the Examination of Multidimensional Ultra-Wide band Data
  3. Effectiveness of the world network of biosphere reserves in maintaining forest ecosystem functions
  4. Sliding Mode Control of an Inductive Power Transmission System with Maximum Efficiency
  5. Cascade PID Controllers Applied on Level and Flow Systems in a SMAR Didactic Plant
  6. Audio-Hacks
  7. A Two-Stage Sliding-Mode High-Gain Observer to Reduce Uncertainties and Disturbances Effects for Sensorless Control in Automotive Applications
  8. What motivates people to use energy feedback systems? A multiple goal approach to predict long-term usage behaviour in daily life
  9. An Outcome-Oriented, Social-Ecological Framework for Assessing Protected Area Effectiveness
  10. Dealing with inclusion–teachers’ assessment of internal and external resources
  11. A utilitarian notion of responsibility for sustainability
  12. Emancipative Values and Non-violent Protest
  13. Pushing the Envelope: Creating Public Value in the Labor Market
  14. A generalized α-level decomposition concept for numerical fuzzy calculus
  15. Disentangling who is who during rhizosphere acidification in root interactions: combining fluorescence with optode techniques
  16. The Mobile Phone: From an Instrument of Microcoordination to a Universal Control Device
  17. Modeling Grounding Processes in Chat-based CSCL
  18. Choice and quantity demand for improved and unimproved public water sources in rural areas
  19. Forging of Mg–3Sn–2Ca–0.4Al Alloy Assisted by Its Processing Map and Validation Through Analytical Modeling
  20. Collaborative modelling for active involvement of stakeholders in urban flood risk management
  21. The Challenge of Democratic Representation in the European Union
  22. Dynamic Inversion-Enhanced U-Control of Quadrotor Trajectory Tracking
  23. What role for frames in scalar conflicts?
  24. Material utilization of organic residues
  25. Linked Accomplishment Of Order Management And Production Planning And Control. An Integrated Model-based Approach
  26. Short run comovement, persistent shocks and the business cycle
  27. Computerspiele
  28. Transparency in an Age of Digitalization and Responsibility