Analyzing User Journey Data In Digital Health: Predicting Dropout From A Digital CBT-I Intervention

Research output: Journal contributionsConference abstract in journalResearchpeer-review

Authors

Intervention dropout is an important factor for the evaluation and implementation of digital therapeutics, including in insomnia. Large amounts of individualized data (logins, questionnaires, EMA data) in these interventions can combine to create user journeys - the data generated by the path an individual takes to navigate the digital therapeutic. User journeys can provide insight about how likely users are to drop out of an intervention on an individual level and lead to increased prediction performance. Thus, the goal of this study is to provide a step-by-step guide for the analysis of user journeys and utilize this guide to predict intervention dropout, illustrated with an example from a data in a RCT of digital therapeutic for chronic insomnia, for which outcomes have previously been published.
Methods
Analysis of user journeys includes data transformation, feature engineering, and statistical model analysis, using machine learning techniques. A framework is established to leverage user journeys to predict various behaviors. For this study, the framework was applied to predict dropouts of 151 participants from a fully automated web-based program (SHUTi) that delivered cognitive behavioral therapy for insomnia. For this task, support vector machines, logistic regression with regularization, and boosted decision trees were applied at different points in 9-week intervention. These techniques were evaluated based on their predictive performance.
Results
After model evaluation, a decision tree ensemble achieved AUC values ranging between 0.6-0.9 based on application of machine earning techniques. Various handcrafted and theory-driven features (e.g., time to complete certain intervention steps, time to get out of bed after arising, and days since last system interaction contributed to prediction performance.
Conclusion
Results indicate that utilizing a user journey framework and analysis can predict intervention dropout. Further, handcrafted theory-driven features can increase prediction performance. This prediction of dropout could lead to an enhanced clinical decision-making in digital therapeutics.
Support
The original study evaluating the efficacy of this intervention has been reported elsewhere and was funded by grant R01 MH86758 from the National Institute of Mental Health.
Original languageEnglish
Article number1204
JournalSleep
Volume43
Issue numberSupplement 1
Pages (from-to)A460
Number of pages1
ISSN0161-8105
DOIs
Publication statusPublished - 27.05.2020
EventAnnual Meeting of the Associated Professional Sleep Societies 2020 - VIRTUAL
Duration: 27.08.202030.08.2020
Conference number: 34
https://www.sleepmeeting.org/

Recently viewed

Publications

  1. Optimized neural networks for modeling of loudspeaker directivity diagrams
  2. Model-based logistic controlling of converging material flows
  3. Trajectory-based computational study of coherent behavior in flows
  4. A Matlab/Simulink toolbox for inversion of local linear model trees
  5. Dynamically changing sequencing rules with reinforcement learning in a job shop system with stochastic influences
  6. Lyapunov stability analysis to set up a PI controller for a mass flow system in case of a non-saturating input
  7. Influence of Process Parameters and Die Design on the Microstructure and Texture Development of Direct Extruded Magnesium Flat Products
  8. Use of Machine-Learning Algorithms Based on Text, Audio and Video Data in the Prediction of Anxiety and Post-Traumatic Stress in General and Clinical Populations
  9. Combining a PI Controller with an Adaptive Feedforward Control in PMSM
  10. “Ideation is Fine, but Execution is Key”
  11. A Multilevel Inverter Bridge Control Structure with Energy Storage Using Model Predictive Control for Flat Systems
  12. Sliding mode and model predictive control for inverse pendulum
  13. Model predictive control for switching gain adaptation in a sliding mode controller of a DC drive with nonlinear friction
  14. Restoring Causal Analysis to Structural Equation ModelingReview of Causality: Models, Reasoning, and Inference (2nd Edition), by Judea Pearl
  15. Effects of diversity versus segregation on automatic approach and avoidance behavior towards own and other ethnic groups
  16. Understanding Low-Code Evolution, Adoption and Ecosystem for Software Development
  17. An observer for sensorless variable valve control in camless internal combustion engines
  18. The Low-Code Phenomenon: Mapping the Intellectual Structure of Research
  19. Evaluating a Bayesian Student Model of Decimal Misconceptions
  20. Operational integration of EMIS and ERP systems
  21. The identification of up-And downstream industries using input-output tables and a firm-level application to minority shareholdings
  22. Concepts
  23. Moving Towards Measuring Multifunctionality in Ecosystems: FieldScreen – A Mobile Positioning System for Non-Invasive Measurement of Plant Traits in Field Experiments
  24. Document assignment in multi-site search engines
  25. German Utilities and distributed PV