Analyzing User Journey Data In Digital Health: Predicting Dropout From A Digital CBT-I Intervention

Research output: Journal contributionsConference abstract in journalResearchpeer-review

Authors

Intervention dropout is an important factor for the evaluation and implementation of digital therapeutics, including in insomnia. Large amounts of individualized data (logins, questionnaires, EMA data) in these interventions can combine to create user journeys - the data generated by the path an individual takes to navigate the digital therapeutic. User journeys can provide insight about how likely users are to drop out of an intervention on an individual level and lead to increased prediction performance. Thus, the goal of this study is to provide a step-by-step guide for the analysis of user journeys and utilize this guide to predict intervention dropout, illustrated with an example from a data in a RCT of digital therapeutic for chronic insomnia, for which outcomes have previously been published.
Methods
Analysis of user journeys includes data transformation, feature engineering, and statistical model analysis, using machine learning techniques. A framework is established to leverage user journeys to predict various behaviors. For this study, the framework was applied to predict dropouts of 151 participants from a fully automated web-based program (SHUTi) that delivered cognitive behavioral therapy for insomnia. For this task, support vector machines, logistic regression with regularization, and boosted decision trees were applied at different points in 9-week intervention. These techniques were evaluated based on their predictive performance.
Results
After model evaluation, a decision tree ensemble achieved AUC values ranging between 0.6-0.9 based on application of machine earning techniques. Various handcrafted and theory-driven features (e.g., time to complete certain intervention steps, time to get out of bed after arising, and days since last system interaction contributed to prediction performance.
Conclusion
Results indicate that utilizing a user journey framework and analysis can predict intervention dropout. Further, handcrafted theory-driven features can increase prediction performance. This prediction of dropout could lead to an enhanced clinical decision-making in digital therapeutics.
Support
The original study evaluating the efficacy of this intervention has been reported elsewhere and was funded by grant R01 MH86758 from the National Institute of Mental Health.
Original languageEnglish
Article number1204
JournalSleep
Volume43
Issue numberSupplement 1
Pages (from-to)A460
Number of pages1
ISSN0161-8105
DOIs
Publication statusPublished - 27.05.2020
EventAnnual Meeting of the Associated Professional Sleep Societies 2020 - VIRTUAL
Duration: 27.08.202030.08.2020
Conference number: 34
https://www.sleepmeeting.org/

Recently viewed

Publications

  1. The Use of Factorization and Multimode Parametric Spectra in Estimating Frequency and Spectral Parameters of Signal
  2. Structure and dynamics laboratory testing of an indirectly controlled full variable valve train for camless engines
  3. Implementing ERP systems in multinational projects
  4. OKBQA framework towards an open collaboration for development of natural language question-answering systems over knowledge bases
  5. Agency and structure in a sociotechnical transition
  6. Quality Assurance Methods and the Open Source Model
  7. NH4+ ad-/desorption in sequencing batch reactors
  8. Is too much help an obstacle? Effects of interactivity and cognitive style on learning with dynamic versus non-dynamic visualizations with narrative explanations
  9. Facing complexity through informed simplifications
  10. Hierarchical trait filtering at different spatial scales determines beetle assemblages in deadwood
  11. Accounting and Modeling as Design Metaphors for CEMIS
  12. A guided simulated annealing search for solving the pick-up and delivery problem with time windows and capacity constraints
  13. A Lean Convolutional Neural Network for Vehicle Classification
  14. HAWK - hybrid question answering using linked data
  15. Modelling and implementation of an Order2Cash Process in distributed systems
  16. Factor structure and measurement invariance of the Students’ Self-report Checklist of Social and Learning Behaviour (SSL)
  17. Interactive Media as Fields of Transduction
  18. Using data mining techniques to investigate the correlation between surface cracks and flange lengths in deep drawn sheet metals
  19. From entity to process
  20. Geometric structures for the parameterization of non-interacting dynamics for multi-body mechanisms
  21. Predicate‐based model of problem‐solving for robotic actions planning
  22. Vergütung, variable
  23. Mechanism of dynamic recrystallization and evolution of texture in the hot working domains of the processing map for Mg-4Al-2Ba-2Ca Alloy
  24. A cascade controller structure using an internal PID controller for a hybrid piezo-hydraulic actuator in camless internal combustion engines
  25. Editorial: Machine Learning and Data Mining in Materials Science
  26. Computing regression statistics from grouped data
  27. Quantum Computing and the Analog/Digital Distinction
  28. The representative turn in EU studies
  29. Temporal dynamics of conflict monitoring and the effects of one or two conflict sources on error-(related) negativity
  30. Users’ handedness and performance when controlling integrated input devices
  31. Differences in adjustment flexibility between regular and temporary agency work
  32. Modelling, explaining, enacting and getting feedback: How can the acquisition of core practices in teacher education be optimally fostered?
  33. Petri net based EMIS-mappers for flexible manufacturing systems
  34. An Overview of Electro Hydraulic Full Variable Valve Train Systems to Reduce Emissions in Internal Combustion Engines
  35. Reciprocal Relationships Between Dispositional Optimism and Work Experiences
  36. A dialectical perspective on innovation: Conflicting demands, multiple pathways, and ambidexterity
  37. A Lyapunov Approach to Set the Parameters of a PI-Controller to Minimise Velocity Oscillations in a Permanent Magnet Synchronous Motor Using Chopper Control for Electrical Vehicles
  38. Assessment of cognitive load in multimedia learning using dual-task methodology
  39. Cognitive load in reading a foreign language text with multimedia aids and the influence of verbal and spatial abilities