Analyzing User Journey Data In Digital Health: Predicting Dropout From A Digital CBT-I Intervention

Research output: Journal contributionsConference abstract in journalResearchpeer-review

Authors

Intervention dropout is an important factor for the evaluation and implementation of digital therapeutics, including in insomnia. Large amounts of individualized data (logins, questionnaires, EMA data) in these interventions can combine to create user journeys - the data generated by the path an individual takes to navigate the digital therapeutic. User journeys can provide insight about how likely users are to drop out of an intervention on an individual level and lead to increased prediction performance. Thus, the goal of this study is to provide a step-by-step guide for the analysis of user journeys and utilize this guide to predict intervention dropout, illustrated with an example from a data in a RCT of digital therapeutic for chronic insomnia, for which outcomes have previously been published.
Methods
Analysis of user journeys includes data transformation, feature engineering, and statistical model analysis, using machine learning techniques. A framework is established to leverage user journeys to predict various behaviors. For this study, the framework was applied to predict dropouts of 151 participants from a fully automated web-based program (SHUTi) that delivered cognitive behavioral therapy for insomnia. For this task, support vector machines, logistic regression with regularization, and boosted decision trees were applied at different points in 9-week intervention. These techniques were evaluated based on their predictive performance.
Results
After model evaluation, a decision tree ensemble achieved AUC values ranging between 0.6-0.9 based on application of machine earning techniques. Various handcrafted and theory-driven features (e.g., time to complete certain intervention steps, time to get out of bed after arising, and days since last system interaction contributed to prediction performance.
Conclusion
Results indicate that utilizing a user journey framework and analysis can predict intervention dropout. Further, handcrafted theory-driven features can increase prediction performance. This prediction of dropout could lead to an enhanced clinical decision-making in digital therapeutics.
Support
The original study evaluating the efficacy of this intervention has been reported elsewhere and was funded by grant R01 MH86758 from the National Institute of Mental Health.
Original languageEnglish
Article number1204
JournalSleep
Volume43
Issue numberSupplement 1
Pages (from-to)A460
Number of pages1
ISSN0161-8105
DOIs
Publication statusPublished - 27.05.2020
EventAnnual Meeting of the Associated Professional Sleep Societies 2020 - VIRTUAL
Duration: 27.08.202030.08.2020
Conference number: 34
https://www.sleepmeeting.org/

Recently viewed

Publications

  1. Using transition management concepts for the evaluation of intersecting policy domains ('grand challenges')
  2. Global temporal typing patterns in foreign language writing
  3. Complex problem solving and intelligence
  4. Finding Creativity in Predictability: Seizing Kairos in Chronos Through Temporal Work in Complex Innovation Processes
  5. XOperator - An extensible semantic agent for instant messaging networks
  6. Proxies
  7. Conceptualizing Role Development in Agile Transformations
  8. Integration of Environmental Management Information Systems and ERP systems using Integration Platforms
  9. Enhancing the Building Information Modeling Lifecycle of Complex Structures with IoT
  10. Intersection tests for the cointegrating rank in dependent panel data
  11. On robustness properties in permanent magnet machine control by using decoupling controller
  12. Visualization of the Plasma Frequency by means of a Particle Simulation using a Normalized Periodic Model
  13. The role of learners’ memory in app-based language instruction: the case of Duolingo.
  14. Global Finite-Time Stabilization of Planar Linear Systems With Actuator Saturation
  15. A Class of Simple Stochastic Online Bin Packing Algorithms
  16. Recognition and approach responses toward threatening objects
  17. Towards a spatial understanding of identity play
  18. Lagged Multidimensional Recurrence Quantification Analysis for Determining Leader–Follower Relationships Within Multidimensional Time Series
  19. Supporting the Development and Implementation of a Digitalization Strategy in SMEs through a Lightweight Architecture-based Method
  20. Kalman Filter for Predictive Maintenance and Anomaly Detection
  21. Volume of Imbalance Container Prediction using Kalman Filter and Long Short-Term Memory
  22. Active and semi-supervised data domain description
  23. Formative Perspectives on the Relation Between CSR Communication and CSR Practices
  24. Experimentally established correlation of friction surfacing process temperature and deposit geometry
  25. Development of a scoring parameter to characterize data quality of centroids in high-resolution mass spectra
  26. Partitioned beta diversity patterns of plants across sharp and distinct boundaries of quartz habitat islands
  27. What can conservation strategies learn from the ecosystem services approach?
  28. An Adaptive and Optimized Switching Observer for Sensorless Control of an Electromagnetic Valve Actuator in Camless Internal Combustion Engines
  29. ASSESS — automatic self-assessment using linked data
  30. Wavelet functions for rejecting spurious values
  31. Preventive Diagnostics for cardiovascular diseases based on probabilistic methods and description logic
  32. Knowledge-Enhanced Language Models Are Not Bias-Proof
  33. An analytical approach to evaluating monotonic functions of fuzzy numbers