Analyzing User Journey Data In Digital Health: Predicting Dropout From A Digital CBT-I Intervention
Research output: Journal contributions › Conference abstract in journal › Research › peer-review
Standard
In: Sleep, Vol. 43, No. Supplement 1, 1204, 27.05.2020, p. A460.
Research output: Journal contributions › Conference abstract in journal › Research › peer-review
Harvard
APA
Vancouver
Bibtex
}
RIS
TY - JOUR
T1 - Analyzing User Journey Data In Digital Health: Predicting Dropout From A Digital CBT-I Intervention
AU - Bremer, Vincent
AU - Chow, Philip
AU - Funk, Burkhardt
AU - Thorndike, F
AU - Ritterband, Lee
N1 - Conference code: 34
PY - 2020/5/27
Y1 - 2020/5/27
N2 - Intervention dropout is an important factor for the evaluation and implementation of digital therapeutics, including in insomnia. Large amounts of individualized data (logins, questionnaires, EMA data) in these interventions can combine to create user journeys - the data generated by the path an individual takes to navigate the digital therapeutic. User journeys can provide insight about how likely users are to drop out of an intervention on an individual level and lead to increased prediction performance. Thus, the goal of this study is to provide a step-by-step guide for the analysis of user journeys and utilize this guide to predict intervention dropout, illustrated with an example from a data in a RCT of digital therapeutic for chronic insomnia, for which outcomes have previously been published.MethodsAnalysis of user journeys includes data transformation, feature engineering, and statistical model analysis, using machine learning techniques. A framework is established to leverage user journeys to predict various behaviors. For this study, the framework was applied to predict dropouts of 151 participants from a fully automated web-based program (SHUTi) that delivered cognitive behavioral therapy for insomnia. For this task, support vector machines, logistic regression with regularization, and boosted decision trees were applied at different points in 9-week intervention. These techniques were evaluated based on their predictive performance.ResultsAfter model evaluation, a decision tree ensemble achieved AUC values ranging between 0.6-0.9 based on application of machine earning techniques. Various handcrafted and theory-driven features (e.g., time to complete certain intervention steps, time to get out of bed after arising, and days since last system interaction contributed to prediction performance.ConclusionResults indicate that utilizing a user journey framework and analysis can predict intervention dropout. Further, handcrafted theory-driven features can increase prediction performance. This prediction of dropout could lead to an enhanced clinical decision-making in digital therapeutics.SupportThe original study evaluating the efficacy of this intervention has been reported elsewhere and was funded by grant R01 MH86758 from the National Institute of Mental Health.
AB - Intervention dropout is an important factor for the evaluation and implementation of digital therapeutics, including in insomnia. Large amounts of individualized data (logins, questionnaires, EMA data) in these interventions can combine to create user journeys - the data generated by the path an individual takes to navigate the digital therapeutic. User journeys can provide insight about how likely users are to drop out of an intervention on an individual level and lead to increased prediction performance. Thus, the goal of this study is to provide a step-by-step guide for the analysis of user journeys and utilize this guide to predict intervention dropout, illustrated with an example from a data in a RCT of digital therapeutic for chronic insomnia, for which outcomes have previously been published.MethodsAnalysis of user journeys includes data transformation, feature engineering, and statistical model analysis, using machine learning techniques. A framework is established to leverage user journeys to predict various behaviors. For this study, the framework was applied to predict dropouts of 151 participants from a fully automated web-based program (SHUTi) that delivered cognitive behavioral therapy for insomnia. For this task, support vector machines, logistic regression with regularization, and boosted decision trees were applied at different points in 9-week intervention. These techniques were evaluated based on their predictive performance.ResultsAfter model evaluation, a decision tree ensemble achieved AUC values ranging between 0.6-0.9 based on application of machine earning techniques. Various handcrafted and theory-driven features (e.g., time to complete certain intervention steps, time to get out of bed after arising, and days since last system interaction contributed to prediction performance.ConclusionResults indicate that utilizing a user journey framework and analysis can predict intervention dropout. Further, handcrafted theory-driven features can increase prediction performance. This prediction of dropout could lead to an enhanced clinical decision-making in digital therapeutics.SupportThe original study evaluating the efficacy of this intervention has been reported elsewhere and was funded by grant R01 MH86758 from the National Institute of Mental Health.
KW - Business informatics
KW - Health sciences
UR - https://academic.oup.com/sleep/article-abstract/43/Supplement_1/A460/5846921
U2 - 10.1093/sleep/zsaa056.1198
DO - 10.1093/sleep/zsaa056.1198
M3 - Conference abstract in journal
VL - 43
SP - A460
JO - Sleep
JF - Sleep
SN - 0161-8105
IS - Supplement 1
M1 - 1204
T2 - Annual Meeting of the Associated Professional Sleep Societies 2020
Y2 - 27 August 2020 through 30 August 2020
ER -