Making the most out of timeseries symptom data: A machine learning study on symptom predictions of internet-based CBT
Research output: Journal contributions › Journal articles › Research › peer-review
Standard
In: Internet Interventions, Vol. 38, 100773, 12.2024.
Research output: Journal contributions › Journal articles › Research › peer-review
Harvard
APA
Vancouver
Bibtex
}
RIS
TY - JOUR
T1 - Making the most out of timeseries symptom data
T2 - A machine learning study on symptom predictions of internet-based CBT
AU - Hentati Isacsson, Nils
AU - Zantvoort, Kirsten
AU - Forsell, Erik
AU - Boman, Magnus
AU - Kaldo, Viktor
N1 - Publisher Copyright: © 2024 The Authors
PY - 2024/12
Y1 - 2024/12
N2 - Objective: Predicting who will not benefit enough from Internet-Based Cognitive Behavioral (ICBT) Therapy early on can assist in better allocation of limited mental health care resources. Repeated measures of symptoms during treatment is the strongest predictor of outcome, and we want to investigate if methods that explicitly account for time-dependency are superior to methods that do not, with data from (a) only two pre-treatment timepoints and (b) the pre-treatment timepoints and three timepoints during initial treatment. Methods: We use 1) commonly used time-independent methods (i.e., Linear Regression and Random Forest models) and 2) time-dependent methods (i.e., multilevel model regression, mixed-effects random forest, and a Long Short-Term Memory model) to predict symptoms during treatment, including the final outcome. This is done with symptom scores from 6436 ICBT patients from regular care, using robust multiple imputation and nested cross-validation methods. Results: The models had a 14 %–12 % root mean squared error (RMSE) in predicting the post-treatment outcome, corresponding to a balanced accuracy of 67–74 %. Time-dependent models did not have higher accuracies. Using data for the initial treatment period (b) instead of only from before treatment (a) increased prediction results by 1.3 % percentage points (12 % to 10.7 %) RMSE and 6 % percentage points BACC (69 % to 75 %). Conclusion: Training prediction models on only symptom scores of the first few weeks is a promising avenue for symptom predictions in treatment, regardless of which model is used. Further research is necessary to better understand the interaction between model complexity, dataset length and width, and the prediction tasks at hand.
AB - Objective: Predicting who will not benefit enough from Internet-Based Cognitive Behavioral (ICBT) Therapy early on can assist in better allocation of limited mental health care resources. Repeated measures of symptoms during treatment is the strongest predictor of outcome, and we want to investigate if methods that explicitly account for time-dependency are superior to methods that do not, with data from (a) only two pre-treatment timepoints and (b) the pre-treatment timepoints and three timepoints during initial treatment. Methods: We use 1) commonly used time-independent methods (i.e., Linear Regression and Random Forest models) and 2) time-dependent methods (i.e., multilevel model regression, mixed-effects random forest, and a Long Short-Term Memory model) to predict symptoms during treatment, including the final outcome. This is done with symptom scores from 6436 ICBT patients from regular care, using robust multiple imputation and nested cross-validation methods. Results: The models had a 14 %–12 % root mean squared error (RMSE) in predicting the post-treatment outcome, corresponding to a balanced accuracy of 67–74 %. Time-dependent models did not have higher accuracies. Using data for the initial treatment period (b) instead of only from before treatment (a) increased prediction results by 1.3 % percentage points (12 % to 10.7 %) RMSE and 6 % percentage points BACC (69 % to 75 %). Conclusion: Training prediction models on only symptom scores of the first few weeks is a promising avenue for symptom predictions in treatment, regardless of which model is used. Further research is necessary to better understand the interaction between model complexity, dataset length and width, and the prediction tasks at hand.
KW - Adaptive treatment strategy
KW - Machine learning
KW - Precision psychiatry
KW - Prediction
KW - Timeseries symptom
KW - Treatment outcome
UR - http://www.scopus.com/inward/record.url?scp=85203524573&partnerID=8YFLogxK
UR - https://www.mendeley.com/catalogue/ed931dae-dfd8-3eb1-9b76-00bd47b6d332/
U2 - 10.1016/j.invent.2024.100773
DO - 10.1016/j.invent.2024.100773
M3 - Journal articles
AN - SCOPUS:85203524573
VL - 38
JO - Internet Interventions
JF - Internet Interventions
SN - 2214-7829
M1 - 100773
ER -