Dataset size versus homogeneity: A machine learning study on pooling intervention data in e-mental health dropout predictions
Research output: Journal contributions › Journal articles › Research › peer-review
Standard
In: Digital Health, Vol. 10, 15.05.2024.
Research output: Journal contributions › Journal articles › Research › peer-review
Harvard
APA
Vancouver
Bibtex
}
RIS
TY - JOUR
T1 - Dataset size versus homogeneity
T2 - A machine learning study on pooling intervention data in e-mental health dropout predictions
AU - Zantvoort, Kirsten
AU - Hentati Isacsson, Nils
AU - Funk, Burkhardt
AU - Kaldo, Viktor
N1 - Publisher Copyright: © The Author(s) 2024.
PY - 2024/5/15
Y1 - 2024/5/15
N2 - Objective: This study proposes a way of increasing dataset sizes for machine learning tasks in Internet-based Cognitive Behavioral Therapy through pooling interventions. To this end, it (1) examines similarities in user behavior and symptom data among online interventions for patients with depression, social anxiety, and panic disorder and (2) explores whether these similarities suffice to allow for pooling the data together, resulting in more training data when prediction intervention dropout. Methods: A total of 6418 routine care patients from the Internet Psychiatry in Stockholm are analyzed using (1) clustering and (2) dropout prediction models. For the latter, prediction models trained on each individual intervention's data are compared to those trained on all three interventions pooled into one dataset. To investigate if results vary with dataset size, the prediction is repeated using small and medium dataset sizes. Results: The clustering analysis identified three distinct groups that are almost equally spread across interventions and are instead characterized by different activity levels. In eight out of nine settings investigated, pooling the data improves prediction results compared to models trained on a single intervention dataset. It is further confirmed that models trained on small datasets are more likely to overestimate prediction results. Conclusion: The study reveals similar patterns of patients with depression, social anxiety, and panic disorder regarding online activity and intervention dropout. As such, this work offers pooling different interventions’ data as a possible approach to counter the problem of small dataset sizes in psychological research.
AB - Objective: This study proposes a way of increasing dataset sizes for machine learning tasks in Internet-based Cognitive Behavioral Therapy through pooling interventions. To this end, it (1) examines similarities in user behavior and symptom data among online interventions for patients with depression, social anxiety, and panic disorder and (2) explores whether these similarities suffice to allow for pooling the data together, resulting in more training data when prediction intervention dropout. Methods: A total of 6418 routine care patients from the Internet Psychiatry in Stockholm are analyzed using (1) clustering and (2) dropout prediction models. For the latter, prediction models trained on each individual intervention's data are compared to those trained on all three interventions pooled into one dataset. To investigate if results vary with dataset size, the prediction is repeated using small and medium dataset sizes. Results: The clustering analysis identified three distinct groups that are almost equally spread across interventions and are instead characterized by different activity levels. In eight out of nine settings investigated, pooling the data improves prediction results compared to models trained on a single intervention dataset. It is further confirmed that models trained on small datasets are more likely to overestimate prediction results. Conclusion: The study reveals similar patterns of patients with depression, social anxiety, and panic disorder regarding online activity and intervention dropout. As such, this work offers pooling different interventions’ data as a possible approach to counter the problem of small dataset sizes in psychological research.
KW - dropout
KW - e-mental health
KW - ICBT
KW - machine learning
KW - prediction
KW - Informatics
KW - Business informatics
UR - http://www.scopus.com/inward/record.url?scp=85193326208&partnerID=8YFLogxK
UR - https://www.mendeley.com/catalogue/49a70e86-edf7-383b-9bfe-25b73aec3f8f/
U2 - 10.1177/20552076241248920
DO - 10.1177/20552076241248920
M3 - Journal articles
C2 - 38757087
AN - SCOPUS:85193326208
VL - 10
JO - Digital Health
JF - Digital Health
SN - 2055-2076
ER -