TY - JOUR

T1 - Dataset size versus homogeneity

T2 - A machine learning study on pooling intervention data in e-mental health dropout predictions

AU - Zantvoort, Kirsten

AU - Hentati Isacsson, Nils

AU - Funk, Burkhardt

AU - Kaldo, Viktor

N1 - Publisher Copyright: © The Author(s) 2024.

PY - 2024/5/15

Y1 - 2024/5/15

N2 - Objective: This study proposes a way of increasing dataset sizes for machine learning tasks in Internet-based Cognitive Behavioral Therapy through pooling interventions. To this end, it (1) examines similarities in user behavior and symptom data among online interventions for patients with depression, social anxiety, and panic disorder and (2) explores whether these similarities suffice to allow for pooling the data together, resulting in more training data when prediction intervention dropout. Methods: A total of 6418 routine care patients from the Internet Psychiatry in Stockholm are analyzed using (1) clustering and (2) dropout prediction models. For the latter, prediction models trained on each individual intervention's data are compared to those trained on all three interventions pooled into one dataset. To investigate if results vary with dataset size, the prediction is repeated using small and medium dataset sizes. Results: The clustering analysis identified three distinct groups that are almost equally spread across interventions and are instead characterized by different activity levels. In eight out of nine settings investigated, pooling the data improves prediction results compared to models trained on a single intervention dataset. It is further confirmed that models trained on small datasets are more likely to overestimate prediction results. Conclusion: The study reveals similar patterns of patients with depression, social anxiety, and panic disorder regarding online activity and intervention dropout. As such, this work offers pooling different interventions’ data as a possible approach to counter the problem of small dataset sizes in psychological research.

AB - Objective: This study proposes a way of increasing dataset sizes for machine learning tasks in Internet-based Cognitive Behavioral Therapy through pooling interventions. To this end, it (1) examines similarities in user behavior and symptom data among online interventions for patients with depression, social anxiety, and panic disorder and (2) explores whether these similarities suffice to allow for pooling the data together, resulting in more training data when prediction intervention dropout. Methods: A total of 6418 routine care patients from the Internet Psychiatry in Stockholm are analyzed using (1) clustering and (2) dropout prediction models. For the latter, prediction models trained on each individual intervention's data are compared to those trained on all three interventions pooled into one dataset. To investigate if results vary with dataset size, the prediction is repeated using small and medium dataset sizes. Results: The clustering analysis identified three distinct groups that are almost equally spread across interventions and are instead characterized by different activity levels. In eight out of nine settings investigated, pooling the data improves prediction results compared to models trained on a single intervention dataset. It is further confirmed that models trained on small datasets are more likely to overestimate prediction results. Conclusion: The study reveals similar patterns of patients with depression, social anxiety, and panic disorder regarding online activity and intervention dropout. As such, this work offers pooling different interventions’ data as a possible approach to counter the problem of small dataset sizes in psychological research.

KW - dropout

KW - e-mental health

KW - ICBT

KW - machine learning

KW - prediction

KW - Informatics

KW - Business informatics

UR - http://www.scopus.com/inward/record.url?scp=85193326208&partnerID=8YFLogxK

UR - https://www.mendeley.com/catalogue/49a70e86-edf7-383b-9bfe-25b73aec3f8f/

U2 - 10.1177/20552076241248920

DO - 10.1177/20552076241248920

M3 - Journal articles

C2 - 38757087

AN - SCOPUS:85193326208

VL - 10

JO - Digital Health

JF - Digital Health

SN - 2055-2076

ER -