Predicting recurrent chat contact in a psychological intervention for the youth using natural language processing
Research output: Journal contributions › Journal articles › Research › peer-review
Standard
In: npj Digital Medicine, Vol. 7, No. 1, 132, 12.2024.
Research output: Journal contributions › Journal articles › Research › peer-review
Harvard
APA
Vancouver
Bibtex
}
RIS
TY - JOUR
T1 - Predicting recurrent chat contact in a psychological intervention for the youth using natural language processing
AU - Hornstein, Silvan
AU - Scharfenberger, Jonas
AU - Lueken, Ulrike
AU - Wundrack, Richard
AU - Hilbert, Kevin
N1 - Publisher Copyright: © The Author(s) 2024.
PY - 2024/12
Y1 - 2024/12
N2 - Chat-based counseling hotlines emerged as a promising low-threshold intervention for youth mental health. However, despite the resulting availability of large text corpora, little work has investigated Natural Language Processing (NLP) applications within this setting. Therefore, this preregistered approach (OSF: XA4PN) utilizes a sample of approximately 19,000 children and young adults that received a chat consultation from a 24/7 crisis service in Germany. Around 800,000 messages were used to predict whether chatters would contact the service again, as this would allow the provision of or redirection to additional treatment. We trained an XGBoost Classifier on the words of the anonymized conversations, using repeated cross-validation and bayesian optimization for hyperparameter search. The best model was able to achieve an AUROC score of 0.68 (p < 0.01) on the previously unseen 3942 newest consultations. A shapely-based explainability approach revealed that words indicating younger age or female gender and terms related to self-harm and suicidal thoughts were associated with a higher chance of recontacting. We conclude that NLP-based predictions of recurrent contact are a promising path toward personalized care at chat hotlines.
AB - Chat-based counseling hotlines emerged as a promising low-threshold intervention for youth mental health. However, despite the resulting availability of large text corpora, little work has investigated Natural Language Processing (NLP) applications within this setting. Therefore, this preregistered approach (OSF: XA4PN) utilizes a sample of approximately 19,000 children and young adults that received a chat consultation from a 24/7 crisis service in Germany. Around 800,000 messages were used to predict whether chatters would contact the service again, as this would allow the provision of or redirection to additional treatment. We trained an XGBoost Classifier on the words of the anonymized conversations, using repeated cross-validation and bayesian optimization for hyperparameter search. The best model was able to achieve an AUROC score of 0.68 (p < 0.01) on the previously unseen 3942 newest consultations. A shapely-based explainability approach revealed that words indicating younger age or female gender and terms related to self-harm and suicidal thoughts were associated with a higher chance of recontacting. We conclude that NLP-based predictions of recurrent contact are a promising path toward personalized care at chat hotlines.
KW - Informatics
UR - http://www.scopus.com/inward/record.url?scp=85193567276&partnerID=8YFLogxK
UR - https://www.mendeley.com/catalogue/827faade-aab3-33c2-af97-fd7b135056ba/
U2 - 10.1038/s41746-024-01121-9
DO - 10.1038/s41746-024-01121-9
M3 - Journal articles
C2 - 38762694
AN - SCOPUS:85193567276
VL - 7
JO - npj Digital Medicine
JF - npj Digital Medicine
SN - 2398-6352
IS - 1
M1 - 132
ER -