Predicting recurrent chat contact in a psychological intervention for the youth using natural language processing

Research output: Journal contributionsJournal articlesResearchpeer-review

Standard

Predicting recurrent chat contact in a psychological intervention for the youth using natural language processing. / Hornstein, Silvan; Scharfenberger, Jonas; Lueken, Ulrike et al.
In: npj Digital Medicine, Vol. 7, 132, 12.2024.

Research output: Journal contributionsJournal articlesResearchpeer-review

Harvard

APA

Vancouver

Bibtex

@article{cefe076ae3194fdab8e903499f3dd24d,
title = "Predicting recurrent chat contact in a psychological intervention for the youth using natural language processing",
abstract = "Chat-based counseling hotlines emerged as a promising low-threshold intervention for youth mental health. However, despite the resulting availability of large text corpora, little work has investigated Natural Language Processing (NLP) applications within this setting. Therefore, this preregistered approach (OSF: XA4PN) utilizes a sample of approximately 19,000 children and young adults that received a chat consultation from a 24/7 crisis service in Germany. Around 800,000 messages were used to predict whether chatters would contact the service again, as this would allow the provision of or redirection to additional treatment. We trained an XGBoost Classifier on the words of the anonymized conversations, using repeated cross-validation and bayesian optimization for hyperparameter search. The best model was able to achieve an AUROC score of 0.68 (p < 0.01) on the previously unseen 3942 newest consultations. A shapely-based explainability approach revealed that words indicating younger age or female gender and terms related to self-harm and suicidal thoughts were associated with a higher chance of recontacting. We conclude that NLP-based predictions of recurrent contact are a promising path toward personalized care at chat hotlines.",
keywords = "Informatics",
author = "Silvan Hornstein and Jonas Scharfenberger and Ulrike Lueken and Richard Wundrack and Kevin Hilbert",
note = "Publisher Copyright: {\textcopyright} The Author(s) 2024.",
year = "2024",
month = dec,
doi = "10.1038/s41746-024-01121-9",
language = "English",
volume = "7",
journal = "npj Digital Medicine",
issn = "2398-6352",
publisher = "Nature Publishing Group",

}

RIS

TY - JOUR

T1 - Predicting recurrent chat contact in a psychological intervention for the youth using natural language processing

AU - Hornstein, Silvan

AU - Scharfenberger, Jonas

AU - Lueken, Ulrike

AU - Wundrack, Richard

AU - Hilbert, Kevin

N1 - Publisher Copyright: © The Author(s) 2024.

PY - 2024/12

Y1 - 2024/12

N2 - Chat-based counseling hotlines emerged as a promising low-threshold intervention for youth mental health. However, despite the resulting availability of large text corpora, little work has investigated Natural Language Processing (NLP) applications within this setting. Therefore, this preregistered approach (OSF: XA4PN) utilizes a sample of approximately 19,000 children and young adults that received a chat consultation from a 24/7 crisis service in Germany. Around 800,000 messages were used to predict whether chatters would contact the service again, as this would allow the provision of or redirection to additional treatment. We trained an XGBoost Classifier on the words of the anonymized conversations, using repeated cross-validation and bayesian optimization for hyperparameter search. The best model was able to achieve an AUROC score of 0.68 (p < 0.01) on the previously unseen 3942 newest consultations. A shapely-based explainability approach revealed that words indicating younger age or female gender and terms related to self-harm and suicidal thoughts were associated with a higher chance of recontacting. We conclude that NLP-based predictions of recurrent contact are a promising path toward personalized care at chat hotlines.

AB - Chat-based counseling hotlines emerged as a promising low-threshold intervention for youth mental health. However, despite the resulting availability of large text corpora, little work has investigated Natural Language Processing (NLP) applications within this setting. Therefore, this preregistered approach (OSF: XA4PN) utilizes a sample of approximately 19,000 children and young adults that received a chat consultation from a 24/7 crisis service in Germany. Around 800,000 messages were used to predict whether chatters would contact the service again, as this would allow the provision of or redirection to additional treatment. We trained an XGBoost Classifier on the words of the anonymized conversations, using repeated cross-validation and bayesian optimization for hyperparameter search. The best model was able to achieve an AUROC score of 0.68 (p < 0.01) on the previously unseen 3942 newest consultations. A shapely-based explainability approach revealed that words indicating younger age or female gender and terms related to self-harm and suicidal thoughts were associated with a higher chance of recontacting. We conclude that NLP-based predictions of recurrent contact are a promising path toward personalized care at chat hotlines.

KW - Informatics

UR - http://www.scopus.com/inward/record.url?scp=85193567276&partnerID=8YFLogxK

U2 - 10.1038/s41746-024-01121-9

DO - 10.1038/s41746-024-01121-9

M3 - Journal articles

C2 - 38762694

AN - SCOPUS:85193567276

VL - 7

JO - npj Digital Medicine

JF - npj Digital Medicine

SN - 2398-6352

M1 - 132

ER -