Finding the Best Match — a Case Study on the (Text‑) Feature and Model Choice in Digital Mental Health Interventions

Research output: Journal contributionsJournal articlesResearchpeer-review

Authors

With the need for psychological help long exceeding the supply, finding ways of
scaling, and better allocating mental health support is a necessity. This paper contributes by investigating how to best predict intervention dropout and failure to allow for a need-based adaptation of treatment. We systematically compare the predictive power of different text representation methods (metadata, TF-IDF, sentiment and topic analysis, and word embeddings) in combination with supplementary numerical inputs (socio-demographic, evaluation, and closed-question data). Additionally, we address the research gap of which ML model types — ranging from linear to sophisticated deep learning models — are best suited for different features and outcome variables. To this end, we analyze nearly 16.000 open-text answers from 849 German-speaking users in a Digital Mental Health Intervention (DMHI) for stress. Our research proves that — contrary to previous findings — there is great promise in using neural network approaches on DMHI text data. We propose a task-specific LSTM-based model architecture to tackle the challenge of long input sequences and thereby demonstrate the potential of word embeddings (AUC scores of up to 0.7) for
predictions in DMHIs. Despite the relatively small data set, sequential deep learning models, on average, outperform simpler features such as metadata and bag-of-words approaches when predicting dropout. The conclusion is that user-generated text of the first two sessions carries predictive power regarding patients’ dropout and intervention failure risk. Furthermore, the match between the sophistication of features and models needs to be closely considered to optimize results, and additional nontext features increase prediction results.
Original languageEnglish
Article number00148
JournalJournal of Healthcare Informatics Research
Volume7
Issue number4
Pages (from-to)447-479
Number of pages33
DOIs
Publication statusPublished - 12.2023

Bibliographical note

Funding Information:
Open Access funding enabled and organized by Projekt DEAL. The present study has been funded by Leuphana University. The original RCTs were funded by the European Union (project EFRE: CCI 2007DE161PR001).

Publisher Copyright:
© 2023, The Author(s).

    Research areas

  • E-mental health, Health care analytics, Machine learning, Natural language processing, Precision psychiatry

Recently viewed

Researchers

  1. Jonathan Pargätzi

Publications

  1. Learning Analytics and Personalized Learning
  2. Generalized self-efficacy as a mediator and moderator between control and complexity at work and personal initiative
  3. Heterogenous activation of dynamic recrystallization and twinning during friction stir processing of a Cu-4Nb alloy
  4. Sustainable Wireless Sensor Networks for Railway Systems Powered by Energy Harvesting from Vibration
  5. I share because of who I am: values, identities, norms, and attitudes explain sharing intentions
  6. Portuguese part-of-speech tagging with large margin structure learning
  7. Mining for critical stock price movements using temporal power laws and integrated autoregressive models
  8. DECODING SUSTAINABILITY IN THE HEALTHCARE SYSTEM. TEACHING STUDENTS HOW TO PROBLEMATIZE COMPLEX CONCEPTS
  9. Folding into being
  10. How cognitive issue bracketing affects interdependent decision-making in negotiations
  11. Subsistence and substitutability in consumer preferences
  12. Challenging the status quo of accelerator research: Concluding remarks
  13. Worse is worse and better doesn't matter?
  14. An introduction to sliding mode control for interdisciplinary education
  15. A microsystem for growth inhibition test of Enterococcus faecalis based on impedance measurement
  16. Process characteristics of constrained friction processing of AM50 magnesium alloy
  17. Implicit and explicit horizons
  18. Learning to collaborate from diverse interactions in project-based sustainability courses
  19. Comparison of Supervised versus Self-Administered Stretching on Bench Press Maximal Strength and Force Development
  20. An introductional lecture on chaotic systems through Lorenz attractor and forced Lotka Volterra equation for interdisciplinary education
  21. CSR
  22. Exploring governance learning
  23. Digital language teaching after COVID-19: what can we learn from the crisis?