Proxy Indicators for the Quality of Open-domain Dialogues

Research output: Contributions to collected editions/worksArticle in conference proceedingsResearchpeer-review

Authors

The automatic evaluation of open-domain dialogues remains a largely unsolved challenge. Thus, despite the abundance of work done in the field, human judges have to evaluate dialogues' quality. As a consequence, performing such evaluations at scale is usually expensive. This work investigates using a deep-learning model trained on the General Language Understanding Evaluation (GLUE) benchmark to serve as a quality indication of open-domain dialogues. The aim is to use the various GLUE tasks as different perspectives on judging the quality of conversation, thus reducing the need for additional training data or responses that serve as quality references. Due to this nature, the method can infer various quality metrics and derive a component-based overall score. We achieve statistically significant correlation coefficients of up to 0.7.

Original languageEnglish
Title of host publicationEMNLP 2021 - 2021 Conference on Empirical Methods in Natural Language Processing, Proceedings
EditorsMarie-Francine Moens, Xuanjing Huang, Lucia Specia, Scott Wen-tau Yih
Number of pages22
PublisherAssociation for Computational Linguistics (ACL)
Publication date01.01.2021
Pages7834-7855
ISBN (electronic)9781955917094
DOIs
Publication statusPublished - 01.01.2021
Externally publishedYes
Event2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021 - ONLINE, Punta Cana, Dominican Republic
Duration: 07.11.202111.11.2021
https://2021.emnlp.org

Bibliographical note

Publisher Copyright:
© 2021 Association for Computational Linguistics

Recently viewed

Researchers

  1. Matthias Klöppner

Publications

  1. Phosphorus uptake from struvite is modulated by the nitrogen form applied
  2. Erratum: Formalised and non-formalised methods in resource management-knowledge and social learning in participatory processes
  3. Machine Learning Applications
  4. Theory-based course design for professional master's degree program in business engineering
  5. Typewriting Dynamics
  6. Variational pragmatics in the foreign language classroom
  7. Subverting Autocracy
  8. Basic analysis of the incremental profile forming process
  9. New trends in pragmatics
  10. The Crowd in Flux
  11. Overyielding in experimental grassland communities - Irrespective of species pool or spatial scale
  12. The role of supervisor support for dealing with customer verbal aggression. Differences between ethnic minority and ethnic majority workers
  13. Ästhetikkolumne
  14. Mouseology – Ludic Interfaces – Zero Interfaces
  15. SAP exchange infrastructure for developers
  16. Idiosyncratic volatility, option-based measures of informed trading, and investor attention
  17. Lesetechnik
  18. Over here and over there
  19. Numerical investigation of laser beam-welded AA2198 joints under different artificial ageing conditions
  20. Attention and the Speed of Information Processing
  21. Futures loss, despair and empowerment work in the University of Vechta: an action research project
  22. Credit constraints, endogenous innovations, and price setting in international trade
  23. Multiculturalism in Canada
  24. Studien zu einer Ethik der Enttäuschung
  25. Two-pass friction stir welding of cladded API X65
  26. Asynchrone Objekte
  27. Tritheism
  28. Schwarz-weiß in Farbe
  29. All production is joint production - A thermodynamic analysis
  30. Introduction: Children's Literature Global and Local
  31. Das Ethos reiner Fraulichkeit
  32. Rechtliche Aspekte
  33. The F.D.P.
  34. Testing Lazear's jack-of-all-trades view of entrepreneurship with German micro data
  35. Didactics of Mathematics in Higher Education as a Scientific Discipline - Conference Proceedings
  36. Vom Abfall zum Einfall
  37. READY! Un programma per stimolare la prontezza decisionale
  38. Größen bauen auf Längen
  39. Let’s talk about money! Assessing the link between firm performance and voluntary Say-on-Pay votes
  40. Impulse für die Migrationsgesellschaft
  41. Negotiated third party access