Treating dialogue quality evaluation as an anomaly detection problem

Publikation: Beiträge in SammelwerkenAufsätze in KonferenzbändenForschungbegutachtet

Authors

Dialogue systems for interaction with humans have been enjoying increased popularity in the research and industry fields. To this day, the best way to estimate their success is through means of human evaluation and not automated approaches, despite the abundance of work done in the field. In this paper, we investigate the effectiveness of perceiving dialogue evaluation as an anomaly detection task. The paper looks into four dialogue modeling approaches and how their objective functions correlate with human annotation scores. A high-level perspective exhibits negative results. However, a more in-depth look shows limited potential for using anomaly detection for evaluating dialogues.

OriginalspracheEnglisch
TitelLREC 2020 - 12th International Conference on Language Resources and Evaluation, Conference Proceedings
HerausgeberNicoletta Calzolari, Frederic Bechet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, Helene Mazo, Asuncion Moreno, Jan Odijk, Stelios Piperidis
Anzahl der Seiten5
VerlagEuropean Language Resources Association (ELRA)
Erscheinungsdatum2020
Seiten508-512
ISBN (elektronisch)9791095546344
PublikationsstatusErschienen - 2020
Extern publiziertJa
Veranstaltung12th International Conference on Language Resources and Evaluation, LREC 2020 - Le Palais du Pharao, Marseille, Frankreich
Dauer: 11.05.202016.05.2020
https://lrec2020.lrec-conf.org/en/about/organizers/index.html

Bibliographische Notiz

Publisher Copyright:
© European Language Resources Association (ELRA), licensed under CC-BY-NC

Links

Zuletzt angesehen

Aktivitäten

  1. Draw the line? How boundary creation behaviors at the end of work relate to recovery and next-day work performance
  2. Carbon Dioxide Treatment, Summary and Presentation of the Final Version of the Computerprogram CO2
  3. Experiencing Nature of Science – Discover your own understanding of NOS, mit Kerstin Oschatz
  4. ICSS Group (Externe Organisation)
  5. The systematic approach: Residue management according to ISO 14 001 and EMAS-Scheme of the European Union.
  6. Twitter as a virtual stage. An enactment perspective on co-creative networks
  7. Zootechnologies. A Media History of Swarm Intelligence
  8. The Not-So-Simple Structure of Personality: What is the Impact on Selection Rates?
  9. Student Consumer Learning in Schools‘ „Cultures of Consumption“: Results of an Empirical Study
  10. Gutachter für The World Academy of Sciences for the advancement of science in developing countries (TWAS) der UNESCO
  11. Technological Forecasting and Social Change (Fachzeitschrift)
  12. Organization Studies (Fachzeitschrift)
  13. D-Grid All Hands Meeting - 2007
  14. Unpolitics in the EU: Evidence from various policy areas, states and institutions
  15. Statusreport des Teilprojekts I.2 "Organized Creativity"
  16. Academy of Management Journal (Fachzeitschrift)
  17. Photodegradation and aerobic biodegradability of angiotensin-converting enzyme inhibitor Captopril and identification of transformation products by LC MS/MS.
  18. Videographiebasierte Forschungsmethoden
  19. The Wheel of Writing
  20. Probenahme, Analytik und Qualitätssicherung
  21. Klausurtagung der AG SchriftSPRACHerwerb
  22. Ergebnisse eines Bodenringversuches
  23. Offenheit eher eng gefasst: Open Educational Practices and niedersächsischen Hochschulen
  24. International Pragmatics Conference - IPrA 2017