Toward supervised anomaly detection

Research output: Journal contributionsJournal articlesResearchpeer-review

Authors

Anomaly detection is being regarded as an unsupervised learning task as anomalies stem from adversarial or unlikely events with unknown distributions. However, the predictive performance of purely unsupervised anomaly detection often fails to match the required detection rates in many tasks and there exists a need for labeled data to guide the model generation. Our first contribution shows that classical semi-supervised approaches, originating from a supervised classifier, are inappropriate and hardly detect new and unknown anomalies. We argue that semi-supervised anomaly detection needs to ground on the unsupervised learning paradigm and devise a novel algorithm that meets this requirement. Although being intrinsically non-convex, we further show that the optimization problem has a convex equivalent under relatively mild assumptions. Additionally, we propose an active learning strategy to automatically filter candidates for labeling. In an empirical study on network intrusion detection data, we observe that the proposed learning methodology requires much less labeled data than the state-of-the-art, while achieving higher detection accuracies.

Original languageEnglish
JournalJournal of Artificial Intelligence Research
Volume46
Pages (from-to)235-262
Number of pages28
ISSN1076-9757
DOIs
Publication statusPublished - 20.02.2013
Externally publishedYes

    Research areas

  • Informatics - learning strategies, Detection accuracy, Empirical studies, Network intrusion detection, Optimization problems, redictive performance, Supervised classifiers, Unsupervised anomaly detection
  • Business informatics

DOI

Recently viewed

Activities

  1. Being there? The social construction of (physical and virtual) copresence
  2. Fracking comes to Germany: Interpreting societal conflicts, the politics of expertise and strategies for conflict resolution in a federal system: with Basil Bornemann
  3. Workshop: Windkanäle - 2019
  4. Leuphana Universität Lüneburg (Organisation)
  5. ECPR Joint Sessions, Workshop No. 6: ‚Comparative Perspectives on Local Party Politics’ 2007
  6. From Pity to Control: Regulated Humanitarianism in News Coverage on Refugees and Asylum
  7. Vortrag: Groteske Figurationen Europas
  8. Veranstaltungsreihe "Brown Bag Lectures" am Institute of English Studies
  9. Animal Behaviour (Zeitschrift)
  10. Veranstaltungsreihe "Brown Bag Lectures" am Institute of English Studies
  11. Conference presentation: Integrated Reporting: Current state of empirical research, limitations and future research implications
  12. Whole Person Development – Building Students as Academic Leaders
  13. “...and then the students wäre very fett”:Task performance and ‘(not)-doing-being-an-expert’ in a TBLL setting
  14. 1. Integrationskonferenz "Werkstatt Sprache" 2016
  15. SASE 2017
  16. Trap-nesting insects´ interactions in an urban-rural gradient
  17. Invitation-only expert seminar
  18. Digital Badges zur Optimierung der Studieneingangsphase - Anwendungsbeispiel für die Hochschullehre
  19. Key Lecture: "Human Empowerment as an Analytical Framework to Study Social and Political Change"
  20. Verbraucherforschungsforum 2017
  21. Abfallvermeidungs- Dialoge 4/5 - 2015
  22. Kunst & Vermittlung #13
  23. Leuphana Universität Lüneburg (Organisation)
  24. Forschungskompetenz von (Lehramts-) Studierenden in der Studieneingangsphase
  25. Keynote: "From Sacrificing Life to Living It: Moral Evolution and the Emerging Mass Basis of Global Peace"
  26. Wellness as a Hybrid Phenomenon
  27. 77th Annual Meeting of the Academy of Management - AOM 2017 (Veranstaltung)
  28. Leuphana Universität Lüneburg (Organisation)
  29. Jahrestagung des Arbeitskreises Parteienforschung 2016
  30. Nachhaltiges Privatrecht
  31. Agrobiodiversity and culture: an analysis from place and agroecosystems in the Tunjuelo river
  32. Enquiry-based Science Education in Austrian Teacher Professional Development Courses