Toward supervised anomaly detection

Research output: Journal contributionsJournal articlesResearchpeer-review

Authors

Anomaly detection is being regarded as an unsupervised learning task as anomalies stem from adversarial or unlikely events with unknown distributions. However, the predictive performance of purely unsupervised anomaly detection often fails to match the required detection rates in many tasks and there exists a need for labeled data to guide the model generation. Our first contribution shows that classical semi-supervised approaches, originating from a supervised classifier, are inappropriate and hardly detect new and unknown anomalies. We argue that semi-supervised anomaly detection needs to ground on the unsupervised learning paradigm and devise a novel algorithm that meets this requirement. Although being intrinsically non-convex, we further show that the optimization problem has a convex equivalent under relatively mild assumptions. Additionally, we propose an active learning strategy to automatically filter candidates for labeling. In an empirical study on network intrusion detection data, we observe that the proposed learning methodology requires much less labeled data than the state-of-the-art, while achieving higher detection accuracies.

Original languageEnglish
JournalJournal of Artificial Intelligence Research
Volume46
Pages (from-to)235-262
Number of pages28
ISSN1076-9757
DOIs
Publication statusPublished - 20.02.2013
Externally publishedYes

    Research areas

  • Informatics - learning strategies, Detection accuracy, Empirical studies, Network intrusion detection, Optimization problems, redictive performance, Supervised classifiers, Unsupervised anomaly detection
  • Business informatics

DOI

Recently viewed

Publications

  1. Who commits virtual identity suicide? Differences in privacy concerns, internet addiction, and personality between facebook users and quitters
  2. Stakeholder Governance – An analysis of BITC Corporate Responsibility Index Data on Stakeholder Engagement and Governance
  3. Kickback Payments under MiFID:
  4. Process limits of extrusion of multimaterial components
  5. Trust Centrality in Online Social Networks
  6. Bush encroachment control and risk management in semi-arid rangelands
  7. Visualizing stakeholders’ willingness for collective action in participatory scenario planning
  8. Integration of Material Flow Management into Company Processes within the Automotive Industry
  9. Supportive Mental Health Self-Monitoring among Smartphone Users with Psychological Distress
  10. The emotional spectrum in traffic situations: Results of two online-studies
  11. Notting Hill Gate 4 Basic
  12. The distribution of power within the community
  13. Mediengenealogie
  14. How much does agriculture depend on pollinators?
  15. Motivation related to work
  16. Inhibition of foam cell formation using a soluble CD68-Fc fusion protein
  17. Der Raum des Cyberspace
  18. Mad speculation and absolute inhumanism
  19. Every single word
  20. Analytical model to determine the strength of form-fit connection joined by die-less hydroforming
  21. Attention on the source of influence reverses the impact of cross-contextual imitation
  22. Transcending the transmission model
  23. Einleitung
  24. Methan - das unterschätzte Klimagas
  25. Rats dying for mice: Modelling the competitor release effect
  26. § 23 Wasserkraft
  27. Linked Data-driven Resilience Research 2023
  28. Community awareness and engagement for arsenic management
  29. The application of environmental ethics in biological conservation