HyperUCB: Hyperparameter optimization using contextual bandits

Research output: Contributions to collected editions/worksArticle in conference proceedingsResearchpeer-review

Authors

Setting the optimal hyperparameters of a learning algorithm is a crucial task. Common approaches such as a grid search over the hyperparameter space or randomly sampling hyperparameters require many configurations to be evaluated in order to perform well. Hence, they either yield suboptimal hyperparameter configurations or are expensive in terms of computational resources. As a remedy, Hyperband, an exploratory bandit-based algorithm, introduces an early-stopping strategy to quickly provide competitive configurations given a resource budget which often outperforms Bayesian optimization approaches. However, Hyperband keeps sampling iid configurations for assessment without taking previous evaluations into account. We propose HyperUCB, a UCB extension of Hyperband which assesses the sampled configurations and only evaluates promising samples. We compare our approach on MNIST data against Hyperband and show that we perform better in most cases.

Original languageEnglish
Title of host publicationMachine Learning and Knowledge Discovery in Databases : International Workshops of ECML PKDD 2019, Würzburg, Germany, September 16–20, 2019, Proceedings, Part I
EditorsPeggy Cellier, Kurt Driessens
Number of pages7
Volume1
Place of PublicationCham
PublisherSpringer Nature
Publication date28.03.2020
Pages44-50
ISBN (print)978-3-030-43822-7
ISBN (electronic)978-3-030-43823-4
DOIs
Publication statusPublished - 28.03.2020
Event19th Joint European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases - 2019 - Wurzburg, Germany
Duration: 16.09.201920.09.2019
Conference number: 19
https://ecmlpkdd2019.org/submissions/researchAndADSTrack/

Recently viewed

Publications

  1. Guest editorial
  2. Creativity in Spaces of Possibilities for Sustainable Urban Development
  3. An automated, modular system for organic waste utilization using heterotrophic alga Galdieria sulphuraria
  4. Belief in Free Will Is Related to Internal Attribution in Self-Perception
  5. Biological Computer Laboratory
  6. States and traits
  7. Towards a thick understanding of sustainability transitions - Linking transition management, capabilities and social practices
  8. Response of saproxylic beetles to small-scale habitat connectivity depends on trophic levels
  9. Exemplary versus statistical evidence?
  10. Perceptual latency priming
  11. Hot working parameters and mechanisms in as-cast Mg-3Sn-1Ca alloy
  12. Negotiation complexity
  13. Influence of Long-Lasting Static Stretching on Maximal Strength, Muscle Thickness and Flexibility
  14. Does ESG performance have an impact on financial performance?
  15. Thermal synthesis of a thermochemical heat storage with heat exchanger optimization
  16. Adaptive Environments
  17. Exploring the motivations of protesters in contingent valuation
  18. Reading Marx
  19. Contested Promises
  20. Introduction
  21. On the Power of an Open Scientific Approach to Actions
  22. The Eschatical Perfection of the World in God
  23. Internet of Things-Specific Challenges for Enterprise Architectures
  24. Strangely Familiar
  25. CAN BUSINESS MODEL COMPONENTS EXPLAIN DIGITAL START-UP SUCCESS?
  26. Mindfulness as self-confirmation? An exploratory intervention study on potentials and limitations of mindfulness-based interventions in the context of environmental and sustainability education
  27. Joint production, externalities, and the regulation of production networks
  28. The emotional spectrum in traffic situations: Results of two online-studies
  29. Unsettling bodies of knowledge
  30. Facilitative-competitive interactions in an Old-Growth Forest: The Importance of Large-Diameter Trees as Benefactors and Stimulators for Forest Community Assembly
  31. Eye Movements During Mathematical Word Problem Solving-Global Measures and Individual Differences
  32. A hysteresis hybrid extended kalman filter as an observer for sensorless valve control in camless internal combustion engines