HyperUCB: Hyperparameter optimization using contextual bandits

Research output: Contributions to collected editions/worksArticle in conference proceedingsResearchpeer-review

Standard

HyperUCB: Hyperparameter optimization using contextual bandits. / Tavakol, Maryam; Mair, Sebastian; Morik, Katharina.
Machine Learning and Knowledge Discovery in Databases: International Workshops of ECML PKDD 2019, Würzburg, Germany, September 16–20, 2019, Proceedings, Part I. ed. / Peggy Cellier; Kurt Driessens. Vol. 1 Cham: Springer Nature AG, 2020. p. 44-50 ( Communications in Computer and Information Science; Vol. 1167).

Research output: Contributions to collected editions/worksArticle in conference proceedingsResearchpeer-review

Harvard

Tavakol, M, Mair, S & Morik, K 2020, HyperUCB: Hyperparameter optimization using contextual bandits. in P Cellier & K Driessens (eds), Machine Learning and Knowledge Discovery in Databases: International Workshops of ECML PKDD 2019, Würzburg, Germany, September 16–20, 2019, Proceedings, Part I. vol. 1, Communications in Computer and Information Science, vol. 1167, Springer Nature AG, Cham, pp. 44-50, 19th Joint European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases - 2019, Wurzburg, Germany, 16.09.19. https://doi.org/10.1007/978-3-030-43823-4_4

APA

Tavakol, M., Mair, S., & Morik, K. (2020). HyperUCB: Hyperparameter optimization using contextual bandits. In P. Cellier, & K. Driessens (Eds.), Machine Learning and Knowledge Discovery in Databases: International Workshops of ECML PKDD 2019, Würzburg, Germany, September 16–20, 2019, Proceedings, Part I (Vol. 1, pp. 44-50). ( Communications in Computer and Information Science; Vol. 1167). Springer Nature AG. https://doi.org/10.1007/978-3-030-43823-4_4

Vancouver

Tavakol M, Mair S, Morik K. HyperUCB: Hyperparameter optimization using contextual bandits. In Cellier P, Driessens K, editors, Machine Learning and Knowledge Discovery in Databases: International Workshops of ECML PKDD 2019, Würzburg, Germany, September 16–20, 2019, Proceedings, Part I. Vol. 1. Cham: Springer Nature AG. 2020. p. 44-50. ( Communications in Computer and Information Science). doi: 10.1007/978-3-030-43823-4_4

Bibtex

@inbook{be2d96cb1eb34c8d84ab5c1583ca95ae,
title = "HyperUCB: Hyperparameter optimization using contextual bandits",
abstract = "Setting the optimal hyperparameters of a learning algorithm is a crucial task. Common approaches such as a grid search over the hyperparameter space or randomly sampling hyperparameters require many configurations to be evaluated in order to perform well. Hence, they either yield suboptimal hyperparameter configurations or are expensive in terms of computational resources. As a remedy, Hyperband, an exploratory bandit-based algorithm, introduces an early-stopping strategy to quickly provide competitive configurations given a resource budget which often outperforms Bayesian optimization approaches. However, Hyperband keeps sampling iid configurations for assessment without taking previous evaluations into account. We propose HyperUCB, a UCB extension of Hyperband which assesses the sampled configurations and only evaluates promising samples. We compare our approach on MNIST data against Hyperband and show that we perform better in most cases.",
keywords = "Business informatics, Hyperparameter optimization, Multi-armed bandits",
author = "Maryam Tavakol and Sebastian Mair and Katharina Morik",
year = "2020",
month = mar,
day = "28",
doi = "10.1007/978-3-030-43823-4_4",
language = "English",
isbn = "978-3-030-43822-7",
volume = "1",
series = " Communications in Computer and Information Science",
publisher = "Springer Nature AG",
pages = "44--50",
editor = "Peggy Cellier and Kurt Driessens",
booktitle = "Machine Learning and Knowledge Discovery in Databases",
address = "Germany",
note = "19th Joint European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases - 2019, 19th ECML PKDD - 2019 ; Conference date: 16-09-2019 Through 20-09-2019",
url = "https://ecmlpkdd2019.org/submissions/researchAndADSTrack/",

}

RIS

TY - CHAP

T1 - HyperUCB

T2 - 19th Joint European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases - 2019

AU - Tavakol, Maryam

AU - Mair, Sebastian

AU - Morik, Katharina

N1 - Conference code: 19

PY - 2020/3/28

Y1 - 2020/3/28

N2 - Setting the optimal hyperparameters of a learning algorithm is a crucial task. Common approaches such as a grid search over the hyperparameter space or randomly sampling hyperparameters require many configurations to be evaluated in order to perform well. Hence, they either yield suboptimal hyperparameter configurations or are expensive in terms of computational resources. As a remedy, Hyperband, an exploratory bandit-based algorithm, introduces an early-stopping strategy to quickly provide competitive configurations given a resource budget which often outperforms Bayesian optimization approaches. However, Hyperband keeps sampling iid configurations for assessment without taking previous evaluations into account. We propose HyperUCB, a UCB extension of Hyperband which assesses the sampled configurations and only evaluates promising samples. We compare our approach on MNIST data against Hyperband and show that we perform better in most cases.

AB - Setting the optimal hyperparameters of a learning algorithm is a crucial task. Common approaches such as a grid search over the hyperparameter space or randomly sampling hyperparameters require many configurations to be evaluated in order to perform well. Hence, they either yield suboptimal hyperparameter configurations or are expensive in terms of computational resources. As a remedy, Hyperband, an exploratory bandit-based algorithm, introduces an early-stopping strategy to quickly provide competitive configurations given a resource budget which often outperforms Bayesian optimization approaches. However, Hyperband keeps sampling iid configurations for assessment without taking previous evaluations into account. We propose HyperUCB, a UCB extension of Hyperband which assesses the sampled configurations and only evaluates promising samples. We compare our approach on MNIST data against Hyperband and show that we perform better in most cases.

KW - Business informatics

KW - Hyperparameter optimization

KW - Multi-armed bandits

UR - http://www.scopus.com/inward/record.url?scp=85083719265&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-43823-4_4

DO - 10.1007/978-3-030-43823-4_4

M3 - Article in conference proceedings

AN - SCOPUS:85083719265

SN - 978-3-030-43822-7

VL - 1

T3 - Communications in Computer and Information Science

SP - 44

EP - 50

BT - Machine Learning and Knowledge Discovery in Databases

A2 - Cellier, Peggy

A2 - Driessens, Kurt

PB - Springer Nature AG

CY - Cham

Y2 - 16 September 2019 through 20 September 2019

ER -