An Entropy Search Portfolio for Bayesian Optimization

Bobak Shahriari, Ziyu Wang, Matthew W. Hoffman, Alexandre Bouchard-Côté, Nando de Freitas

Published 2014-06-18, updated 2014-10-27Version 2

Portfolio methods provide an effective, principled way of combining a collection of acquisition functions in the context of Bayesian optimization. We introduce a novel approach to this problem motivated by an information theoretic consideration. Our construction additionally provides an extension of Thompson sampling to continuous domains with GP priors. We show that our method outperforms a range of other portfolio methods on several synthetic problems, automated machine learning tasks, and a simulated control task. Finally, the effectiveness of even the random portfolio strategy suggests that portfolios in general should play a more pivotal role in Bayesian optimization.