Title: FunBO: Discovering Acquisition Functions for Bayesian Optimization with FunSearch
Speaker: Virginia Aglietti (https://virgiagl.github.io/)
Abstract:
The sample efficiency of Bayesian optimization algorithms depends on carefully crafted acquisition functions (AFs) guiding the sequential collection of function evaluations. The best-performing AF can vary significantly across optimization problems, often requiring ad-hoc and problem-specific choices. Defining methodologies that automatically identify new AFs capable of outperforming general-purpose and function-specific alternatives, both in and out of the training distribution, remains a significant and unaddressed challenge. In this talk I will formulate the problem of learning novel AFs as an algorithm discovery problem and address it by extending FunSearch, a recently proposed algorithm that uses LLMs to solve open problems in mathematical sciences. I’ll introduce FunBO, a novel method that explores the space of AFs written in computer code by leveraging access to a limited number of evaluations for a set of objective functions. I will show how FunBO identifies AFs that generalize well in and out of the training distribution of functions, thus outperforming established general-purpose AFs and achieving competitive performance against AFs that are customized to specific function types and are learned via transfer-learning algorithms.