A memetic algorithm using a trust-region derivative-free optimization with quadratic modelling for optimization of expensive and noisy black-box functions

A novel algorithm integrates evolutionary optimization, clustering, and the trust-region derivative-free optimization framework for global minimization of black-box functions whose evaluation is computationally resource intensive and where uncertainty exist in the objective function value, i.e. the latter contains noise. On the global scale the EA efficiently explores the search space; no global model of the objective function is generated. On the local scale the objective function is modeled by a series of quadratic models which are checked for agreement with the objective function and are updated if necessary. The algorithm incorporates numerous new techniques to enhance both its global and its local search stages. The performance of the algorithm was evaluated by using functions of dimension 2-20, with and without noise. The algorithm performed well; its performance in the presence of noise in the objective function is attributed both to the mild effect of noise on the evolutionary algorithm and to mechanics of the trust-region algorithm. The latter uses quadratic models and an interpolation technique which generates spatially spaced points; both of these diminish the effect of noise in derivatives-based trust-region minimization. Accordingly, the memetic algorithm presented here efficiently minimized black-box functions with up to 20 variables which also contain noise in the objective function value.