A versatile surrogate-assisted memetic algorithm for optimization of computationally expensive functions and its engineering applications

A core element of modern engineering is optimization with computationally expensive 'black-box' functions. We identify three open critical issues in optimization of such functions: (a) how to generate accurate surrogate-models (b) how to handle points where the simulation fails to converge and (c) how to balance exploration vs. exploitation. In this work we propose a novel surrogate-assisted memetic algorithm which addresses these issues and analyze its performance. The proposed algorithm contains four novelties: (a) it autonomously generates an accurate surrogate-models using multiple cross-validation tests (b) it uses interpolation to generate a global penalty function which biases the global search away from points where the simulation did not converge (c) it uses a method based on the Metropolis criterion from statistical physics to manage exploration vs. exploitation and (d) it combines an EA with the trust-region derivative-free algorithm. The proposed algorithm generates global surrogate-models of the expensive objective function and searches for an optimum using an EA; it then uses a trust-region local-search which combines local quadratic surrogate-models and sequential quadratic programming with the trust-region framework, and converges to an accurate optimum of the true objective function. While the global-local search approach has been studied previously, the improved performance in our algorithm is due to the four novelties just mentioned. We present a detailed performance analysis, in which the proposed algorithm significantly outperformed four variants of a real-coded EA on three difficult optimization problems which include mathematical test functions with a complicated landscape, discontinuities and a tight limit of only 100 function evaluations. The proposed algorithm was also benchmarked against a recent memetic algorithm and a recent EA and significantly outperformed both, showing it is competitive with some of the best available algorithms. Lastly, the proposed algorithm was also applied to a difficult real-world engineering problem of airfoil shape optimization; also in this experiment it consistently performed well, and in spite of a many points where the simulation did not converge, and the tight limit on function evaluations it obtained an airfoil which satisfies the requirements very well. Overall, the proposed algorithm offers a solution to important open issues in optimization of computationally-expensive black-box functions: generating accurate surrogate-models, handling points where the simulation does not converge and managing exploration-exploitation to improve the optimization search when only a small number of function evaluations are allowed, and it efficiently obtains a good approximation to an optimum of such functions in difficult real-world optimization problems.