Towards robust model selection using estimation and approximation error bounds

One of the main problems in machine learning and statistical inference is selecting an appropriate model by which a set of data can be explained. A novel model selection criterion based on the uniform convergence of empirical processes combined with the results concerning the approximation ability of non-linear manifolds of functions is introduced. A coherent and robust framework for model selection was elucidated and a lower bound on the approximation error was established, giving a well specified sense for most functions of interest.