The internal structure of the model depends on the type of the There is a variety of approaches that can be used to learning algorithm, so complex data-driven models can consist identify the optimal design of the data-driven model. For of several semi-independent blocks - this approach is usually instance, AutoML solutions can be based on random search referred to as ensembling . There are several techniques to , Bayesian optimisation , reinforcement learning (RL) build complex models: for example, blending allows creating , Monte Carlo tree search , sequential model-based single-level ensembles of machine learning (ML) models, and optimization , gradient-based approaches . However, stacking allows creating multi-level ones. Other approaches are most of them are less flexible than evolutionary approaches to based on the representation of a model structure (or even the the model design (implemented e.g. in ). Their conceptual whole modeling pipeline) as a directed acyclic graph (DAG).
Scalarizing functions have been widely used to convert a multiobjective optimization problem into a single objective optimization problem. However, their use in solving (computationally) expensive multi- and many-objective optimization problems in Bayesian multiobjective optimization is scarce. Scalarizing functions can play a crucial role on the quality and number of evaluations required when doing the optimization. In this article, we study and review 15 different scalarizing functions in the framework of Bayesian multiobjective optimization and build Gaussian process models (as surrogates, metamodels or emulators) on them. We use expected improvement as infill criterion (or acquisition function) to update the models. In particular, we compare different scalarizing functions and analyze their performance on several benchmark problems with different number of objectives to be optimized. The review and experiments on different functions provide useful insights when using and selecting a scalarizing function when using a Bayesian multiobjective optimization method.
Highly non-linear machine learning algorithms have the capacity to handle large, complex datasets. However, the predictive performance of a model usually critically depends on the choice of multiple hyperparameters. Optimizing these (often) constitutes an expensive black-box problem. Model-based optimization is one state-of-the-art method to address this problem. Furthermore, resulting models often lack interpretability, as models usually contain many active features with non-linear effects and higher-order interactions. One model-agnostic way to enhance interpretability is to enforce sparse solutions through feature selection. It is in many applications desirable to forego a small drop in performance for a substantial gain in sparseness, leading to a natural treatment of feature selection as a multi-objective optimization task. Despite the practical relevance of both hyperparameter optimization and feature selection, they are often carried out separately from each other, which is neither efficient, nor does it take possible interactions between hyperparameters and selected features into account. We present, discuss and compare two algorithmically different approaches for joint and multi-objective hyperparameter optimization and feature selection: The first uses multi-objective model-based optimization to tune a feature filter ensemble. The second is an evolutionary NSGA-II-based wrapper-approach to feature selection which incorporates specialized sampling, mutation and recombination operators for the joint decision space of included features and hyperparameter settings. We compare and discuss the approaches on a variety of benchmark tasks. While model-based optimization needs fewer objective evaluations to achieve good performance, it incurs significant overhead compared to the NSGA-II-based approach. The preferred choice depends on the cost of training the ML model on the given data.
This paper deals with these problems by using a new decomposition-based algorithm called: "Fractal geometric decomposition base algorithm" (FDA). It is a deterministic metaheuristic developed to solve large-scale continuous optimization problems . It can be noticed, that we call large scale problems those having the dimension greater than 1000. In this research, we are interested in using FDA to deal with MOPs because in the literature decomposition based algorithms have been with more less success applied to solve these problems, their main problem is related to their complexity. In this work, the goal is to deal with this complexity problem by keeping the same level of efficiency. FDA is based on "divide-and-conquer" paradigm where the sub-regions are hyperspheres rather than hypercubes on classical approaches. In order to identify the Pareto optimal solutions, we propose to extend FDA using the scalarization approach. We called the proposed algorithm Mo-FDA.
Soil characteristics are extremely important when determining yield potential. Fertilization and liming are commonly used to adapt soils to the nutritional requirements of the crops to be cultivated. Planting the crop that will best fit the soil characteristics is an interesting alternative to minimize the need for soil treatment, reducing costs and potential environmental damages. In addition, farmers usually look for investments that offer the greatest potential earnings with the least possible risks. Regarding the objectives to be considered, the crop-selection problem may be difficult to solve using traditional tools.