This book brings together - in an informal and tutorial fashion - the computer techniques, mathematical tools, and research results that will enable both students and practitioners to apply genetic algorithms to problems in many fields. Major concepts are illustrated with running examples, and major algorithms are illustrated by Pascal computer programs. No prior knowledge of GAs or genetics is assumed, and only a minimum of computer programming and mathematics background is required.
With this approach, candidate solutions to an optimization problem are randomly generated and act as individuals interacting with a larger population. A fitness function determines the quality of the solutions the candidates find as they move about in each iteration. The "best fit" individuals are then chosen for reproduction in the next iteration. This generational process is repeated until the algorithm has evolved to find the optimal solution to the problem.
Genetic algorithms (GA) are a family of heuristics which are empirically good at providing a decent answer in many cases, although they are rarely the best option for a given domain. You mention derivative-based algorithms, but even in the absence of derivatives there are plenty of derivative-free optimization algorithms that perform way better than GAs. See this and this answer for some ideas. What many standard optimization algorithms have in common (even derivative-free methods) is the assumption that the underlying space is a smooth manifold (perhaps with a few discrete dimensions), and the function to optimize is somewhat well-behaved. However, not all functions are defined on a smooth manifold.
Highly non-linear machine learning algorithms have the capacity to handle large, complex datasets. However, the predictive performance of a model usually critically depends on the choice of multiple hyperparameters. Optimizing these (often) constitutes an expensive black-box problem. Model-based optimization is one state-of-the-art method to address this problem. Furthermore, resulting models often lack interpretability, as models usually contain many active features with non-linear effects and higher-order interactions. One model-agnostic way to enhance interpretability is to enforce sparse solutions through feature selection. It is in many applications desirable to forego a small drop in performance for a substantial gain in sparseness, leading to a natural treatment of feature selection as a multi-objective optimization task. Despite the practical relevance of both hyperparameter optimization and feature selection, they are often carried out separately from each other, which is neither efficient, nor does it take possible interactions between hyperparameters and selected features into account. We present, discuss and compare two algorithmically different approaches for joint and multi-objective hyperparameter optimization and feature selection: The first uses multi-objective model-based optimization to tune a feature filter ensemble. The second is an evolutionary NSGA-II-based wrapper-approach to feature selection which incorporates specialized sampling, mutation and recombination operators for the joint decision space of included features and hyperparameter settings. We compare and discuss the approaches on a variety of benchmark tasks. While model-based optimization needs fewer objective evaluations to achieve good performance, it incurs significant overhead compared to the NSGA-II-based approach. The preferred choice depends on the cost of training the ML model on the given data.
What is happening in the field of evolutionary and genetic algorithms today? Are there any cutting edge scientific projects in terms of AI/AGI? I'd very much appreciate it if someone could link me the relevant websites, researches, papers regarding the subject along with respective books or monographs. I'm just trying to find things out and getting back on track.