An Adaptive and Near Parameter-free Evolutionary Computation Approach Towards True Automation in AutoML Benjamin Patrick Evans, Bing Xue, and Mengjie Zhang School of Engineering and Computer Science Victoria University of Wellington New Zealand { benjamin.evans,bing.xue,mengjie.zhang}@ecs.vuw.ac.nz Abstract A common claim of evolutionary computation methods is that they can achieve good results without the need for human intervention. However, one criticism of this is that there are still hyperparameters which must be tuned in order to achieve good performance. In this work, we propose a near "parameter-free" genetic programming approach, which adapts the hyperparameter values throughout evolution without ever needing to be specified manually. We apply this to the area of automated machine learning (by extending TPOT), to produce pipelines which can effectively be claimed to be free from human input, and show that the results are competitive with existing state-of-the-art which use hand-selected hyper-parameter values. Pipelines begin with a randomly chosen estimator and evolve to competitive pipelines automatically. This work moves towards a truly automatic approach to AutoML. 1 Introduction In recent years, machine learning has made its way into many application areas, which has attracted a wide variety of interest from many users from outside the machine learning world. This demand for machine learning has spurred the area of automated machine learning (AutoML), which aims to make machine learning accessible to non-experts [1], or allows experts to focus on other aspects of the machine learning process rather than pipeline design [2]. However, while two of the goals of AutoML are automation and ease of use, most current state-of-the-art methods become a new optimisation problem themselves: rather than searching for pipelines, one must search for appropriate 1 arXiv:2001.10178v1 Granted, this is a simpler search space than the original one, but is still an undesirable property and prevents true human-free automation.

Meet the xenobots: Tiny living robots have been created using cells taken from frog embryos. Each so-called xenobot is less than a millimeter across, but one can propel itself through water using two stumpy limbs, while another has a kind of pouch that it could use to carry a small load. The early research, published in Proceedings of the National Academy of Sciences, could help the development of useful soft robots that can heal themselves when damaged. Because they are made of living tissue, they also decay once they stop working. The researchers, from Tufts University, the University of Vermont, and the Wyss Institute at Harvard, hope such living robots could one day be used to clean up microplastics, digest toxic materials, or even deliver drugs inside our bodies (although this is obviously still all a long way off). The robots are constructed from heart cells, which spontaneously contract and relax like tiny pistons, and skin cells that provide more rigid structure.

Solving many-objective problems (MaOPs) is still a significant challenge in the multi-objective optimization (MOO) field. One way to measure algorithm performance is through the use of benchmark functions (also called test functions or test suites), which are artificial problems with a well-defined mathematical formulation, known solutions and a variety of features and difficulties. In this paper we propose a parameterized generator of scalable and customizable benchmark problems for MaOPs. It is able to generate problems that reproduce features present in other benchmarks and also problems with some new features. We propose here the concept of generative benchmarking, in which one can generate an infinite number of MOO problems, by varying parameters that control specific features that the problem should have: scalability in the number of variables and objectives, bias, deceptiveness, multimodality, robust and non-robust solutions, shape of the Pareto front, and constraints. The proposed Generalized Position-Distance (GPD) tunable benchmark generator uses the position-distance paradigm, a basic approach to building test functions, used in other benchmarks such as Deb, Thiele, Laumanns and Zitzler (DTLZ), Walking Fish Group (WFG) and others. It includes scalable problems in any number of variables and objectives and it presents Pareto fronts with different characteristics. The resulting functions are easy to understand and visualize, easy to implement, fast to compute and their Pareto optimal solutions are known.

Technical singularity is defined as a hypothetical future of superhuman machines with a cognitive capability far beyond the capacity of human minds. In the journey toward this potential technology revolution is something that I have been focused on called artificial swarm intelligence. A starling murmuration, something that people have told me is awe-inspiring, is a marvel of nature similar to an army of ants or a swarm of bees. How do all these individual entities organize around a common mission that includes a form of collaboration and unified orchestration as a team? When thinking about swarms of AI bots or even nanobots, the foundational concept we want to define is what exactly AI bot are.

Nowadays, we are immersed in tens of newly-proposed evolutionary and swam-intelligence metaheuristics, which makes it very difficult to choose a proper one to be applied on a specific optimization problem at hand. On the other hand, most of these metaheuristics are nothing but slightly modified variants of the basic metaheuristics. For example, Differential Evolution (DE) or Shuffled Frog Leaping (SFL) are just Genetic Algorithms (GA) with a specialized operator or an extra local search, respectively. Therefore, what comes to the mind is whether the behavior of such newly-proposed metaheuristics can be investigated on the basis of studying the specifications and characteristics of their ancestors. In this paper, a comprehensive evaluation study on some basic metaheuristics i.e. Genetic Algorithm (GA), Particle Swarm Optimization (PSO), Artificial Bee Colony (ABC), Teaching-Learning-Based Optimization (TLBO), and Cuckoo Optimization algorithm (COA) is conducted, which give us a deeper insight into the performance of them so that we will be able to better estimate the performance and applicability of all other variations originated from them. A large number of experiments have been conducted on 20 different combinatorial optimization benchmark functions with different characteristics, and the results reveal to us some fundamental conclusions besides the following ranking order among these metaheuristics, {ABC, PSO, TLBO, GA, COA} i.e. ABC and COA are the best and the worst methods from the performance point of view, respectively. In addition, from the convergence perspective, PSO and ABC have significant better convergence for unimodal and multimodal functions, respectively, while GA and COA have premature convergence to local optima in many cases needing alternative mutation mechanisms to enhance diversification and global search.

A common problem machine learning developers are faced with is overfitting, that is, fitting a pipeline too closely to the training data that the performance degrades for unseen data. Automated machine learning aims to free (or at least ease) the developer from the burden of pipeline creation, but this overfitting problem can persist. In fact, this can become more of a problem as we look to iteratively optimise the performance of an internal cross-validation (most often \textit{k}-fold). While this internal cross-validation hopes to reduce this overfitting, we show we can still risk overfitting to the particular folds used. In this work, we aim to remedy this problem by introducing dynamic fitness evaluations which approximate repeated \textit{k}-fold cross-validation, at little extra cost over single \textit{k}-fold, and far lower cost than typical repeated \textit{k}-fold. The results show that when time equated, the proposed fitness function results in significant improvement over the current state-of-the-art baseline method which uses an internal single \textit{k}-fold. Furthermore, the proposed extension is very simple to implement on top of existing evolutionary computation methods, and can provide essentially a free boost in generalisation/testing performance.

Recently, the term explainable AI became known as an approach to produce models from artificial intelligence which allow interpretation. Since a long time, there are models of symbolic regression in use that are perfectly explainable and mathematically tractable: in this contribution we demonstrate how to use symbolic regression methods to infer the optimal control of a dynamical system given one or several optimization criteria, or cost functions. In previous publications, network control was achieved by automatized machine learning control using genetic programming. Here, we focus on the subsequent analysis of the analytical expressions which result from the machine learning. In particular, we use AUTO to analyze the stability properties of the controlled oscillator system which served as our model. As a result, we show that there is a considerable advantage of explainable models over less accessible neural networks.

Search-Based Software Engineering (SBSE) is a promising paradigm that exploits computational search to optimize different processes when engineering complex software systems. Self-adaptive system (SAS) is one category of such complex systems that permits to optimize different functional and non-functional objectives/criteria under changing environment (e.g., requirements and workload), which involves problems that are subject to search. In this regard, over years, there have been a considerable amount of work that investigates SBSE for SASs. In this paper, we provide the first systematic and comprehensive survey exclusively on SBSE for SASs, covering 3,740 papers in 27 venues from 7 repositories, which eventually leads to several key statistics from the most notable 73 primary studies in this particular field of research. Our results, surprisingly, have revealed five disappointed issues that are of utmost importance, but have been overwhelmingly ignored in existing studies. We provide evidences to justify our arguments against the disappointments and highlight six emergent, but currently under-explored opportunities for future work on SBSE for SASs. By mitigating the disappointed issues revealed in this work, together with the highlighted opportunities, we hope to be able to excite a much more significant growth on this particular research direction.

There's a dire need to speed the planet's shift to clean energy - and the power of Artificial Intelligence can help. The world has gone through a number of energy transformations – from wood to coal, then to oil, gas and (partly) nuclear. These shifts were gradual and contingent on economic conditions. Now major efforts are under way to reform the global energy sector to make it low-carbon, non-nuclear and climate-compatible. But, unlike the previous transformations, the ongoing restructuring process is driven by elevated awareness of the disastrous consequences of climate change. Notwithstanding the global efforts made to revolutionise the energy business (to make it capable of coping with the variability inherent in most renewable energy generation technologies), there is still a dire need to speed up the shift to clean energy solutions.

Ensemble learning is a powerful paradigm that has been usedin the top state-of-the-art machine learning methods like Random Forestsand XGBoost. Inspired by the success of such methods, we have devel-oped a new Genetic Programming method called Ensemble GP. The evo-lutionary cycle of Ensemble GP follows the same steps as other GeneticProgramming systems, but with differences in the population structure,fitness evaluation and genetic operators. We have tested this method oneight binary classification problems, achieving results significantly betterthan standard GP, with much smaller models. Although other methodslike M3GP and XGBoost were the best overall, Ensemble GP was able toachieve exceptionally good generalization results on a particularly hardproblem where none of the other methods was able to succeed.