Collaborating Authors

Evolutionary Systems: Instructional Materials

C++ Machine Learning Algorithms Inspired by Nature


This online course is for students and software developers who want to level up their skills by learning interesting optimization algorithms in C . You will learn some of the most famous AI algorithms by writing it in C from scratch, so we will not use any libraries. We will start with the Genetic Algorithm (GA), continue with Simulated Annealing (SA) and then touch on a less known one: Differential Evolution. Finally, we will look at Ant Colony Optimization (ACO). The Genetic Algorithm is the most famous one in a class called metaheuristics or optimization algorithms. You will learn what optimization algorithms are, when to use them, and then you will solve two problems with the Genetic Algorithm(GA).

Art in the Age of Machine Learning


An examination of machine learning art and its practice in new media art and music.Over the past decade, an artistic movement has emerged that draws on machine learning as both inspiration and medium. In this book, transdisciplinary artist-researcher Sofian Audry examines artistic practices at the intersection of machine learning and new media art, providing conceptual tools and historical perspectives for new media artists, musicians, composers, writers, curators, and theorists. Audry looks at works from a broad range of practices, including new media installation, robotic art, visual art, electronic music and sound, and electronic literature, connecting machine learning art to such earlier artistic practices as cybernetics art, artificial life art, and evolutionary art. Machine learning underlies computational systems that are biologically inspired, statistically driven, agent-based networked entities that program themselves. Audry explains the fundamental design of machine learning algorithmic structures in terms accessible to the nonspecialist while framing these technologies within larger historical and conceptual spaces. Audry debunks myths about machine learning art, including the ideas that machine learning can create art without artists and that machine learning will soon bring about superhuman intelligence and creativity. Audry considers learning procedures, describing how artists hijack the training process by playing with evaluative functions; discusses trainable machines and models, explaining how different types of machine learning systems enable different kinds of artistic practices; and reviews the role of data in machine learning art, showing how artists use data as a raw material to steer learning systems and arguing that machine learning allows for novel forms of algorithmic remixes.

The Weights can be Harmful: Pareto Search versus Weighted Search in Multi-Objective Search-Based Software Engineering Artificial Intelligence

In presence of multiple objectives to be optimized in Search-Based Software Engineering (SBSE), Pareto search has been commonly adopted. It searches for a good approximation of the problem's Pareto optimal solutions, from which the stakeholders choose the most preferred solution according to their preferences. However, when clear preferences of the stakeholders (e.g., a set of weights which reflect relative importance between objectives) are available prior to the search, weighted search is believed to be the first choice since it simplifies the search via converting the original multi-objective problem into a single-objective one and enable the search to focus on what only the stakeholders are interested in. This paper questions such a "weighted search first" belief. We show that the weights can, in fact, be harmful to the search process even in the presence of clear preferences. Specifically, we conduct a large scale empirical study which consists of 38 systems/projects from three representative SBSE problems, together with two types of search budget and nine sets of weights, leading to 604 cases of comparisons. Our key finding is that weighted search reaches a certain level of solution quality by consuming relatively less resources at the early stage of the search; however, Pareto search is at the majority of the time (up to 77% of the cases) significantly better than its weighted counterpart, as long as we allow a sufficient, but not unrealistic search budget. This, together with other findings and actionable suggestions in the paper, allows us to codify pragmatic and comprehensive guidance on choosing weighted and Pareto search for SBSE under the circumstance that clear preferences are available. All code and data can be accessed at:

Stop Oversampling for Class Imbalance Learning: A Critical Review Artificial Intelligence

For the last two decades, oversampling has been employed to overcome the challenge of learning from imbalanced datasets. Many approaches to solving this challenge have been offered in the literature. Oversampling, on the other hand, is a concern. That is, models trained on fictitious data may fail spectacularly when put to real-world problems. The fundamental difficulty with oversampling approaches is that, given a real-life population, the synthesized samples may not truly belong to the minority class. As a result, training a classifier on these samples while pretending they represent minority may result in incorrect predictions when the model is used in the real world. We analyzed a large number of oversampling methods in this paper and devised a new oversampling evaluation system based on hiding a number of majority examples and comparing them to those generated by the oversampling process. Based on our evaluation system, we ranked all these methods based on their incorrectly generated examples for comparison. Our experiments using more than 70 oversampling methods and three imbalanced real-world datasets reveal that all oversampling methods studied generate minority samples that are most likely to be majority. Given data and methods in hand, we argue that oversampling in its current forms and methodologies is unreliable for learning from class imbalanced data and should be avoided in real-world applications.

Separating Rule Discovery and Global Solution Composition in a Learning Classifier System Artificial Intelligence

The utilization of digital agents to support crucial decision making is increasing in many industrial scenarios. However, trust in suggestions made by these agents is hard to achieve, though essential for profiting from their application, resulting in a need for explanations for both the decision making process as well as the model itself. For many systems, such as common deep learning black-box models, achieving at least some explainability requires complex post-processing, while other systems profit from being, to a reasonable extent, inherently interpretable. In this paper we propose an easily interpretable rule-based learning system specifically designed and thus especially suited for these scenarios and compare it on a set of regression problems against XCSF, a prominent rule-based learning system with a long research history. One key advantage of our system is that the rules' conditions and which rules compose a solution to the problem are evolved separately. We utilise independent rule fitnesses which allows users to specifically tailor their model structure to fit the given requirements for explainability. We find that the results of SupRB2's evaluation are comparable to XCSF's while allowing easier control of model structure and showing a substantially smaller sensitivity to random seeds and data splits. This increased control aids in subsequently providing explanations for both the training and the final structure of the model.

The First AI4TSP Competition: Learning to Solve Stochastic Routing Problems Artificial Intelligence

The TSP is one of the classical combinatorial optimization problems, with many variants inspired by real-world applications. This first competition asked the participants to develop algorithms to solve a time-dependent orienteering problem with stochastic weights and time windows (TD-OPSWTW). It focused on two types of learning approaches: surrogate-based optimization and deep reinforcement learning. In this paper, we describe the problem, the setup of the competition, the winning methods, and give an overview of the results. The winning methods described in this work have advanced the state-of-the-art in using AI for stochastic routing problems. Overall, by organizing this competition we have introduced routing problems as an interesting problem setting for AI researchers. The simulator of the problem has been made open-source and can be used by other researchers as a benchmark for new AI methods.

Online AutoML: An adaptive AutoML framework for online learning Artificial Intelligence

Automated Machine Learning (AutoML) has been used successfully in settings where the learning task is assumed to be static. In many real-world scenarios, however, the data distribution will evolve over time, and it is yet to be shown whether AutoML techniques can effectively design online pipelines in dynamic environments. This study aims to automate pipeline design for online learning while continuously adapting to data drift. For this purpose, we design an adaptive Online Automated Machine Learning (OAML) system, searching the complete pipeline configuration space of online learners, including preprocessing algorithms and ensembling techniques. This system combines the inherent adaptation capabilities of online learners with the fast automated pipeline (re)optimization capabilities of AutoML. Focusing on optimization techniques that can adapt to evolving objectives, we evaluate asynchronous genetic programming and asynchronous successive halving to optimize these pipelines continually. We experiment on real and artificial data streams with varying types of concept drift to test the performance and adaptation capabilities of the proposed system. The results confirm the utility of OAML over popular online learning algorithms and underscore the benefits of continuous pipeline redesign in the presence of data drift.

Differential evolution outside the box Artificial Intelligence

Consequently, any optimisation algorithm, including nonlinear optimisation heuristics, should be able to deal with such constraints by means of a constraint handling method. Such a method deals with infeasible solution (IS) candidates x R D by means of a suitable approach, involving concepts such as, e.g., ignoring or repairing them. In nonlinear optimisation heuristics inspired by nature, the infeasible components of a solution are generated by the mutation operator, which is expected to help explore regions of the search space outside the scope of the crossover operator and then converge towards solution candidates for which f is minimised or maximised. Intuitively, this search process is disrupted and thus lacks the ability to adapt itself to the properties of the objective function f when it generates many infeasible solutions during the course of the search. In this paper, we present an empirical investigation of the proportion of infeasible solutions generated for various variants and parameter settings of Differential Evolution. The algorithm variants under consideration are introduced in Section 2 while the adopted methods of dealing with generated infeasible solutions, as well as the experimental setup, are introduced in Section 3. The results are discussed in Section 4 and conclusions are drawn in Section 5. 2. Differential evolution Originally intended for a simple fitting problem [36, 31], Differential Evolution (DE) has soon become an established metaheuristic method for general-purpose real-valued optimisation, finding its place among other optimisation methods for real-world applications in engineering, robotics and other fields [35, 30, 41]. Besides the effectiveness of the DE optimisation framework, its success is attributed to the simplicity of its algorithmic structure. As can be seen from the pseudocode in Algorithm 1, it requires tuning only three parameters: the population size N (i.e., number of candidate solutions), the scaling factor F (i.e., a prefixed scalar multiplier in the range p0,2s involved in the mutation process) and the crossover rate C

Time Series Forecasting Using Fuzzy Cognitive Maps: A Survey Artificial Intelligence

Increasing complexity comes from some factors including uncertainty, ambiguity, inconsistency, multiple dimensionalities, increasing the number of effective factors and relation between them. Some of these features are common among most real-world problems which are considered complex and dynamic problems. In other words, since the data and relations in real world applications are usually highly complex and inaccurate, modeling real complex systems based on observed data is a challenging task especially for large scale, inaccurate and non stationary datasets. Therefore, to cover and address these difficulties, the existence of a computational system with the capability of extracting knowledge from the complex system with the ability to simulate its behavior is essential. In other words, it is needed to find a robust approach and solution to handle real complex problems in an easy and meaningful way [1]. Hard computing methods depend on quantitative values with expensive solutions and lack of ability to represent the problem in real life due to some uncertainties. In contrast, soft computing approaches act as alternative tools to deal with the reasoning of complex problems [2]. Using soft computing methods such as fuzzy logic, neural network, genetic algorithms or a combination of these allows achieving robustness, tractable and more practical solutions. Generally, two types of methods are used for analyzing and modeling dynamic systems including quantitative and qualitative approaches.

Forecasting: theory and practice Machine Learning

Forecasting has always been at the forefront of decision making and planning. The uncertainty that surrounds the future is both exciting and challenging, with individuals and organisations seeking to minimise risks and maximise utilities. The large number of forecasting applications calls for a diverse set of forecasting methods to tackle real-life challenges. This article provides a non-systematic review of the theory and the practice of forecasting. We provide an overview of a wide range of theoretical, state-of-the-art models, methods, principles, and approaches to prepare, produce, organise, and evaluate forecasts. We then demonstrate how such theoretical concepts are applied in a variety of real-life contexts. We do not claim that this review is an exhaustive list of methods and applications. However, we wish that our encyclopedic presentation will offer a point of reference for the rich work that has been undertaken over the last decades, with some key insights for the future of forecasting theory and practice. Given its encyclopedic nature, the intended mode of reading is non-linear. We offer cross-references to allow the readers to navigate through the various topics. We complement the theoretical concepts and applications covered by large lists of free or open-source software implementations and publicly-available databases.