Collaborating Authors


Bayesian Hyperparameter Optimization with tune-sklearn in PyCaret


In this post, I will show you how easy it is to use other state-of-the-art algorithms with PyCaret thanks to tune-sklearn, a drop-in replacement for scikit-learn's model selection module with cutting edge hyperparameter tuning techniques. I'll also report results from a series of benchmarks, showing how tune-sklearn is able to easily improve classification model performance. Hyperparameter optimization algorithms can vary greatly in efficiency. Random search has been a machine learning staple and for a good reason: it's easy to implement, understand and gives good results in reasonable time. However, as the name implies, it is completely random -- a lot of time can be spent on evaluating bad configurations.

An Introduction to Problem-Solving using Search Algorithms for Beginners


In computer science, problem-solving refers to artificial intelligence techniques, including various techniques such as forming efficient algorithms, heuristics, and performing root cause analysis to find desirable solutions. The basic crux of artificial intelligence is to solve problems just like humans. In today's fast-paced digitized world, artificial intelligence techniques are used widely to automate systems that can use the resource and time efficiently. Some of the well-known problems experienced in everyday life are games and puzzles. Using AI techniques, we can solve these problems efficiently. In artificial intelligence, problems can be solved by using searching algorithms, evolutionary computations, knowledge representations, etc.

Hyperparameter Tuning with Grid Search and Random Search


Hyperparameters are parameters that are defined before training to specify how we want model training to happen. We have full control over hyperparameter settings and by doing that we control the learning process. It can be set to any integer value but of course, setting it to 10 or 1000 changes the learning process significantly. Parameters, on the other hand, are found during the training. We have no control over parameter values as they are the result of model training.

Half a Dozen Real-World Applications of Evolutionary Multitasking and More Artificial Intelligence

Until recently, the potential to transfer evolved skills across distinct optimization problem instances (or tasks) was seldom explored in evolutionary computation. The concept of evolutionary multitasking (EMT) fills this gap. It unlocks a population's implicit parallelism to jointly solve a set of tasks, hence creating avenues for skills transfer between them. Despite it being early days, the idea of EMT has begun to show promise in a range of real-world applications. In the backdrop of recent advances, the contribution of this paper is twofold. We first present a review of several application-oriented explorations of EMT in the literature, assimilating them into half a dozen broad categories according to their respective application areas. Within each category, the fundamental motivations for multitasking are discussed, together with an illustrative case study. Second, we present a set of recipes by which general problem formulations of practical interest, those that cut across different disciplines, could be transformed in the new light of EMT. We intend our discussions to not only underscore the practical utility of existing EMT methods, but also spark future research toward novel algorithms crafted for real-world deployment.

A dynamic programming algorithm for informative measurements and near-optimal path-planning Artificial Intelligence

Observing the outcomes of a sequence of measurements usually increases our knowledge about the state of a particular system we might be interested in. An informative measurement is the most efficient way of gaining this information, having the largest possible statistical dependence between the state being measured and the observed measurement outcome. Lindley first introduced the notion of the amount of information in an experiment, and suggested the following greedy rule for experimentation: perform that experiment for which the expected gain in information is the greatest, and continue experimentation until a preassigned amount of information has been attained [Lindley, 1955]. Greedy methods are still the most common approaches for finding informative measurements, being both simple to implement and efficient to compute. For example, in a weighing problem where an experimenter has a two-pan balance and is given a set of balls of equal weight except for a single odd ball that is heavier or lighter than the others (see Figure 1), the experimenter would like to find the odd ball in the fewest weighings. MacKay suggested that for useful information to be gained as quickly as possible, each stage of an optimal measurement sequence should have measurement outcomes as close as possible to equiprobable [MacKay, 2003].

Chess AI: Competing Paradigms for Machine Intelligence Artificial Intelligence

Endgame studies have long served as a tool for testing human creativity and intelligence. We find that they can serve as a tool for testing machine ability as well. Two of the leading chess engines, Stockfish and Leela Chess Zero (LCZero), employ significantly different methods during play. We use Plaskett's Puzzle, a famous endgame study from the late 1970s, to compare the two engines. Our experiments show that Stockfish outperforms LCZero on the puzzle. We examine the algorithmic differences between the engines and use our observations as a basis for carefully interpreting the test results. Drawing inspiration from how humans solve chess problems, we ask whether machines can possess a form of imagination. On the theoretical side, we describe how Bellman's equation may be applied to optimize the probability of winning. To conclude, we discuss the implications of our work on artificial intelligence (AI) and artificial general intelligence (AGI), suggesting possible avenues for future research.

Artificial Intelligence 2018: Build the Most Powerful AI


Artificial Intelligence 2018: Build the Most Powerful AI - Learn, build and implement the most powerful AI model at home. Two months ago we discovered that a very new kind of AI was invented. The kind of AI which is based on a genius idea and that you can build from scratch and without the need for any framework. We checked that out, we built it, and... the results are absolutely insane! This game-changing AI called Augmented Random Search, ARS for short.

Minimax Rates for STIT and Poisson Hyperplane Random Forests Machine Learning

In [12], Mourtada, Ga\"{i}ffas and Scornet showed that, under proper tuning of the complexity parameters, random trees and forests built from the Mondrian process in $\mathbb{R}^d$ achieve the minimax rate for $\beta$-H\"{o}lder continuous functions, and random forests achieve the minimax rate for $(1+\beta)$-H\"{o}lder functions in arbitrary dimension. In this work, we show that a much larger class of random forests built from random partitions of $\mathbb{R}^d$ also achieve these minimax rates. This class includes STIT random forests, the most general class of random forests built from a self-similar and stationary partition of $\mathbb{R}^d$ by hyperplane cuts possible, as well as forests derived from Poisson hyperplane tessellations. Our proof technique relies on classical results as well as recent advances on stationary random tessellations in stochastic geometry.

Multi-Objective Bayesian Optimization over High-Dimensional Search Spaces Machine Learning

The ability to optimize multiple competing objective functions with high sample efficiency is imperative in many applied problems across science and industry. Multi-objective Bayesian optimization (BO) achieves strong empirical performance on such problems, but even with recent methodological advances, it has been restricted to simple, low-dimensional domains. Most existing BO methods exhibit poor performance on search spaces with more than a few dozen parameters. In this work we propose MORBO, a method for multi-objective Bayesian optimization over high-dimensional search spaces. MORBO performs local Bayesian optimization within multiple trust regions simultaneously, allowing it to explore and identify diverse solutions even when the objective functions are difficult to model globally. We show that MORBO significantly advances the state-of-the-art in sample-efficiency for several high-dimensional synthetic and real-world multi-objective problems, including a vehicle design problem with 222 parameters, demonstrating that MORBO is a practical approach for challenging and important problems that were previously out of reach for BO methods.

Search For Deep Graph Neural Networks Artificial Intelligence

Current GNN-oriented NAS methods focus on the search for different layer aggregate components with shallow and simple architectures, which are limited by the 'over-smooth' problem. To further explore the benefits from structural diversity and depth of GNN architectures, we propose a GNN generation pipeline with a novel two-stage search space, which aims at automatically generating high-performance while transferable deep GNN models in a block-wise manner. Meanwhile, to alleviate the 'over-smooth' problem, we incorporate multiple flexible residual connection in our search space and apply identity mapping in the basic GNN layers. For the search algorithm, we use deep-q-learning with epsilon-greedy exploration strategy and reward reshaping. Extensive experiments on real-world datasets show that our generated GNN models outperforms existing manually designed and NAS-based ones.