We consider black-box optimization in which only an extremely limited number of function evaluations, on the order of around 100, are affordable and the function evaluations must be performed in even fewer batches of a limited number of parallel trials. This is a typical scenario when optimizing variable settings that are very costly to evaluate, for example in the context of simulation-based optimization or machine learning hyperparameterization. We propose an original method that uses established approaches to propose a set of points for each batch and then down-selects from these candidate points to the number of trials that can be run in parallel. The key novelty of our approach lies in the introduction of a hyperparameterized method for down-selecting the number of candidates to the allowed batch-size, which is optimized offline using automated algorithm configuration. We tune this method for black box optimization and then evaluate on classical black box optimization benchmarks. Our results show that it is possible to learn how to combine evaluation points suggested by highly diverse black box optimization methods conditioned on the progress of the optimization. Compared with the state of the art in black box minimization and various other methods specifically geared towards few-shot minimization, we achieve an average reduction of 50\% of normalized cost, which is a highly significant improvement in performance.
Bio-inspired computing focuses on extracting computational models for problem solving from in-depth understanding of behaviour and mechanisms of biological systems. In recent years, cellular computational models based on the structure and the processes of living cells, such as bacterial colonies  and viral models  have become an important line of research in bio-inspired computing. Physarum-computing, as an example of cellular computing model, has attracted the attention of many researchers . Physarum polycephalum (Physarum for short) is an example of plasmodial slime moulds that are classified as a fungus "Myxomycetes" . In recent years, research on Physarum-inspired computing has become more popular since Nakagaki et al. (2000) performed their well-known experiments showing that Physarum was able to find the shortest route through a maze . Recent research has confirmed the ability of Physarum-inspired algorithms to solve a wide range of problems [103, 78]. Physarum can be modelled as a reaction-diffusion system (cytoplasmic liquid) encapsulated in an elastic growing membrane of actin-myosin cytoskeleton .
The no free lunch theorem states that no model is better suited to every problem. A question that arises from this is how to design methods that propose optimizers tailored to specific problems achieving state-of-the-art performance. This paper addresses this issue by proposing the use of meta-learning to infer population-based black-box optimizers that can automatically adapt to specific classes of problems. We suggest a general modeling of population-based algorithms that result in Learning-to-Optimize POMDP (LTO-POMDP), a meta-learning framework based on a specific partially observable Markov decision process (POMDP). From that framework's formulation, we propose to parameterize the algorithm using deep recurrent neural networks and use a meta-loss function based on stochastic algorithms' performance to train efficient data-driven optimizers over several related optimization tasks. The learned optimizers' performance based on this implementation is assessed on various black-box optimization tasks and hyperparameter tuning of machine learning models. Our results revealed that the meta-loss function encourages a learned algorithm to alter its search behavior so that it can easily fit into a new context. Thus, it allows better generalization and higher sample efficiency than state-of-the-art generic optimization algorithms, such as the Covariance matrix adaptation evolution strategy (CMA-ES).
Network models provide an efficient way to represent many real life problems mathematically. In the last few decades, the field of network optimization has witnessed an upsurge of interest among researchers and practitioners. The network models considered in this thesis are broadly classified into four types including transportation problem, shortest path problem, minimum spanning tree problem and maximum flow problem. Quite often, we come across situations, when the decision parameters of network optimization problems are not precise and characterized by various forms of uncertainties arising from the factors, like insufficient or incomplete data, lack of evidence, inappropriate judgements and randomness. Considering the deterministic environment, there exist several studies on network optimization problems. However, in the literature, not many investigations on single and multi objective network optimization problems are observed under diverse uncertain frameworks. This thesis proposes seven different network models under different uncertain paradigms. Here, the uncertain programming techniques used to formulate the uncertain network models are (i) expected value model, (ii) chance constrained model and (iii) dependent chance constrained model. Subsequently, the corresponding crisp equivalents of the uncertain network models are solved using different solution methodologies. The solution methodologies used in this thesis can be broadly categorized as classical methods and evolutionary algorithms. The classical methods, used in this thesis, are Dijkstra and Kruskal algorithms, modified rough Dijkstra algorithm, global criterion method, epsilon constraint method and fuzzy programming method. Whereas, among the evolutionary algorithms, we have proposed the varying population genetic algorithm with indeterminate crossover and considered two multi objective evolutionary algorithms.
Bai, Ruibin, Chen, Xinan, Chen, Zhi-Long, Cui, Tianxiang, Gong, Shuhui, He, Wentao, Jiang, Xiaoping, Jin, Huan, Jin, Jiahuan, Kendall, Graham, Li, Jiawei, Lu, Zheng, Ren, Jianfeng, Weng, Paul, Xue, Ning, Zhang, Huayan
The Vehicle Routing Problem (VRP) is one of the most intensively studied combinatorial optimisation problems for which numerous models and algorithms have been proposed. To tackle the complexities, uncertainties and dynamics involved in real-world VRP applications, Machine Learning (ML) methods have been used in combination with analytical approaches to enhance problem formulations and algorithmic performance across different problem solving scenarios. However, the relevant papers are scattered in several traditional research fields with very different, sometimes confusing, terminologies. This paper presents a first, comprehensive review of hybrid methods that combine analytical techniques with ML tools in addressing VRP problems. Specifically, we review the emerging research streams on ML-assisted VRP modelling and ML-assisted VRP optimisation. We conclude that ML can be beneficial in enhancing VRP modelling, and improving the performance of algorithms for both online and offline VRP optimisations. Finally, challenges and future opportunities of VRP research are discussed.
Nature-inspired algorithms are commonly used for solving the various optimization problems. In past few decades, various researchers have proposed a large number of nature-inspired algorithms. Some of these algorithms have proved to be very efficient as compared to other classical optimization methods. A young researcher attempting to undertake or solve a problem using nature-inspired algorithms is bogged down by a plethora of proposals that exist today. Not every algorithm is suited for all kinds of problem. Some score over others. In this paper, an attempt has been made to summarize various leading research proposals that shall pave way for any new entrant to easily understand the journey so far. Here, we classify the nature-inspired algorithms as natural evolution based, swarm intelligence based, biological based, science based and others. In this survey, widely acknowledged nature-inspired algorithms namely- ACO, ABC, EAM, FA, FPA, GA, GSA, JAYA, PSO, SFLA, TLBO and WCA, have been studied. The purpose of this review is to present an exhaustive analysis of various nature-inspired algorithms based on its source of inspiration, basic operators, control parameters, features, variants and area of application where these algorithms have been successfully applied. It shall also assist in identifying and short listing the methodologies that are best suited for the problem.
This paper explores the state of the art on to methods to verify and validate navigation algorithms for autonomous surface ships. We perform a systematic mapping study to find research works published in the last 10 years proposing new algorithms for autonomous navigation and collision avoidance and we have extracted what verification and validation approaches have been applied on these algorithms. We observe that most research works use simulations to validate their algorithms. However, these simulations often involve just a few scenarios designed manually. This raises the question if the algorithms have been validated properly. To remedy this, we propose the use of a systematic scenario-based testing approach to validate navigation algorithms extensively.
Predictions obtained by, e.g., artificial neural networks have a high accuracy but humans often perceive the models as black boxes. Insights about the decision making are mostly opaque for humans. Particularly understanding the decision making in highly sensitive areas such as healthcare or finance, is of paramount importance. The decision-making behind the black boxes requires it to be more transparent, accountable, and understandable for humans. This survey paper provides essential definitions, an overview of the different principles and methodologies of explainable Supervised Machine Learning (SML). We conduct a state-of-the-art survey that reviews past and recent explainable SML approaches and classifies them according to the introduced definitions. Finally, we illustrate principles by means of an explanatory case study and discuss important future directions.
Full truckload transportation (FTL) in the form of freight containers represents one of the most important transportation modes in international trade. Due to large volume and scale, in FTL, delivery time is often less critical but cost and service quality are crucial. Therefore, efficiently solving large scale multiple shift FTL problems is becoming more and more important and requires further research. In one of our earlier studies, a set covering model and a three-stage solution method were developed for a multi-shift FTL problem. This paper extends the previous work and presents a significantly more efficient approach by hybridising pricing and cutting strategies with metaheuristics (a variable neighbourhood search and a genetic algorithm). The metaheuristics were adopted to find promising columns (vehicle routes) guided by pricing and cuts are dynamically generated to eliminate infeasible flow assignments caused by incompatible commodities. Computational experiments on real-life and artificial benchmark FTL problems showed superior performance both in terms of computational time and solution quality, when compared with previous MIP based three-stage methods and two existing metaheuristics. The proposed cutting and heuristic pricing approach can efficiently solve large scale real-life FTL problems.
Predictions obtained by, e.g., artificial neural networks have a high accuracy but humans often perceive the models as black boxes. Insights about the decision making are mostly opaque for humans. Particularly understanding the decision making in highly sensitive areas such as healthcare or fifinance, is of paramount importance. The decision-making behind the black boxes requires it to be more transparent, accountable, and understandable for humans. This survey paper provides essential definitions, an overview of the different principles and methodologies of explainable Supervised Machine Learning (SML). We conduct a state-of-the-art survey that reviews past and recent explainable SML approaches and classifies them according to the introduced definitions. Finally, we illustrate principles by means of an explanatory case study and discuss important future directions.