Evolution by natural selection is a scientific theory which aims to explain how natural systems evolved over time into more complex systems. In evolutionary algorithms, a fitness value can be used as a guide to indicate how close we are to a solution (eg. the higher the value, the closer we are to our desired objective). By grouping closer together all the elements in a population which share a similar fitnesses and further apart all the dissimilar elements, we can then construct a Fitness Landscape (Figure 1). One of the main problems faced by evolutionary algorithms is the presence of local optima in the fitness landscape. Local optima, can, in fact, mislead our algorithm to not reach our desired global maxima in favour of a less optimal solution.

Federated Learning (FL) has been recently presented as a new technique for training shared machine learning models in a distributed manner while respecting data privacy. However, implementing FL in wireless networks may significantly reduce the lifetime of energy-constrained mobile devices due to their involvement in the construction of the shared learning models. To handle this issue, we propose a novel approach at the physical layer based on the application of lightwave power transfer in the FL-based wireless network and a resource allocation scheme to manage the network's power efficiency. Hence, we formulate the corresponding optimization problem and then propose a method to obtain the optimal solution. Numerical results reveal that, the proposed scheme can provide sufficient energy to a mobile device for performing FL tasks without using any power from its own battery. Hence, the proposed approach can support the FL-based wireless network to overcome the issue of limited energy in mobile devices.

In this paper we accelerate the column generation (CG) solution to expanded linear programming (LP) relaxations (Barnhart et al. 1996) using dual optimal inequalities (Ben Amor et al. 2006) (DOI). Expanded LP relaxations are used to solve integer linear programs (ILPs) for which compact LP relaxations are very loose. In contrast to compact LP relaxations, which contain a small number of variables, expanded LP relaxations contain a massive number of variables (called columns). However an expanded LP relaxation is often much tighter than the corresponding compact LP relaxation, and permits efficient (often in practice exact) optimization (Yarkony et al. 2019) of the corresponding ILP. To solve expanded LP relaxations, CG is used. Since the set of all feasible columns is enormous and can not be easily enumerated, a sufficient set is constructed iteratively using CG. The process of identifying negative reduced cost columns is called pricing.

It is already reported in the literature that the performance of a machine learning algorithm is greatly impacted by performing proper Hyper-Parameter optimization. One of the ways to perform Hyper-Parameter optimization is by manual search but that is time consuming. Some of the common approaches for performing Hyper-Parameter optimization are Grid search Random search and Bayesian optimization using Hyperopt. In this paper, we propose a brand new approach for hyperparameter improvement i.e. Randomized-Hyperopt and then tune the hyperparameters of the XGBoost i.e. the Extreme Gradient Boosting algorithm on ten datasets by applying Random search, Randomized-Hyperopt, Hyperopt and Grid Search. The performances of each of these four techniques were compared by taking both the prediction accuracy and the execution time into consideration. We find that the Randomized-Hyperopt performs better than the other three conventional methods for hyper-paramter optimization of XGBoost.

There are diverse topics in the field of Artificial Intelligence and Machine learning. Research is required to find optimal solutions in this field. In Deep learning, various neural networks are used but optimization has been a very important step to find out the best solution for a good model. In the field of AI, many complex algorithms have been used. It is also important to find out an optimal solution.

We consider an NP-hard selective and periodic inventory routing problem (SPIRP) in a waste vegetable oil collection environment. This SPIRP arises in the context of reverse logistics where a biodiesel company has daily requirements of oil to be used as raw material in its production process. These requirements can be fulfilled by using the available inventory, collecting waste vegetable oil or purchasing virgin oil. The problem consists in determining a period (cyclic) planning for the collection and purchasing of oil such that the total collection, inventory and purchasing costs are minimized, while meeting the company's oil requirements and all the operational constraints. We propose a MIP-based heuristic which solves a relaxed model without routing, constructs routes taking into account the relaxation's solution and then improves these routes by solving the capacitated vehicle routing problem associated to each period. Following this approach, an a posteriori performance guarantee is ensured, as the approach provides both a lower bound and a feasible solution. The performed computational experiments show that the MIP-based heuristic is very fast and effective as it is able to encounter near optimal solutions with low gaps within seconds, improving several of the best known results using just a fraction of the time spent by a state-of-the-art heuristic. A remarkable fact is that the proposed MIP-based heuristic improves over the best known results for all the large instances available in the literature.

Given a separation oracle for a convex set $K \subset \mathbb{R}^n$ that is contained in a box of radius $R$, the goal is to either compute a point in $K$ or prove that $K$ does not contain a ball of radius $\epsilon$. We propose a new cutting plane algorithm that uses an optimal $O(n \log (\kappa))$ evaluations of the oracle and an additional $O(n^2)$ time per evaluation, where $\kappa = nR/\epsilon$. $\bullet$ This improves upon Vaidya's $O( \text{SO} \cdot n \log (\kappa) + n^{\omega+1} \log (\kappa))$ time algorithm [Vaidya, FOCS 1989a] in terms of polynomial dependence on $n$, where $\omega < 2.373$ is the exponent of matrix multiplication and $\text{SO}$ is the time for oracle evaluation. $\bullet$ This improves upon Lee-Sidford-Wong's $O( \text{SO} \cdot n \log (\kappa) + n^3 \log^{O(1)} (\kappa))$ time algorithm [Lee, Sidford and Wong, FOCS 2015] in terms of dependence on $\kappa$. For many important applications in economics, $\kappa = \Omega(\exp(n))$ and this leads to a significant difference between $\log(\kappa)$ and $\mathrm{poly}(\log (\kappa))$. We also provide evidence that the $n^2$ time per evaluation cannot be improved and thus our running time is optimal. A bottleneck of previous cutting plane methods is to compute leverage scores, a measure of the relative importance of past constraints. Our result is achieved by a novel multi-layered data structure for leverage score maintenance, which is a sophisticated combination of diverse techniques such as random projection, batched low-rank update, inverse maintenance, polynomial interpolation, and fast rectangular matrix multiplication. Interestingly, our method requires a combination of different fast rectangular matrix multiplication algorithms.

In the contemporary big data realm, Deep Neural Networks (DNNs) are evolving towards more complex architectures to achieve higher inference accuracy. Model compression techniques can be leveraged to efficiently deploy such compute-intensive architectures on resource-limited mobile devices. Such methods comprise various hyper-parameters that require per-layer customization to ensure high accuracy. Choosing such hyper-parameters is cumbersome as the pertinent search space grows exponentially with model layers. This paper introduces GeneCAI, a novel optimization method that automatically learns how to tune per-layer compression hyper-parameters. We devise a bijective translation scheme that encodes compressed DNNs to the genotype space. The optimality of each genotype is measured using a multi-objective score based on accuracy and number of floating point operations. We develop customized genetic operations to iteratively evolve the non-dominated solutions towards the optimal Pareto front, thus, capturing the optimal trade-off between model accuracy and complexity. GeneCAI optimization method is highly scalable and can achieve a near-linear performance boost on distributed multi-GPU platforms. Our extensive evaluations demonstrate that GeneCAI outperforms existing rule-based and reinforcement learning methods in DNN compression by finding models that lie on a better accuracy-complexity Pareto curve.

In multi-label learning, the issue of missing labels brings a major challenge. Many methods attempt to recovery missing labels by exploiting low-rank structure of label matrix. However, these methods just utilize global low-rank label structure, ignore both local low-rank label structures and label discriminant information to some extent, leaving room for further performance improvement. In this paper, we develop a simple yet effective discriminant multi-label learning (DM2L) method for multi-label learning with missing labels. Specifically, we impose the low-rank structures on all the predictions of instances from the same labels (local shrinking of rank), and a maximally separated structure (high-rank structure) on the predictions of instances from different labels (global expanding of rank). In this way, these imposed low-rank structures can help modeling both local and global low-rank label structures, while the imposed high-rank structure can help providing more underlying discriminability. Our subsequent theoretical analysis also supports these intuitions. In addition, we provide a nonlinear extension via using kernel trick to enhance DM2L and establish a concave-convex objective to learn these models. Compared to the other methods, our method involves the fewest assumptions and only one hyper-parameter. Even so, extensive experiments show that our method still outperforms the state-of-the-art methods.

There is a clear need for efficient algorithms to tune hyperparameters for statistical learning schemes, since the commonly applied search methods (such as grid search with N-fold cross-validation) are inefficient and/or approximate. Previously existing algorithms that efficiently search for hyperparameters relying on the smoothness of the cost function cannot be applied in problems such as Lasso regression. In this contribution, we develop a hyperparameter optimization method that relies on the structure of proximal gradient methods and does not require a smooth cost function. Such a method is applied to Leave-one-out (LOO)-validated Lasso and Group Lasso to yield efficient, data-driven, hyperparameter optimization algorithms. Numerical experiments corroborate the convergence of the proposed method to a local optimum of the LOO validation error curve, and the efficiency of its approximations.