A recent trend of fair machine learning is to define fairness as causality-based notions which concern the causal connection between protected attributes and decisions. However, one common challenge of all causality-based fairness notions is identifiability, i.e., whether they can be uniquely measured from observational data, which is a critical barrier to applying these notions to real-world situations. In this paper, we develop a framework for measuring different causality-based fairness. We propose a unified definition that covers most of previous causality-based fairness notions, namely the path-specific counterfactual fairness (PC fairness). Based on that, we propose a general method in the form of a constrained optimization problem for bounding the path-specific counterfactual fairness under all unidentifiable situations. Experiments on synthetic and real-world datasets show the correctness and effectiveness of our method.

While often stated as an instance of the likelihood ratio trick [Rubinstein, 1989], the original policy gradient theorem [Sutton, 1999] involves an integral over the action space. When this integral can be computed, the resulting "all-action" estimator [Sutton, 2001] provides a conditioning effect [Bratley, 1987] reducing the variance significantly compared to the REINFORCE estimator [Williams, 1992]. In this paper, we adopt a numerical integration perspective to broaden the applicability of the all-action estimator to general spaces and to any function class for the policy or critic components, beyond the Gaussian case considered by [Ciosek, 2018]. In addition, we provide a new theoretical result on the effect of using a biased critic which offers more guidance than the previous "compatible features" condition of [Sutton, 1999]. We demonstrate the benefit of our approach in continuous control tasks with nonlinear function approximation. Our results show improved performance and sample efficiency.

The Travelling Salesman Problem (TSP) is one of the most popularCombinatorial Optimization Problem. It is well solicited for the large variety ofapplications that it can solve, but also for its difficulty to find optimal solutions. Oneof the variants of the TSP is the Generalized TSP (GTSP), where the TSP isconsidered as a special case which makes the GTSP harder to solve. We propose inthis paper a new memetic algorithm based on the well-known Breakout Local Search(BLS) metaheuristic to provide good solutions for GTSP instances. Our approach iscompetitive compared to other recent memetic algorithms proposed for the GTSPand gives at the same time some improvements to BLS to reduce its runtime.Keywords: Generalized Travelling Salesman Problem, Breakout Local Search,Memetic Algorithms, Iterated Local Search

W e explore machine learning methods for AC Optimal Powerflow (ACOPF) - the task of optimizing power generation in a transmission network according while respecting physical and engineering constraints. W e present two formulations of ACOPF as a machine learning problem: 1) an end-to-end prediction task where we directly predict the optimal generator settings, and 2) a constraint prediction task where we predict the set of active constraints in the optimal solution.

The Quadratic Assignment Problem (QAP) is a well-known permutation-based combinatorial optimization problem with real applications in industrial and logistics environments. Motivated by the challenge that this NPhard problem represents, it has captured the attention of the evolutionary computation community for decades. As a result, a large number of algorithms have been proposed to optimize this algorithm. Among these, exact methods are only able to solve instances of size n 40, and thus, many heuristic and metaheuristic methods have been applied to the QAP. In this work, we follow this direction by approaching the QAP through Estimation of Distribution Algorithms (EDAs). Particularly, a nonparametric distance-based exponential probabilistic model is used. Based on the analysis of the characteristics of the QAP, and previous work in the area, we introduce Kernels of Mallows Model under the Hamming distance to the context of EDAs. Conducted experiments point out that the performance of the proposed algorithm in the QAP is superior to (i) the classical EDAs adapted to deal with the QAP, and also (ii) to the specific EDAs proposed in the literature to deal with permutation problems.1. Introduction The Quadratic Assignment Problem (QAP) [30] is a well known combinatorial optimization problem. Along with other problems, such as the traveling salesman problem, the linear ordering problem and the flowshop scheduling problem, it belongs to the family of permutation-based (a permutation is a bijection of the set {1,...,n } onto itself) problems [10]. The QAP has been applied in many different environments over the years, to name but a few notable examples, selecting optimal hospital layouts [24], optimally placing components on circuit boards [44], assigning gates at airports [23] or optimizing data transmission [38]. Sahni and Gonzalez [45] proved that the QAP is an NPhard optimization problem, and as such, no polynomial-time exact algorithm can solve this problem unless P NP. M: The size of the set of selected solutions. S: The number of new solutions generated at each iteration.

Dynamic Multi-objective Optimization Problems (DMOPs) refer to optimization problems that objective functions will change with time. Solving DMOPs implies that the Pareto Optimal Set (POS) at different moments can be accurately found, and this is a very difficult job due to the dynamics of the optimization problems. The POS that have been obtained in the past can help us to find the POS of the next time more quickly and accurately. Therefore, in this paper we present a Support Vector Machine (SVM) based Dynamic Multi-Objective Evolutionary optimization Algorithm, called SVM-DMOEA. The algorithm uses the POS that has been obtained to train a SVM and then take the trained SVM to classify the solutions of the dynamic optimization problem at the next moment, and thus it is able to generate an initial population which consists of different individuals recognized by the trained SVM. The initial populuation can be fed into any population based optimization algorithm, e.g., the Nondominated Sorting Genetic Algorithm II (NSGA-II), to get the POS at that moment. The experimental results show the validity of our proposed approach.

Real-world problems typically require the simultaneous op timization of several, often conflicting objectives. Many of these multi-objective optimization problems are characterized by wide ranges of uncertainties in their deci sion variables or objective functions, which further increases the complexity of optim ization. To cope with such uncertainties, robust optimization is widely studied aiming to distinguish candidate solutions with uncertain objectives specified by confidence intervals, probability distributions or sampled data. However, existing techniques most ly either fail to consider the actual distributions or assume uncertainty as instance s of uniform or Gaussian distributions. This paper introduces an empirical approac h that enables an efficient comparison of candidate solutions with uncertain objectiv es that can follow arbitrary distributions. Given two candidate solutions under compar ison, this operator calculates the probability that one solution dominates the other in terms of each uncertain objective. It can substitute for the standard comparison op erator of existing optimization techniques such as evolutionary algorithms to ena ble discovering robust solutions to problems with multiple uncertain objectives. Th is paper also proposes to incorporate various uncertainties in well-known multi-ob jective problems to provide a benchmark for evaluating uncertainty-aware optimization techniques. The proposed comparison operator and benchmark suite are integrated int o an existing optimization tool that features a selection of multi-objective optimiza tion problems and algorithms. Experiments show that in comparison with existing techniqu es, the proposed approach achieves higher optimization quality at lower overheads. The authors are with the Department of Computer Science, Friedr ich-Alexander-Universit at Erlangen-N urnberg (FAU), Erlangen 91058, Germany. A a candidate solution B a candidate solution B beta distribution C comparison operator c positive constant E expected value e approximation error f objective function N Gaussian distribution N number of samples or quantile cuts n number of decision variables m number of objective functions S sequence of samples s sample from an uncertain objective's distribution U uniform distribution u uncertainty added to an optimization problem V ar variance X random variable x decision variable γ comparison threshold δ tolerance (bound on an error) σ standard deviation ω interval width in a histogram 2 1 Introduction Real-world problems typically demand solutions that are optimized with respect to multiple criteria called objectives. In these so-called multi-objective optimization problems, the objectives often conflict with each other such that no single solution can b e found to be optimal in all objectives. Instead, one usually searches for a set of non-d ominated solutions known as Pareto front or Pareto set that provide decent tradeoffs among objectives.

We propose a novel discrete-time dynamical system-based framework for achieving adversarial robustness in machine learning models. Our algorithm is originated from robust optimization, which aims to find the saddle point of a min-max optimization problem in the presence of uncertainties. The robust learning problem is formulated as a robust optimization problem, and we introduce a discrete-time algorithm based on a saddle-point dynamical system (SDS) to solve this problem. Under the assumptions that the cost function is convex and uncertainties enter concavely in the robust learning problem, we analytically show that using a diminishing step-size, the stochastic version of our algorithm, SSDS converges asymptotically to the robust optimal solution. The algorithm is deployed for the training of adversarially robust deep neural networks. Although such training involves highly non-convex non-concave robust optimization problems, empirical results show that the algorithm can achieve significant robustness for deep learning. We compare the performance of our SSDS model to other state-of-the-art robust models, e.g., trained using the projected gradient descent (PGD)-training approach. From the empirical results, we find that SSDS training is computationally inexpensive (compared to PGD-training) while achieving comparable performances. SSDS training also helps robust models to maintain a relatively high level of performance for clean data as well as under black-box attacks.

Learning causal graphical models based on directed acyclic graphs is an important task in causal discovery and causal inference. We consider a general framework towards efficient causal structure learning with potentially large graphs. Within this framework, we propose a masked gradient-based structure learning method based on binary adjacency matrix that exists for any structural equation model. To enable first-order optimization methods, we use Gumbel-Softmax approach to approximate the binary valued entries of the adjacency matrix, which usually results in real values that are close to zero or one. The proposed method can readily include any differentiable score function and model function for learning causal structures. Experiments on both synthetic and real-world datasets are conducted to show the effectiveness of our approach.

We study the problem of fitting task-specific learning rate schedules from the perspective of hyperparameter optimization. This allows us to explicitly search for schedules that achieve good generalization. We describe the structure of the gradient of a validation error w.r.t. the learning rate, the hypergradient, and based on this we introduce a novel online algorithm. Our method adaptively interpolates between the recently proposed techniques of Franceschi et al. (2017) and Baydin et al. (2017), featuring increased stability and faster convergence. We show empirically that the proposed method compares favourably with baselines and related methods in terms of final test accuracy.