Optimization
Sparse convolutional coding for neuronal assembly detection
Sven Peter, Elke Kirschbaum, Martin Both, Lee Campbell, Brandon Harvey, Conor Heins, Daniel Durstewitz, Ferran Diego, Fred A. Hamprecht
Cell assemblies, originally proposed by Donald Hebb (1949), are subsets of neurons firing in a temporally coordinated way that gives rise to repeated motifs supposed to underly neural representations and information processing. Although Hebb's original proposal dates back many decades, the detection of assemblies and their role in coding is still an open and current research topic, partly because simultaneous recordings from large populations of neurons became feasible only relatively recently. Most current and easy-to-apply computational techniques focus on the identification of strictly synchronously spiking neurons. In this paper we propose a new algorithm, based on sparse convolutional coding, for detecting recurrent motifs of arbitrary structure up to a given length. Testing of our algorithm on synthetically generated datasets shows that it outperforms established methods and accurately identifies the temporal structure of embedded assemblies, even when these contain overlapping neurons or when strong background noise is present. Moreover, exploratory analysis of experimental datasets from hippocampal slices and cortical neuron cultures have provided promising results.
On Optimal Generalizability in Parametric Learning
Ahmad Beirami, Meisam Razaviyayn, Shahin Shahrampour, Vahid Tarokh
We consider the parametric learning problem, where the objective of the learner is determined by a parametric loss function. Employing empirical risk minimization with possibly regularization, the inferred parameter vector will be biased toward the training samples. Such bias is measured by the cross validation procedure in practice where the data set is partitioned into a training set used for training and a validation set, which is not used in training and is left to measure the outof-sample performance. A classical cross validation strategy is the leave-one-out cross validation (LOOCV) where one sample is left out for validation and training is done on the rest of the samples that are presented to the learner, and this process is repeated on all of the samples. LOOCV is rarely used in practice due to the high computational complexity. In this paper, we first develop a computationally efficient approximate LOOCV (ALOOCV) and provide theoretical guarantees for its performance. Then we use ALOOCV to provide an optimization algorithm for finding the regularizer in the empirical risk minimization framework. In our numerical experiments, we illustrate the accuracy and efficiency of ALOOCV as well as our proposed framework for the optimization of the regularizer.
Efficient Online Linear Optimization with Approximation Algorithms
We revisit the problem of online linear optimization in case the set of feasible actions is accessible through an approximated linear optimization oracle with a factor multiplicative approximation guarantee. This setting is in particular interesting since it captures natural online extensions of well-studied offline linear optimization problems which are NP-hard, yet admit efficient approximation algorithms. The goal here is to minimize the -regret which is the natural extension of the standard regret in online learning to this setting. We present new algorithms with significantly improved oracle complexity for both the full information and bandit variants of the problem.
On the Optimization Landscape of Tensor Decompositions
Non-convex optimization with local search heuristics has been widely used in machine learning, achieving many state-of-art results. It becomes increasingly important to understand why they can work for these NP-hard problems on typical data. The landscape of many objective functions in learning has been conjectured to have the geometric property that "all local optima are (approximately) global optima", and thus they can be solved efficiently by local search algorithms. However, establishing such property can be very difficult. In this paper, we analyze the optimization landscape of the random over-complete tensor decomposition problem, which has many applications in unsupervised leaning, especially in learning latent variable models.
Certified Defenses for Data Poisoning Attacks
Jacob Steinhardt, Pang Wei W. Koh, Percy S. Liang
Machine learning systems trained on user-provided data are susceptible to data poisoning attacks, whereby malicious users inject false training data with the aim of corrupting the learned model. While recent work has proposed a number of attacks and defenses, little is understood about the worst-case loss of a defense in the face of a determined attacker. We address this by constructing approximate upper bounds on the loss across a broad family of attacks, for defenders that first perform outlier removal followed by empirical risk minimization. Our approximation relies on two assumptions: (1) that the dataset is large enough for statistical concentration between train and test error to hold, and (2) that outliers within the clean (nonpoisoned) data do not have a strong effect on the model. Our bound comes paired with a candidate attack that often nearly matches the upper bound, giving us a powerful tool for quickly assessing defenses on a given dataset. Empirically, we find that even under a simple defense, the MNIST-1-7 and Dogfish datasets are resilient to attack, while in contrast the IMDB sentiment dataset can be driven from 12% to 23% test error by adding only 3% poisoned data.
Optimal Ground Station Selection for Low-Earth Orbiting Satellites
Eddy, Duncan, Ho, Michelle, Kochenderfer, Mykel J.
This paper presents a solution to the problem of optimal ground station selection for low-Earth orbiting (LEO) space missions that enables mission operators to precisely design their ground segment performance and costs. Space mission operators are increasingly turning to Ground-Station-as-a-Service (GSaaS) providers to supply the terrestrial communications segment to reduce costs and increase network size. However, this approach leads to a new challenge of selecting the optimal service providers and station locations for a given mission. We consider the problem of ground station selection as an optimization problem and present a general solution framework that allows mission designers to set their overall optimization objective and constrain key mission performance variables such as total data downlink, total mission cost, recurring operational cost, and maximum communications time-gap. We solve the problem using integer programming (IP). To address computational scaling challenges, we introduce a surrogate optimization approach where the optimal station selection is determined based on solving the problem over a reduced time domain. Two different IP formulations are evaluated using randomized selections of LEO satellites of varying constellation sizes. We consider the networks of the commercial GSaaS providers Atlas Space Operations, Amazon Web Services (AWS) Ground Station, Azure Orbital Ground Station, Kongsberg Satellite Services (KSAT), Leaf Space, and Viasat Real-Time Earth. We compare our results against standard operational practices of integrating with one or two primary ground station providers.
Green vehicle routing problem that jointly optimizes delivery speed and routing based on the characteristics of electric vehicles
The abundance of materials and the development of the economy have led to the flourishing of the logistics industry, but have also caused certain pollution. The research on GVRP (Green vehicle routing problem) for planning vehicle routes during transportation to reduce pollution is also increasingly developing. Further exploration is needed on how to integrate these research findings with real vehicles. This paper establishes an energy consumption model using real electric vehicles, fully considering the physical characteristics of each component of the vehicle. To avoid the distortion of energy consumption models affecting the results of route planning. The energy consumption model also incorporates the effects of vehicle start/stop, speed, distance, and load on energy consumption. In addition, a load first speed optimization algorithm was proposed, which selects the most suitable speed between every two delivery points while planning the route. In order to further reduce energy consumption while meeting the time window. Finally, an improved Adaptive Genetic Algorithm is used to solve for the most energy-efficient route. The experiment shows that the results of using this speed optimization algorithm are generally more energy-efficient than those without using this algorithm. The average energy consumption of constant speed delivery at different speeds is 17.16% higher than that after speed optimization. Provided a method that is closer to reality and easier for logistics companies to use. It also enriches the GVRP model.
A Survey on Group Fairness in Federated Learning: Challenges, Taxonomy of Solutions and Directions for Future Research
Salazar, Teresa, Araújo, Helder, Cano, Alberto, Abreu, Pedro Henriques
Group fairness in machine learning is a critical area of research focused on achieving equitable outcomes across different groups defined by sensitive attributes such as race or gender. Federated learning, a decentralized approach to training machine learning models across multiple devices or organizations without sharing raw data, amplifies the need for fairness due to the heterogeneous data distributions across clients, which can exacerbate biases. The intersection of federated learning and group fairness has attracted significant interest, with 47 research works specifically dedicated to addressing this issue. However, no dedicated survey has focused comprehensively on group fairness in federated learning. In this work, we present an in-depth survey on this topic, addressing the critical challenges and reviewing related works in the field. We create a novel taxonomy of these approaches based on key criteria such as data partitioning, location, and applied strategies. Additionally, we explore broader concerns related to this problem and investigate how different approaches handle the complexities of various sensitive groups and their intersections. Finally, we review the datasets and applications commonly used in current research. We conclude by highlighting key areas for future research, emphasizing the need for more methods to address the complexities of achieving group fairness in federated systems.