Convex (specifically semidefinite) relaxation provides a powerful approach to constructing robust machine perception systems, enabling the recovery of certifiably globally optimal solutions of challenging estimation problems in many practical settings. However, solving the large-scale semidefinite relaxations underpinning this approach remains a formidable computational challenge. A dominant cost in many state-of-the-art (Burer-Monteiro factorization-based) certifiable estimation methods is solution verification (testing the global optimality of a given candidate solution), which entails computing a minimum eigenpair of a certain symmetric certificate matrix. In this letter, we show how to significantly accelerate this verification step, and thereby the overall speed of certifiable estimation methods. First, we show that the certificate matrices arising in the Burer-Monteiro approach generically possess spectra that make the verification problem expensive to solve using standard iterative eigenvalue methods. We then show how to address this challenge using preconditioned eigensolvers; specifically, we design a specialized solution verification algorithm based upon the locally optimal block preconditioned conjugate gradient (LOBPCG) method together with a simple yet highly effective algebraic preconditioner. Experimental evaluation on a variety of simulated and real-world examples shows that our proposed verification scheme is very effective in practice, accelerating solution verification by up to 280x, and the overall Burer-Monteiro method by up to 16x, versus the standard Lanczos method when applied to relaxations derived from large-scale SLAM benchmarks.

The optimality and sensitivity of the empirical risk minimization problem with relative entropy regularization (ERM-RER) are investigated for the case in which the reference is a sigma-finite measure instead of a probability measure. This generalization allows for a larger degree of flexibility in the incorporation of prior knowledge over the set of models. In this setting, the interplay of the regularization parameter, the reference measure, the risk function, and the empirical risk induced by the solution of the ERM-RER problem is characterized. This characterization yields necessary and sufficient conditions for the existence of a regularization parameter that achieves an arbitrarily small empirical risk with arbitrarily high probability. The sensitivity of the expected empirical risk to deviations from the solution of the ERM-RER problem is studied. The sensitivity is then used to provide upper and lower bounds on the expected empirical risk. Moreover, it is shown that the expectation of the sensitivity is upper bounded, up to a constant factor, by the square root of the lautum information between the models and the datasets.

Neural ordinary differential equations (Neural ODEs) define continuous time dynamical systems with neural networks. The interest in their application for modelling has sparked recently, spanning hybrid system identification problems and time series analysis. In this work we propose the use of a neural control policy capable of satisfying state and control constraints to solve nonlinear optimal control problems. The control policy optimization is posed as a Neural ODE problem to efficiently exploit the availability of a dynamical system model. We showcase the efficacy of this type of deterministic neural policies in two constrained systems: the controlled Van der Pol system and a bioreactor control problem. This approach represents a practical approximation to the intractable closed-loop solution of nonlinear control problems.

Bayesian optimization has emerged at the forefront of expensive black-box optimization due to its data efficiency. Recent years have witnessed a proliferation of studies on the development of new Bayesian optimization algorithms and their applications. Hence, this paper attempts to provide a comprehensive and updated survey of recent advances in Bayesian optimization and identify interesting open problems. We categorize the existing work on Bayesian optimization into nine main groups according to the motivations and focus of the proposed algorithms. For each category, we present the main advances with respect to the construction of surrogate models and adaptation of the acquisition functions. Finally, we discuss the open questions and suggest promising future research directions, in particular with regard to heterogeneity, privacy preservation, and fairness in distributed and federated optimization systems.

The DEtection TRansformer (DETR) approach, which uses a transformer encoder-decoder architecture and a set-based global loss, has become a building block in many transformer based applications. However, as originally presented, the assignment cost and the global loss are not aligned, i.e., reducing the former is likely but not guaranteed to reduce the latter. And the issue of gradient is ignored when a combinatorial solver such as Hungarian is used. In this paper we show that the global loss can be expressed as the sum of an assignment-independent term, and an assignment-dependent term which can be used to define the assignment cost matrix. Recent results on generalized gradients of optimal assignment cost with respect to parameters of an assignment problem are then used to define generalized gradients of the loss with respect to network parameters, and backpropagation is carried out properly. Our experiments using the same loss weights show interesting convergence properties and a potential for further performance improvements.

The Facility for Rare Isotope Beams (FRIB) at Michigan State University (MSU) is designed for various kinds of rare isotope production. This involves the frequent switch of the ion source species. Therefore, fast tuning of the accelerator Front-End (FE) to maintain optimal beam optics is one of the key performance requirements. Breaking-through the tuning performance over the traditional black-box optimization algorithm may be possible if historical or simulated data can be incorporated into the optimization algorithm in a computationally feasible way. However, we experienced significant machine status changes (a.k.a. 'distribution shift' or'machine drift') whenever ion source species are switched or the ion source is re-started (after overnight turn-off).

For certain diseases or traits, genes play a crucial role in their expression. And the interactions between genes, i.e., gene regulatory networks (GRNs), have become a recent research hotspot. With the availability of high-throughput gene expression data and the substantial increase in arithmetic power, it is possible to reconstruct large-scale gene regulatory networks[1]. Due to the nature of gene expression data, providing information about the abundance of mRNAs only rather than binding information, gene regulatory networks defined in the above sense provide information about regulatory interactions between regulators and their potential targets; genegene interactions, and potential protein-protein interactions. In this paper, we refer to the network inferred in this way as a gene regulatory network[32]. In gene regulatory networks, genes can be divided into two categories. Transcription factors (TF), also known as trans-acting factors, are DNA-binding proteins that specifically interact with the cis-acting elements of eukaryotic genes and have an activating or inhibiting effect on gene transcription. The gene that receives this activation or repression is referred to as the target gene. It is important to note that transcription factors themselves may also be target genes, i.e., there may be mutual regulation in the regulatory network.

Soft robots are interesting examples of hyper-redundancy in robotics, however, the nonlinear continuous dynamics of these robots and the use of hyper-elastic and visco-elastic materials makes modeling of these robots more complicated. This study presents a geometric Inverse Kinematic (IK) model for trajectory tracking of multi-segment extensible soft robots, where, each segment of the soft actuator is geometrically approximated with multiple rigid links connected with rotary and prismatic joints. Using optimization methods, the desired configuration variables of the soft actuator for the desired end-effector positions are obtained. Also, the redundancy of the robot is applied for second task applications, such as tip angle control. The model's performance is investigated through simulations, numerical benchmarks, and experimental validations and results show lower computational costs and higher accuracy compared to most existing methods. The method is easy to apply to multi segment soft robots, both in 2D and 3D. As a case study, a fully 3D-printed soft robot manipulator is tested using a control unit and the model predictions show good agreement with the experimental results.

This paper focuses on stochastic saddle point problems with decision-dependent distributions. These are problems whose objective is the expected value of a stochastic payoff function and whose data distribution drifts in response to decision variables--a phenomenon represented by a distributional map. A common approach to accommodating distributional shift is to retrain optimal decisions once a new distribution is revealed, or repeated retraining. We introduce the notion of equilibrium points, which are the fixed points of this repeated retraining procedure, and provide sufficient conditions for their existence and uniqueness. To find equilibrium points, we develop deterministic and stochastic primal-dual algorithms and demonstrate their convergence with constant step-size in the former and polynomial decay step-size schedule in the latter. By modeling errors emerging from a stochastic gradient estimator as sub-Weibull random variables, we provide error bounds in expectation and in high probability that hold for each iteration. Without additional knowledge of the distributional map, computing saddle points is intractable. Thus we propose a condition on the distributional map--which we call opposing mixture dominance--that ensures that the objective is strongly-convex-strongly-concave. Finally, we demonstrate that derivative-free algorithms with a single function evaluation are capable of approximating saddle points

Backpropagation applied to handwritten zip code recognition.