In recent years, nonlinear model predictive control (NMPC) has been extensively used for solving automotive motion control and planning tasks. In order to formulate the NMPC problem, different coordinate systems can be used with different advantages. We propose and compare formulations for the NMPC related optimization problem, involving a Cartesian and a Frenet coordinate frame (CCF/ FCF) in a single nonlinear program (NLP). We specify costs and collision avoidance constraints in the more advantageous coordinate frame, derive appropriate formulations and compare different obstacle constraints. With this approach, we exploit the simpler formulation of opponent vehicle constraints in the CCF, as well as road aligned costs and constraints related to the FCF. Comparisons to other approaches in a simulation framework highlight the advantages of the proposed approaches.

The real-time operation of large-scale infrastructure networks requires scalable optimization capabilities. Decomposition schemes can help achieve scalability; classical decomposition approaches such as the alternating direction method of multipliers (ADMM) and distributed Newtons schemes, however, often either suffer from slow convergence or might require high degrees of communication. In this work, we present new primal decomposition schemes for solving large-scale, strongly convex QPs. These approaches have global convergence guarantees and require limited communication. We benchmark their performance against the off-the-shelf interior-point method Ipopt and against ADMM on infrastructure networks that contain up to 300,000 decision variables and constraints. Overall, we find that the proposed approaches solve problems as fast as Ipopt but with reduced communication. Moreover, we find that the proposed schemes achieve higher accuracy than ADMM approaches.

A new form of ML knowledge representation with high generalization power is developed and implemented numerically. Initial $\mathit{IN}$ attributes and $\mathit{OUT}$ class label are transformed into the corresponding Hilbert spaces by considering localized wavefunctions. A partially unitary operator optimally converting a state from $\mathit{IN}$ Hilbert space into $\mathit{OUT}$ Hilbert space is then built from an optimization problem of transferring maximal possible probability from $\mathit{IN}$ to $\mathit{OUT}$, this leads to the formulation of a new algebraic problem. Constructed Knowledge Generalizing Operator $\mathcal{U}$ can be considered as a $\mathit{IN}$ to $\mathit{OUT}$ quantum channel; it is a partially unitary rectangular matrix of the dimension $\mathrm{dim}(\mathit{OUT}) \times \mathrm{dim}(\mathit{IN})$ transforming operators as $A^{\mathit{OUT}}=\mathcal{U} A^{\mathit{IN}} \mathcal{U}^{\dagger}$. Whereas only operator $\mathcal{U}$ projections squared are observable $\left\langle\mathit{OUT}|\mathcal{U}|\mathit{IN}\right\rangle^2$ (probabilities), the fundamental equation is formulated for the operator $\mathcal{U}$ itself. This is the reason of high generalizing power of the approach; the situation is the same as for the Schr\"{o}dinger equation: we can only measure $\psi^2$, but the equation is written for $\psi$ itself.

This thesis considers sequential decision problems, where the loss/reward incurred by selecting an action may not be inferred from observed feedback. A major part of this thesis focuses on the unsupervised sequential selection problem, where one can not infer the loss incurred for selecting an action from observed feedback. We also introduce a new setup named Censored Semi Bandits, where the loss incurred for selecting an action can be observed under certain conditions. Finally, we study the channel selection problem in the communication networks, where the reward for an action is only observed when no other player selects that action to play in the round. These problems find applications in many fields like healthcare, crowd-sourcing, security, adaptive resource allocation, among many others. This thesis aims to address the above-described sequential decision problems by exploiting specific structures these problems exhibit. We develop provably optimal algorithms for each of these setups with weak feedback and validate their empirical performance on different problem instances derived from synthetic and real datasets.

Probabilistic Law Discovery (PLD) is a logic based Machine Learning method, which implements a variant of probabilistic rule learning. In several aspects, PLD is close to Decision Tree/Random Forest methods, but it differs significantly in how relevant rules are defined. The learning procedure of PLD solves the optimization problem related to the search for rules (called probabilistic laws), which have a minimal length and relatively high probability. At inference, ensembles of these rules are used for prediction. Probabilistic laws are human-readable and PLD based models are transparent and inherently interpretable. Applications of PLD include classification/clusterization/regression tasks, as well as time series analysis/anomaly detection and adaptive (robotic) control. In this paper, we outline the main principles of PLD, highlight its benefits and limitations and provide some application guidelines.

The issue of left before treatment complete (LBTC) patients is common in emergency departments (EDs). This issue represents a medico-legal risk and may cause a revenue loss. Thus, understanding the factors that cause patients to leave before treatment is complete is vital to mitigate and potentially eliminate these adverse effects. This paper proposes a framework for studying the factors that affect LBTC outcomes in EDs. The framework integrates machine learning, metaheuristic optimization, and model interpretation techniques. Metaheuristic optimization is used for hyperparameter optimization--one of the main challenges of machine learning model development. Three metaheuristic optimization algorithms are employed for optimizing the parameters of extreme gradient boosting (XGB), which are simulated annealing (SA), adaptive simulated annealing (ASA), and adaptive tabu simulated annealing (ATSA). The optimized XGB models are used to predict the LBTC outcomes for the patients under treatment in ED. The designed algorithms are trained and tested using four data groups resulting from the feature selection phase. The model with the best predictive performance is interpreted using SHaply Additive exPlanations (SHAP) method. The findings show that ATSA-XGB outperformed other mode configurations with an accuracy, area under the curve (AUC), sensitivity, specificity, and F1-score of 86.61%, 87.50%, 85.71%, 87.51%, and 86.60%, respectively. The degree and the direction of effects of each feature were determined and explained using the SHAP method.

Designing a local planner to control tractor-trailer vehicles in forward and backward maneuvering is a challenging control problem in the research community of autonomous driving systems. Considering a critical situation in the stability of tractor-trailer systems, a practical and novel approach is presented to design a non-linear MPC(NMPC) local planner for tractor-trailer autonomous vehicles in both forward and backward maneuvering. The tractor velocity and steering angle are considered to be control variables. The proposed NMPC local planner is designed to handle jackknife situations, avoiding multiple static obstacles, and path following in both forward and backward maneuvering. The challenges mentioned above are converted into a constrained problem that can be handled simultaneously by the proposed NMPC local planner. The direct multiple shooting approach is used to convert the optimal control problem(OCP) into a non-linear programming problem(NLP) that IPOPT solvers can solve in CasADi. The controller performance is evaluated through different backup and forward maneuvering scenarios in the Gazebo simulation environment in real-time. It achieves asymptotic stability in avoiding static obstacles and accurate tracking performance while respecting path constraints. Finally, the proposed NMPC local planner is integrated with an open-source autonomous driving software stack called AutowareAi.

This paper presents a methodology for integrating machine learning techniques into metaheuristics for solving combinatorial optimization problems. Namely, we propose a general machine learning framework for neighbor generation in metaheuristic search. We first define an efficient neighborhood structure constructed by applying a transformation to a selected subset of variables from the current solution. Then, the key of the proposed methodology is to generate promising neighbors by selecting a proper subset of variables that contains a descent of the objective in the solution space. To learn a good variable selection strategy, we formulate the problem as a classification task that exploits structural information from the characteristics of the problem and from high-quality solutions. We validate our methodology on two metaheuristic applications: a Tabu Search scheme for solving a Wireless Network Optimization problem and a Large Neighborhood Search heuristic for solving Mixed-Integer Programs. The experimental results show that our approach is able to achieve a satisfactory trade-off between the exploration of a larger solution space and the exploitation of high-quality solution regions on both applications.

In this work, we propose a robust framework that employs adversarially robust training to safeguard the machine learning models against perturbed testing data. We achieve this by incorporating the worst-case additive adversarial error within a fixed budget for each sample during model estimation. Our main focus is to provide a plug-and-play solution that can be incorporated in the existing machine learning algorithms with minimal changes. To that end, we derive the ready-to-use solution for several widely used loss functions with a variety of norm constraints on adversarial perturbation for various supervised and unsupervised ML problems, including regression, classification, two-layer neural networks, graphical models, and matrix completion. The solutions are either in closed-form, 1-D optimization, semidefinite programming, difference of convex programming or a sorting-based algorithm. Finally, we validate our approach by showing significant performance improvement on real-world datasets for supervised problems such as regression and classification, as well as for unsupervised problems such as matrix completion and learning graphical models, with very little computational overhead.

In this paper, we studied the federated bilevel optimization problem, which has widespread applications in machine learning. In particular, we developed two momentum-based algorithms for optimizing this kind of problem and established the convergence rate of our two algorithms, providing the sample and communication complexities. Importantly, to the best of our knowledge, our convergence rate is the first one achieving the linear speedup with respect to the number of devices for federated bilevel optimization algorithms. At last, our extensive experimental results confirm the effectiveness of our two algorithms.