Optimization
A novel decision fusion approach for sale price prediction using Elastic Net and MOPSO
Price prediction algorithms propose prices for every product or service according to market trends, projected demand, and other characteristics, including government rules, international transactions, and speculation and expectation. As the dependent variable in price prediction, it is affected by several independent and correlated variables which may challenge the price prediction. To overcome this challenge, machine learning algorithms allow more accurate price prediction without explicitly modeling the relatedness between variables. However, as inputs increase, it challenges the existing machine learning approaches regarding computing efficiency and prediction effectiveness. Hence, this study introduces a novel decision level fusion approach to select informative variables in price prediction. The suggested metaheuristic algorithm balances two competitive objective functions, which are defined to improve the prediction utilized variables and reduce the error rate simultaneously. To generate Pareto optimal solutions, an Elastic net approach is employed to eliminate unrelated and redundant variables to increase the accuracy. Afterward, we propose a novel method for combining solutions and ensuring that a subset of features is optimal. Two various real datasets evaluate the proposed price prediction method. The results support the suggested superiority of the model concerning its relative root mean square error and adjusted correlation coefficient.
SURESTEP: An Uncertainty-Aware Trajectory Optimization Framework to Enhance Visual Tool Tracking for Robust Surgical Automation
Shinde, Nikhil U., Chiu, Zih-Yun, Richter, Florian, Lim, Jason, Zhi, Yuheng, Herbert, Sylvia, Yip, Michael C.
Inaccurate tool localization is one of the main reasons for failures in automating surgical tasks. Imprecise robot kinematics and noisy observations caused by the poor visual acuity of an endoscopic camera make tool tracking challenging. Previous works in surgical automation adopt environment-specific setups or hard-coded strategies instead of explicitly considering motion and observation uncertainty of tool tracking in their policies. In this work, we present SURESTEP, an uncertainty-aware trajectory optimization framework for robust surgical automation. We model the uncertainty of tool tracking with the components motivated by the sources of noise in typical surgical scenes. Using a Gaussian assumption to propagate our uncertainty models through a given tool trajectory, SURESTEP provides a general framework that minimizes the upper bound on the entropy of the final estimated tool distribution. We compare SURESTEP with a baseline method on a real-world suture needle regrasping task under challenging environmental conditions, such as poor lighting and a moving endoscopic camera. The results over 60 regrasps on the da Vinci Research Kit (dVRK) demonstrate that our optimized trajectories significantly outperform the un-optimized baseline.
Improving Generalization via Meta-Learning on Hard Samples
Jain, Nishant, Suggala, Arun S., Shenoy, Pradeep
Learned reweighting (LRW) approaches to supervised learning use an optimization criterion to assign weights for training instances, in order to maximize performance on a representative validation dataset. We pose and formalize the problem of optimized selection of the validation set used in LRW training, to improve classifier generalization. In particular, we show that using hard-to-classify instances in the validation set has both a theoretical connection to, and strong empirical evidence of generalization. We provide an efficient algorithm for training this meta-optimized model, as well as a simple train-twice heuristic for careful comparative study. We demonstrate that LRW with easy validation data performs consistently worse than LRW with hard validation data, establishing the validity of our meta-optimization problem. Our proposed algorithm outperforms a wide range of baselines on a range of datasets and domain shift challenges (Imagenet-1K, CIFAR-100, Clothing-1M, CAMELYON, WILDS, etc.), with ~1% gains using VIT-B on Imagenet. We also show that using naturally hard examples for validation (Imagenet-R / Imagenet-A) in LRW training for Imagenet improves performance on both clean and naturally hard test instances by 1-2%. Secondary analyses show that using hard validation data in an LRW framework improves margins on test data, hinting at the mechanism underlying our empirical gains. We believe this work opens up new research directions for the meta-optimization of meta-learning in a supervised learning context.
A Sequential Quadratic Programming Approach to the Solution of Open-Loop Generalized Nash Equilibria for Autonomous Racing
Zhu, Edward L., Borrelli, Francesco
Dynamic games can be an effective approach for modeling interactive behavior between multiple competitive agents in autonomous racing and they provide a theoretical framework for simultaneous prediction and control in such scenarios. In this work, we propose DG-SQP, a numerical method for the solution of local generalized Nash equilibria (GNE) for open-loop general-sum dynamic games for agents with nonlinear dynamics and constraints. In particular, we formulate a sequential quadratic programming (SQP) approach which requires only the solution of a single convex quadratic program at each iteration. The three key elements of the method are a non-monotonic line search for solving the associated KKT equations, a merit function to handle zero sum costs, and a decaying regularization scheme for SQP step selection. We show that our method achieves linear convergence in the neighborhood of local GNE and demonstrate the effectiveness of the approach in the context of head-to-head car racing, where we show significant improvement in solver success rate when comparing against the state-of-the-art PATH solver for dynamic games. An implementation of our solver can be found at https://github.com/zhu-edward/DGSQP.
Simple inverse kinematics computation considering joint motion efficiency
Yonezawa, Ansei, Yonezawa, Heisei, Kajiwara, Itsuro
Inverse kinematics is an important and challenging problem in the operation of industrial manipulators. This study proposes a simple inverse kinematics calculation scheme for an industrial serial manipulator. The proposed technique can calculate appropriate values of the joint variables to realize the desired end-effector position and orientation while considering the motion costs of each joint. Two scalar functions are defined for the joint variables: one is to evaluate the end-effector position and orientation, whereas the other is to evaluate the motion efficiency of the joints. By combining the two scalar functions, the inverse kinematics calculation of the manipulator is formulated as a numerical optimization problem. Furthermore, a simple algorithm for solving the inverse kinematics via the aforementioned optimization is constructed on the basis of the simultaneous perturbation stochastic approximation with a norm-limited update vector (NLSPSA). The proposed scheme considers not only the accuracy of the position and orientation of the end-effector but also the efficiency of the robot movement. Therefore, it yields a practical result of the inverse problem. Moreover, the proposed algorithm is simple and easy to implement owing to the high calculation efficiency of NLSPSA. Finally, the effectiveness of the proposed method is verified through numerical examples using a redundant manipulator.
Distributed Swarm Learning for Edge Internet of Things
Wang, Yue, Tian, Zhi, Fan, FXin, Cai, Zhipeng, Nowzari, Cameron, Zeng, Kai
The rapid growth of Internet of Things (IoT) has led to Challenge-2: Non-convex optimization. Gradient-based algorithms the widespread deployment of smart IoT devices at wireless get trapped in local optima when tackling non-convex edge for collaborative machine learning tasks, ushering in a problems, e.g., training neural networks with nonlinear activation. With a huge number of hardwareconstrained This problem worsens in distributed learning, particularly IoT devices operating in resource-limited wireless in IoT scenarios where edge devices access limited data. Edge learning including communication and computation bottlenecks, device faces statistical heterogeneity in local training data across and data heterogeneity, security risks, privacy leakages, nonconvex workers, also known as the non-i.i.d. To heterogeneity in IoT hardware capability and link quality, address these issues, this article explores a novel framework which degrades edge learning performance significantly.
Fully Zeroth-Order Bilevel Programming via Gaussian Smoothing
Aghasi, Alireza, Ghadimi, Saeed
We are particularly interested in the setting where neither ex plicit knowledge about f,g are available nor their unbiased stochastic derivatives. In this zeroth-order setting, we assume that only noisy evaluations of f and g are available upon query to an oracle. The BLP problem was first introduced by Bracken and McGill in t he 1970s [7] followed by a more general form of problem involving joint constraints of outer and inner variables. This is a fundamental problem in engineering and economics with dire ct applications in problems such as decision making [48], supply chain [61, 59], network design [51, 43], transportation and planning [16, 83], and optimal design [4, 32]. More recently, BLP has f ound applications in many areas of machine learning and artificial intelligence. Zeroth-order methods apply to many optimization problems ( including the BLP) where for various reasons such as complexity, lack of access to an accurat e model, or computational limitations, there is no or limited access to the objective gradient.
Beyond Suspension: A Two-phase Methodology for Concluding Sports Leagues
Hassanzadeh, Ali, Hosseini, Mojtaba, Turner, John G.
Problem definition: Professional sports leagues may be suspended due to various reasons such as the recent COVID-19 pandemic. A critical question the league must address when re-opening is how to appropriately select a subset of the remaining games to conclude the season in a shortened time frame. Academic/practical relevance: Despite the rich literature on scheduling an entire season starting from a blank slate, concluding an existing season is quite different. Our approach attempts to achieve team rankings similar to that which would have resulted had the season been played out in full. Methodology: We propose a data-driven model which exploits predictive and prescriptive analytics to produce a schedule for the remainder of the season comprised of a subset of originally-scheduled games. Our model introduces novel rankings-based objectives within a stochastic optimization model, whose parameters are first estimated using a predictive model. We introduce a deterministic equivalent reformulation along with a tailored Frank-Wolfe algorithm to efficiently solve our problem, as well as a robust counterpart based on min-max regret. Results: We present simulation-based numerical experiments from previous National Basketball Association (NBA) seasons 2004--2019, and show that our models are computationally efficient, outperform a greedy benchmark that approximates a non-rankings-based scheduling policy, and produce interpretable results. Managerial implications: Our data-driven decision-making framework may be used to produce a shortened season with 25-50\% fewer games while still producing an end-of-season ranking similar to that of the full season, had it been played.
An Optimization-Based Planner with B-spline Parameterized Continuous-Time Reference Signals
Tao, Chuyuan, Cheng, Sheng, Zhao, Yang, Wang, Fanxin, Hovakimyan, Naira
For the cascaded planning and control modules implemented for robot navigation, the frequency gap between the planner and controller has received limited attention. In this study, we introduce a novel B-spline parameterized optimization-based planner (BSPOP) designed to address the frequency gap challenge with limited onboard computational power in robots. The proposed planner generates continuous-time control inputs for low-level controllers running at arbitrary frequencies to track. Furthermore, when considering the convex control action sets, BSPOP uses the convex hull property to automatically constrain the continuous-time control inputs within the convex set. Consequently, compared with the discrete-time optimization-based planners, BSPOP reduces the number of decision variables and inequality constraints, which improves computational efficiency as a byproduct. Simulation results demonstrate that our approach can achieve a comparable planning performance to the high-frequency baseline optimization-based planners while demanding less computational power. Both simulation and experiment results show that the proposed method performs better in planning compared with baseline planners in the same frequency.
Dual Simplex Volume Maximization for Simplex-Structured Matrix Factorization
Abdolali, Maryam, Barbarino, Giovanni, Gillis, Nicolas
Simplex-structured matrix factorization (SSMF) is a generalization of nonnegative matrix factorization, a fundamental interpretable data analysis model, and has applications in hyperspectral unmixing and topic modeling. To obtain identifiable solutions, a standard approach is to find minimum-volume solutions. By taking advantage of the duality/polarity concept for polytopes, we convert minimum-volume SSMF in the primal space to a maximum-volume problem in the dual space. We first prove the identifiability of this maximum-volume dual problem. Then, we use this dual formulation to provide a novel optimization approach which bridges the gap between two existing families of algorithms for SSMF, namely volume minimization and facet identification. Numerical experiments show that the proposed approach performs favorably compared to the state-of-the-art SSMF algorithms.