Robotic perception often requires solving large nonlinear least-squares (NLS) problems. While sparsity has been well-exploited to scale solvers, a complementary and underexploited structure is \emph{separability} -- where some variables (e.g., visual landmarks) appear linearly in the residuals and, for any estimate of the remaining variables (e.g., poses), have a closed-form solution. Variable projection (VarPro) methods are a family of techniques that exploit this structure by analytically eliminating the linear variables and presenting a reduced problem in the remaining variables that has favorable properties. However, VarPro has seen limited use in robotic perception; a major challenge arises from gauge symmetries (e.g., cost invariance to global shifts and rotations), which are common in perception and induce specific computational challenges in standard VarPro approaches. We present a VarPro scheme designed for problems with gauge symmetries that jointly exploits separability and sparsity. Our method can be applied as a one-time preprocessing step to construct a \emph{matrix-free Schur complement operator}. This operator allows efficient evaluation of costs, gradients, and Hessian-vector products of the reduced problem and readily integrates with standard iterative NLS solvers. We provide precise conditions under which our method applies, and describe extensions when these conditions are only partially met. Across synthetic and real benchmarks in SLAM, SNL, and SfM, our approach achieves up to \textbf{2$\times$--35$\times$ faster runtimes} than state-of-the-art methods while maintaining accuracy. We release an open-source C++ implementation and all datasets from our experiments.

This paper presents an overview of the Burer-Monteiro method (BM), a technique that has been applied to solve robot perception problems to certifiable optimality in real-time. BM is often used to solve semidefinite programming relaxations, which can be used to perform global optimization for non-convex perception problems. Specifically, BM leverages the low-rank structure of typical semidefinite programs to dramatically reduce the computational cost of performing optimization. This paper discusses BM in certifiable perception, with three main objectives: (i) to consolidate information from the literature into a unified presentation, (ii) to elucidate the role of the linear independence constraint qualification (LICQ), a concept not yet well-covered in certifiable perception literature, and (iii) to share practical considerations that are discussed among practitioners but not thoroughly covered in the literature. Our general aim is to offer a practical primer for applying BM towards certifiable perception.

Geometric perception problems are fundamental tasks in robotics and computer vision. In real-world applications, they often encounter the inevitable issue of outliers, preventing traditional algorithms from making correct estimates. In this paper, we present a novel general-purpose robust estimator TIVM (Thresholding with Intra-class Variance Maximization) that can collaborate with standard non-minimal solvers to efficiently reject outliers for geometric perception problems. First, we introduce the technique of intra-class variance maximization to design a dynamic 2-group thresholding method on the measurement residuals, aiming to distinctively separate inliers from outliers. Then, we develop an iterative framework that robustly optimizes the model by approaching the pure-inlier group using a multi-layered dynamic thresholding strategy as subroutine, in which a self-adaptive mechanism for layer-number tuning is further employed to minimize the user-defined parameters. We validate the proposed estimator on 3 classic geometric perception problems: rotation averaging, point cloud registration and category-level perception, and experiments show that it is robust against 70--90\% of outliers and can converge typically in only 3--15 iterations, much faster than state-of-the-art robust solvers such as RANSAC, GNC and ADAPT. Furthermore, another highlight is that: our estimator can retain approximately the same level of robustness even when the inlier-noise statistics of the problem are fully unknown.

For machine learning perception problems, human-level classification performance is used as an estimate of top algorithm performance. Thus, it is important to understand as precisely as possible the factors that impact human-level performance. Knowing this 1) provides a benchmark for model performance, 2) tells a project manager what type of data to obtain for human labelers in order to get accurate labels, and 3) enables ground-truth analysis--largely conducted by humans--to be carried out smoothly. In this empirical study, we explored the relationship between resolution and human classification performance using the MNIST data set down-sampled to various resolutions. The quantitative heuristic we derived could prove useful for predicting machine model performance, predicting data storage requirements, and saving valuable resources in the deployment of machine learning projects. It also has the potential to be used in a wide variety of fields such as remote sensing, medical imaging, scientific imaging, and astronomy.

What inspired you to take an interest in robots? I've always played with robots. For example, I remember a competition in high school where my friends and I built a robotic arm to move the chess pieces on the chessboard. It seems very trivial now, but way back then, the robots were all primitive and as high school students, we thought that building a robot that could do that was a big deal. Graduate students Ashutosh Saxena, left, and Morgan Quigley, center, and Ng were part of a large effort to develop a robot to see an unfamiliar object and ascertain the best spot to grasp it.

"Andrew Ingram" is a digital assistant that scans your emails, gives scheduling ideas for the meetings and appointments you discuss with your coworkers, sets up tasks, and sends invites to the relevant parties with very little assistance. It uses the advanced artificial-intelligence capabilities of X.ai, a New York–based startup that specializes in developing AI assistants. The problems it solves can save a lot of time and frustration for people (like me) who have a messy schedule. But according to a Wired story published in May, the intelligence behind Andrew Ingram is not totally artificial. It's backed by a group of 40 Filipinos in a highly secured building on the outskirts of Manila who monitor the AI's behavior and take over whenever the assistant runs into a case it can't handle.

For self-driving cars and other smart transport to be successfully integrated in the real-world, the safety of passengers and pedestrians must be ensured. In the world of intelligent machines, perception answers the question: what is around me? This situational awareness is paramount for safe operation of autonomous vehicles in real-world environments. Scientists working in this field point to robotic perception as fundamental in equipping machines with a semantic understanding of the world, so that they can reliably identify objects and make informed predictions and actions. Michael Milford, Associate Professor at Queensland University of Technology (QUT), is a leading robotics researcher working to improve perception and more in autonomous vehicles, conducting his research at the intersection of robotics, neuroscience and computer vision.

Almost all algorithmic assistants that utilize unsupervised machine learning have several skill sets based on modern data science. They can baseline normal behavior by accurately modeling time series data (any series of data with a time stamp on it – usually log data from servers, devices, endpoints, and applications); they can identify data points that are anomalous or "outliers;" and they can score the level of anomalousness of these outliers. Generally, you'll hear this set of skills packaged up under the term "machine learning anomaly detection."