A new job figure is silently emerging: teleoperators, that is human piloting robots. While remotely completing a task is useful by itself, there is much more to it. Every successful trial can be logged, building a dataset of experiences. If done properly, we can build an infinitely reusable learning resource to train any number of autonomous robots to perform the same tasks. Here I will go over the potential and unsolved problems in teleoperation, review selected projects in data-driven learning and speculate on the evolution and opportunities in the nascent teleoperation industry, with focus on manipulation. Introduction Building useful AI agents is hard. Since we didn’t figure out a priori how to build general intelligence, the best we can as of today is to take a statistical regressive approach: build a huge dataset which incorporates the behaviour we would like to see, take a predictive function with a lot of free parameters and finally write an algorithm to tune these parameters so that the function is often correct when faced with a data point similar to the ones in the dataset. You may not like it, but the bitter lesson is that it works reasonably well, since we have powerful computers. Now, building useful physical AI agents, a.k.a robots, is harder. Traditional approaches require online interactions: the robot is learning while performing the actions. While this works well in virtual environments such as video games, deploying a baby robot to learn in the real world is costly and dangerous; moreover such online algorithms are not suited to reuse past experiences. To overcome these problems, a promising direction is data-driven reinforcement learning (also named offline-RL or batch-RL), in which we train agents offline on a dataset of already collected experiences. The training can be done virtually and then the learned skills, usually in the form of a neural network, can be deployed to a real robot. If we assume that offline-RL works well we are now able to reuse past-experiences indefinitely, but we still haven’t solved the most crucial problem: the access to a large dataset. As of today, we simply miss the large datasets of robot-experiences needed to power the learning, in the same way as large amounts of labelled pictures and text powered advances in computer vision and natural language processing. Teleoperation has entered the chat. Teleoperation Teleoperation, or Telerobotics, is about separating the brain and body: the human operators control the movements and take decisions, the robot executes. In fact there are different degrees of teleoperation: Direct Control: The operator is controlling the motion of the robot directly and without any automated help. Shared Control: ​​Some degree of autonomy or automated help is available to assist the user. Such autonomy is set in advance and is fixed. Shared Autonomy: Same as Shared Control, but the level of autonomy is adjusted dynamically (and autonomously!) according to the situation. Supervisory Control: The control happens at a very high level, the robot executes nearly all the functions autonomously. (to go deeper, check Autonomy in Physical Human-Robot Interaction: a Brief Survey and the classic reference (cap 43)). The robot can really be anything: a drone, a manipulator, a vehicle or a humanoid. The robot hardware often dictates how we command the robot; a non exhaustive list of controllers includes joysticks, steering wheels, virtual reality kits, twin robotic arms, haptic controllers and suites, electromyography sensors and tracking the operator body with a camera, using fiducial markers or AI body tracking! Despite being old, with roots going back to the 1940s and 1950s in the remote manipulation of radioactive waste, teleoperation is still not a mature technology. My favourite overview of teleoperation shortcomings remains this 2007 paper, which mentions 8 limiting factors: cameras narrow field of view, figuring out the robot orientation and attitude, multi camera setups logistic, low frame rates, degradation due to motion, egocentric-vs-exocentric camera view tradeoffs, depth perception and stream latency. While all these directions still need to be perfected, in my experience the latency and its unpredictability, that is time delays in the sensor stream, remains the largest bottleneck to a fluid teleoperation experience. In this spirit, teleoperating fixed robots such as manipulators is going to be much easier than mobile robots in outdoor environments since the former can use wired connections, while the latter are forced to rely on cellular connections. Currently there is a significant amount of research into improving the operator User Interface. For instance to alleviate latency it’s possible to overlay a predictive model to the real time feed, such as a “ghost“ of the future robot state. In this way the operator has the illusion to control a zero latency robot. Virtual fixtures and augmented reality markers can also help. It is also possible to train an AI to map low dimensional inputs into complex actions, so that the operator can perform complex tasks with a simple joystick. For instance if a manipulator needs to grasp a cup on a table, the operator can simply instruct the manipulator to get closer to the cup and the AI will infer from the camera scene that the operator is looking to grasp it, therefore controlling all the fine motor skills required in the grasping. As a side effect, easier controls allow low skilled teleoperators to operate complex scenarios, which is important since today there are few expert operators. It’s worth stressing that all this area around shared control and UI needs careful engineering, indeed paradoxically shared control can be harder than just automating, since we need to take into account how the operator reacts to the partial automation. It must be done well, keeping automated and human tasks separated and smoothly glued, so that the automation does not surprise the operator, causing the typical wait and see behaviour (input a command, wait for the robot to finish the movement, input another command, repeat). In closing this intro, we can very crudely divide teleoperation tasks into two macro categories: driving and manipulation. These two classes present opposite challenges: driving (which includes piloting drones, cars and robot dogs walking) difficulty comes from the unpredictability of the environment and the requirement for fast reaction times, while in terms of control it’s easy (brake, accelerate, turn the steering wheel and not much more). Manipulation instead usually operates in slowly varying or fixed environments and in non time critical scenarios, but the controls are very nuanced and high dimensional. So for the rest of the article I will focus on robot manipulators, since that’s where the biggest learning challenges lie. The Nascent Teleoperation Industry and Bootstrapping Automation Today teleoperation is mainly being used in high touch use cases, such as medical surgery, nuclear decommissioning, space and undersea robotics. The operators are expensive professional domain experts, but their cost is justified since the alternative is too expensive or dangerous. With the increased quality and dropping cost of collaborative robots and the advancements in artificial intelligence in the next few years teleoperation will expand to service use cases, such as in warehouses, light manufacturing, commercial kitchens and labs. How is this possible? Having a teleoperator, even if low skilled, continuously tele operating the robot will rarely make sense. In fact what will happen is that teleoperation will be used to bootstrap automation and then for remote assistance. To understand why, it’s important to recall why automation is hard in the first place. Robots have superhuman precision already, so automating a fixed scenario it’s always possible with superhuman performance. The problem is that in real life no two environments are the same: different lighting, different objects to interact with, different arrangements, different success criteria. Considering these needs for autonomy and flexibility, here’s the example of how the development and deployment of a kitchen robot manipulator which is tasked to assemble rice bowls may look like: The robot manipulator is trained to learn from a mixture of teleoperated, simulated and unsupervised experiences, from a standardised kitchen, using popular ingredients, tools and appliances. Every experience is divided into a set of basic actions, such as pick&place, mix, pour, sprinkle and it is logged as camera streams, position and velocity of the robot joints, force sensors and any other sensor measurement available. To every experience a score is also assigned, so that the robot learns what is an acceptable end state. After months of development the robot is able to reliably prepare bowls in the training scenario, but the performance in a different kitchen would be poor. The system is deployed in a real kitchen, but for the first weeks a teleoperator has direct control of the robot, getting feedback from the kitchen owner. Every single experience with the new setup is logged and the AI is trained to imitate the operator. The AI performance is continuously tested, by asking in real time what action it would take and then confronting with the action that the operator actually takes. After a few weeks the AI is accurate enough to be left in control. The teleperator is called a few times a day to iron out the edge cases in which the AI keeps making mistakes. After some time the error rate is so low that a single teleoperator can assist more than 50 deployed robots at the same time. The aggregate experiences are trained offline and the robot firmware is routinely updated, so that the robot reaches superhuman performances. Future deployments proceed as in step 2, but the coaching time of the teleoperator keeps decreasing as the global dataset of experience increases in size. Eventually a few hours of demonstrations are enough to onboard a new kitchen. Also deploying the robot for different tasks becomes gradually easier, as basic actions such as pick&place can be reused. In a first instance, robotic companies will vertically integrate and have their own fleet of teleoperators. Eventually specialised infrastructure providers will leverage economies of scale to provide teleoperation-as-a-service, providing flexible fleets of operators when needed. This is similar to how companies providing dataset labelling services operate today, but teleoperation is destined to be a much larger industry as the value of the market size being automated is bigger and the need for teleoperation assistance persists after the initial training. Where the teleoperators are actually located will depend on how critical latency is and what are the regulations and liabilities around remote work for physical tasks, something which at the moment is pretty niche. Besides professional services with strict accuracy standards and trained operators, it will be possible to crowdsource demonstrations from the public, perhaps even inside gamified environments. In the long run, as teleoperation tooling and humanoid robots get cheaper, teleoperation will be rolled out to consumers for tele-existence. Hopefully by then we will not worry about work, and the main use cases will be around entertainment, social connections and exploration. Data-Driven Reinforcement Learning Today As said, offline reinforcement learning will be critical to scale since having a real robot to learn in a real environment is too slow and dangerous. Here I will scratch the tip of the research iceberg and mention a few approaches to tackle the offline reinforcement learning pipeline. A crucial element in building a dataset for offline RL is establishing the reward of each experience. This is in my view the strongest explanation as to why a teleoperator is needed to bootstrap automation: the operator needs to understand what “good” means for every single deployment and act accordingly, iterating over the feedback of the new robot owner. In this light, I find the ideas in Scaling data-driven robotics with reward sketching and batch reinforcement learning pretty interesting. Firstly they provide an intuitive mechanism to sketch the reward of a given experience, so that every single camera frame is rated according to how close we are to the desired goal. This helps having a more granular reward distribution than just rating trajectories as good or bad, even though it introduces some degree of subjectivity, since different operators will have different definitions of being close to the goal. More importantly, based on the human labelled rewards, they propose a mechanism to automatically relabel all the dataset accumulated over previous experiences, so that a large amount of data can be leveraged to learn a new task out a few initial demonstrations. Basically the reward annotations produced by the sketching procedure are used to train a reward model, which is then used to predict the reward of all the past data, according to the new definition of success. Ideally a dataset of 10.000 demonstrations for task A can be relabelled to be a dataset of 10.000 demonstrations for task B, assuming that the tasks are not too different. This approach is somewhat opposite to the usual deep learning paradigm of pretraining an AI agent on a large heterogeneous dataset and then fine-tuning using a small amount of data coming from the use case of interest. A more traditional approach is followed in Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasets, where a medium-size dataset (7200 demonstrations) collected from 71 kitchen tasks is used to bootstrap training on 10 unseed tasks, resulting in a 2x performance improvement with respect to just training the new tasks from scratch. It remains an open research question to quantify how much we can push the performance improvements if we leverage a truly large dataset, containing millions of demonstrations. To my knowledge the easiest resource to get started with offline RL is Robomimic, an open source framework to learn from demonstrations. It provides a set of standardised datasets containing action-state-reward trajectories, with emphasis on human-provided demonstrations and support for multiple observation spaces, including visuomotor policies. The datasets available also contain different sources such as single expert teleoperators, multiple teleoperators and machine-generated trajectories across several simulated and real-world tasks. It contains implementations for several offline learning and imitation learning algorithms, including Behaviour Cloning, Behaviour Cloning-RNN, HBC, IRIS, BCQ, CQL, and TD3-BC (by the way, when to use offline reinforcement learning vs imitation learning?). In the same ecosystem we find RoboTurk, a project to lower the barrier to create large scale crowdsourced datasets and RoboSuite a Mujoco-based simulation framework with benchmark environments. Robomimic is well structured, with a clear documentation instructing on how to create a dataset and train an agent. It is still a bit rough, but hopefully the project will keep being maintained so that other projects can avoid duplicating efforts when training robots. Finally I want to mention d3rlpy, a library for offline and online reinforcement learning. The best thing about it is that it’s intuitive and well documented, with clear examples already available in the README. Outro Bringing artificial intelligence to the real world is going to be hard, many and many practitioners agree. But the takeaway message I want to convey is the following: there is a way to bootstrap automation in the short term, relying heavily on human-in-the-loop teleoperation. Such reliance on humans should be seen as a feature, not as a bug. Commercialising a novel product is always a massive undertaking, but in robotics this is exacerbated by the slow hardware development cycle. A teleop-first approach shortens the iteration cycles, incorporates feedback from the end-user and creates a pool of experiences on which to build scalable solutions.

Bayesian optimization (BO) is a sample-efficient approach for tuning design parameters to optimize expensive-to-evaluate, black-box performance metrics. In many manufacturing processes, the design parameters are subject to random input noise, resulting in a product that is often less performant than expected. Although BO methods have been proposed for optimizing a single objective under input noise, no existing method addresses the practical scenario where there are multiple objectives that are sensitive to input perturbations. In this work, we propose the first multi-objective BO method that is robust to input noise. We formalize our goal as optimizing the multivariate value-at-risk (MVaR), a risk measure of the uncertain objectives. Since directly optimizing MVaR is computationally infeasible in many settings, we propose a scalable, theoretically-grounded approach for optimizing MVaR using random scalarizations. Empirically, we find that our approach significantly outperforms alternative methods and efficiently identifies optimal robust designs that will satisfy specifications across multiple metrics with high probability.

The world is changing at a fast pace and almost the entire credit goes to artificial intelligence. AI has penetrated our professional and personal lives in such a way that it becomes inherent for us to need more innovative and automated solutions for our daily activities. Advanced technologies like AI and machine learning have their own disadvantages and banes, including the potential for bias, exploitation, and such other misuses, nonetheless, it has opened unprecedented ways of approaching every facet of business and our daily lives. Researchers and scientists are relentlessly striving to mitigate all technological constraints and innovate new branches of artificial intelligence to seamlessly automate processes. One such innovative branch of AI is artificial general intelligence (AGI).

We address a three-tier data-driven approach to solve the inverse problem in complex systems modelling from spatio-temporal data produced by microscopic simulators using machine learning. In the first step, we exploit manifold learning and in particular parsimonious Diffusion Maps using leave-one-out cross-validation (LOOCV) to both identify the intrinsic dimension of the manifold where the emergent dynamics evolve and for feature selection over the parametric space. In the second step, based on the selected features, we learn the right-hand-side of the effective partial differential equations (PDEs) using two machine learning schemes, namely shallow Feedforward Neural Networks (FNNs) with two hidden layers and single-layer Random Projection Networks(RPNNs) which basis functions are constructed using an appropriate random sampling approach. Finally, based on the learned black-box PDE model, we construct the corresponding bifurcation diagram, thus exploiting the numerical bifurcation analysis toolkit. For our illustrations, we implemented the proposed method to construct the one-parameter bifurcation diagram of the 1D FitzHugh-Nagumo PDEs from data generated by $D1Q3$ Lattice Boltzmann simulations. The proposed method was quite effective in terms of numerical accuracy regarding the construction of the coarse-scale bifurcation diagram. Furthermore, the proposed RPNN scheme was $\sim$ 20 to 30 times less costly regarding the training phase than the traditional shallow FNNs, thus arising as a promising alternative to deep learning for solving the inverse problem for high-dimensional PDEs.

Accurate electricity demand forecasts play a crucial role in sustainable power systems. To enable better decision-making especially for demand flexibility of the end-user, it is necessary to provide not only accurate but also understandable and actionable forecasts. To provide accurate forecasts Global Forecasting Models (GFM) trained across time series have shown superior results in many demand forecasting competitions and real-world applications recently, compared with univariate forecasting approaches. We aim to fill the gap between the accuracy and the interpretability in global forecasting approaches. In order to explain the global model forecasts, we propose Local Interpretable Model-agnostic Rule-based Explanations for Forecasting (LIMREF), a local explainer framework that produces k-optimal impact rules for a particular forecast, considering the global forecasting model as a black-box model, in a model-agnostic way. It provides different types of rules that explain the forecast of the global model and the counterfactual rules, which provide actionable insights for potential changes to obtain different outputs for given instances. We conduct experiments using a large-scale electricity demand dataset with exogenous features such as temperature and calendar effects. Here, we evaluate the quality of the explanations produced by the LIMREF framework in terms of both qualitative and quantitative aspects such as accuracy, fidelity, and comprehensibility and benchmark those against other local explainers.

Composing Graph Neural Networks (GNNs) of operations that respect physical symmetries has been suggested to give better model performance with a smaller number of learnable parameters. However, real-world applications, such as in high energy physics have not born this out. We present the novel architecture VecNet that combines both symmetry-respecting and unconstrained operations to study and tune the degree of physics-informed GNNs. We introduce a novel metric, the \textit{ant factor}, to quantify the resource-efficiency of each configuration in the search-space. We find that a generalized architecture such as ours can deliver optimal performance in resource-constrained applications.

Smart grid systems are critical to the power industry, however their sophisticated architectural design and operations expose them to a number of cybersecurity threats, such as data tampering, data eavesdropping, and Denial of Service, among others. Artificial Intelligence (AI)-based technologies are becoming increasingly popular for detecting cyber assaults in a variety of computer settings, and several efforts have been made to secure various systems. The present AI systems are being exposed and vanquished because of the recent emergence of sophisticated adversarial systems such as Generative Adversarial Networks (GAN). The purpose of this short review is to outline some of the initiatives to protect smart grid systems, their obstacles, and what might be a potential future AI research direction

Battery discharge capacity forecasting is critically essential for the applications of lithium-ion batteries. The capacity degeneration can be treated as the memory of the initial battery state of charge from the data point of view. The streaming sensor data collected by battery management systems (BMS) reflect the usable battery capacity degradation rates under various operational working conditions. The battery capacity in different cycles can be measured with the temporal patterns extracted from the streaming sensor data based on the attention mechanism. The attention-based similarity regarding the first cycle can describe the battery capacity degradation in the following cycles. The deep degradation network (DDN) is developed with the attention mechanism to measure similarity and predict battery capacity. The DDN model can extract the degeneration-related temporal patterns from the streaming sensor data and perform the battery capacity prediction efficiently online in real-time. Based on the MIT-Stanford open-access battery aging dataset, the root-mean-square error of the capacity estimation is 1.3 mAh. The mean absolute percentage error of the proposed DDN model is 0.06{\%}. The DDN model also performance well in the Oxford Battery Degradation Dataset with dynamic load profiles. Therefore, the high accuracy and strong robustness of the proposed algorithm are verified.

The rising popularity of self-driving cars has led to the emergence of a new research field in the recent years: Autonomous racing. Researchers are developing software and hardware for high performance race vehicles which aim to operate autonomously on the edge of the vehicles limits: High speeds, high accelerations, low reaction times, highly uncertain, dynamic and adversarial environments. This paper represents the first holistic survey that covers the research in the field of autonomous racing. We focus on the field of autonomous racecars only and display the algorithms, methods and approaches that are used in the fields of perception, planning and control as well as end-to-end learning. Further, with an increasing number of autonomous racing competitions, researchers now have access to a range of high performance platforms to test and evaluate their autonomy algorithms. This survey presents a comprehensive overview of the current autonomous racing platforms emphasizing both the software-hardware co-evolution to the current stage. Finally, based on additional discussion with leading researchers in the field we conclude with a summary of open research challenges that will guide future researchers in this field.

Li-Ion Solid-State Electrolytes (Li-SSEs) are a promising solution that resolves the critical issues of conventional Li-Ion Batteries (LIBs) such as poor ionic conductivity, interfacial instability, and dendrites growth. In this study, a platform consisting of a high-throughput screening and a machine-learning surrogate model for discovering superionic Li-SSEs among 20,237 Li-containing materials is developed. For the training database, the ionic conductivity of Na SuperIonic CONductor (NASICON) and Li SuperIonic CONductor (LISICON) type SSEs are obtained from the previous literature. Then, the chemical descriptor (CD) and additional structural properties are used as machine-readable features. Li-SSE candidates are selected through the screening criteria, and the prediction on the ionic conductivity of those is followed. Then, to reduce uncertainty in the surrogate model, the ensemble method by considering the best-performing two models is employed, whose mean prediction accuracy is 0.843 and 0.829, respectively. Furthermore, first-principles calculations are conducted for confirming the ionic conductivity of the strong candidates. Finally, six potential superionic Li-SSEs that have not previously been investigated are proposed. We believe that the constructed platform can accelerate the search for Li-SSEs with high ionic conductivity at minimum cost.