A policy is said to be robust if it maximizes the reward while considering a bad, or even adversarial, model. In this work we formalize two new criteria of robustness to action uncertainty. Specifically, we consider two scenarios in which the agent attempts to perform an action $\mathbf{a}$, and (i) with probability $\alpha$, an alternative adversarial action $\bar{\mathbf{a}}$ is taken, or (ii) an adversary adds a perturbation to the selected action in the case of continuous action space. We show that our criteria are related to common forms of uncertainty in robotics domains, such as the occurrence of abrupt forces, and suggest algorithms in the tabular case. Building on the suggested algorithms, we generalize our approach to deep reinforcement learning (DRL) and provide extensive experiments in the various MuJoCo domains. Our experiments show that not only does our approach produce robust policies, but it also improves the performance in the absence of perturbations. This generalization indicates that action-robustness can be thought of as implicit regularization in RL problems.

We prove that for an $L$-layer fully-connected linear neural network, if the width of every hidden layer is $\tilde\Omega (L \cdot r \cdot d_{\mathrm{out}} \cdot \kappa^3 )$, where $r$ and $\kappa$ are the rank and the condition number of the input data, and $d_{\mathrm{out}}$ is the output dimension, then gradient descent with Gaussian random initialization converges to a global minimum at a linear rate. The number of iterations to find an $\epsilon$-suboptimal solution is $O(\kappa \log(\frac{1}{\epsilon}))$. Our polynomial upper bound on the total running time for wide deep linear networks and the $\exp\left(\Omega\left(L\right)\right)$ lower bound for narrow deep linear neural networks [Shamir, 2018] together demonstrate that wide layers are necessary for optimizing deep models.

We study the training process of Deep Neural Networks (DNNs) from the Fourier analysis perspective. Our starting point is a Frequency Principle (F-Principle) --- DNNs initialized with small parameters often fit target functions from low to high frequencies --- which was first proposed by Xu et al. (2018) and Rahaman et al. (2018) on synthetic datasets. In this work, we first show the universality of the F-Principle by demonstrating this phenomenon on high-dimensional benchmark datasets, such as MNIST and CIFAR10. Then, based on experiments, we show that the F-Principle provides insight into both the success and failure of DNNs in different types of problems. Based on the F-Principle, we further propose that DNN can be adopted to accelerate the convergence of low frequencies for scientific computing problems, in which most of the conventional methods (e.g., Jacobi method) exhibit the opposite convergence behavior --- faster convergence for higher frequencies. Finally, we prove a theorem for DNNs of one hidden layer as a first step towards a mathematical explanation of the F-Principle. Our work indicates that the F-Principle with Fourier analysis is a promising approach to the study of DNNs because it seems ubiquitous, applicable, and explainable.

Learning with noisy labels is one of the hottest problems in weakly-supervised learning. Based on memorization effects of deep neural networks, training on small-loss instances becomes very promising for handling noisy labels. This fosters the state-of-the-art approach "Co-teaching" that cross-trains two deep neural networks using the small-loss trick. However, with the increase of epochs, two networks converge to a consensus and Co-teaching reduces to the self-training MentorNet. To tackle this issue, we propose a robust learning paradigm called Co-teaching+, which bridges the "Update by Disagreement" strategy with the original Co-teaching. First, two networks feed forward and predict all data, but keep prediction disagreement data only. Then, among such disagreement data, each network selects its small-loss data, but back propagates the small-loss data from its peer network and updates its own parameters. Empirical results on benchmark datasets demonstrate that Co-teaching+ is much superior to many state-of-the-art methods in the robustness of trained models.

In the 21st century, governments cannot ignore how changes in technology will affect employment and political stability. The automation of work – principally through robotics, artificial intelligence (AI) and the Internet of things (IoT), collectively known as the Fourth Industrial Revolution – will provide an unprecedented boost to productivity and profit. It will also threaten the stability of low- and mid-skilled jobs in many developing and middle-income countries. Developing countries must begin seriously considering how technological changes will impact labour trends. Technology now looms just as large a disruptive force, if not larger, than the whims of global capital.

If you want to understand more about the power and potential of AI and machine learning, TED Talks are a great start. Presented by thought leaders in the field of artificial intelligence, TED Talks give anyone the opportunity to gain insider knowledge from an expert's perspective. In an inspiring conversation with TED Curator Chris Anderson, educator and entrepreneur Sebastian Thrun discusses the progress of deep learning, why we shouldn't fear AI, and how society at large will benefit from machine learning technology. Gerbert walks through the fundamentals of AI and what it can mean for your business. How can we help kids excel at things that humans will always do better than AI? AI expert Noriko Arai and her team created Todai Robot, whose sole purpose is to pass the entrance exam for the University of Tokyo.

As concerns over security risks for connected vehicles continue to build, automotive cybersecurity company SafeRide Technologies believes unsupervised machine learning will help keep threat actors out of the driver's seats. Earlier this month, SafeRide launched its vXRay technology for connected vehicles' security operations center (SOC), which uses unsupervised machine learning technology to provide behavioral profiling and anomaly detection to improve connected vehicle security. Gil Reiter, vice president of product management and marketing at SafeRide, based in Tel Aviv, Israel, said vXRay is available for OEMs and fleet managers to integrate in their vehicles' SOC. "The vXRay technology establishes the normal behavior of the vehicle without any dependencies or without any knowledge of the specific electronic control unit properties," Reiter said. "Once the behavioral baseline of the vehicle is established, the technology can accurately detect and then flag any abnormal behavior of the vehicle system and report the abnormal behavior to the connected vehicle's SOC for further analysis."

Video Friday is your weekly selection of awesome robotics videos, collected by your Automaton bloggers. We'll also be posting a weekly calendar of upcoming robotics events for the next few months; here's what we have so far (send us your events!): Let us know if you have suggestions for next week, and enjoy today's videos. At Amazon, we continually invest in new technologies to benefit customers. We've been hard at work developing a new, fully-electric delivery system – Amazon Scout – designed to safely get packages to customers using autonomous delivery devices.

Machine learning is growing ever more sophisticated, thanks to algorithms which pit two artificial intelligences against each other. These algorithms, known as generative adversarial networks (GANs), have already been used to create art, crack encryption codes, and produce uncannily real pictures of faces and animals. Researchers have now combined GANs with another hot technology: quantum computing. Luyan Sun at Tsinghua University in Beijing, China and his colleagues have created a GAN on a quantum circuit.

Kids are particularly terrible for robots. At least, that's what researchers in Japan discovered when they let a robot roam around a shopping center in Osaka in 2015. A group of kids antagonized the robot, forcing the researchers to program an algorithm that would give the bot the agency to evade abuse. That's just one example of challenging social interactions between humans and robots, and one that technologists have almost certainly considered when building and designing delivery bots. Including the folks at Amazon: This week, the e-commerce behemoth dropped a web page for Scout, its new delivery robot.