This paper studies how encouraging semantically-aligned features during deep neural network training can increase network robustness. Recent works observed that Adversarial Training leads to robust models, whose learnt features appear to correlate with human perception. Inspired by this connection from robustness to semantics, we study the complementary connection: from semantics to robustness. To do so, we provide a tight robustness certificate for distance-based classification models (clustering-based classifiers), which we leverage to propose ClusTR (Clustering Training for Robustness), a clustering-based and adversary-free training framework to learn robust models. Interestingly, ClusTR outperforms adversarially-trained networks by up to 4\% under strong PGD attacks. Moreover, it can be equipped with simple and fast adversarial training to improve the current state-of-the-art in robustness by 16\%-29\% on CIFAR10, SVHN, and CIFAR100.

If we compute the partial derivatives of the cross-entropy relatively to all the weights and all the biases we obtain a "gradient", computed for a given image, label, and present value of weights and biases. Remember that we can have millions of weights and biases so computing the gradient sounds like a lot of work. Fortunately, TensorFlow does it for us. The mathematical property of a gradient is that it points "up". Since we want to go where the cross-entropy is low, we go in the opposite direction.

Everybody dances, every culture throughout history has danced. But our days of monopolizing the move busting market could soon be coming to an end, as Facebook AI has become the latest team to teach an AI to bop along to the beat. "In this work, we focus on designing interesting choreographies by combining the best of what humans are naturally good at – heuristics of'good' dance that an audience might find appealing – and what machines are good at – optimizing well-defined objective functions," the team wrote in a study published Tuesday. This isn't the first time we've tried to teach AIs to dance. In 2016, Swedish Choreographer Louise Crnkovic-Friis and her husband, Peltarion CEO Luka Crnkovic-Friis, trained a recurrent neural network, dubbed Chor-rnn, on 48 hours of Louise's movements.

One of the many benefits of using artificial intelligence (AI) is to help us view societal problems from a different perspective. While there's been much hubbub about how AI might be misused, we must not overlook the many ways AI can be used for good. Our global issues are complex, and AI provides us with a valuable tool to augment human efforts to come up with solutions to vexing problems. Here are 10 of the best ways artificial intelligence is used for good. Artificial intelligence, powered by deep-learning algorithms, is already in use in healthcare.

In today's world, with the advancement of technologies and science, autonomous cars and AI-based self-driving cars are made into the present. But to work it perfectly, vehicles should be able to classify or interpret traffic signals and signs. In this tutorial, we'll be going through a project that is based on Image classification with Deep Learning. We'll be classifying 43 classes of traffic signs images. We will build a deep neural network model to classify which class a traffic sign belongs to.

Behind these smart drones are well-trained deep-learning models based on Baidu's PaddlePaddle, the first open-source deep-learning platform in China. Like mainstream AI frameworks such as Google's TensorFlow and Facebook's PyTorch, PaddlePaddle, which was open sourced in 2016, provides software developers of all skill levels with the tools, services, and resources they need to rapidly adopt and implement deep learning at scale. PaddlePaddle is being used by more than 1.9 million developers and 84,000 enterprises globally. Industries throughout China are using the platform to create specialized applications for their sectors, from the automotive industry's acceleration of autonomous vehicles to the health-care industry's applications for fighting covid-19. Indeed, the coronavirus pandemic, which has spread over 150 countries and caused a worldwide economic shock, is increasing demands for AI transformation.

Throughout this paper, we focus on the improvement of the direct feedback alignment (DFA) algorithm and extend the usage of the DFA to convolutional and recurrent neural networks (CNNs and RNNs). Even though the DFA algorithm is biologically plausible and has a potential of high-speed training, it has not been considered as the substitute for back-propagation (BP) due to the low accuracy in the CNN and RNN training. In this work, we propose a new DFA algorithm for BP-level accurate CNN and RNN training. Firstly, we divide the network into several modules and apply the DFA algorithm within the module. Second, the DFA with the sparse backward weight is applied. It comes with a form of dilated convolution in the CNN case, and in a form of sparse matrix multiplication in the RNN case. Additionally, the error propagation method of CNN becomes simpler through the group convolution. Finally, hybrid DFA increases the accuracy of the CNN and RNN training to the BP-level while taking advantage of the parallelism and hardware efficiency of the DFA algorithm.

Despite being the workhorse of deep learning, the backpropagation algorithm is no panacea. It enforces sequential layer updates, thus preventing efficient parallelization of the training process. Furthermore, its biological plausibility is being challenged. Alternative schemes have been devised; yet, under the constraint of synaptic asymmetry, none have scaled to modern deep learning tasks and architectures. Here, we challenge this perspective, and study the applicability of Direct Feedback Alignment (DFA) to neural view synthesis, recommender systems, geometric learning, and natural language processing. In contrast with previous studies limited to computer vision tasks, our findings show that it successfully trains a large range of state-of-the-art deep learning architectures, with performance close to fine-tuned backpropagation. When a larger gap between DFA and backpropagation exists, like in Transformers, we attribute this to a need to rethink common practices for large and complex architectures. At variance with common beliefs, our work supports that challenging tasks can be tackled in the absence of weight transport.

The ability to predict and plan into the future is fundamental for agents acting in the world. To reach a faraway goal, we predict trajectories at multiple timescales, first devising a coarse plan towards the goal and then gradually filling in details. In contrast, current learning approaches for visual prediction and planning fail on long-horizon tasks as they generate predictions (1) without considering goal information, and (2) at the finest temporal resolution, one step at a time. In this work we propose a framework for visual prediction and planning that is able to overcome both of these limitations. First, we formulate the problem of predicting towards a goal and propose the corresponding class of latent space goal-conditioned predictors (GCPs). GCPs significantly improve planning efficiency by constraining the search space to only those trajectories that reach the goal. Further, we show how GCPs can be naturally formulated as hierarchical models that, given two observations, predict an observation between them, and by recursively subdividing each part of the trajectory generate complete sequences. This divide-and-conquer strategy is effective at long-term prediction, and enables us to design an effective hierarchical planning algorithm that optimizes trajectories in a coarse-to-fine manner. We show that by using both goal-conditioning and hierarchical prediction, GCPs enable us to solve visual planning tasks with much longer horizon than previously possible.

In countries that enabled patients to choose their own providers, a common problem is that the patients did not make rational decisions, and hence, fail to use healthcare resources efficiently. This might cause problems such as overwhelming tertiary facilities with mild condition patients, thus limiting their capacity of treating acute and critical patients. To address such maldistributed patient volume, it is essential to oversee patients choices before further evaluation of a policy or resource allocation. This study used nationwide insurance data, accumulated possible features discussed in existing literature, and used a deep neural network to predict the patients choices of hospital levels. This study also used explainable artificial intelligence methods to interpret the contribution of features for the general public and individuals. In addition, we explored the effectiveness of changing data representations. The results showed that the model was able to predict with high area under the receiver operating characteristics curve (AUC) (0.90), accuracy (0.90), sensitivity (0.94), and specificity (0.97) with highly imbalanced label. Generally, social approval of the provider by the general public (positive or negative) and the number of practicing physicians serving per ten thousand people of the located area are listed as the top effecting features. The changing data representation had a positive effect on the prediction improvement. Deep learning methods can process highly imbalanced data and achieve high accuracy. The effecting features affect the general public and individuals differently. Addressing the sparsity and discrete nature of insurance data leads to better prediction. Applications using deep learning technology are promising in health policy making. More work is required to interpret models and practice implementation.