Roundup Welcome to our roundup of major AI news from the past two weeks. Machine learning is so hyped right now, it doesn't help when companies such as Intel and Nvidia announce new chips and reveal little information about the specs, but make lofty claims of increased speed and precision. It's also difficult to keep track of all the different software frameworks and hardware options. Outfits like ARM, AMD, Amazon and Facebook are aware of this and are trying to make it easier to transfer models written in one language to another and optimize the models across various chips. Google's'surprise' Pixel 2 chip It's the first smartphone chip Google has ever designed, and it wasn't announced during the launch of the Pixel 2, which features the new silicon, because, er, it isn't enabled nor supported by applications yet.

Researchers from Purdue University developed a model that can decode what the human brain is seeing by using deep learning to interpret fMRI scans from people watching videos, representing a sort of mind-reading technology. "That type of network (convolutional neural network) has made an enormous impact in the field of computer vision in recent years," said Zhongming Liu, an assistant professor in Purdue University's Weldon School of Biomedical Engineering and School of Electrical and Computer Engineering. "Our technique uses the neural network to understand what you are seeing." According to the researcher's paper, the new findings represent the first time such an approach has been used to see how the brain processes movies of natural scenes, a step toward decoding the brain while people are trying to make sense of complex and dynamic visual surroundings Using GTX 1080 GPUs and the cuDNN-accelerated Caffe deep learning framework, the researchers trained their convolutional neural network model on more than 11 hours of fMRI data from each of three women subjects watching 972 video clips, including those showing people or animals in action and nature scenes. Once trained, they used the model to decode fMRI data from the subjects to reconstruct the videos, even ones the model had never watched before.

Scientists at Columbia and Lehigh Universities have effectively created a method for error-correcting deep learning networks. With the tool, they've been able to reverse-engineer complex AI, thus providing a work-around for the mysterious'black box' problem. Deep learning AI systems often make decisions inside a black box – meaning humans can't readily understand why a neural-network chose one solution over another. This exists because machines can perform millions of tests in short amounts of time, come up with a solution, and move on to performing millions more tests to come up with a better solution. The researchers created DeepXplore, software that exposes flaws in a neural-network by tricking it into making mistakes.

Welcome back to Mind Over Money. In our last episode from October 17, we explored the powerful new computing technologies known as machine learning and deep learning. These AI "genies" were let out of the bottle by computer scientists and engineers who learned to harness semiconductors that were originally designed for advanced gaming graphics. What the researchers discovered was that the GPU (graphical processing units) chips made by NVIDIA NVDA and AMD AMD provided the parallel processing required to teach machines to go beyond replicating simple human intelligence tasks like filtering your email for junk and to do things such as win at "extensive" -- or, extremely complicated -- games like the Chinese Go or Texas Hold'em. With lots of data -- both structured and messy -- a computer can be trained to automatically analyze language, images, faces, and even behavior that seems like it could be financial fraud.

The first instances to include NVIDIA Tesla V100 GPUs, P3 instances are the most powerful GPU instances available in the cloud. P3 instances allow customers to build and deploy advanced applications with up to 14 times better performance than previous-generation Amazon EC2 GPU compute instances, and reduce training of machine learning applications from days to hours. With up to eight NVIDIA Tesla V100 GPUs, P3 instances provide up to one petaflop of mixed-precision, 125 teraflops of single-precision, and 62 teraflops of double-precision floating point performance, as well as a 300 GB/s second-generation NVIDIA NVLink interconnect that enables high-speed, low-latency GPU-to-GPU communication. P3 instances also feature up to 64 vCPUs based on custom Intel Xeon E5 (Broadwell) processors, 488 GB of DRAM, and 25 Gbps of dedicated aggregate network bandwidth using the Elastic Network Adapter (ENA). "When we launched our P2 instances last year, we couldn't believe how quickly people adopted them," said Matt Garman, Vice President of Amazon EC2.

Amazon Web Services is the first cloud to launch new compute instances that allow developers to build applications that tap into Nvidia's new generation of Volta GPUs, which are designed to provide high-performance acceleration for applications like AI computation. Companies all over are turning to machine learning to help propel their businesses, but building new models often requires a great deal of computation. The Volta is supposed to be a good deal faster at that than past generations of Nvidia's silicon, and making it available through Amazon's cloud means that companies will be able to get started using them right away. Customers will be able to run instances with up to 8 V100 GPUs, which will be made available initially from AWS's Northern Virginia, Oregon, Ireland, and Tokyo datacenters. Nvidia launched a new GPU Cloud offering alongside AWS, which is designed to provide companies with the most optimized environment for running deep learning applications on top of the company's hardware in a public cloud.

In this study, we propose a novel deep neural network and its supervised learning method that uses a feedforward supervisory signal. The method is inspired by the human visual system and performs human-like association-based learning without any backward error propagation. The feedforward supervisory signal that produces the correct result is preceded by the target signal and associates its confirmed label with the classification result of the target signal. It effectively uses a large amount of information from the feedforward signal, and forms a continuous and rich learning representation. The method is validated using visual recognition tasks on the MNIST handwritten dataset.

We describe an approach to understand the peculiar and counterintuitive generalization properties of deep neural networks. The approach involves going beyond worst-case theoretical capacity control frameworks that have been popular in machine learning in recent years to revisit old ideas in the statistical mechanics of neural networks. Within this approach, we present a prototypical Very Simple Deep Learning (VSDL) model, whose behavior is controlled by two control parameters, one describing an effective amount of data, or load, on the network (that decreases when noise is added to the input), and one with an effective temperature interpretation (that increases when algorithms are early stopped). Using this model, we describe how a very simple application of ideas from the statistical mechanics theory of generalization provides a strong qualitative description of recently-observed empirical results regarding the inability of deep neural networks not to overfit training data, discontinuous learning and sharp transitions in the generalization properties of learning algorithms, etc.

The concepts of unitary evolution matrices and associative memory have boosted the field of Recurrent Neural Networks (RNN) to state-of-the-art performance in a variety of sequential tasks. However, RNN still have a limited capacity to manipulate long-term memory. To bypass this weakness the most successful applications of RNN use external techniques such as attention mechanisms. In this paper we propose a novel RNN model that unifies the state-of-the-art approaches: Rotational Unit of Memory (RUM). The core of RUM is its rotational operation, which is, naturally, a unitary matrix, providing architectures with the power to learn long-term dependencies by overcoming the vanishing and exploding gradients problem. Moreover, the rotational unit also serves as associative memory. We evaluate our model on synthetic memorization, question answering and language modeling tasks. RUM learns the Copying Memory task completely and improves the state-of-the-art result in the Recall task. RUM's performance in the bAbI Question Answering task is comparable to that of models with attention mechanism. We also improve the state-of-the-art result to 1.189 bits-per-character (BPC) loss in the Character Level Penn Treebank (PTB) task, which is to signify the applications of RUM to real-world sequential data. The universality of our construction, at the core of RNN, establishes RUM as a promising approach to language modeling, speech recognition and machine translation.

Material recognition enables robots to incorporate knowledge of material properties into their interactions with everyday objects. For example, material recognition opens up opportunities for clearer communication with a robot, such as "bring me the metal coffee mug", and recognizing plastic versus metal is crucial when using a microwave or oven. However, collecting labeled training data with a robot is often more difficult than unlabeled data. We present a semi-supervised learning approach for material recognition that uses generative adversarial networks (GANs) with haptic features such as force, temperature, and vibration. Our approach achieves state-of-the-art results and enables a robot to estimate the material class of household objects with ~90% accuracy when 92% of the training data are unlabeled. We explore how well this approach can recognize the material of new objects and we discuss challenges facing generalization. To motivate learning from unlabeled training data, we also compare results against several common supervised learning classifiers. In addition, we have released the dataset used for this work which consists of time-series haptic measurements from a robot that conducted thousands of interactions with 72 household objects.