The recently developed variational autoencoders (VAEs) have proved to be an effective confluence of the rich representational power of neural networks with Bayesian methods. However, most work on VAEs use a rather simple prior over the latent variables such as standard normal distribution, thereby restricting its applications to relatively simple phenomena. In this work, we propose hierarchical nonparametric variational autoencoders, which combines tree-structured Bayesian nonparametric priors with VAEs, to enable infinite flexibility of the latent representation space. Both the neural parameters and Bayesian priors are learned jointly using tailored variational inference. The resulting model induces a hierarchical structure of latent semantic concepts underlying the data corpus, and infers accurate representations of data instances. We apply our model in video representation learning. Our method is able to discover highly interpretable activity hierarchies, and obtain improved clustering accuracy and generalization capacity based on the learned rich representations.

This paper presents incremental network quantization (INQ), a novel method, targeting to efficiently convert any pre-trained full-precision convolutional neural network (CNN) model into a low-precision version whose weights are constrained to be either powers of two or zero. Unlike existing methods which are struggled in noticeable accuracy loss, our INQ has the potential to resolve this issue, as benefiting from two innovations. On one hand, we introduce three interdependent operations, namely weight partition, group-wise quantization and re-training. A well-proven measure is employed to divide the weights in each layer of a pre-trained CNN model into two disjoint groups. The weights in the first group are responsible to form a low-precision base, thus they are quantized by a variable-length encoding method. The weights in the other group are responsible to compensate for the accuracy loss from the quantization, thus they are the ones to be re-trained. On the other hand, these three operations are repeated on the latest re-trained group in an iterative manner until all the weights are converted into low-precision ones, acting as an incremental network quantization and accuracy enhancement procedure. Extensive experiments on the ImageNet classification task using almost all known deep CNN architectures including AlexNet, VGG-16, GoogleNet and ResNets well testify the efficacy of the proposed method. Specifically, at 5-bit quantization, our models have improved accuracy than the 32-bit floating-point references. Taking ResNet-18 as an example, we further show that our quantized models with 4-bit, 3-bit and 2-bit ternary weights have improved or very similar accuracy against its 32-bit floating-point baseline. Besides, impressive results with the combination of network pruning and INQ are also reported. The code is available at https://github.com/Zhouaojun/Incremental-Network-Quantization.

Earlier this week, our research team reached that 5.1 percent error rate with our speech recognition system – a new industry milestone that substantially surpasses the accuracy we achieved last year. We reduced our error rate by 12 percent from last year's level, using improvements to our neural net-based acoustic and language models. We introduced an additional convolutional neural network combined with bidirectional long-short-term memory (CNN-BLSTM) model for improved acoustic modeling. Additionally, our approach to combine predictions from multiple acoustic models now does so at both the frame/senone and word levels. We published a technical report that has the full system details.

Douglas Hofstadter in his book "I am a Strange Loop" coined this idea: In the end, we are self-perceiving, self-inventing, locked-in mirages that are little miracles of self-reference. Loops are not typical in Deep Learning systems. This is not hyperbole, this is happening today where researchers are training'narrow' intelligence systems to create very capable specialist automation that surpass human capabilities. For more on this in this "strange loop" please consult:

Microsoft has developed a new system for deep learning, dubbed Project Brainwave, that embeds deep neural network technology into the company's high-tech programmable computer chips to accelerate one of the key processes for artificial intelligence. The company announced the initiative this afternoon as an extension of its existing work in field programmable gate arrays, or FPGAs, data center processors that can be reprogrammed on the fly to best serve whatever computing task is at hand. "We designed the system for real-time AI, which means the system processes requests as fast as it receives them, with ultra-low latency," said Doug Burger, Microsoft distinguished engineer, in a post describing the technical details of the system. "Real-time AI is becoming increasingly important as cloud infrastructures process live data streams, whether they be search queries, videos, sensor streams, or interactions with users." This is part of a broader effort by Microsoft to advance the state of the art in artificial intelligence, and bring those new approaches to market.

Henri Waelbroeck, director of research at machine learning trade execution system Portware, says rather poetically that the system "reads the tea leaves" in market data to distinguish different sorts of orders and execute trades more efficiently. Portware uses artificial intelligence to help traders select the best algorithm for particular market conditions, asset class, broker, venue etc., interacting with the order flow and computing a mind-boggling array of variables in real time. Say you are buying a stock, and you predict there is likely to be more orders hitting the bid side of the spread in the next five minutes, you should be able to operate an efficient algorithm that only posts limit orders and collects the spread as it executes. Using an algorithm that crosses the spread in this instance would be wasteful since you expect order flow to be coming your way. Waelbroeck, formerly a professor at the Institute of Nuclear Sciences at the National University of Mexico, whose specialisms include genetic algorithms and chaos theory, said: "Just throwing machine learning at problems usually doesn't give a very good answer. You need to have a good analytical understanding of what's going on and this usually gives you a baseline model and then you find opportunities to insert machine learning tactically to exploit opportunities to improve the models."

Gaurav Chakravorty, co-founder of AI investment advisors qplum, likes to use sporting analogies to illustrate changing trends within finance. The way high frequency trading (HFT) seemed to work like magic in the old days reminds him of Manchester United under Sir Alex Ferguson. Between 1993 and 2013 Manchester United won the English Premier League 13 times, an incredible record. The truth was Ferguson used a machinery that other clubs had not yet happened upon. He would scout clubs in Europe for talented youngsters and be willing to pay top dollar for young stars without a proven track record at a big club.

Discovering the secrets of the universe is not a task for the timid and the impatient; there's a need to peer into the deepest reaches of outer space and try to make sense of distant galaxies, stars, gas clouds, quasars, halos, and black holes. "Understanding how these objects behave and how they interact gives us answers to how the universe was formed and how it works," says Kevin Schawinski, an astrophysicist and assistant professor in the Institute for Astronomy at ETH Zurich, the Swiss Federal Institute of Technology. The problem is that traditional tools such as telescopes can see only so far, even with radical advances in optics and the placement of observatories in space, where they are free of the light and dust of Earth. For instance, the Hubble Telescope changed the way astrophysicists and astronomers viewed deep space by delivering far clearer images than previously possible. Of course, in this context, distance and time are inextricably linked.

This post describes how temporally-sensitive saliency maps can be obtained for deep neural networks designed for video recognition. It is evident from the previous works [2, 3, 4] that saliency maps help visualize why a model produced a given prediction and can uncover artifacts in the data and point towards better model architectures. Task: Recognizing human actions in videos from our recently released dataset requires a fine-grained understanding of concepts like three-dimensional geometry, material properties, object permanence, affordance and gravity [1]. The dataset, dubbed "Something-Something", consists of 100,000 videos across 174 categories containing concepts such as dropping, picking, pushing etc. Grad-CAM or Gradient-weighted Class Activation Mapping, proposed by [4], allows us to obtain a localization map for any target class. Please refer [4] for more details.

Below is a list of materials that introduces the reader to the field of Artificial Intelligence. These materials are targeted at the novice AI practitioner. Students who are just learning the field may also find this material useful. Artificial intelligence (AI) is the intelligence exhibited by machines or software. It is also the name of the academic field of study which studies how to create computers and computer software that are capable of intelligent behavior.