Deep conditional generative models are developed to simultaneously learn the temporal dependencies of multiple sequences. The model is designed by introducing a three-way weight tensor to capture the multiplicative interactions between side information and sequences. The proposed model builds on the Temporal Sigmoid Belief Network (TSBN), a sequential stack of Sigmoid Belief Networks (SBNs). The transition matrices are further factored to reduce the number of parameters and improve generalization. When side information is not available, a general framework for semi-supervised learning based on the proposed model is constituted, allowing robust sequence classification. Experimental results show that the proposed approach achieves state-of-the-art predictive and classification performance on sequential data, and has the capacity to synthesize sequences, with controlled style transitioning and blending.

With the enormous growth of data size n and model complexity, asynchronous parallel algorithms [1, 2, 3, 4, 5, 6] have become an important tool and received significant successes for solving large scale machine learning problems in the form of (1). Asynchronous parallel algorithms distribute computation on multicore systems (shared memory architecture) or multi-machine system (parameter server architecture), whose computation power generally scales up with the increasing number of cores or machines. As a consequence, effective design and implementation of asynchronous parallel algorithms is critical for large scale machine learning. Numerous efforts have been devoted to this topic. Among them, asynchronous stochastic gradient descent is proposed in [1, 2], and its performance is guaranteed by theoretical convergence analyses. An asynchronous proximal gradient descent algorithm is designed on the parameter server architecture in [3] with a distributed optimization software provided. Convergence rate of asynchronous stochastic gradient descent with a nonconvex objective is analyzed in [4].

Dimensionality reduction techniques play an essential role in data analytics, signal processing and machine learning. Dimensionality reduction is usually performed in a preprocessing stage that is separate from subsequent data analysis, such as clustering or classification. Finding reduced-dimension representations that are well-suited for the intended task is more appealing. This paper proposes a joint factor analysis and latent clustering framework, which aims at learning cluster-aware low-dimensional representations of matrix and tensor data. The proposed approach leverages matrix and tensor factorization models that produce essentially unique latent representations of the data to unravel latent cluster structure -- which is otherwise obscured because of the freedom to apply an oblique transformation in latent space. At the same time, latent cluster structure is used as prior information to enhance the performance of factorization. Specific contributions include several custom-built problem formulations, corresponding algorithms, and discussion of associated convergence properties. Besides extensive simulations, real-world datasets such as Reuters document data and MNIST image data are also employed to showcase the effectiveness of the proposed approaches.

Formal synthesis is the process of generating a program satisfying a high-level formal specification. In recent times, effective formal synthesis methods have been proposed based on the use of inductive learning. We refer to this class of methods that learn programs from examples as formal inductive synthesis. In this paper, we present a theoretical framework for formal inductive synthesis. We discuss how formal inductive synthesis differs from traditional machine learning. We then describe oracle-guided inductive synthesis (OGIS), a framework that captures a family of synthesizers that operate by iteratively querying an oracle. An instance of OGIS that has had much practical impact is counterexample-guided inductive synthesis (CEGIS). We present a theoretical characterization of CEGIS for learning any program that computes a recursive language. In particular, we analyze the relative power of CEGIS variants where the types of counterexamples generated by the oracle varies. We also consider the impact of bounded versus unbounded memory available to the learning algorithm. In the special case where the universe of candidate programs is finite, we relate the speed of convergence to the notion of teaching dimension studied in machine learning theory. Altogether, the results of the paper take a first step towards a theoretical foundation for the emerging field of formal inductive synthesis.

Google has developed its own accelerator chips for artificial intelligence it calls tensor processing units (TPUs) after the open source TensorFlow algorithms it released last year. The news was the big surprise saved for the end of a two-hour keynote at the search giant s annual Google IO event in the heart of Silicon Valley. We have started building tensor processing units TPUs are an order of magnitude higher performance per Watt than commercial FPGAs and GPUs, they powered the AlphaGo system, said Sundar Pichai, Google s chief executive, citing the Google computer that beat a human Go champion. The accelerators have been running in Google s data centers for more than a year, according to a blog by Norm Jouppi, a distinguished hardware engineer at Google. TPUs already power many applications at Google, including RankBrain, used to improve the relevancy of search results and Street View, to improve the accuracy and quality of our maps and navigation, he said.

When he introduced the TPU at the I/O conference, Google CEO Sundar Pichai said it provides an order of magnitude better performance per watt than existing chips for machine learning tasks. While it's unlikely to usurp CPUs and GPUs already in use in the machine learning world, the TPU could potentially speed the machine learning process without using much more energy. Google has been carefully guarding the details of the TPU project, but it's been generally understood that the project was in progress. Based on the company's job postings, it had become evident over the past year that Google was working on a chip of some kind. Now, Google confirms the chip has been under development for about two years.

Transition Between The Current Market And Where The Market Is Going 1.3.3

Life would be pretty dull without imagination. In fact, maybe the biggest problem for computers is that they don't have any. That's the belief motivating the founders of Vicarious, an enigmatic AI company backed by some of the most famous and successful names in Silicon Valley. Vicarious is developing a new way of processing data, inspired by the way information seems to flow through the brain. The company's leaders say this gives computers something akin to imagination, which they hope will help make the machines a lot smarter.

ICRA is almost over, and we hope you've been enjoying our coverage, which so far has featured robot moths, zipper actuators, machine learning, and duckies. We'll have lots more from the converence over the next few weeks, but for you impatient types, we're cramming Video Friday this week with a painstakingly curated selection of ICRA videos--emphasis on pain: there were nearly 500 videos! We tried to include videos from many different areas of robotics: control, sensing, humanoids, actuators, exoskeletons, manipulators, prosthetics, aerial vehicles, grasping, AI, VR, haptics, vision, and microrobots. We're posting the abstracts along with the videos, but if you have any questions about these projects, let us know and we'll get more details from the authors. Have a great weekend everyone! We present an adaptive filter model of cerebellar function applied to the calibration of a tactile sensory map to improve the accuracy of directed movements of a robotic manipulator.

This year, Google did something entirely different with Google I/O than it has in years past. Rather than host it inside the stuffy, fluorescent-lighted halls of the Moscone Center in downtown San Francisco, it took place 40 miles away at an outside concert venue in Mountain View. The result--for me, at least--was a hoard of sunburned developers slogging through 90-degree heat. But regardless of the weather, the massive venue afforded Google plenty of space to set up tents for sessions and art installations. Its other products were also on display, like Project Loon and the self-driving car, and there were even tents devoted to some of the smaller, lesser-known programs, like Google.org's After hours, the Shoreline transformed into a very mild rave of sorts, with neon lights galore and delicious food trucks on standby to feed hungry developers.