Sequence-to-sequence attention-based models on subword units allow simple open-vocabulary end-to-end speech recognition. In this work, we show that such models can achieve competitive results on the Switchboard 300h and LibriSpeech 1000h tasks. In particular, we report the state-of-the-art word error rates (WER) of 3.54% on the dev-clean and 3.82% on the test-clean evaluation subsets of LibriSpeech. We introduce a new pretraining scheme by starting with a high time reduction factor and lowering it during training, which is crucial both for convergence and final performance. In some experiments, we also use an auxiliary CTC loss function to help the convergence. In addition, we train long short-term memory (LSTM) language models on subword units. By shallow fusion, we report up to 27% relative improvements in WER over the attention baseline without a language model.

The Statistical Learning Theory (SLT) provides the theoretical guarantees for supervised machine learning based on the Empirical Risk Minimization Principle (ERMP). Such principle defines an upper bound to ensure the uniform convergence of the empirical risk Remp(f), i.e., the error measured on a given data sample, to the expected value of risk R(f) (a.k.a. actual risk), which depends on the Joint Probability Distribution P(X x Y) mapping input examples x in X to class labels y in Y. The uniform convergence is only ensured when the Shattering coefficient N(F,2n) has a polynomial growing behavior. This paper proves the Shattering coefficient for any Hilbert space H containing the input space X and discusses its effects in terms of learning guarantees for supervised machine algorithms.

Dictionary learning is a popular approach for inferring a hidden basis or dictionary in which data has a sparse representation. Data generated from the dictionary A (an n by m matrix, with m > n in the over-complete setting) is given by Y = AX where X is a matrix whose columns have supports chosen from a distribution over k-sparse vectors, and the non-zero values chosen from a symmetric distribution. Given Y, the goal is to recover A and X in polynomial time. Existing algorithms give polytime guarantees for recovering incoherent dictionaries, under strong distributional assumptions both on the supports of the columns of X, and on the values of the non-zero entries. In this work, we study the following question: Can we design efficient algorithms for recovering dictionaries when the supports of the columns of X are arbitrary? To address this question while circumventing the issue of non-identifiability, we study a natural semirandom model for dictionary learning where there are a large number of samples $y=Ax$ with arbitrary k-sparse supports for x, along with a few samples where the sparse supports are chosen uniformly at random. While the few samples with random supports ensures identifiability, the support distribution can look almost arbitrary in aggregate. Hence existing algorithmic techniques seem to break down as they make strong assumptions on the supports. Our main contribution is a new polynomial time algorithm for learning incoherent over-complete dictionaries that works under the semirandom model. Additionally the same algorithm provides polynomial time guarantees in new parameter regimes when the supports are fully random. Finally using these techniques, we also identify a minimal set of conditions on the supports under which the dictionary can be (information theoretically) recovered from polynomial samples for almost linear sparsity, i.e., $k=\tilde{O}(n)$.

While tree methods have been popular in practice, researchers and practitioners are also looking for simple algorithms which can reach similar accuracy of trees. In 2010, (Ping Li UAI'10) developed the method of "abc-robust-logitboost" and compared it with other supervised learning methods on datasets used by the deep learning literature. In this study, we propose a series of "tunable GMM kernels" which are simple and perform largely comparably to tree methods on the same datasets. Note that "abc-robust-logitboost" substantially improved the original "GDBT" in that (a) it developed a tree-split formula based on second-order information of the derivatives of the loss function; (b) it developed a new set of derivatives for multi-class classification formulation. In the prior study in 2017, the "generalized min-max" (GMM) kernel was shown to have good performance compared to the "radial-basis function" (RBF) kernel. However, as demonstrated in this paper, the original GMM kernel is often not as competitive as tree methods on the datasets used in the deep learning literature. Since the original GMM kernel has no parameters, we propose tunable GMM kernels by adding tuning parameters in various ways. Three basic (i.e., with only one parameter) GMM kernels are the "$e$GMM kernel", "$p$GMM kernel", and "$\gamma$GMM kernel", respectively. Extensive experiments show that they are able to produce good results for a large number of classification tasks. Furthermore, the basic kernels can be combined to boost the performance.

It's no secret that our planet is struggling with the pressures humans are putting it under. Discarding plastic in oceans, burning fuels and using an abundance of petrol are only some of the ways we're melting the ice caps and thinning the ozone layer. Warnings from experts are becoming more severe and many organisations are working on methods to help combat global warming. Microsoft has just announced that they are broadening their AI for Earth program and'investing $50 million over the next five years to put artificial intelligence technology in the hands of individuals and organisations around the world who are working to protect our planet.' With advances in artificial intelligence, we can learn more and more about the planet and are able to identify elements such as the conditions of the oceans, the well-being of wildlife, and the pollution levels of the air to help us make informed decisions on how to reduce this.

I describe here our recent ICLR paper [1] [code] [talk], which introduces a novel method for model-based reinforcement learning. The main author of this work is Stefan Depeweg, a phd student at Technical University in Munich who I am co-supervising. The key contribution is in our models: Bayesian neural networks with random inputs, whose input layer contains both input features but also random variables which are propagated forward through the network and transformed into an arbitrary noise signal at the output layer. The random inputs enable our models to automatically capture complex noise patterns, improving the quality of our model-based simulations and producing better policies in practice. We address the problem of policy search in stochastic dynamical systems.

For many of us, the very mention of artificial intelligence (AI) conjures up futuristic notions of Skynet and the malevolent robots that rose up to destroy humankind in the Terminator film series. In reality, however, artificial intelligence is already here and it's not out to kill us -- well, not yet, anyway. Our computers are smarter than most of us realize and they're getting smarter all the time. I had an eerie moment of this realization recently when searching my iPhone camera roll for some photos of a trip to the Cotswolds in the U.K. a few years ago. I opened up the search bar and started typing the word but only got up to the third letter when an album of images popped up featuring me and my heavily pregnant wife assembling a cot for our soon-to-be-born son.

Microsoft CEO Satya Nadella says "privacy is a human right" and that internet users should be in control of their data. Microsoft CEO Satya Nadella delivers the keynote address at Build, the company's annual conference for software developers Monday, May 7, 2018, in Seattle. Nadella outlined the company's ethical principles Monday as he kicked off its annual conference for software developers. Nadella didn't mention Facebook's privacy scandals or the data collection practices of rival tech companies, but his comments at the Build conference in Seattle further staked out Microsoft's message that technology should be built for social good. He says Microsoft is prepared for stricter privacy rules taking effect in Europe on May 25, calling it "sound, good regulation."

AI applications demand new architectures to meet the need for unprecedented performance, sophisticated functionality and ultra-low power. Interconnect technology is a critical component of these architectures. "We are still in the early stages of AI maturity, but what we've learned is that sophisticated neural networks require tremendous performance," said Linley Gwennap, principal analyst at The Linley Group. "Generating this level of performance often requires a large number of cores, but these architectures can be difficult to design and test. NoC technology, such as NetSpeed's, has become an important enabling technology for AI because it automates the process of connecting CPUs and accelerators on a complex SoC." "Esperanto is putting thousands of processors and accelerators on a single chip, and our challenge was how to interconnect them. NetSpeed provided a compelling solution for interconnecting our cores with high performance and by licensing their IP we can reduce our time to market as well," said Dave Ditzel, CEO of Esperanto Technologies.

For many of us, the very mention of artificial intelligence (AI) conjures up futuristic notions of Skynet and the malevolent robots that rose up to destroy humankind in the Terminator film series. In reality, however, artificial intelligence is already here and it's not out to kill us -- well, not yet, anyway. Our computers are smarter than most of us realize and they're getting smarter all the time. I had an eerie moment of this realization recently when searching my iPhone camera roll for some photos of a trip to the Cotswolds in the U.K. a few years ago. I opened up the search bar and started typing the word but only got up to the third letter when an album of images popped up featuring me and my heavily pregnant wife assembling a cot for our soon-to-be-born son.