Say hello to Joe Rogan: podcaster, entertainer of problematic views, and man who believes that feeding his all chimp hockey team a diet of bone broth and elk meat will give them the power to rip your balls off. Or, at least that's what the unaware listener might believe after listening to an entirely AI-generated clip of the popular podcaster. Unlike Rogan's typical totally coherent rants, this one is a total fabrication. "The replica of Rogan's voice the team created was produced using a text-to-speech deep learning system they developed called RealTalk," explained the researchers behind the clip in a blog post, "which generates life-like speech using only text inputs." This obviously calls to mind deepfakes, the video editing tech that can convincingly edit videos to make it look like people did or said things they in fact did not.

Recently we have discussed the emerging concept of smart farming that makes agriculture more efficient and effective with the help of high-precision algorithms. The mechanism that drives it is Machine Learning -- the scientific field that gives machines the ability to learn without being strictly programmed. It has emerged together with big data technologies and high-performance computing to create new opportunities to unravel, quantify, and understand data intensive processes in agricultural operational environments. Machine learning is everywhere throughout the whole growing and harvesting cycle. It begins with a seed being planted in the soil -- from the soil preparation, seeds breeding and water feed measurement -- and it ends when robots pick up the harvest determining the ripeness with the help of computer vision.

Punchh, the leader in digital marketing solutions for physical retailers, today announced the launch of Punchh Deep Sentiment Analysis. The new product allows brands to extract valuable insights from customer reviews using Punchh's natural language comprehension engine built with industry-leading deep learning and artificial intelligence. Its natural language processing model achieves human-level performance, defined as more than 93 percent accurate, and features multi-language support. "In today's hyper-competitive climate, brands need to do everything they can to foster and nurture direct customer relationships, and paying attention to customer reviews is an essential part of that," said Shyam Rao, CEO of Punchh. "Manually reading every review is prohibitively time-consuming for most retailers, which leads to slower response times and poor customer experiences. Our solution uses AI and machine learning to help brands analyze reviews at scale and immediately identify critical information so they can focus on high-level insights and make quick decisions to strengthen customer relationships and increase loyalty."

On March 10, 2016, one of the strongest Go players in the world, Lee Sedol, stared at one of the oddest moves in the history of professional Go. His opponent -- the computer program AlphaGo, from Google-owned DeepMind -- had, in the 37th move of the game, placed its stone in what the Go community calls a "shoulder hit"; a move professional Go players seldom use. Stunned, Lee just walked out of the room. AlphaGo appeared to demonstrate creative initiative exceeding the best human players. Lee returned a few moments later and played a brilliant game, though he still conceded defeat after 211 moves.

Machine learning has become ubiquitous and a key technology on mining electronic health records (EHRs) for facilitating clinical research and practice. Unsupervised machine learning, as opposed to supervised learning, has shown promise in identifying novel patterns and relations from EHRs without using human created labels. In this paper, we investigate the application of unsupervised machine learning models in discovering latent disease clusters and patient subgroups based on EHRs. We utilized Latent Dirichlet Allocation (LDA), a generative probabilistic model, and proposed a novel model named Poisson Dirichlet Model (PDM), which extends the LDA approach using a Poisson distribution to model patients' disease diagnoses and to alleviate age and sex factors by considering both observed and expected observations. In the empirical experiments, we evaluated LDA and PDM on three patient cohorts with EHR data retrieved from the Rochester Epidemiology Project (REP), for the discovery of latent disease clusters and patient subgroups. We compared the effectiveness of LDA and PDM in identifying latent disease clusters through the visualization of disease representations learned by two approaches. We also tested the performance of LDA and PDM in differentiating patient subgroups through survival analysis, as well as statistical analysis. The experimental results show that the proposed PDM could effectively identify distinguished disease clusters by alleviating the impact of age and sex, and that LDA could stratify patients into more differentiable subgroups than PDM in terms of p-values. However, the subgroups discovered by PDM might imply the underlying patterns of diseases of greater interest in epidemiology research due to the alleviation of age and sex. Both unsupervised machine learning approaches could be leveraged to discover patient subgroups using EHRs but with different foci.

Variational autoencoders learn unsupervised data representations, but these models frequently converge to minima that fail to preserve meaningful semantic information. For example, variational autoencoders with autoregressive decoders often collapse into autodecoders, where they learn to ignore the encoder input. In this work, we demonstrate that adding an auxiliary decoder to regularize the latent space can prevent this collapse, but successful auxiliary decoding tasks are domain dependent. Auxiliary decoders can increase the amount of semantic information encoded in the latent space and visible in the reconstructions. The semantic information in the variational autoencoder's representation is only weakly correlated with its rate, distortion, or evidence lower bound. Compared to other popular strategies that modify the training objective, our regularization of the latent space generally increased the semantic information content.

Despite its huge empirical success, deep learning still preserves many features of alchemy [Rahimi, 2017]: progress in this field is obtained mainly by trial and error, and our intuition about how do neural networks actually work often misleads us. Alchemy, in order to become usual chemistry, needs a theoretical ground. For now, a solid theoretical ground for deep learning is lacking, however, fortunately, many pieces of theory appeared from different directions during several past years. The purpose of this essay is not to provide a comprehensive review, but to draw connections between some works on this topic. The list of works mentioned here is by no means representative, or, all the more so, complete. Since the theory of deep learning is lacking, some features of neural networks learning seem "mysterious". We emphasize two mysteries of deep learning: 1. Generalization mystery. It is very common for contemporary neural networks to have many more parameters than the number of training examples at hand.

Much research work in computer vision is being spent on optimizing existing network architectures to obtain a few more percentage points on benchmarks. Recent AutoML approaches promise to relieve us from this effort. However, they are mainly designed for comparatively small-scale classification tasks. In this work, we show how to use and extend existing AutoML techniques to efficiently optimize large-scale U-Net-like encoder-decoder architectures. In particular, we leverage gradient-based neural architecture search and Bayesian optimization for hyperparameter search. The resulting optimization does not require a large company-scale compute cluster. We show results on disparity estimation that clearly outperform the manually optimized baseline and reach state-of-the-art performance.

Convolutional neural networks have been widely deployed in various application scenarios. In order to extend the applications' boundaries to some accuracy-crucial domains, researchers have been investigating approaches to boost accuracy through either deeper or wider network structures, which brings with them the exponential increment of the computational and storage cost, delaying the responding time. In this paper, we propose a general training framework named self distillation, which notably enhances the performance (accuracy) of convolutional neural networks through shrinking the size of the network rather than aggrandizing it. Different from traditional knowledge distillation - a knowledge transformation methodology among networks, which forces student neural networks to approximate the softmax layer outputs of pre-trained teacher neural networks, the proposed self distillation framework distills knowledge within network itself. The networks are firstly divided into several sections. Then the knowledge in the deeper portion of the networks is squeezed into the shallow ones. Experiments further prove the generalization of the proposed self distillation framework: enhancement of accuracy at average level is 2.65%, varying from 0.61% in ResNeXt as minimum to 4.07% in VGG19 as maximum. In addition, it can also provide flexibility of depth-wise scalable inference on resource-limited edge devices.Our codes will be released on github soon.

Truncated backpropagation through time (TBPTT) is a popular method for learning in recurrent neural networks (RNNs) that saves computation and memory at the cost of bias by truncating backpropagation after a fixed number of lags. In practice, choosing the optimal truncation length is difficult: TBPTT will not converge if the truncation length is too small, or will converge slowly if it is too large. We propose an adaptive TBPTT scheme that converts the problem from choosing a temporal lag to one of choosing a tolerable amount of gradient bias. For many realistic RNNs, the TBPTT gradients decay geometrically for large lags; under this condition, we can control the bias by varying the truncation length adaptively. For RNNs with smooth activation functions, we prove that this bias controls the convergence rate of SGD with biased gradients for our non-convex loss. Using this theory, we develop a practical method for adaptively estimating the truncation length during training. We evaluate our adaptive TBPTT method on synthetic data and language modeling tasks and find that our adaptive TBPTT ameliorates the computational pitfalls of fixed TBPTT.