It's December - you know what that means. Everyone wants to talk about the new year. I guess you can include me in "everyone", because my main goal this December is to get you and your marketing strategy ready for 2018. We're going to kick it off by talking about marketing trends we expect to see in the coming year. Some of these are general marketing predictions for 2018, while others have a specific local focus; still, they will all probably affect your business in some way.

I know how Beam Search work, and i know that at each step of decoder, we keep k top result and continue decode with them. The thing i want to ask is beam search is applied to the test time only or in both test and train????????

Both of the major smart home platforms have a long-running problem with "discoverability": it's very hard to let users know what their devices can do, particularly if they're always improving thanks to rapid software updates. Amazon and Google are constantly experimenting with ways to get around this, but so far they have been timid. Amazon sends a weekly email, while Google includes some tips in its app. Expect to see them be bolder, particularly as powerful rivals such as Apple appear on the scene with worse AI but better sound. So don't be surprised if your Google Home or Amazon Echo begin to talk back, rather than simply following commands.

My pick for this year: "The Shattered Gradients Problem: If resnets are the answer, then what is the question?" Honorable mentions: 1. Poincaré Embeddings for Learning Hierarchical Representations (for elegance) 2. Inferring and Executing Programs for Visual Reasoning (for trying to tackle an important problem (not just VQA itself) the hard, but ultimately right way) 3. Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods (for bringing empirical joy to my heart)

ZDNet's Sandra Vogel posted a formal review of the Huawei Mate 10 Pro, giving it an outstanding 9/10 rating. I've been spending quality time with this business-ready powerhouse and think that the AI found in the camera is worth discussing in a bit more detail. Huawei's partnership with Leica has resulted in some fantastic cameras and performance that is tough to beat. DxOMark awarded the Mate 10 Pro it second highest overall score, 97, and best still image score, 100. Keep in mind, these scores are not scaled to 100.

Results aren't very good, but I wonder if that's a result in and of itself? One thing that's been annoying me recently is that with all the advancements in the field in the past few years many people have started to think of research only being about finding the next AlexNet that will make waves throughout the field. I feel like research should be approached in a scientific manner with the goal of experimenting with the application of models and that all results are good results. Knowing that a specific approach doesn't work allows for us to find different methods without having to repeat the work other's have already done. I'd say these results definitely could be worth publishing, but I think you should do more work and testing before you consider that.

Label distribution learning (LDL) is a general learning framework, which assigns to an instance a distribution over a set of labels rather than a single label or multiple labels. Current LDL methods have either restricted assumptions on the expression form of the label distribution or limitations in representation learning, e.g., to learn deep features in an end-to-end manner. This paper presents label distribution learning forests (LDLFs) - a novel label distribution learning algorithm based on differentiable decision trees, which have several advantages: 1) Decision trees have the potential to model any general form of label distributions by a mixture of leaf node predictions. 2) The learning of differentiable decision trees can be combined with representation learning. We define a distribution-based loss function for a forest, enabling all the trees to be learned jointly, and show that an update function for leaf node predictions, which guarantees a strict decrease of the loss function, can be derived by variational bounding. The effectiveness of the proposed LDLFs is verified on several LDL tasks and a computer vision application, showing significant improvements to the state-of-the-art LDL methods.

Low-rank matrix approximation (LRMA) methods have achieved excellent accuracy among today's collaborative filtering (CF) methods. In existing LRMA methods, the rank of user/item feature matrices is typically fixed, i.e., the same rank is adopted to describe all users/items. However, our studies show that submatrices with different ranks could coexist in the same user-item rating matrix, so that approximations with fixed ranks cannot perfectly describe the internal structures of the rating matrix, therefore leading to inferior recommendation accuracy. In this paper, a mixture-rank matrix approximation (MRMA) method is proposed, in which user-item ratings can be characterized by a mixture of LRMA models with different ranks. Meanwhile, a learning algorithm capitalizing on iterated condition modes is proposed to tackle the non-convex optimization problem pertaining to MRMA. Experimental studies on MovieLens and Netflix datasets demonstrate that MRMA can outperform six state-of-the-art LRMA-based CF methods in terms of recommendation accuracy.

Many efforts have been devoted to training generative latent variable models with autoregressive decoders, such as recurrent neural networks (RNN). Stochastic recurrent models have been successful in capturing the variability observed in natural sequential data such as speech. We unify successful ideas from recently proposed architectures into a stochastic recurrent model: each step in the sequence is associated with a latent variable that is used to condition the recurrent dynamics for future steps. Training is performed with amortised variational inference where the approximate posterior is augmented with a RNN that runs backward through the sequence. In addition to maximizing the variational lower bound, we ease training of the latent variables by adding an auxiliary cost which forces them to reconstruct the state of the backward recurrent network. This provides the latent variables with a task-independent objective that enhances the performance of the overall model. We found this strategy to perform better than alternative approaches such as KL annealing. Although being conceptually simple, our model achieves state-of-the-art results on standard speech benchmarks such as TIMIT and Blizzard and competitive performance on sequential MNIST. Finally, we apply our model to language modeling on the IMDB dataset where the auxiliary cost helps in learning interpretable latent variables.

When used as a surrogate objective for maximum likelihood estimation in latent variable models, the evidence lower bound (ELBO) produces state-of-the-art results. Inspired by this, we consider the extension of the ELBO to a family of lower bounds defined by a particle filter's estimator of the marginal likelihood, the filtering variational objectives (FIVOs). FIVOs take the same arguments as the ELBO, but can exploit a model's sequential structure to form tighter bounds. We present results that relate the tightness of FIVO's bound to the variance of the particle filter's estimator by considering the generic case of bounds defined as log-transformed likelihood estimators. Experimentally, we show that training with FIVO results in substantial improvements over training the same model architecture with the ELBO on sequential data.