Although we don't know how brain functions yet, we have the feeling that it must have a logic unit and a memory unit. We make decisions by reasoning and by experience. So do computers, we have the logic units, CPUs and GPUs and we also have memories. But when you look at a neural network, it functions like a black box. You feed in some inputs from one side, you receive some outputs from the other side.

I attended my first AI-focused conference in several years at MLConf. It was great catching up on the state-of-the-art in machine learning and data science. While much of the conference was focused on deep learning, there were broader lessons to takeaway from the event. I'll discuss each of these points in more detail below. In the coming weeks, all of the slides will be available on SlideShare.

These authors contributed equally to this work. The authors declare no competing financial interests.

We are living in an era driven by algorithms and more specifically deep learning algorithms which are beginning to pervading and potentially intruding every single facet of our personal and professional lives. When algorithms begin playing a commanding role in our everyday personal choices including clothes, shoes, movies, music, content, jobs, whatever and start dictating what is best for us and what is not, we have to concede that we are already in the midst of algorithms driven enlightenment, based on whichever camp we want to be in. Consider their application in myriad esoteric use cases - sorting and grading cucumbers; creating movie trailers; writing news articles; measuring productivity of cows; predicting students likely to drop out; optimizing soil nutrient levels and many more such use cases. To a battle for supremacy on the'senses' dimension against the human race including vision, speech and text, these algorithms are proving their mettle in every walk of human life. So it is really high time that we bow to the powers of these very powerful algorithms and be led by them in this algorithms-driven insights economy.

The release of two machine learning (ML) model builders have made it easier for software engineers to create and run ML models, even without specialized training. Microsoft and Amazon Web Services' (AWS) Gluon is an open source project that eliminates some of the difficult work required to develop artificial intelligence (AI) systems. It provides training algorithms and neural network models, two important components of a deep learning system, that developers can use to develop their own ML systems. Google's ML engine is part of its cloud platform and is offered as a managed service for developers to build ML models that work on any type of data, of any size. Similar to Gluon, Google's service provides pre-trained models for developers to generate their own tailored ML models.

The technology is a form of deep-learning artificial intelligence software developed to fit onto mobile computer chips. This allows artificial intelligence to be used in a range of devices, from smartphones to industrial robots. This portability would enable devices to operate independent of the Internet while using artificial intelligence that performs equivalent to tethered neural networks. With this, a hosting chip embedded in a smartphone could run a speech-activated virtual assistant and undertake other intelligent features, such as controlling data usage. Other applications include operating drones and surveillance cameras in remote areas.

In recent years, tech giants such as Amazon, Google, Microsoft and IBM (along with a slew of startups) have all begun to offer what's known as Artificial Intelligence as a service (AIaaS). These services, in a nutshell, make a wide range of AI algorithms available to the public. Examples for this are algorithms for classification, regression, and Deep Learning – a modern learning algorithm that relies on Artificial Deep Neural Nets. As more and more companies begin to make use of AlaaS, a better understanding of how it can be best integrated into your own business is the difference between having a massive cost-saver and a massive headache. Companies once had to spend a lot of time producing their own AI applications, and did so at great expenditure.

Recurrent neural networks like long short-term memory (LSTM) are important architectures for sequential prediction tasks. LSTMs (and RNNs in general) model sequences along the forward time direction. Bidirectional LSTMs (Bi-LSTMs) on the other hand model sequences along both forward and backward directions and are generally known to perform better at such tasks because they capture a richer representation of the data. In the training of Bi-LSTMs, the forward and backward paths are learned independently. We propose a variant of the Bi-LSTM architecture, which we call Variational Bi-LSTM, that creates a channel between the two paths (during training, but which may be omitted during inference); thus optimizing the two paths jointly. We arrive at this joint objective for our model by minimizing a variational lower bound of the joint likelihood of the data sequence. Our model acts as a regularizer and encourages the two networks to inform each other in making their respective predictions using distinct information. We perform ablation studies to better understand the different components of our model and evaluate the method on various benchmarks, showing state-of-the-art performance.

The use of covariance kernels is ubiquitous in the field of spatial statistics. Kernels allow data to be mapped into high-dimensional feature spaces and can thus extend simple linear additive methods to nonlinear methods with higher order interactions. However, until recently, there has been a strong reliance on a limited class of stationary kernels such as the Matern or squared exponential, limiting the expressiveness of these modelling approaches. Recent machine learning research has focused on spectral representations to model arbitrary stationary kernels and introduced more general representations that include classes of nonstationary kernels. In this paper, we exploit the connections between Fourier feature representations, Gaussian processes and neural networks to generalise previous approaches and develop a simple and efficient framework to learn arbitrarily complex nonstationary kernel functions directly from the data, while taking care to avoid overfitting using state-of-the-art methods from deep learning. We highlight the very broad array of kernel classes that could be created within this framework. We apply this to a time series dataset and a remote sensing problem involving land surface temperature in Eastern Africa. We show that without increasing the computational or storage complexity, nonstationary kernels can be used to improve generalisation performance and provide more interpretable results.

Many modern unsupervised or semi-supervised machine learning algorithms rely on Bayesian probabilistic models. These models are usually intractable and thus require approximate inference. Variational inference (VI) lets us approximate a high-dimensional Bayesian posterior with a simpler variational distribution by solving an optimization problem. This approach has been successfully used in various models and large-scale applications. In this review, we give an overview of recent trends in variational inference. We first introduce standard mean field variational inference, then review recent advances focusing on the following aspects: (a) scalable VI, which includes stochastic approximations, (b) generic VI, which extends the applicability of VI to a large class of otherwise intractable models, such as non-conjugate models, (c) accurate VI, which includes variational models beyond the mean field approximation or with atypical divergences, and (d) amortized VI, which implements the inference over local latent variables with inference networks. Finally, we provide a summary of promising future research directions.