Dialogue assistants are rapidly becoming an indispensable daily aid. To avoid the significant effort needed to hand-craft the required dialogue flow, the Dialogue Management (DM) module can be cast as a continuous Markov Decision Process (MDP) and trained through Reinforcement Learning (RL). Several RL models have been investigated over recent years. However, the lack of a common benchmarking framework makes it difficult to perform a fair comparison between different models and their capability to generalise to different environments. Therefore, this paper proposes a set of challenging simulated environments for dialogue model development and evaluation. To provide some baselines, we investigate a number of representative parametric algorithms, namely deep reinforcement learning algorithms - DQN, A2C and Natural Actor-Critic and compare them to a non-parametric model, GP-SARSA. Both the environments and policy models are implemented using the publicly available PyDial toolkit and released on-line, in order to establish a testbed framework for further experiments and to facilitate experimental reproducibility.

We propose a novel computational strategy based on deep and reinforcement learning techniques for de-novo design of molecules with desired properties. This strategy integrates two deep neural networks - generative and predictive - that are trained separately but employed jointly to generate novel chemical structures with the desired properties. Generative models are trained to produce chemically feasible SMILES, and predictive models are derived to forecast the desired compound properties. One example of such an approach is the broad use of Lipinski's rules of bioavailability (15, 16) to filter molecules that possess the desired bioactivity in vitro. Indeed, it has been acknowledged that the broad use of these rules has substantially reduced the failure rate in experimental ADME studies of drug candidates (17). The crucial step in many new drug discovery projects is the formulation of a well-motivated hypothesis for new lead compound generation (de novo design) or compound selection from available or synthetically feasible chemical libraries based on the available SAR data. Commonly, an interdisciplinary team of scientists generates the new hypothesis by employing computational models of drug action and relying on their expertise and medicinal chemistry intuition. Therefore, the design hypothesis is often biased towards preferred chemistry (18) or driven by model interpretation (19). Automated approaches for designing compounds with desired properties de novo have become an active field of research in the last 15 years (20, 21). In an attempt to design new compounds, both medicinal and computational chemists face virtually infinite chemical space. Great advances in both computational algorithms(24, 25), hardware, and high-throughput screening (HTS) technologies (16) notwithstanding, the size of this virtual library prohibits its exhaustive sampling and testing by systematic construction and evaluation of each individual compound. Local optimization approaches have been proposed but they do not ensure the optimal solution, as the design process converges on a local or'practical' optimum by stochastic sampling, or restrict the search to a defined section of chemical space which can be screened exhaustively (20, 26-28).

This paper studies directed exploration for reinforcement learning agents by tracking uncertainty about the value of each available action. We identify two sources of uncertainty that are relevant for exploration. The first originates from limited data (parametric uncertainty), while the second originates from the distribution of the returns (return uncertainty). We identify methods to learn these distributions with deep neural networks, where we estimate parametric uncertainty with Bayesian drop-out, while return uncertainty is propagated through the Bellman equation as a Gaussian distribution. Then, we identify that both can be jointly estimated in one network, which we call the Double Uncertain Value Network. The policy is directly derived from the learned distributions based on Thompson sampling. Experimental results show that both types of uncertainty may vastly improve learning in domains with a strong exploration challenge.

We propose a multiagent distributed actor-critic algorithm for multitask reinforcement learning (MRL), named Diff-DAC. The agents are connected, forming a (possibly sparse) network. Each agent is assigned a task and has access to data from this local task only. During the learning process, the agents are able to communicate some parameters to their neighbors. Since the agents incorporate their neighbors' parameters into their own learning rules, the information is diffused across the network, and they can learn a common policy that generalizes well across all tasks. Diff-DAC is scalable since the computational complexity and communication overhead per agent grow with the number of neighbors, rather than with the total number of agents. Moreover, the algorithm is fully distributed in the sense that agents self-organize, with no need for coordinator node. Diff-DAC follows an actor-critic scheme where the value function and the policy are approximated with deep neural networks, being able to learn expressive policies from raw data. As a by-product of Diff-DAC's derivation from duality theory, we provide novel insights into the standard actor-critic framework, showing that it is actually an instance of the dual ascent method to approximate the solution of a linear program. Experiments illustrate the performance of the algorithm in the cart-pole, inverted pendulum, and swing-up cart-pole environments.

We analyse multimodal time-series data corresponding to weight, sleep and steps measurements. We focus on predicting whether a user will successfully achieve his/her weight objective. For this, we design several deep long short-term memory (LSTM) architectures, including a novel cross-modal LSTM (X-LSTM), and demonstrate their superiority over baseline approaches. The X-LSTM improves parameter efficiency by processing each modality separately and allowing for information flow between them by way of recurrent cross-connections. We present a general hyperparameter optimisation technique for X-LSTMs, which allows us to significantly improve on the LSTM and a prior state-of-the-art cross-modal approach, using a comparable number of parameters. Finally, we visualise the model's predictions, revealing implications about latent variables in this task.

One of the biggest elements of this will be the increased capabilities of self-driving cars, which are going to spread in both use and performance in more geographies. For instance the UK Chancellor announced that self-driving cars will be allowed on the roads from 2024, where at the moment they are allowed only with a human behind the wheel. We will also see a big increase in the amount of experimentation with AI for considerably more complex problems, with DeepMind at the forefront of this, even if at the moment this only takes the form of games like GO and chess. However, 2018 may well be the point at which we see these kind of elements becoming the foundation of even further AI development.

Over our last few blog posts, we've gone through the various ways you can use BigML's new Deepnet resource via the Dashboard, programmatically, and via download on your local machine. Is there a little wizard pulling an elaborate console with cartoonish-looking levers and dials? Well, as we'll see, Deepnets certainly do have a lot of levers and dials. So many, in fact, that using them can be pretty intimidating. Thankfully, BigML is here to be your wizard so that you aren't the one looked shamefacedly at Dorothy when she realizes you're not as all-powerful as you might have thought.

The demand for data science talent in the capital markets space has seen portfolio managers and discretionary traders attending night classes in a bid to safeguard their jobs amid a rising tide of automation. The whole asset management industry is moving in the direction of being more systematic, being more quantitative and using new unique data. A lot of firms are struggling in many ways to get their heads around that. While there may be many vanilla data science training courses out there, finding a solid introduction for quantitative finance training with the nuanced level of applicability is not so easy. Leigh Drogen, the founder of crowdsourced financial analysis platform Estimize, designed and created the L2Q (Learn to Quant) programme as a rudimentary introduction for discretionary managers.

Samsung may have botched the launch of its virtual assistant Bixby, but it has already promised to patch things up with the next release, which will leverage "deep linking capabilities and enhanced natural language abilities" in order to deliver "a predictive, personalized experience." In addition to integrating Viv's technology for Bixby 2.0, the Korean giant has now acquired Fluenty, an AI startup co-founded by local accelerator FuturePlay. Fluenty is best known for its chatbot and assistant service that suggest smart replies, which is similar to what Google and LinkedIn offer but is instead made for various social networking services, including WhatsApp, Facebook Messenger, Telegram, Line, Kakao Talk, Naver Talk and more. As simple as it sounds, Fluenty relies on a deep learning model initially based on "over 700 million public chat conversations, searching for patterns in replies most frequently used by people." The startup then spent over two years to reduce the loading times from seven seconds to a mere 50 ms.

"Deep Learning has had a huge impact on computer science, making it possible to explore new frontiers of research and to develop amazingly useful products that millions of people use every day." With innovation driving business success, the demand for community-based, open-source software that incorporates AI & deep learning is taking over start-ups and enterprises alike. We've rounded up a few successful deep learning technologies that are making a big impact. TensorFlow is an open source software library that uses data flow graphs for numerical computation. Nodes in the graph represent mathematical operations, while the graph edges represent the multidimensional data arrays communicated between them.