Goto

Collaborating Authors

 Genre


Multiresolution Recurrent Neural Networks: An Application to Dialogue Response Generation

arXiv.org Machine Learning

We introduce the multiresolution recurrent neural network, which extends the sequence-to-sequence framework to model natural language generation as two parallel discrete stochastic processes: a sequence of high-level coarse tokens, and a sequence of natural language tokens. There are many ways to estimate or learn the high-level coarse tokens, but we argue that a simple extraction procedure is sufficient to capture a wealth of high-level discourse semantics. Such procedure allows training the multiresolution recurrent neural network by maximizing the exact joint log-likelihood over both sequences. In contrast to the standard log- likelihood objective w.r.t. natural language tokens (word perplexity), optimizing the joint log-likelihood biases the model towards modeling high-level abstractions. We apply the proposed model to the task of dialogue response generation in two challenging domains: the Ubuntu technical support domain, and Twitter conversations. On Ubuntu, the model outperforms competing approaches by a substantial margin, achieving state-of-the-art results according to both automatic evaluation metrics and a human evaluation study. On Twitter, the model appears to generate more relevant and on-topic responses according to automatic evaluation metrics. Finally, our experiments demonstrate that the proposed model is more adept at overcoming the sparsity of natural language and is better able to capture long-term structure.


De-biasing the Lasso: Optimal Sample Size for Gaussian Designs

arXiv.org Machine Learning

Performing statistical inference in high-dimension is an outstanding challenge. A major source of difficulty is the absence of precise information on the distribution of high-dimensional estimators. Here, we consider linear regression in the high-dimensional regime $p\gg n$. In this context, we would like to perform inference on a high-dimensional parameters vector $\theta^*\in{\mathbb R}^p$. Important progress has been achieved in computing confidence intervals for single coordinates $\theta^*_i$. A key role in these new methods is played by a certain debiased estimator $\hat{\theta}^{\rm d}$ that is constructed from the Lasso. Earlier work establishes that, under suitable assumptions on the design matrix, the coordinates of $\hat{\theta}^{\rm d}$ are asymptotically Gaussian provided $\theta^*$ is $s_0$-sparse with $s_0 = o(\sqrt{n}/\log p )$. The condition $s_0 = o(\sqrt{n}/ \log p )$ is stronger than the one for consistent estimation, namely $s_0 = o(n/ \log p)$. We study Gaussian designs with known or unknown population covariance. When the covariance is known, we prove that the debiased estimator is asymptotically Gaussian under the nearly optimal condition $s_0 = o(n/ (\log p)^2)$. Note that earlier work was limited to $s_0 = o(\sqrt{n}/\log p)$ even for perfectly known covariance. The same conclusion holds if the population covariance is unknown but can be estimated sufficiently well, e.g. under the same sparsity conditions on the inverse covariance as assumed by earlier work. For intermediate regimes, we describe the trade-off between sparsity in the coefficients and in the inverse covariance of the design. We further discuss several applications of our results to high-dimensional inference. In particular, we propose a new estimator that is minimax optimal up to a factor $1+o_n(1)$ for i.i.d. Gaussian designs.


Modal-set estimation with an application to clustering

arXiv.org Machine Learning

We present a first procedure that can estimate -- with statistical consistency guarantees -- any local-maxima of a density, under benign distributional conditions. The procedure estimates all such local maxima, or $\textit{modal-sets}$, of any bounded shape or dimension, including usual point-modes. In practice, modal-sets can arise as dense low-dimensional structures in noisy data, and more generally serve to better model the rich variety of locally-high-density structures in data. The procedure is then shown to be competitive on clustering applications, and moreover is quite stable to a wide range of settings of its tuning parameter.


Micro-interventions in urban transport from pattern discovery on the flow of passengers and on the bus network

arXiv.org Artificial Intelligence

In this paper, we describe a case study in a big metropolis, in which from data collected by digital sensors, we tried to understand mobility patterns of persons using buses and how this can generate knowledge to suggest interventions that are applied incrementally into the transportation network in use. We have first estimated an Origin-Destination matrix of buses users from datasets about the ticket validation and GPS positioning of buses. Then we represent the supply of buses with their routes through bus stops as a complex network, which allowed us to understand the bottlenecks of the current scenario and, in particular, applying community discovery techniques, to identify clusters that the service supply infrastructure has. Finally, from the superimposing of the flow of people represented in the OriginDestination matrix in the supply network, we exemplify how micro-interventions can be prospected by means of an example of the introduction of express routes.


Using Virtual Humans to Understand Real Ones

arXiv.org Artificial Intelligence

Human interactions are characterized by explicit as well as implicit channels of communication. While the explicit channel transmits overt messages, the implicit ones transmit hidden messages about the communicator (e.g., his/her intentions and attitudes). There is a growing consensus that providing a computer with the ability to manipulate implicit affective cues should allow for a more meaningful and natural way of studying particular non-verbal signals of human-human communications by human-computer interactions. In this pilot study, we created a non-dynamic human-computer interaction while manipulating three specific non-verbal channels of communication: gaze pattern, facial expression, and gesture. Participants rated the virtual agent on affective dimensional scales (pleasure, arousal, and dominance) while their physiological signal (electrodermal activity, EDA) was captured during the interaction. Assessment of the behavioral data revealed a significant and complex three-way interaction between gaze, gesture, and facial configuration on the dimension of pleasure, as well as a main effect of gesture on the dimension of dominance. These results suggest a complex relationship between different non-verbal cues and the social context in which they are interpreted. Qualifying considerations as well as possible next steps are further discussed in light of these exploratory findings.


Machine learning for financial prediction: experimentation with David Aronson's latest work โ€“ part 1

#artificialintelligence

The results are a little different to those obtained using RMSE as the objective function. The focus is still well and truly on the volatility indicators, but in this case the best cross validated performance occurred when selecting only 2 out of the 15 candidate variables. Here's a plot of the cross validated performance of the best feature set for various numbers of features: The model clearly performs better in terms of absolute return for a smaller number of predictors. Performance bottoms at 8 predictors and then improves, but never again achieves the performance obtained with 2-4 predictors. This is consistent with Aronson's assertion that we should stick with at most 3-4 variables otherwise overfitting is almost unavoidable.


NIPS 2015 Workshop (Duvenaud) 15644 Probabilistic Integration

#artificialintelligence

Integration is the central numerical operation required for Bayesian machine learning (in the form of marginalization and conditioning). Sampling algorithms still abound in this area, although it has long been known that Monte Carlo methods are fundamentally sub-optimal. The challenges for the development of better performing integration methods are mostly algorithmic. Moreover, recent algorithms have begun to outperform MCMC and its siblings, in wall-clock time, on realistic problems from machine learning. A community website for probabilistic numerics can be found at http://probabilistic-numerics.org.


Google Sets Sights on NHS with AI-driven Apps - Mobile Marketing

#artificialintelligence

The NHS could be applying machine learning-style processing to its patient, doctor and hospital data in an effort to improve efficiency within five years if plans by Google/DeepMind to push into the healthcare sector are approved. According to New Scientist, which has obtained a Memorandum of Understanding drawn up between DeepMind and the Royal Free NHS Trust in London, the two organisations are attempting to form a "broad ranging, mutually beneficial partnership, engaging in high levels of collaborative activity and maximising the potential to work on genuinely innovative and transformative projects." Among the areas the project aims to touch on are making improvements in clinical outcomes and patient safety, and reducing costs throughout the organisation. The memo also sets out a long list of "areas of mutual interest" where the two organisations could work together over the next five years, including bed and demand management software, financial control products, private messaging and task management for junior doctors, and even real-time health prediction. In fact, health prediction has formed the basis of the first project between the two partners, with Google/DeepMind creating an app called Streams that aims to study healthcare data to try to identify patients at risk of deterioration, readmission or even death.


WWDC 2016: Apple set to unveil artificial intelligence plans and showcase iOS 10 at annual conference

The Independent - Tech

Nasa has announced that it has found evidence of flowing water on Mars. Scientists have long speculated that Recurring Slope Lineae -- or dark patches -- on Mars were made up of briny water but the new findings prove that those patches are caused by liquid water, which it has established by finding hydrated salts. Several hundred camped outside the London store in Covent Garden. The 6s will have new features like a vastly improved camera and a pressure-sensitive "3D Touch" display


Predictive Models with Supervised learning in R

#artificialintelligence

The concept of statistical learning started from the method of least squares in the early 1900s has led to the invention of linear regression method. Most of the concepts at those times were applied to astronomical science. The evolution of linear and multiple regression methods gave rise to quantitative statistical computing. Statistical computing divides the majority of the conundrums into two categories. Those are supervised and unsupervised learning categories.