We propose a simple algorithm to train stochastic neural networks to draw samples from given target distributions for probabilistic inference. Our method is based on iteratively adjusting the neural network parameters so that the output changes along a Stein variational gradient that maximumly decreases the KL divergence with the target distribution. Our method works for any target distribution specified by their unnormalized density function, and can train any black-box architectures that are differentiable in terms of the parameters we want to adapt. As an application of our method, we propose an amortized MLE algorithm for training deep energy model, where a neural sampler is adaptively trained to approximate the likelihood function. Our method mimics an adversarial game between the deep energy model and the neural sampler, and obtains realistic-looking images competitive with the state-of-the-art results.

Asynchronous methods are widely used in deep learning, but have limited theoretical justification when applied to non-convex problems. We show that running stochastic gradient descent (SGD) in an asynchronous manner can be viewed as adding a momentum-like term to the SGD iteration. Our result does not assume convexity of the objective function, so it is applicable to deep learning systems. We observe that a standard queuing model of asynchrony results in a form of momentum that is commonly used by deep learning practitioners. This forges a link between queuing theory and asynchrony in deep learning systems, which could be useful for systems builders. For convolutional neural networks, we experimentally validate that the degree of asynchrony directly correlates with the momentum, confirming our main result. An important implication is that tuning the momentum parameter is important when considering different levels of asynchrony. We assert that properly tuned momentum reduces the number of steps required for convergence. Finally, our theory suggests new ways of counteracting the adverse effects of asynchrony: a simple mechanism like using negative algorithmic momentum can improve performance under high asynchrony. Since asynchronous methods have better hardware efficiency, this result may shed light on when asynchronous execution is more efficient for deep learning systems.

Quantum control is valuable for various quantum technologies such as high-fidelity gates for universal quantum computing, adaptive quantum-enhanced metrology, and ultra-cold atom manipulation. Although supervised machine learning and reinforcement learning are widely used for optimizing control parameters in classical systems, quantum control for parameter optimization is mainly pursued via gradient-based greedy algorithms. Although the quantum fitness landscape is often compatible with greedy algorithms, sometimes greedy algorithms yield poor results, especially for large-dimensional quantum systems. We employ differential evolution algorithms to circumvent the stagnation problem of non-convex optimization. We improve quantum control fidelity for noisy system by averaging over the objective function. To reduce computational cost, we introduce heuristics for early termination of runs and for adaptive selection of search subspaces. Our implementation is massively parallel and vectorized to reduce run time even further. We demonstrate our methods with two examples, namely quantum phase estimation and quantum gate design, for which we achieve superior fidelity and scalability than obtained using greedy algorithms.

The aim of this paper is to investigate the interplay between knowledge shared by a group of agents and its coalition ability. We investigate this relation in the standard context of imperfect information concurrent game. We assume that whenever a set of agents form a coalition to achieve a goal, they share their knowledge before acting. Based on this assumption, we propose a new semantics for alternating-time temporal logic with imperfect information and perfect recall. It turns out that this semantics is sufficient to preserve all the desirable properties of coalition ability in traditional coalitional logics. Meanwhile, we investigate how knowledge sharing within a group of agents contributes to its coalitional ability through the interplay of epistemic and coalition modalities. This work provides a partial answer to the question: which kind of group knowledge is required for a group to achieve their goals in the context of imperfect information.

Goal models have been widely used in Computer Science to represent software requirements, business objectives, and design qualities. Existing goal modelling techniques, however, have shown limitations of expressiveness and/or tractability in coping with complex real-world problems. In this work, we exploit advances in automated reasoning technologies, notably Satisfiability and Optimization Modulo Theories (SMT/OMT), and we propose and formalize: (i) an extended modelling language for goals, namely the Constrained Goal Model (CGM), which makes explicit the notion of goal refinement and of domain assumption, allows for expressing preferences between goals and refinements, and allows for associating numerical attributes to goals and refinements for defining constraints and optimization goals over multiple objective functions, refinements and their numerical attributes; (ii) a novel set of automated reasoning functionalities over CGMs, allowing for automatically generating suitable refinements of input CGMs, under user-specified assumptions and constraints, that also maximize preferences and optimize given objective functions. We have implemented these modelling and reasoning functionalities in a tool, named CGM-Tool, using the OMT solver OptiMathSAT as automated reasoning backend. Moreover, we have conducted an experimental evaluation on large CGMs to support the claim that our proposal scales well for goal models with thousands of elements.

With the development of community based question answering (Q&A) services, a large scale of Q&A archives have been accumulated and are an important information and knowledge resource on the web. Question and answer matching has been attached much importance to for its ability to reuse knowledge stored in these systems: it can be useful in enhancing user experience with recurrent questions. In this paper, we try to improve the matching accuracy by overcoming the lexical gap between question and answer pairs. A Word Embedding based Correlation (WEC) model is proposed by integrating advantages of both the translation model and word embedding, given a random pair of words, WEC can score their co-occurrence probability in Q&A pairs and it can also leverage the continuity and smoothness of continuous space word representation to deal with new pairs of words that are rare in the training parallel text. An experimental study on Yahoo! Answers dataset and Baidu Zhidao dataset shows this new method's promising potential.

StartupSource's Liam Tracey-Raymont recently spoke with Andrew Arruda, CEO and co-founder of ROSS Intelligence Inc. ("ROSS"), to discuss ROSS's growth and to shed some light on the developmental timeline and hurdles associated with launching a tech startup. ROSS, initially developed in Canada and founded by University of Toronto ("U of T") students, is a cloud-based software program that assists users in answering legal questions efficiently and without relying on complicated boolean queries and keywords. While ROSS is still a relatively new product, it has quickly attracted significant media attention and immense interest from U.S. clients, which are predominantly law firms. Liam caught up with the ROSS CEO after Andrew returned to his new home in San Francisco, California, following a week on the road. Andrew, it's been a while. What have you been up to recently?

Every student who has panicked while reading the same page of a textbook over and over again may suspect this. But stress cramming for an exam does not work, because the facts are likely to be lost from your memory. Instead it is best to learn through practice tests to protect your brain from the effects of stress, with a study showing that we remember more this way. Every student who has panicked while reading the same page of a textbook over and over again may suspect this. Some 120 students were asked to learn a set of 30 words and 30 images. Each item was displayed for a few seconds on a computer screen.

Talking Machines, 12 episodes, iTunes An interview format based podcast with Hosts, Katherine Gorman and Ryan Adams, who bring clear conversations with experts in the machine learning field. Partially derivative, 23 episodes, iTunes A show about data science, interesting new projects, latest data news and all these conversations over a beer which makes it a good entertainer!! [Latest] Episode 23: Political Science Rulez This week Chris overcompensates for his love of political science while Jonathon continues to be unimpressive. The Data Skeptic, 56 episodes, iTunes This podcast features conversations on topics related to data science, statistics, machine learning, artificial intelligence. It alternates between mini episodes which are quick introductions to concepts and long form episodes which are usually interviews with experts in the field. CyArk is a non-profit focused on using technology and data to preserve the world's important historic and cultural locations digitally.

IBM wants to make the connected products, robots, and other gizmos of the future as smart as they can be -- and it's Jeopardy!-champion That's why the company announced last week that Watson will be made available to more developers than ever thanks to Project Intu. If people take advantage of this program, Watson could make sure their supposedly "smart" products aren't quite as elementary as they are now. Watson has already been used to do everything from enable robot concierges to help a professor at Georgia Tech manage the online forums of his A.I. course. Project Intu is supposed to expand Watson's influence even further.