Reinforcement learning studies how to balance exploration and exploitation in real-world systems, optimizing interactions with the world while simultaneously learning how the world works. One general class of algorithms for such learning is the multi-armed bandit setting (in which sequential interactions are independent and identically distributed) and the related contextual bandit case, in which the distribution depends on different information or 'context' presented with each interaction. Thompson sampling, though introduced in the 1930s, has recently been shown to perform well and to enjoy provable optimality properties, while at the same time permitting generative, interpretable modeling. In a Bayesian setting, prior knowledge is incorporated and the computed posteriors naturally capture the full state of knowledge. In several application domains, for example in health and medicine, each interaction with the world can be expensive and invasive, whereas drawing samples from the model is relatively inexpensive. Exploiting this viewpoint, we develop a double-sampling technique driven by the uncertainty in the learning process. The proposed algorithm does not make any distributional assumption and it is applicable to complex reward distributions, as long as Bayesian posterior updates are computable. We empirically show that it out-performs (in the sense of regret) Thompson sampling in two classical illustrative cases, i.e., the multi-armed bandit problem with and without context.

In many biomedical, science, and engineering problems, one must sequentially decide which action to take next so as to maximize rewards. Reinforcement learning is an area of machine learning that studies how this maximization balances exploration and exploitation, optimizing interactions with the world while simultaneously learning how the world operates. One general class of algorithms for this type of learning is the multi-armed bandit setting and, in particular, the contextual bandit case, in which observed rewards are dependent on each action as well as on given information or 'context' available at each interaction with the world. The Thompson sampling algorithm has recently been shown to perform well in real-world settings and to enjoy provable optimality properties for this set of problems. It facilitates generative and interpretable modeling of the problem at hand, though complexity of the model limits its application, since one must both sample from the distributions modeled and calculate their expected rewards. We here show how these limitations can be overcome using variational approximations, applying to the reinforcement learning case advances developed for the inference case in the machine learning community over the past two decades. We consider bandit applications where the true reward distribution is unknown and approximate it with a mixture model, whose parameters are inferred via variational inference.

China's government has announced a goal of becoming a global leader in artificial intelligence in just over a decade, putting political muscle behind growing investment by Chinese companies in developing self-driving cars and other advances. Communist leaders see AI as key to making China an "economic power," said a Cabinet statement on Thursday. It calls for developing skills and research and educational resources to achieve "major breakthroughs" by 2025 and make China a world leader by 2030. It might not be long before machines begin thinking for themselves -- creatively, independently, and sometimes with better judgment than a human.... Artificial intelligence is one of the emerging fields along with renewable energy, robotics and electric cars where communist leaders hope to take an early lead and help transform China from a nation of factory workers and farmers into a technology pioneer. They have issued a series of development plans over the past decade, some of which have prompted complaints Beijing improperly subsidizes its technology developers and shields them from competition in violation of its free-trade commitments.

One of the modern computer's first killer apps was predicting the weather. John von Neumann, who built the initial ENIAC computer, became fascinated with predicting weather in the 1930s. He called it "the most complex, interactive, and highly nonlinear problem that had ever been conceived of." In 1948, he assembled a team of meteorologists to create a mathematical model that would describe what weather would occur based on conditions in the atmosphere. These first weather predictions took more than 24 hours to compute, but proved the idea was possible--and that we needed faster computers.

Oil prices have fallen dramatically over last few years, forcing some major oil companies to take drastic actions such as layoffs, cutting investments and budgets, and more. In view of falling oil prices and the resulting squeeze on cash flows, the oil and gas industry has been challenged to adapt and optimize its performance to remain profitable while maintaining a long-term investment and operating outlook. Currently, oil and gas companies find it difficult to maintain the same level of investment in exploration and production as when crude prices were at their peak. Operations in the oil and gas industry today means balancing a dizzying array of trade-offs in the drive for competitive advantage while maximizing return on investment. The result is a dire need to optimize performance and optimize the cost of production per barrel.

Avitas Systems, a GE subsidiary based in Boston, is now using drones and robots to automate the inspection of infrastructure such as pipelines, power lines, and transportation systems. The company is using off-the-shelf machine-learning technology from Nvidia (50 Smartest Companies 2017) to guide the checkups, and to automatically identify anomalies in the data collected. The effort shows how low-cost drones and robotic systems--combined with rapid advances in machine learning--are making it possible to automate whole sectors of low-skill work. While there is plenty of worry about the automation of jobs in manufacturing and offices, routine security and safety inspections may be one of the first big areas to be undermined by advances in AI. Drones have been used on some industrial sites for a while (see "New Boss on Construction Sites Is a Drone"), and various companies, such as Kespry, Flyability, and CyPhy, offer aerial systems for monitoring mines, inspecting wind turbines, and assessing building insurance claims.

Crowdsourcing has been successfully applied in many domains including astronomy, cryptography and biology. In order to test its potential for useful application in a Smart Grid context, this paper investigates the extent to which a crowd can contribute predictive hypotheses to a model of residential electric energy consumption. In this experiment, the crowd generated hypotheses about factors that make one home different from another in terms of monthly energy usage. To implement this concept, we deployed a web-based system within which 627 residential electricity customers posed 632 questions that they thought predictive of energy usage. While this occurred, the same group provided 110,573 answers to these questions as they accumulated. Thus users both suggested the hypotheses that drive a predictive model and provided the data upon which the model is built. We used the resulting question and answer data to build a predictive model of monthly electric energy consumption, using random forest regression. Because of the sparse nature of the answer data, careful statistical work was needed to ensure that these models are valid. The results indicate that the crowd can generate useful hypotheses, despite the sparse nature of the dataset.

Convolutional sparse representations are a form of sparse representation with a dictionary that has a structure that is equivalent to convolution with a set of linear filters. While effective algorithms have recently been developed for the convolutional sparse coding problem, the corresponding dictionary learning problem is substantially more challenging. Furthermore, although a number of different approaches have been proposed, the absence of thorough comparisons between them makes it difficult to determine which of them represents the current state of the art. The present work both addresses this deficiency and proposes some new approaches that outperform existing ones in certain contexts. A thorough set of performance comparisons indicates a very wide range of performance differences among the existing and proposed methods, and clearly identifies those that are the most effective.

Over-fitting.If you perform a regression with 200 predictors (with strong cross-correlations among predictors), use meta regression coefficients: that is, use coefficients of the form f[Corr(Var, Response), a,b, c] where a, b, c are three meta-parameters (e.g. This will reduce your number of parameters from 200 to 3, and eliminate most of the over-fitting Perform the right type of cross-validation. If your training set has 400,000 observations distributed across 50 clients, and your test data set (used for cross-validation) has 200,000 observations but only 3 clients or 5 days worth of historical data, then your cross-validation methodology is very flawed. Better, split your cross-validation data set in 5 subsets to compute confidence intervals. Make sure you've eliminated outliers and cleaned your data set.

SAN FRANCISCO, Sept. 3 (Xinhua) -- Researchers from the U.S. Department of Energy's SLAC National Accelerator Laboratory and Stanford University have shown that neural networks, a form of artificial intelligence, can analyze the complex distortions in spacetime known as gravitational lenses 10 million times faster than traditional methods. The work, by a research team at the Kavli Institute for Particle Astrophysics and Cosmology (KIPAC), a joint institute of SLAC and Stanford, was detailed in a study published in Nature. The researchers used neural networks to analyze images of strong gravitational lensing, where the image of a faraway galaxy is multiplied and distorted into rings and arcs by the gravity of a massive object, such as a galaxy cluster. The distortions provide clues about how mass is distributed in space and how that distribution changes over time, which are linked to invisible dark matter that makes up 85 percent of all matter in the universe and to dark energy that is accelerating the expansion of the universe. Until now, analyzing such images has been a tedious process that involves comparing actual images of lenses with a large number of computer simulations of mathematical lensing models, according to a news release from SLAC, originally named Stanford Linear Accelerator Center.