In many sequential decision-making problems one is interested in minimizing an expected cumulative cost while taking into account \emph{risk}, i.e., increased awareness of events of small probability and high consequences. Accordingly, the objective of this paper is to present efficient reinforcement learning algorithms for risk-constrained Markov decision processes (MDPs), where risk is represented via a chance constraint or a constraint on the conditional value-at-risk (CVaR) of the cumulative cost. We collectively refer to such problems as percentile risk-constrained MDPs. Specifically, we first derive a formula for computing the gradient of the Lagrangian function for percentile risk-constrained MDPs. Then, we devise policy gradient and actor-critic algorithms that (1) estimate such gradient, (2) update the policy in the descent direction, and (3) update the Lagrange multiplier in the ascent direction. For these algorithms we prove convergence to locally optimal policies. Finally, we demonstrate the effectiveness of our algorithms in an optimal stopping problem and an online marketing application.

Finding the music of the moment can often be a challenging problem, even for humans with well-versed musical tastes. These challenges further explode into a myriad of complexities when attempting to construct algorithmic approaches for automatic playlist generation. A variety of factors play a role in influencing a listener's perception of what music is appropriate on a given seed (e.g. Erik Schmidt, Senior Scientist at Pandora will be presenting at the Machine Intelligence Summit in San Francisco, 23-24 March. Erik will present an overview of recommendations at Pandora, followed by a deep dive into the challenges of recommending content.

Whether you've just gotten rid of cable or want to supplement your TV package with online video, now's an excellent time to buy a media streaming device. Compared to the typical smart TV, standalone streamers such as the Roku Streaming Stick and Amazon Fire TV tend to have bigger app selections, faster performance, and more features. And with so much competition between device makers, the hardware is becoming faster, more capable, and more affordable. We constantly test all the latest devices, including Roku players, Fire TV devices, Android TV devices, Apple TV, and Chromecast. We review each new generation of hardware and constantly revisit the software and app selection so we can help you determine which platform is right for you.

AI applications have been deployed and used for industrial, government, and consumer purposes for many years. The experiences have been documented in IAAI conference proceedings since 1989. Over the years, the breadth of applications has expanded many times over and AI systems have become more commonplace. Indeed, AI has recently become a focal point in the industrial and consumer consciousness. This article focuses on changes in the world of computing over the last three decades that made building AI applications more feasible. We then examine lessons learned during this time and distill these lessons into succinct advice for future application builders.

Poaching is considered a major driver for the population drop of key species such as tigers, elephants, and rhinos, which can be detrimental to whole ecosystems. While conducting foot patrols is the most commonly used approach in many countries to prevent poaching, such patrols often do not make the best use of the limited patrolling resources.

The stochastic block model (SBM) is a random graph model with planted clusters. It is widely employed as a canonical model to study clustering and community detection, and provides generally a fertile ground to study the statistical and computational tradeoffs that arise in network and data sciences. This note surveys the recent developments that establish the fundamental limits for community detection in the SBM, both with respect to information-theoretic and computational thresholds, and for various recovery requirements such as exact, partial and weak recovery (a.k.a., detection). The main results discussed are the phase transitions for exact recovery at the Chernoff-Hellinger threshold, the phase transition for weak recovery at the Kesten-Stigum threshold, the optimal distortion-SNR tradeoff for partial recovery, the learning of the SBM parameters and the gap between information-theoretic and computational thresholds. The note also covers some of the algorithms developed in the quest of achieving the limits, in particular two-round algorithms via graph-splitting, semi-definite programming, linearized belief propagation, classical and nonbacktracking spectral methods. A few open problems are also discussed.

Marketing and sales seemed to be much easier in the past. Customers simply visited a retail shop, where they could ask a knowledgeable salesperson about a product they discovered in a local newspaper. In recent years, the ubiquity of the internet and a state-of-the-art technology changed everything. Customers became prosumers, well informed about the product before the purchase. And what is even more important, customers frequently use a variety of channels: online and traditional stores, mobile apps, online auctions, price comparison websites, social media and more. Today, in spite of all the available technologies, life is more challenging for both marketers and salespeople.

Let's consider what factors are driving artificial intelligence applications for payments and transaction processing: Digital banking and ecommerce channels are growing exponentially as more and more people use apps and mobile connectivity for transactions. For retailers, newer business models are evolving every day, from instant delivery of goods to digital downloads. Commerce is now operating in an omni-channel environment across multiple devices and touchpoints. Growth comes with a price, as it has led to a corresponding rise in fraud - and fraud loss - in online marketplaces that connect buyers and sellers. That's especially true in e-commerce, where it is harder and more complex to prevent fraud than in person transactions.

Military delegates arrive at the Great Hall of the People for a meeting ahead of Saturday's opening ceremony of the National People's Congress (NPC), in Beijing, China March 4, 2016. As Washington fiddles, China is investing billions in U.S. startups with cutting-edge products that could have military applications at the same time it is dialing back investments in less critical American industries such as entertainment. A New York Times story this week says that among the startups are companies working on artificial intelligence for military robots, rocket engines, ship sensors and printers that could produce high-tech components such as computer screens for military jets. Many of the firms making such investments are owned by companies controlled by the Chinese government or connected to its leaders. A blog post last December on the website of CB Insights, which tracks startup investments, says that China poured $9.9 billion into new Silicon Valley firms in 2015 and made an additional $3.5 billion in tech investments in the first nine months of last year.

More and more companies are putting drones to work, including tech giants, manufacturers, utilities, and news organizations. With a broad range of practical applications and rapidly evolving technology, drones offer huge untapped potential, but not every market offers equal opportunities for growth. Here are seven facts and forecasts to know before investing. The demand for drones in the U.S. is projected to rise 10% annually to $4.4 billion in 2020, and the number of vehicles sold will more than double to 5.5 million. Drones sold to commercial and consumer users can cost less than $100 on the low end for toy drones to $10,000 or more on the high end for professional drones with sophisticated sensors and controls. Civilian markets are in the early stages of development, and over the next decade sales are expected to grow at explosive rates, similar to those posted by smartphones following the introduction of the iPhone in 2007.