Overview
Pruning Random Forests for Prediction on a Budget
Nan, Feng, Wang, Joseph, Saligrama, Venkatesh
We propose to prune a random forest (RF) for resource-constrained prediction. We first construct a RF and then prune it to optimize expected feature cost & accuracy. We pose pruning RFs as a novel 0-1 integer program with linear constraints that encourages feature re-use. We establish total unimodularity of the constraint set to prove that the corresponding LP relaxation solves the original integer program. We then exploit connections to combinatorial optimization and develop an efficient primal-dual algorithm, scalable to large datasets. In contrast to our bottom-up approach, which benefits from good RF initialization, conventional methods are top-down acquiring features based on their utility value and is generally intractable, requiring heuristics. Empirically, our pruning algorithm outperforms existing state-of-the-art resource-constrained algorithms.
Learning Treewidth-Bounded Bayesian Networks with Thousands of Variables
Scanagatta, Mauro, Corani, Giorgio, Campos, Cassio P. de, Zaffalon, Marco
We present a method for learning treewidth-bounded Bayesian networks from data sets containing thousands of variables. Bounding the treewidth of a Bayesian network greatly reduces the complexity of inferences. Yet, being a global property of the graph, it considerably increases the difficulty of the learning process. Our novel algorithm accomplishes this task, scaling both to large domains and to large treewidths. Our novel approach consistently outperforms the state of the art on experiments with up to thousands of variables.
Stochastic Variance Reduction Methods for Saddle-Point Problems
Palaniappan, Balamurugan, Bach, Francis
We consider convex-concave saddle-point problems where the objective functions may be split in many components, and extend recent stochastic variance reduction methods (such as SVRG or SAGA) to provide the first large-scale linearly convergent algorithms for this class of problems which are common in machine learning. While the algorithmic extension is straightforward, it comes with challenges and opportunities: (a) the convex minimization analysis does not apply and we use the notion of monotone operators to prove convergence, showing in particular that the same algorithm applies to a larger class of problems, such as variational inequalities, (b) there are two notions of splits, in terms of functions, or in terms of partial derivatives, (c) the split does need to be done with convex-concave terms, (d) non-uniform sampling is key to an efficient algorithm, both in theory and practice, and (e) these incremental algorithms can be easily accelerated using a simple extension of the "catalyst" framework, leading to an algorithm which is always superior to accelerated batch algorithms.
AI and Speech Recognition: A Primer for Chatbots
Our smartphone currently represents the most expensive area to be purchased per squared centimeter (even more expensive than the square meters price of houses in Beverly Hills), and it is not hard to envision that having a bot as unique interfaces will make this area worth almost zero. None of these would be possible though without heavily investing in speech recognition research. Deep Reinforcement Learning (DFL) has been the boss in town for the past few years and it has been fed by human feedbacks. However, I personally believe that soon we will move toward a B2B (bot-to-bot) training for a very simple reason: the reward structure. Humans spend time training their bots if they are enough compensated for their effort.
A Review of Multivariate Distributions for Count Data Derived from the Poisson Distribution
Inouye, David I., Yang, Eunho, Allen, Genevera I., Ravikumar, Pradeep
The Poisson distribution has been widely studied and used for modeling univariate count-valued data. Multivariate generalizations of the Poisson distribution that permit dependencies, however, have been far less popular. Yet, real-world high-dimensional count-valued data found in word counts, genomics, and crime statistics, for example, exhibit rich dependencies, and motivate the need for multivariate distributions that can appropriately model this data. We review multivariate distributions derived from the univariate Poisson, categorizing these models into three main classes: 1) where the marginal distributions are Poisson, 2) where the joint distribution is a mixture of independent multivariate Poisson distributions, and 3) where the node-conditional distributions are derived from the Poisson. We discuss the development of multiple instances of these classes and compare the models in terms of interpretability and theory. Then, we empirically compare multiple models from each class on three real-world datasets that have varying data characteristics from different domains, namely traffic accident data, biological next generation sequencing data, and text data. These empirical experiments develop intuition about the comparative advantages and disadvantages of each class of multivariate distribution that was derived from the Poisson. Finally, we suggest new research directions as explored in the subsequent discussion section.
Bank distress in the news: Describing events through deep learning
Rรถnnqvist, Samuel, Sarlin, Peter
While many models are purposed for detecting the occurrence of significant events in financial systems, the task of providing qualitative detail on the developments is not usually as well automated. We present a deep learning approach for detecting relevant discussion in text and extracting natural language descriptions of events. Supervised by only a small set of event information, comprising entity names and dates, the model is leveraged by unsupervised learning of semantic vector representations on extensive text data. We demonstrate applicability to the study of financial risk based on news (6.6M articles), particularly bank distress and government interventions (243 events), where indices can signal the level of bank-stress-related reporting at the entity level, or aggregated at national or European level, while being coupled with explanations. Thus, we exemplify how text, as timely, widely available and descriptive data, can serve as a useful complementary source of information for financial and systemic risk analytics.
Rewriting the Code of Life
Early on an unusually blustery day in June, Kevin Esvelt climbed aboard a ferry at Woods Hole, bound for Nantucket Island. Esvelt, an assistant professor of biological engineering at the Massachusetts Institute of Technology, was on his way to present to local health officials a plan for ridding the island of one of its most persistent problems: Lyme disease. He had been up for much of the night working on his slides, and the fatigue showed. He had misaligned the buttons on his gray pin-striped shirt, and the rings around his deep-blue eyes made him look like a sandy-haired raccoon. Esvelt, who is thirty-four, directs the "sculpting evolution" group at M.I.T., where he and his colleagues are attempting to design molecular tools capable of fundamentally altering the natural world. If the residents of Nantucket agree, Esvelt intends to use those tools to rewrite the DNA of white-footed mice to make them immune to the bacteria that cause Lyme and other tick-borne diseases. He and his team would breed the mice in the laboratory and then, as an initial experiment, release them on an uninhabited island. If the number of infected ticks begins to plummet, he would seek permission to repeat the process on Nantucket and on nearby Martha's Vineyard. More than a quarter of Nantucket's residents have been infected with Lyme, which has become one of the most rapidly spreading diseases in the United States. The illness is often accompanied by a red bull's-eye rash, along with fever and chills. When the disease is caught early enough, it can be cured in most cases with a single course of antibiotics. For many people, though, pain and neurological symptoms can persist for years. In communities throughout the Northeast, the fear of ticks has changed the nature of summer itself--few parents these days would permit a child to run barefoot through the grass or wander blithely into the woods. "What if we could wave our hands and make this problem go away?" Esvelt asked the two dozen officials and members of the public who had assembled at the island's police station for his presentation. He explained that white-footed mice are the principal reservoir of Lyme disease, which they pass, through ticks, to humans.
Changing HR : AI At Work
Data driven recruitment has a significant, positive impact on talent management strategies and business performance. As technology becomes more sophisticated, AI is playing an increasingly essential role in decisions made around hiring and is used by brands such as Facebook as an integral part of the screening and assessment of candidates. This article examines its ongoing effect on the jobs market and the ways in which HR can harness its advantages to better understand, improve and predict hiring needs and potential problems. AI is broadly defined as'machines which perform tasks which humans are capable of performing'. It has been traditionally been regarded as a threat to jobs, with the most drastic predictions suggesting that unemployment rates will reach 50% within 30 years, but perceptions and predictions are changing.
Job Automation Predictions from 2016 Silicon Valley Survey -
Job automation predictions from an individual expert typically draw from years of academic research experience, or time "in the trenches" of industry. With growing interest and speculation on the job market of the next decade, we set out to garner a perspective as to what Silicon Valley thinks about the possibilities of automations in various business tasks. In the infographics and article below, we explore the survey responses from nearly 80 Bay Area investors, founders, and tech folks โ on which business functions have the greatest potential for automation today, and in the coming five years ahead. Together with San Fransisco-based venture firm BootstrapLabs, we designed a simple survey that was handed out during their "Autonomous Corporation" event in November 2016. It is interesting to note all three groups of respondents considered business intelligence to be the business function with the most current automation potential.
This Week's Awesome Stories From Around the Web (Through December 24th)
Big Tech's AI Predictions for 2017 Lolita Taub The Huffington Post "For the final Cognitive Business post of the year, I asked artificial intelligence centric Fortune 500 leaders for their 2017 enterprise AI predictions. Microsoft, IBM, Baidu, NVIDIA, Qualcomm, GE, SAS, and Oracle responded. What they had to say is exciting..." Artificial Intelligence Is Going to Make It Easier Than Ever to Fake Images and Video James Vincent The Verge "Smile Vector is just the tip of the iceberg. It's hard to give a comprehensive overview of all the work being done on multimedia manipulation in AI right now, but here are a few examples: creating 3D face models from a single 2D image; changing the facial expressions of a target on video in real time using a human "puppet"; changing the light source and shadows in any picture... live-streaming the presidential debates but making Trump bald..." Artificial Feathers Let Drones Morph Their Wings Like Birds Evan Ackerman IEEE Spectrum "Thanks to overlapping feathers and a joint at the end of the wing, most birds can fold their primary flight feathers back, which significantly reduces the surface area of their wings... These folding wings can vary their surface area by 41 percent. When the wing is completely retracted, lift decreases by 32 percent, and drag decreases by 40 percent, boosting the top speed of the drone from 6.3 meters per second to 7.6 meters per second."