We introduce a new, high-throughput, synchronous, distributed, data-parallel, stochastic-gradient-descent learning algorithm. This algorithm uses amortized inference in a compute-cluster-specific, deep, generative, dynamical model to perform joint posterior predictive inference of the mini-batch gradient computation times of all worker-nodes in a parallel computing cluster. We show that a synchronous parameter server can, by utilizing such a model, choose an optimal cutoff time beyond which mini-batch gradient messages from slow workers are ignored that maximizes overall mini-batch gradient computations per second. In keeping with earlier findings we observe that, under realistic conditions, eagerly discarding the mini-batch gradient computations of stragglers not only increases throughput but actually increases the overall rate of convergence as a function of wall-clock time by virtue of eliminating idleness. The principal novel contribution and finding of this work goes beyond this by demonstrating that using the predicted run-times from a generative model of cluster worker performance to dynamically adjust the cutoff improves substantially over the static-cutoff prior art, leading to, among other things, significantly reduced deep neural net training times on large computer clusters.

Abstract--In this article, we improve extreme learning machines for regression tasks using a graph signal processing based regularization. We assume that the target signal for prediction or regression is a graph signal. With this assumption, we use the regularization to enforce that the output of an extreme learning machine is smooth over a given graph. Simulation results with real data confirm that such regularization helps significantly when the available training data is limited in size and corrupted by noise. I NTRODUCTION Extreme learning machines (ELMs) have emerged as an active area of research within the machine learning community [1].

We study the problem of detecting the presence of a single unknown spike in a rectangular data matrix, in a high-dimensional regime where the spike has fixed strength and the aspect ratio of the matrix converges to a finite limit. This setup includes Johnstone's spiked covariance model. We analyze the likelihood ratio of the spiked model against an "all noise" null model of reference, and show it has asymptotically Gaussian fluctuations in a region below---but in general not up to---the so-called BBP threshold from random matrix theory. Our result parallels earlier findings of Onatski et al.\ (2013) and Johnstone-Onatski (2015) for spherical spikes. We present a probabilistic approach capable of treating generic product priors. In particular, sparsity in the spike is allowed. Our approach is based on Talagrand's interpretation of the cavity method from spin-glass theory. The question of the maximal parameter region where asymptotic normality is expected to hold is left open. This region is shaped by the prior in a non-trivial way. We conjecture that this is the entire paramagnetic phase of an associated spin-glass model, and is defined by the vanishing of the replica-symmetric solution of Lesieur et al.\ (2015).

We address the problem of finding influential training samples for a particular case of tree ensemble-based models, e.g., Random Forest (RF) or Gradient Boosted Decision Trees (GBDT). A natural way of formalizing this problem is studying how the model's predictions change upon leave-one-out retraining, leaving out each individual training sample. Recent work has shown that, for parametric models, this analysis can be conducted in a computationally efficient way. We propose several ways of extending this framework to non-parametric GBDT ensembles under the assumption that tree structures remain fixed. Furthermore, we introduce a general scheme of obtaining further approximations to our method that balance the trade-off between performance and computational complexity. We evaluate our approaches on various experimental setups and use-case scenarios and demonstrate both the quality of our approach to finding influential training samples in comparison to the baselines and its computational efficiency.

I recently spoke at a panel/debate at the World Government Summit in Dubai and also attended the AI Roundtable organized by the AI society at Harvard Kennedy school of government. The topic discussed was "Is a machine-driven world better? I was on the side of the Machines. Not an easy debate – someone referred to us as the'evil team'! After the panel, I shared my thoughts with Gregory Piatetsky Shapiro. Gregory motivated me to also consider the contra perspective i.e. the risks of an AI driven world.

CREATIVITY: IT'S AT THE HEART OF WHO WE HUMANS ARE… WE HUMANS ARE SPECIAL, RIGHT? Can a robot write a symphony? Can a robot turn a canvas into a beautiful masterpiece? OVER SOME 40,000 YEARS, HUMAN CREATIVITY HAS EXPLODED – FROM DRAWINGS ON CAVE WALLS THROUGH THE GREAT ART OF CENTURIES TO COME…. NOW, SCIENTISTS -- AND ARTISTS –ARE ASKING CAN A ROBOT TRULY IMAGINE AN ORIGINAL MASTERWORK? COMPUTATIONAL CREATIVITY IS LEADING US TO ASK NEW QUESTIONS ABOUT HUMAN CREATIVITY. IS THIS ESSENTIAL HUMAN TRAIT TRULY UNIQUE? WILL ARTIFICIAL INTELLIGENCE BE A COMPETITOR? OR CAN IT BE A COLLABORATOR, HELPING US TOWARD STILL UNIMAGINED CREATIONS? SCHAEFER: My first guest is a member of Google Brain's Magenta team. He is currently working on neural network models of sound and music and recently produced a synthesizer that designed its own sounds. SCHAEFER: Also with us, is an Assistant professor at the University of Illinois at Urbana Champaign in the Dept. of Electrical and Computer Engineering. He focuses on several surprising creative domains including the culinary arts and fashion and the theoretical foundations of creativity. SCHAEFER: Also with us is an Associate Professor of psychological and brain science at Dartmouth College. He's interested in the neural basis of imagination and in the evolution of human creativity. A former research fellow at MIT's Media lab and artist in residence at Google, please welcome Sougwen Chung.

Artificial intelligence is quickly becoming an integral part of every tech company's strategy, so it's no surprise that big firms are ramping up their acquisition of AI startups. M&A activity has already seen a fivefold increase in the number of AI startup acquisitions -- from 22 in 2013 to 115 in 2017. While the race is far from over, Google and Apple have acquired the most AI startups since 2012. With close to 14 acquisitions, Google is currently leading the charge to buy AI startups. The company's most recent acquisition involves the conversational commerce platform Banter, which helps businesses connect with their customers over popular messaging platforms like Facebook Messenger, Twitter, and Snapchat.

Want to watch this again later? Sign in to add this video to a playlist. Report Need to report the video? Sign in to report inappropriate content. Report Need to report the video?

AI could reboot industries and make the economy more productive; it's already infusing many of the products we use daily. But a new report by more than 20 researchers from the Universities of Oxford and Cambridge, OpenAI, and the Electronic Frontier Foundation warns that the same technology creates new opportunities for criminals, political operatives, and oppressive governments--so much so that some AI research may need to be kept secret. Included in the report, The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation, are four dystopian vignettes involving artificial intelligence that seem taken straight out of the Netflix science fiction show Black Mirror. An administrator for a building's robot security system spends some of her time on Facebook during the workday. There she sees an ad for a model train set and downloads a brochure for it.

Rael wanted to combine his years of knowledge in the finance world with advertising. He quit his job soon afterwards and after contacting Dr. Jun Wang who had papers on the very same subject, they both decided to co-found MediaGamma. Rael Cline shares with me how MediaGamma is helping companies harness the power of their customer data to drive better customer-centric products. Hi Rael, thanks for agreeing to share your story on YHP. Can you give us some background information about yourself?