First Amazon announced plans to fully automate its new brick and mortar store with robots. Then we learned that Foxconn plans to automate 30 percent of its factory workforce by 2018. And recently, Wendy's announced plans to add automated kiosks at more than 1,000 stores. One thing is clear -- robots are changing the way we live and work. And this is just the beginning.

Kernel online convex optimization (KOCO) is a framework combining the expressiveness of non-parametric kernel models with the regret guarantees of online learning. First-order KOCO methods such as functional gradient descent require only $\mathcal{O}(t)$ time and space per iteration, and, when the only information on the losses is their convexity, achieve a minimax optimal $\mathcal{O}(\sqrt{T})$ regret. Nonetheless, many common losses in kernel problems, such as squared loss, logistic loss, and squared hinge loss posses stronger curvature that can be exploited. In this case, second-order KOCO methods achieve $\mathcal{O}(\log(\text{Det}(\boldsymbol{K})))$ regret, which we show scales as $\mathcal{O}(d_{\text{eff}}\log T)$, where $d_{\text{eff}}$ is the effective dimension of the problem and is usually much smaller than $\mathcal{O}(\sqrt{T})$. The main drawback of second-order methods is their much higher $\mathcal{O}(t^2)$ space and time complexity. In this paper, we introduce kernel online Newton step (KONS), a new second-order KOCO method that also achieves $\mathcal{O}(d_{\text{eff}}\log T)$ regret. To address the computational complexity of second-order methods, we introduce a new matrix sketching algorithm for the kernel matrix $\boldsymbol{K}_t$, and show that for a chosen parameter $\gamma \leq 1$ our Sketched-KONS reduces the space and time complexity by a factor of $\gamma^2$ to $\mathcal{O}(t^2\gamma^2)$ space and time per iteration, while incurring only $1/\gamma$ times more regret.

This paper proposes a new family of algorithms for training neural networks (NNs). These are based on recent developments in the field of non-convex optimization, going under the general name of successive convex approximation (SCA) techniques. The basic idea is to iteratively replace the original (non-convex, highly dimensional) learning problem with a sequence of (strongly convex) approximations, which are both accurate and simple to optimize. Differently from similar ideas (e.g., quasi-Newton algorithms), the approximations can be constructed using only first-order information of the neural network function, in a stochastic fashion, while exploiting the overall structure of the learning problem for a faster convergence. We discuss several use cases, based on different choices for the loss function (e.g., squared loss and cross-entropy loss), and for the regularization of the NN's weights. We experiment on several medium-sized benchmark problems, and on a large-scale dataset involving simulated physical data. The results show how the algorithm outperforms state-of-the-art techniques, providing faster convergence to a better minimum. Additionally, we show how the algorithm can be easily parallelized over multiple computational units without hindering its performance. In particular, each computational unit can optimize a tailored surrogate function defined on a randomly assigned subset of the input variables, whose dimension can be selected depending entirely on the available computational power.

Determinantal point processes (DPPs) are distributions over sets of items that model diversity using kernels. Their applications in machine learning include summary extraction and recommendation systems. Yet, the cost of sampling from a DPP is prohibitive in large-scale applications, which has triggered an effort towards efficient approximate samplers. We build a novel MCMC sampler that combines ideas from combinatorial geometry, linear programming, and Monte Carlo methods to sample from DPPs with a fixed sample cardinality, also called projection DPPs. Our sampler leverages the ability of the hit-and-run MCMC kernel to efficiently move across convex bodies. Previous theoretical results yield a fast mixing time of our chain when targeting a distribution that is close to a projection DPP, but not a DPP in general. Our empirical results demonstrate that this extends to sampling projection DPPs, i.e., our sampler is more sample-efficient than previous approaches which in turn translates to faster convergence when dealing with costly-to-evaluate functions, such as summary extraction in our experiments.

In this article, I present a few modern techniques that have been used in various business contexts, comparing performance with traditional methods. The advanced techniques in question are math-free, innovative, efficiently process large amounts of unstructured data, and are robust and scalable. Implementations in Python, R, Julia and Perl are provided, but here we focus on an Excel version that does not even require any Excel macros, coding, plug-ins, or anything other than the most basic version of Excel. It is actually easily implemented in standard, basic SQL too, and we invite readers to work on an SQL version. In short, we offer here an Excel template for machine learning and statistical computing, and it is quite powerful for an Excel spreadsheet.

In the technical and largely male-dominated world of Artificial Intelligence, the notion of making emotional connections seems completely counter-intuitive…until you talk with Olga Russakovsky. As co-founder of SAILORS (Stanford Artificial Intelligence Laboratory's Outreach Summer), America's first AI summer camp for teen girls, Olga reflects on what she refers to as a "transformative experience." It was a simple moment that occurred one morning when the SAILORS camp girls were at breakfast. "Girls were sitting there, braiding each other's hair…and discussing AI!" she says. "Watching them do something so girly like braiding hair while they were talking through their research project at SAILORS was so inspiring!"

Something quite spectacular happened during the week: Students have achieved an astounding level of score improvement on a highly complicated machine learning problem - in just three afternoons. They achieved scores that improved more than 70% over the initial solution that were built by a team of experienced domain specialists and senior data scientists (figure 1). Considering that roughly half of the students had no prior exposure to machine learning, and that the other half were mostly beginners, these improvements are impressive. In fact, this is not the first time we observed this kind of results: every time we ran a data challenge using RAMP (rapid analytics and model prototyping) platform, major improvements have been made over the initial solution. So, how does this happen?

Everyone who gets going in Machine Learning (and Deep Learning) gets overwhelmed by the plethora of MOOCs available. Here, I try to give a comprehensive survey of such courses available freely on the internet. You can take this post as an complementary to this and this previous posts. I will try to highlight some important pointers such as the difficulty of the courses, the correct order in which these should to be completed, the right audience for these courses. You will get a feel of how these courses give you a stack of skills in your arsenal and how you can use them to develop practical machine learning systems.

The belief that there is a link between talent and left-handedness has a long history. Leonardo da Vinci was left-handed. So were Mark Twain, Mozart, Marie Curie, Nicola Tesla and Aristotle. It's no different today – former US president Barack Obama is a left-hander, as is business leader Bill Gates and footballer Lionel Messi. But is it really true that left-handers are more likely to be geniuses?

This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.