We propose a new PAC-Bayesian bound and a way of constructing a hypothesis space, so that the bound is convex in the posterior distribution and also convex in a trade-off parameter between empirical performance of the posterior distribution and its complexity. The complexity is measured by the Kullback-Leibler divergence to a prior. We derive an alternating procedure for minimizing the bound. We show that the bound can be rewritten as a one-dimensional function of the trade-off parameter and provide sufficient conditions under which the function has a single global minimum. When the conditions are satisfied the alternating minimization is guaranteed to converge to the global minimum of the bound. We provide experimental results demonstrating that rigorous minimization of the bound is competitive with cross-validation in tuning the trade-off between complexity and empirical performance. In all our experiments the trade-off turned to be quasiconvex even when the sufficient conditions were violated.

In modern large-scale machine learning applications, the training data are often partitioned and stored on multiple machines. It is customary to employ the "data parallelism" approach, where the aggregated training loss is minimized without moving data across machines. In this paper, we introduce a novel distributed dual formulation for regularized loss minimization problems that can directly handle data parallelism in the distributed setting. This formulation allows us to systematically derive dual coordinate optimization procedures, which we refer to as Distributed Alternating Dual Maximization (DADM). The framework extends earlier studies described in (Boyd et al., 2011; Ma et al., 2017; Jaggi et al., 2014; Yang, 2013) and has rigorous theoretical analyses. Moreover with the help of the new formulation, we develop the accelerated version of DADM (Acc-DADM) by generalizing the acceleration technique from (Shalev-Shwartz and Zhang, 2014) to the distributed setting. We also provide theoretical results for the proposed accelerated version and the new result improves previous ones (Yang, 2013; Ma et al., 2017) whose iteration complexities grow linearly on the condition number. Our empirical studies validate our theory and show that our accelerated approach significantly improves the previous state-of-the-art distributed dual coordinate optimization algorithms.

R is smooth, and h and r are convex (but possibly non-smooth). In particular, we target applications in signal-processing, high-dimensional statistics and machine learning. Here, x is viewed as a variable of primary interest, while θ represents a set of auxiliary (nuisance) parameters. In many applications, efficient algorithms have been developed to globally minimize the objective function in x for fixed θ. We provide a provably convergent algorithmic recipe to extend these algorithms to (1). We begin by reviewing the classic Variable Projection (VP) technique for nonlinear least squares problems. Early work on the topic [10] has found numerous applications in chemistry, mechanical systems, neural networks, and telecommunications (see the surveys of [11] and [15], and references therein.)

Structured Query Language (SQL) is an indispensable skill in the data science industry and generally speaking, learning this skill is fairly easy. However, most forget that SQL isn't just about writing queries, which is just the first step down the road. Ensuring that queries are performant or that they fit the context that you're working in is a whole other thing. That's why this SQL tutorial will provide you with a small peek at some steps that you can go through to evaluate your query: Are you interested in an SQL course? Take DataCamp's Intro to SQL for Data Science course!

Serial killer David Berkowitz, also known as the "Son of Sam," famously claimed that he heard voices in the form of a dog telling him to commit murder. In fact, according to the authors of a recent study published in the journal Brain, enhanced attention-related nerual pathways might cause these illusory sounds. People hear them because their brains may be especially primed to pick up speech. "It's true that lots of people who hear voices have serious mental health issues," Ben Alderson-Day, a psychological research at Durham University and lead author on the study told Popular Science. "But roughly 5 to 15 percent of the general population will have some experience of hearing unusual voices at some point in their lives. We think potentially up to one percent might have pretty frequent experiences and just don't really tell anyone and get on with their everyday lives."

Among the 22 Turing Laureates in attendance at the conference were: Front row, from left: Whitfield Diffie (2015), Martin Hellman (2015), Robert Tarjan (1986), Barbara Liskov (2008). Second row, from left: Vinton Cerf (2004), Richard Karp (1985), Richard Stearns (1993), Dana Scott (1976).

Charles William "Charlie" Bachman, the "father of databases" who received the ACM A.M. Turing Award for 1973 for creating the first database management system, died June 13 at the age of 92. Born in Manhattan, KS, in 1924, Bachman earned his B.S. in mechanical engineering in 1948, as well as an M.S. in mechanical engineering from the University of Pennsylvania. He went to work for Dow Chemical in 1950, using mechanical punched-card computing devices to solve networks of simultaneous equations representing data from Dow plants. In 1957, Bachman became head of Dow's Data Processing Department, through which he became a member of Share Inc., and a founding member of the Share Data Processing Committee. In 1960, Bachman joined the General Electric (GE) Production Control Services Group in New York City, using a factory in Philadelphia to test designs for a system to automate factory planning, scheduling, operational control, and inventory control.

Despite the fact that he does not see very well, Alexei Efros, recipient of the 2016 ACM Prize in Computing and a professor at the University of California at Berkeley, has spent most of his career trying to understand, model, and recreate the visual world. Drawing on the massive collection of images on the Internet, he has used machine learning algorithms to manipulate objects in photographs, translate black-and-white images into color, and identify architecturally revealing details about cities. Here, he talks about harnessing the power of visual complexity. You were born in St. Petersburg (Russia), and were 14 when you came to the U.S. What drew you to computer science?

Artificial intelligence is making its way into the messy world of account receivables for those corporates processing large volumes of payments, thanks to Bank of America Merrill Lynch and fintech startup, High Radius. The "Intelligent Receivables" solution is designed to streamline the receivables process, using AI technology developed by High Radius to fill in the gaps or allow companies to upload missing remittance information to invoices, the companies announced today. Using this solution, payers are automatically identified, as is remittance data that might have arrived separate to an invoice. Clients can also set up automatic emails to payers, in order to further streamline the process. The solution is presently available for clients in the United States and Canada, but it is expected to roll out to other markets later this year.

As excited as we are about the forthcoming generation of social home robots (including Jibo, Kuri, and many others), it's hard to ignore the fact that most of them look somewhat similar. They tend to feature lots of shiny white and black plasticky roundness. That's for admittedly very good reasons, but it comes at the cost of both uniqueness and visual and tactile personality. Guy Hoffman, who is well known for the fascinating creativity of his robot designs, has been working on a completely new kind of social robot in a collaboration between his lab at Cornell and Google ZOO's creative technology team in APAC. The robot is called Blossom, and we'd describe it for you, except that it's designed to be handmade out of warm natural materials like wool and wood so that every single one is a little bit different.