We propose the stochastic average gradient (SAG) method for optimizing the sum of a finite number of smooth convex functions. Like stochastic gradient (SG) methods, the SAG method's iteration cost is independent of the number of terms in the sum. However, by incorporating a memory of previous gradient values the SAG method achieves a faster convergence rate than black-box SG methods. The convergence rate is improved from O(1/k^{1/2}) to O(1/k) in general, and when the sum is strongly-convex the convergence rate is improved from the sub-linear O(1/k) to a linear convergence rate of the form O(p^k) for p \textless{} 1. Further, in many cases the convergence rate of the new method is also faster than black-box deterministic gradient methods, in terms of the number of gradient evaluations. Numerical experiments indicate that the new algorithm often dramatically outperforms existing SG and deterministic gradient methods, and that the performance may be further improved through the use of non-uniform sampling strategies.

We continue our analysis of volume and energy measures that are appropriate for quantifying inductive inference systems. We extend logical depth and conceptual jump size measures in AIT to stochastic problems, and physical measures that involve volume and energy. We introduce a graphical model of computational complexity that we believe to be appropriate for intelligent machines. We show several asymptotic relations between energy, logical depth and volume of computation for inductive inference. In particular, we arrive at a "black-hole equation" of inductive inference, which relates energy, volume, space, and algorithmic information for an optimal inductive inference solution. We introduce energy-bounded algorithmic entropy. We briefly apply our ideas to the physical limits of intelligent computation in our universe.

In October 2011, Apple added the emoji keyboard to iOS as an international keyboard. Since then, digital language has evolved such that nearly half of comments and captions on Instagram contain emoji characters. And earlier this week, Instagram also added support for emoji characters in hashtags, which allows people to tag and search content with their favorite emoji # . In Part 1 of this blog post series, we will take a deep dive into emoji usage on Instagram. By applying machine learning and natural language processing techniques, we'll discover the hidden semantics of emoji.

In a conference on Monday, U.S. Deputy Defense Secretary Bob Work outlined a key component of modern warfare strategy called Third Offset. The military intends to take advantage of cutting-edge R&D to incorporate AI-human teams to overcome an enemy's network. At the 2016 Global Strategy forum on Monday, Mr. Work noted that products with potential military applications are fast-tracked to enter the global market. "R&D is going down in the public sector, but up in the private sector. Most things that have to do with AI [artificial intelligence] and autonomy are happening in the private sector. And so all competitors are going to have access to it, it's going to be a world of fast-followers. You're going to have an instance where you're not going to have a lasting advantage."

Update: readers of the post have also pointed out this critique by Ernest Davis and this response to Davis by Rob Bensinger. Update 2: Both Rob Bensinger and Michael Tetelman rightly pointed out that my intelligence definition was sloppily defined. I've added a clarification that the defintion is'for a given task'. This post is a discussion of Nick Bostrom's book "Superintelligence". The book has had an effect on the thinking of many of the world's thought leaders. In that light, and given this series of blog posts is about the "Future of AI", it seemed important to read the book and discuss his ideas. In an ideal world, this post would certainly have contained more summaries of the books arguments and perhaps a later update will improve on that aspect. For the moment the review focuses on counter-arguments and perceived omissions (the post already got too long with just covering those). Bostrom considers various routes we have to forming intelligent machines and what the possible outcomes might be from developing such technologies. He is a professor of philosophy but has an impressive array of background degrees in areas such as mathematics, logic, philosophy and computational neuroscience. So let's start at the beginning and put the book in context by trying to understand what is meant by the term "superintelligence" In common with many contributions to the debate on artificial intelligence, Bostrom never defines what he means by intelligence. Obviously, this can be problematic. On the other hand, superintelligence is defined as outperforming humans in every intelligent capability that they express.

The robot will take its exams in a closed room without anyone else present, except for proctors and a notary.

Artificial intelligence has gained serious attention as a solution for complex problems, but the head of the Defense Advanced Research Projects Agency cautions against viewing it as a panacea. "When we look at what's happening with artificial intelligence, we see something that is very, very powerful, very valuable for military applications, but we also see a technology that is still quite fundamentally limited," DARPA Director Arati Prabhakar said at the Atlantic Council on May 2. Image analysis, Prabhakar said, reveals some of the technology's limitations. While AI and machine learning systems are statistically better than humans at identifying images because they can sift through thousands of images in seconds, "the problem is that when they're wrong, they are wrong in ways that no human would ever be wrong," she said. In one case, a picture of a baby holding a toothbrush was identified by a machine as a baby with a baseball bat. "I think this is a critically important caution about where and how we would use this generation of artificial intelligence," she said.

Some AI proponents argue that Artificial Intelligence will usurp human intelligence or even make us obsolete. That kind of talk must stop, before we lose control of AI. Artificial Intelligence (AI) is one of the leading Internet trends of 2016, particularly with large companies like Google and Facebook pouring resources into it. While there are many benefits to AI -- for example, Facebook using it to make our news feeds smarter -- the hype is getting hubristic. I'm particularly concerned about the language AI proponents are using.

The list of featured speakers includes innovators, business leaders, and entrepreneurs from the Allen Institute for Artificial Intelligence, Amazon Robotics, Baidu, Facebook, GE Software Research, Google, IBM, Pinterest, Tesla, and more. About MIT Technology Review Founded at the Massachusetts Institute of Technology in 1899, MIT Technology Review is a digitally oriented independent media company whose analysis, features, reviews, interviews, and live events explain the commercial, social, and political impact of new technologies. MIT Technology Review readers are curious technology enthusiasts--a global audience of business and thought leaders, innovators and early adopters, entrepreneurs and investors. Every day, we provide an authoritative filter for the flood of information about technology. We are the first to report on a broad range of new technologies, informing our audiences about how important breakthroughs will impact their careers and their lives.

You probably didn't consider basing your Kentucky Derby bets on artificial intelligence -- but maybe you should have. The artificial intelligence company Unanimous tested its new software platform, UNU, on last weekend's Kentucky Derby, as reported by TechRepublic. Twenty participants, convened by the company, first used the software to narrow the field of 20 horses down to four top picks. The participants then used UNU to predict the winning order -- and it turned out to be 100 percent correct. "I placed my 1 bet on the race at the Derby on Saturday and made 542.10 -- the odds of winning the superfecta [the top 4 finishers in order] were 540-1," TechRepublic reporter Hope Reese wrote.