In this paper, we prove a conjecture published in 1989 and also partially address an open problem announced at the Conference on Learning Theory (COLT) 2015. With no unrealistic assumption, we first prove the following statements for the squared loss function of deep linear neural networks with any depth and any widths: 1) the function is non-convex and non-concave, 2) every local minimum is a global minimum, 3) every critical point that is not a global minimum is a saddle point, and 4) there exist "bad" saddle points (where the Hessian has no negative eigenvalue) for the deeper networks (with more than three layers), whereas there is no bad saddle point for the shallow networks (with three layers). Moreover, for deep nonlinear neural networks, we prove the same four statements via a reduction to a deep linear model under the independence assumption adopted from recent work. As a result, we present an instance, for which we can answer the following question: how difficult is it to directly train a deep model in theory? It is more difficult than the classical machine learning models (because of the non-convexity), but not too difficult (because of the nonexistence of poor local minima). Furthermore, the mathematically proven existence of bad saddle points for deeper models would suggest a possible open problem. We note that even though we have advanced the theoretical foundations of deep learning and non-convex optimization, there is still a gap between theory and practice.

Convex optimization is an essential tool for machine learning, as many of its problems can be formulated as minimization problems of specific objective functions. While there is a large variety of algorithms available to solve convex problems, we can argue that it becomes more and more important to focus on efficient, scalable methods that can deal with big data. When the objective function can be written as a sum of "simple" terms, proximal splitting methods are a good choice. UNLocBoX is a MATLAB library that implements many of these methods, designed to solve convex optimization problems of the form $\min_{x \in \mathbb{R}^N} \sum_{n=1}^K f_n(x).$ It contains the most recent solvers such as FISTA, Douglas-Rachford, SDMM as well a primal dual techniques such as Chambolle-Pock and forward-backward-forward. It also includes an extensive list of common proximal operators that can be combined, allowing for a quick implementation of a large variety of convex problems.

An important problem in the field of bioinformatics is to identify interactive effects among profiled variables for outcome prediction. In this paper, a logistic regression model with pairwise interactions among a set of binary covariates is considered. Modeling the structure of the interactions by a graph, our goal is to recover the interaction graph from independently identically distributed (i.i.d.) samples of the covariates and the outcome. When viewed as a feature selection problem, a simple quantity called influence is proposed as a measure of the marginal effects of the interaction terms on the outcome. For the case when the underlying interaction graph is known to be acyclic, it is shown that a simple algorithm that is based on a maximum-weight spanning tree with respect to the plug-in estimates of the influences not only has strong theoretical performance guarantees, but can also outperform generic feature selection algorithms for recovering the interaction graph from i.i.d. samples of the covariates and the outcome. Our results can also be extended to the model that includes both individual effects and pairwise interactions via the help of an auxiliary covariate.

While many models are purposed for detecting the occurrence of significant events in financial systems, the task of providing qualitative detail on the developments is not usually as well automated. We present a deep learning approach for detecting relevant discussion in text and extracting natural language descriptions of events. Supervised by only a small set of event information, comprising entity names and dates, the model is leveraged by unsupervised learning of semantic vector representations on extensive text data. We demonstrate applicability to the study of financial risk based on news (6.6M articles), particularly bank distress and government interventions (243 events), where indices can signal the level of bank-stress-related reporting at the entity level, or aggregated at national or European level, while being coupled with explanations. Thus, we exemplify how text, as timely, widely available and descriptive data, can serve as a useful complementary source of information for financial and systemic risk analytics.

Owing to the data deluge and the Cambrian explosion of machine learning techniques over the past decade, one might have expected the transformation of marketing strategy into a predominantly quantitative discipline by now. The fact that it hasn't happened yet, and the observation that marketing is still influenced by a lot of qualitative inputs can be ascribed to two reasons, in my opinion. The first and principal reason continues to be institutional inertia. Second, there is a significant communication and knowledge gap between data scientists and marketers, owing to their relative lack of familiarity with the other side's perspectives and paradigms. The successful marketer of the next decade is someone who is conversant with management theories of Kotler[1] as well as machine learning advances by Hinton[2]/LeCun[3]/ Ng[4].

Apple Inc. (NASDAQ:AAPL) made waves in the artificial intelligence world when it officially announced that it would start publishing artificial intelligence papers very soon. The Cupertino-based tech giant has done exactly what it promised and published its maiden AI paper. After submitting its artificial intelligence paper on Nov. 15, Apple's AI paper was officially published on Dec. 22. The paper discusses the different techniques of how a developer can improve the training of an algorithm's capability of recognizing different images which are computer generated and not from the real world. The technique is applicable in machine learning research where developers can apply such synthetic images, including those incorporated in video games, to train neural networks.

Scientists at the Pharmaceutical Artificial Intelligence (pharma.AI) group of Insilico Medicine, Inc, have announced the publication of a seminal paper demonstrating the application of generative adversarial autoencoders (AAEs) to generating new molecular fingerprints on demand. The study was published in Oncotarget on 22nd of December, 2016. The study represents the proof of concept for applying Generative Adversarial Networks (GANs) to drug discovery. The authors significantly extended this model to generate new leads according to multiple requested characteristics and plan to launch a comprehensive GAN-based drug discovery engine producing promising therapeutic treatments to significantly accelerate pharmaceutical R&D and improve the success rates in clinical trials. Since 2010 deep learning systems demonstrated unprecedented results in image, voice and text recognition, in many cases surpassing human accuracy and enabling autonomous driving, automated creation of pleasant art and even composition of pleasant music.

Early on an unusually blustery day in June, Kevin Esvelt climbed aboard a ferry at Woods Hole, bound for Nantucket Island. Esvelt, an assistant professor of biological engineering at the Massachusetts Institute of Technology, was on his way to present to local health officials a plan for ridding the island of one of its most persistent problems: Lyme disease. He had been up for much of the night working on his slides, and the fatigue showed. He had misaligned the buttons on his gray pin-striped shirt, and the rings around his deep-blue eyes made him look like a sandy-haired raccoon. Esvelt, who is thirty-four, directs the "sculpting evolution" group at M.I.T., where he and his colleagues are attempting to design molecular tools capable of fundamentally altering the natural world. If the residents of Nantucket agree, Esvelt intends to use those tools to rewrite the DNA of white-footed mice to make them immune to the bacteria that cause Lyme and other tick-borne diseases. He and his team would breed the mice in the laboratory and then, as an initial experiment, release them on an uninhabited island. If the number of infected ticks begins to plummet, he would seek permission to repeat the process on Nantucket and on nearby Martha's Vineyard. More than a quarter of Nantucket's residents have been infected with Lyme, which has become one of the most rapidly spreading diseases in the United States. The illness is often accompanied by a red bull's-eye rash, along with fever and chills. When the disease is caught early enough, it can be cured in most cases with a single course of antibiotics. For many people, though, pain and neurological symptoms can persist for years. In communities throughout the Northeast, the fear of ticks has changed the nature of summer itself--few parents these days would permit a child to run barefoot through the grass or wander blithely into the woods. "What if we could wave our hands and make this problem go away?" Esvelt asked the two dozen officials and members of the public who had assembled at the island's police station for his presentation. He explained that white-footed mice are the principal reservoir of Lyme disease, which they pass, through ticks, to humans.

In this tutorial, you discovered how to establish a baseline performance on time series forecast problems with Python. The importance of establishing a baseline and the persistence algorithm that you can use. How to implement the persistence algorithm in Python from scratch. How to evaluate the forecasts of the persistence algorithm and use them as a baseline. The importance of establishing a baseline and the persistence algorithm that you can use. How to implement the persistence algorithm in Python from scratch. How to evaluate the forecasts of the persistence algorithm and use them as a baseline. Do you have any questions about baseline performance, or about this tutorial? Ask your questions in the comments below and I will do my best to answer.

One of the most important things that the U.S. can do to improve economic growth is to invest in artificial intelligence, or A.I., said the White House, in a new report. A.I.-driven, intelligent systems have the potential to displace millions, such as truck drivers, from their jobs. But potential negative impacts can be offset by investments in education as well as by ensuring there is a safety net to help affected people, the White House argued, in what will likely be the Obama administration's final report on technology policy. Some of the report's recommendations, which include expanded unemployment help and access to healthcare, may be anathema to a Republican-controlled Congress with a focus on tax reductions and spending cuts. But this report -- "Artificial Intelligence, Automation, and the Economy" (PDF) -- which was in the works well before election day, also describes broader, technological-driven changes that will impact jobs and may pose issues for President-elect Donald Trump.