Although there has been a rapid development of practical applications, theoretical explanations of deep learning are in their infancy. A possible starting point suggests that deep learning performs a sophisticated coarse graining. Coarse graining is the foundation of the renormalization group (RG), which provides a systematic construction of the theory of large scales starting from an underlying microscopic theory. In this way RG can be interpreted as providing a mechanism to explain the emergence of large scale structure, which is directly relevant to deep learning. We pursue the possibility that RG may provide a useful framework within which to pursue a theoretical explanation of deep learning. A statistical mechanics model for a magnet, the Ising model, is used to train an unsupervised RBM. The patterns generated by the trained RBM are compared to the configurations generated through a RG treatment of the Ising model. We argue that correlation functions between hidden and visible neurons are capable of diagnosing RG-like coarse graining. Numerical experiments show the presence of RG-like patterns in correlators computed using the trained RBMs. The observables we consider are also able to exhibit important differences between RG and deep learning.

One of my favorite things about deep reinforcement learning is that, unlike supervised learning, it really, really doesn't want to work. Throwing a neural net at a computer vision problem might get you 80% of the way there. Throwing a neural net at an RL problem will probably blow something up in front of your face -- and it will blow up in a different way each time you try. A lot of the biggest challenges in RL revolve around two questions: how we interact with the environment effectively (e.g. In this post, I want to explore a few recent directions in deep RL research that attempt to address these challenges, and do so with particularly elegant parallels to human cognition. This post will begin with a quick review of two canonical deep RL algorithms -- DQN and A3C -- to provide us some intuitions to refer back to, and then jump into a deep dive on a few recent papers and breakthroughs in the categories described above.

Great Learning, India's leading Ed-tech platform for working professionals today announced that it is expanding its geographic footprint globally to Europe, Asia Pacific, Africa and the Middle East. The company will offer three of its most popular programs in Data Science & Business Analytics (PGP-DSBA - a special international variant of its business analytics program PGP-BABI ranked #1 in India for the last 4 years), Artificial Intelligence & Machine Learning (PGP-AIML) and Cyber Security (SACSP - Stanford Advanced Computer Security Program) in these geographies. Offered in association with two of the top universities of the world, Stanford University and The University of Texas, Austin, these online programs have already attracted learners from 17 countries including the UK, Singapore, South Africa, UAE, etc. These programs, designed and developed by the top-notch faculty of UT Austin and Stanford, are delivered online by Great Learning, utilizing its unique mentored-learning model where personalized mentorship is provided by expert instructors from Great Learning's 750 Global Guru network. The mentors include industry veterans from companies like Google, Microsoft, Amazon and Walmart.

There are many surprising and perhaps counter-intuitive properties of optimization of deep neural networks. We propose and experimentally verify a unified phenomenological model of the loss landscape that incorporates many of them. High dimensionality plays a key role in our model. Our core idea is to model the loss landscape as a set of high dimensional \emph{wedges} that together form a large-scale, inter-connected structure and towards which optimization is drawn. We first show that hyperparameter choices such as learning rate, network width and $L_2$ regularization, affect the path optimizer takes through the landscape in a similar ways, influencing the large scale curvature of the regions the optimizer explores. Finally, we predict and demonstrate new counter-intuitive properties of the loss-landscape. We show an existence of low loss subspaces connecting a set (not only a pair) of solutions, and verify it experimentally. Finally, we analyze recently popular ensembling techniques for deep networks in the light of our model.

In this paper, the echo state network (ESN) memory capacity, which represents the amount of input data an ESN can store, is analyzed for a new type of deep ESNs. In particular, two deep ESN architectures are studied. First, a parallel deep ESN is proposed in which multiple reservoirs are connected in parallel allowing them to average outputs of multiple ESNs, thus decreasing the prediction error. Then, a series architecture ESN is proposed in which ESN reservoirs are placed in cascade that the output of each ESN is the input of the next ESN in the series. This series ESN architecture can capture more features between the input sequence and the output sequence thus improving the overall prediction accuracy. Fundamental analysis shows that the memory capacity of parallel ESNs is equivalent to that of a traditional shallow ESN, while the memory capacity of series ESNs is smaller than that of a traditional shallow ESN.In terms of normalized root mean square error, simulation results show that the parallel deep ESN achieves 38.5% reduction compared to the traditional shallow ESN while the series deep ESN achieves 16.8% reduction.

Climate change is one of the greatest challenges facing humanity, and we, as machine learning experts, may wonder how we can help. Here we describe how machine learning can be a powerful tool in reducing greenhouse gas emissions and helping society adapt to a changing climate. From smart grids to disaster management, we identify high impact problems where existing gaps can be filled by machine learning, in collaboration with other fields. Our recommendations encompass exciting research questions as well as promising business opportunities. We call on the machine learning community to join the global effort against climate change.

Depression and anxiety are critical public health issues affecting millions of people around the world. To identify individuals who are vulnerable to depression and anxiety, predictive models have been built that typically utilize data from one source. Unlike these traditional models, in this study, we leverage a rich heterogeneous data set from the University of Notre Dame's NetHealth study that collected individuals' (student participants') social interaction data via smartphones, health-related behavioral data via wearables (Fitbit), and trait data from surveys. To integrate the different types of information, we model the NetHealth data as a heterogeneous information network (HIN). Then, we redefine the problem of predicting individuals' mental health conditions (depression or anxiety) in a novel manner, as applying to our HIN a popular paradigm of a recommender system (RS), which is typically used to predict the preference that a person would give to an item (e.g., a movie or book). In our case, the items are the individuals' different mental health states. We evaluate three state-of-the-art RS approaches. Also, we model the prediction of individuals' mental health as another problem type -- that of node classification (NC) in our HIN, evaluating in the process four node features under logistic regression as a proof-of-concept classifier. We find that our RS and NC network methods produce more accurate predictions than a logistic regression model using the same NetHealth data in the traditional non-network fashion as well as a random-approach. Also, we find that RS outperforms NC. This is the first study to integrate smartphone, wearable sensor, and survey data in an HIN manner and use RS or NC on the HIN to predict individuals' mental health conditions.

This work is done as part of a master's thesis project. The increase in the volume of data has given rise to various issues related to the collection, storage, analysis and exploitation of these data in order to create an added value. In this master, we are interested in the search of frequent closed patterns in the transaction bases. One way to process data is to partition the search space into subcontexts, and then explore the subcontexts simultaneously. In this context, we have proposed a new approach for extracting frequent closed itemsets. The main idea is to update frequent closed patterns with their minimal generators by applying a strategy of partitioning of the initial extraction context. Our new approach called UFCIGs-DAC was designed and implemented to perform a search in the test bases. The main originality of this approach is the simultaneous exploration of the research space by the update of the frequent closed patterns and the minimal generators. Moreover, our approach can be adapted to any algorithm of extraction of the frequent closed patterns with their minimal generators.

Image-processing algorithms to smartly identify, caption and moderate your pictures. Convert spoken audio into text, use voice for verification, or add speaker recognition to your app. Allow your apps to process natural language with pre-built scripts, evaluate sentiment and learn how to recognize what users want. Map complex information and data in order to solve tasks such as intelligent recommendations and semantic search. Add Google or Bing Search APIs to your apps and harness the ability to comb billions of webpages, images, videos, news and much more.

The spread of tuberculosis (TB) has diminished in the developed world, but it is still prevalent in the developing parts of the world such as in Asia and Africa. The rise of HIV in the 1980's also saw an increase in TB infections due to the weakened immune systems of patients with HIV. Currently about 1.6 million people die from TB each year, and 10 million people develop active TB infections, which is also contagious. Tuberculosis is caused by Mycobacterium tuberculosis bacteria and it generally affects the lungs. Individuals can harbor the TB bacteria but show no symptoms.