Instructional Material
Why does Deep Learning work? - A perspective from Group Theory
Paul, Arnab, Venkatasubramanian, Suresh
Why does Deep Learning work? What representations does it capture? How do higher-order representations emerge? We study these questions from the perspective of group theory, thereby opening a new approach towards a theory of Deep learning. One factor behind the recent resurgence of the subject is a key algorithmic step called pre-training: first search for a good generative model for the input samples, and repeat the process one layer at a time. We show deeper implications of this simple principle, by establishing a connection with the interplay of orbits and stabilizers of group actions. Although the neural networks themselves may not form groups, we show the existence of {\em shadow} groups whose elements serve as close approximations. Over the shadow groups, the pre-training step, originally introduced as a mechanism to better initialize a network, becomes equivalent to a search for features with minimal orbits. Intuitively, these features are in a way the {\em simplest}. Which explains why a deep learning network learns simple features first. Next, we show how the same principle, when repeated in the deeper layers, can capture higher order representations, and why representation complexity increases as the layers get deeper.
ggRandomForests: Visually Exploring a Random Forest for Regression
Random Forests [Breiman:2001] (RF) are a fully non-parametric statistical method requiring no distributional assumptions on covariate relation to the response. RF are a robust, nonlinear technique that optimizes predictive accuracy by fitting an ensemble of trees to stabilize model estimates. The randomForestSRC package (http://cran.r-project.org/package=randomForestSRC) is a unified treatment of Breiman's random forests for survival, regression and classification problems. Predictive accuracy make RF an attractive alternative to parametric models, though complexity and interpretability of the forest hinder wider application of the method. We introduce the ggRandomForests package (http://cran.r-project.org/package=ggRandomForests), for visually understand random forest models grown in R with the randomForestSRC package. The vignette is a tutorial for using the ggRandomForests package with the randomForestSRC package for building and post-processing a regression random forest. In this tutorial, we explore a random forest model for the Boston Housing Data, available in the MASS package. We grow a random forest for regression and demonstrate how ggRandomForests can be used when determining variable associations, interactions and how the response depends on predictive variables within the model. The tutorial demonstrates the design and usage of many of ggRandomForests functions and features how to modify and customize the resulting ggplot2 graphic objects along the way. A development version of the ggRandomForests package is available on Github. We invite comments, feature requests and bug reports for this package at (https://github.com/ehrlinger/ggRandomForests).
Mechanisation of Thought Processes
Biology seems to be a science in its own right, or set of sciences having common aims, and so it should have its own language and explanatory concepts; yet when any specifically biological concept is suggested and used as an explanatory concept it seems to be unsatisfactory and even mystical. There are many biological concepts of this kind: Purpose, Drive, elan vital, Entelechy, Gestalten.* Physicists and engineers seem, on the other hand, to have clearly defined concepts having great power within biology.
COGNITIVE SCIENCE 2 361 383 1978
He knows about examples and heuristics and how they are related. He has a sense of what to use and when to use it, and what is worth remembering. He has an intuitive feeling for the subject, how it hangs together, and how it relates to other theories. He knows how not to be swamped by details, but also to reference them when he needs them. This paper is concerned with this important extra-logical knowledge that is often outside of traditional discussions in mathematics.
Lecture Notes it Artificial Intelligence
This paper presents a hybrid case-based reasoning (CBR) and information retrieval (IR) system, called SPIRE, that both retrieves documents from a full-text document corpus and from within individual documents, and locates passages likely to contain information about important problem-solving features of cases. SPIRE uses two case-bases, one containing past precedents, and one containing excerpts from past case texts. Both are used by SPIRE to automatically generate queries, which are then run by the INQUERY full-text retrieval engine on a large text collection in the case of document retrieval and on individual text documents for passage retrieval.
PROCEEDINGS AU ToMA T
Where the accountants have fallen down, however, is in their reluctance and sometimes inability to make intensive studies of different equipment and to specify their requirements for equipment. As one authority in the field of electronic data processing has pointed out, "Accountants, unlike engineers, take the equipment as given without bothering to specify their own particular needs." But after all things are taken into consideration, it is of primary importance that the personnel who are handling the details of the investigation have a good knowledge of the particular application to be studied. Executives in many companies have been dissatisfied with the help received from outsiders who are expert programmers and who know a lot about equipment, but who are unfamiliar with business systems. In some companies executives have found that their own personnel, who know the firm's particular data processing system, after three or four months of experience in which to grasp the logics of the computer and the intricacies of programming, are much more valuable than such outside experts.