Goto

Collaborating Authors

 Instructional Material


Why does Deep Learning work? - A perspective from Group Theory

arXiv.org Machine Learning

Why does Deep Learning work? What representations does it capture? How do higher-order representations emerge? We study these questions from the perspective of group theory, thereby opening a new approach towards a theory of Deep learning. One factor behind the recent resurgence of the subject is a key algorithmic step called pre-training: first search for a good generative model for the input samples, and repeat the process one layer at a time. We show deeper implications of this simple principle, by establishing a connection with the interplay of orbits and stabilizers of group actions. Although the neural networks themselves may not form groups, we show the existence of {\em shadow} groups whose elements serve as close approximations. Over the shadow groups, the pre-training step, originally introduced as a mechanism to better initialize a network, becomes equivalent to a search for features with minimal orbits. Intuitively, these features are in a way the {\em simplest}. Which explains why a deep learning network learns simple features first. Next, we show how the same principle, when repeated in the deeper layers, can capture higher order representations, and why representation complexity increases as the layers get deeper.


ggRandomForests: Visually Exploring a Random Forest for Regression

arXiv.org Machine Learning

Random Forests [Breiman:2001] (RF) are a fully non-parametric statistical method requiring no distributional assumptions on covariate relation to the response. RF are a robust, nonlinear technique that optimizes predictive accuracy by fitting an ensemble of trees to stabilize model estimates. The randomForestSRC package (http://cran.r-project.org/package=randomForestSRC) is a unified treatment of Breiman's random forests for survival, regression and classification problems. Predictive accuracy make RF an attractive alternative to parametric models, though complexity and interpretability of the forest hinder wider application of the method. We introduce the ggRandomForests package (http://cran.r-project.org/package=ggRandomForests), for visually understand random forest models grown in R with the randomForestSRC package. The vignette is a tutorial for using the ggRandomForests package with the randomForestSRC package for building and post-processing a regression random forest. In this tutorial, we explore a random forest model for the Boston Housing Data, available in the MASS package. We grow a random forest for regression and demonstrate how ggRandomForests can be used when determining variable associations, interactions and how the response depends on predictive variables within the model. The tutorial demonstrates the design and usage of many of ggRandomForests functions and features how to modify and customize the resulting ggplot2 graphic objects along the way. A development version of the ggRandomForests package is available on Github. We invite comments, feature requests and bug reports for this package at (https://github.com/ehrlinger/ggRandomForests).


Mechanisation of Thought Processes

AI Classics

Biology seems to be a science in its own right, or set of sciences having common aims, and so it should have its own language and explanatory concepts; yet when any specifically biological concept is suggested and used as an explanatory concept it seems to be unsatisfactory and even mystical. There are many biological concepts of this kind: Purpose, Drive, elan vital, Entelechy, Gestalten.* Physicists and engineers seem, on the other hand, to have clearly defined concepts having great power within biology.


Mechanisation of Thought Processes

AI Classics

If ability to perform complex calculations were a sufficient criterion, then even a conventional digital computor could lay claim to more intelligence than any of usand perhaps we had better let it make away with the word and be done with it.


Irtellige it S stems

AI Classics

In either view, typical examples are closer where (only) positive examples ought to exceptional cases, case memory, analogy, to the prototype, and atypical examples and be.


Arguments and Cases: An Inevitable Intertwining '

AI Classics

We discuss several aspects of legal arguments, primarily arguments about the meaning of statutes. First, we discuss how the requirements of argument guide the specification and selection of supporting cases and how an existing case base influences argument formation.


COGNITIVE SCIENCE 2 361 383 1978

AI Classics

He knows about examples and heuristics and how they are related. He has a sense of what to use and when to use it, and what is worth remembering. He has an intuitive feeling for the subject, how it hangs together, and how it relates to other theories. He knows how not to be swamped by details, but also to reference them when he needs them. This paper is concerned with this important extra-logical knowledge that is often outside of traditional discussions in mathematics.


Lecture Notes it Artificial Intelligence

AI Classics

This paper presents a hybrid case-based reasoning (CBR) and information retrieval (IR) system, called SPIRE, that both retrieves documents from a full-text document corpus and from within individual documents, and locates passages likely to contain information about important problem-solving features of cases. SPIRE uses two case-bases, one containing past precedents, and one containing excerpts from past case texts. Both are used by SPIRE to automatically generate queries, which are then run by the INQUERY full-text retrieval engine on a large text collection in the case of document retrieval and on individual text documents for passage retrieval.


PROCEEDINGS AU ToMA T

AI Classics

Where the accountants have fallen down, however, is in their reluctance and sometimes inability to make intensive studies of different equipment and to specify their requirements for equipment. As one authority in the field of electronic data processing has pointed out, "Accountants, unlike engineers, take the equipment as given without bothering to specify their own particular needs." But after all things are taken into consideration, it is of primary importance that the personnel who are handling the details of the investigation have a good knowledge of the particular application to be studied. Executives in many companies have been dissatisfied with the help received from outsiders who are expert programmers and who know a lot about equipment, but who are unfamiliar with business systems. In some companies executives have found that their own personnel, who know the firm's particular data processing system, after three or four months of experience in which to grasp the logics of the computer and the intricacies of programming, are much more valuable than such outside experts.