Tishby, Naftali
The Power of Amnesia
Ron, Dana, Singer, Yoram, Tishby, Naftali
We propose a learning algorithm for a variable memory length Markov process. Human communication, whether given as text, handwriting, or speech, has multi characteristic time scales. On short scales it is characterized mostly by the dynamics that generate the process, whereas on large scales, more syntactic and semantic information is carried. For that reason the conventionally used fixed memory Markov models cannot capture effectively the complexity of such structures. On the other hand using long memory models uniformly is not practical even for as short memory as four.
Decoding Cursive Scripts
Singer, Yoram, Tishby, Naftali
Online cursive handwriting recognition is currently one of the most intriguing challenges in pattern recognition. This study presents a novel approach to this problem which is composed of two complementary phases. The first is dynamic encoding of the writing trajectory into a compact sequence of discrete motor control symbols. In this compact representation we largely remove the redundancy of the script, while preserving most of its intelligible components. In the second phase these control sequences are used to train adaptive probabilistic acyclic automata (PAA) for the important ingredients of the writing trajectories, e.g.
The Power of Amnesia
Ron, Dana, Singer, Yoram, Tishby, Naftali
We propose a learning algorithm for a variable memory length Markov process. Human communication, whether given as text, handwriting, or speech, has multi characteristic time scales. On short scales it is characterized mostly by the dynamics that generate theprocess, whereas on large scales, more syntactic and semantic informationis carried. For that reason the conventionally used fixed memory Markov models cannot capture effectively the complexity of such structures. On the other hand using long memory modelsuniformly is not practical even for as short memory as four.
Information, Prediction, and Query by Committee
Freund, Yoav, Seung, H. Sebastian, Shamir, Eli, Tishby, Naftali
We analyze the "query by committee" algorithm, a method for filtering informative queries from a random stream of inputs. We show that if the two-member committee algorithm achieves information gain with positive lower bound, then the prediction error decreases exponentially with the number of queries. We show that, in particular, this exponential decrease holds for query learning of thresholded smooth functions.
Statistical Modeling of Cell Assemblies Activities in Associative Cortex of Behaving Monkeys
Gat, Itay, Tishby, Naftali
So far there has been no general method for relating extracellular electrophysiological measured activity of neurons in the associative cortex to underlying network or "cognitive" states. We propose to model such data using a multivariate Poisson Hidden Markov Model. We demonstrate the application of this approach for temporal segmentation of the firing patterns, and for characterization of the cortical responses to external stimuli. Using such a statistical model we can significantly discriminate two behavioral modes of the monkey, and characterize them by the different firing patterns, as well as by the level of coherency of their multi-unit firing activity. Our study utilized measurements carried out on behaving Rhesus monkeys by M. Abeles, E. Vaadia, and H. Bergman, of the Hadassa Medical School of the Hebrew University. 1 Introduction Hebb hypothesized in 1949 that the basic information processing unit in the cortex is a cell-assembly which may include thousands of cells in a highly interconnected network[l].
Information, Prediction, and Query by Committee
Freund, Yoav, Seung, H. Sebastian, Shamir, Eli, Tishby, Naftali
We analyze the "query by committee" algorithm, a method for filtering informativequeries from a random stream of inputs. We show that if the two-member committee algorithm achieves information gainwith positive lower bound, then the prediction error decreases exponentially with the number of queries. We show that, in particular, this exponential decrease holds for query learning of thresholded smooth functions.