Goto

Collaborating Authors

 Industry


Bach in a Box - Real-Time Harmony

Neural Information Processing Systems

The learning and inferencing algorithms presented here speak an extended form of the classical figured bass representation common in Bach's time. Paired with a melody, figured bass provides a sufficient amount of information to reconstruct the harmonic content of a piece of music. Figured bass has several characteristics which make it well-disposed to learning rules. It is a symbolic format which uses a relatively small alphabet of symbols. It is also hierarchical - it specifies first the chord function that is to be played at the current note/timestep, then the scale step to be played by the bass voice, then additional information as needed to specify the alto and tenor scale steps. This allows our algorithm to fire sets of rules sequentially, to first determine the chord function which should be associated with a new melody note, and then to use that chord function as an input attribute to subsequent rulebases which determine the bass, alto, and tenor scale steps. In this way we can build up the final chord from simpler pieces, each governed by a specialized rulebase.


Learning Path Distributions Using Nonequilibrium Diffusion Networks

Neural Information Processing Systems

Department of Mathematics University of California, San Diego La Jolla, CA 92093-0112 Abstract We propose diffusion networks, a type of recurrent neural network with probabilistic dynamics, as models for learning natural signals that are continuous in time and space. We give a formula for the gradient of the log-likelihood of a path with respect to the drift parameters for a diffusion network. This gradient can be used to optimize diffusion networks in the nonequilibrium regime for a wide variety of problems paralleling techniques which have succeeded in engineering fields such as system identification, state estimation and signal filtering. An aspect of this work which is of particular interestto computational neuroscience and hardware design is that with a suitable choice of activation function, e.g., quasi-linear sigmoidal, the gradient formula is local in space and time. 1 Introduction Many natural signals, like pixel gray-levels, line orientations, object position, velocity andshape parameters, are well described as continuous-time continuous-valued stochastic processes; however, the neural network literature has seldom explored the continuous stochastic case. Since the solutions to many decision theoretic problems of interest are naturally formulated using probability distributions, it is desirable to have a flexible framework for approximating probability distributions on continuous pathspaces.


Analog VLSI Model of Intersegmental Coordination with Nearest-Neighbor Coupling

Neural Information Processing Systems

We have a developed an analog VLSI system that models the coordination ofneurobiological segmental oscillators. We have implemented and tested a system that consists of a chain of eleven pattern generating circuits thatare synaptically coupled to their nearest neighbors. Each pattern generating circuit is implemented with two silicon Morris-Lecar neurons that are connected in a reciprocally inhibitory network. We discuss themechanisms of oscillations in the two-cell network and explore system behavior based on isotropic and anisotropic coupling, and frequency gradientsalong the chain of oscillators.


A Revolution: Belief Propagation in Graphs with Cycles

Neural Information Processing Systems

Department of Physics, Cavendish Laboratory Cambridge University Abstract Until recently, artificial intelligence researchers have frowned upon the application of probability propagation in Bayesian belief networks thathave cycles. The probability propagation algorithm is only exact in networks that are cycle-free. However, it has recently been discovered that the two best error-correcting decoding algorithms areactually performing probability propagation in belief networks with cycles. 1 Communicating over a noisy channel Our increasingly wired world demands efficient methods for communicating bits of information over physical channels that introduce errors. Examples of real-world channels include twisted-pair telephone wires, shielded cable-TV wire, fiberoptic cable, deep-space radio, terrestrial radio, and indoor radio. Engineers attempt to correct the errors introduced by the noise in these channels through the use of channel coding which adds protection to the information source, so that some channel errors can be corrected.


The Observer-Observation Dilemma in Neuro-Forecasting

Neural Information Processing Systems

We explain how the training data can be separated into clean information andunexplainable noise. Analogous to the data, the neural network is separated into a time invariant structure used for forecasting, and a noisy part. We propose a unified theory connecting the optimization algorithms forcleaning and learning together with algorithms that control the data noise and the parameter noise. The combined algorithm allows a data-driven local control of the liability of the network parameters and therefore an improvement in generalization. The approach is proven to be very useful at the task of forecasting the German bond market.



Monotonic Networks

Neural Information Processing Systems

Monotonicity is a constraint which arises in many application domains. Wepresent a machine learning model, the monotonic network, for which monotonicity can be enforced exactly, i.e., by virtue offunctional form. A straightforward method for implementing and training a monotonic network is described. Monotonic networks are proven to be universal approximators of continuous, differentiable monotonicfunctions. We apply monotonic networks to a real-world task in corporate bond rating prediction and compare them to other approaches. 1 Introduction Several recent papers in machine learning have emphasized the importance of priors anddomain-specific knowledge. In their well-known presentation of the biasvariance tradeoff(Geman and Bienenstock, 1992)' Geman and Bienenstock conclude by arguing that the crucial issue in learning is the determination of the "right biases" whichconstrain the model in the appropriate way given the task at hand.


MELONET I: Neural Nets for Inventing Baroque-Style Chorale Variations

Neural Information Processing Systems

Given a melody, the system invents afour-part chorale harmonization and a variation of any chorale voice, after being trained on music pieces of composers like J. S. Bach and J .


An Analog VLSI Neural Network for Phase-based Machine Vision

Neural Information Processing Systems

Gabor filters are used as preprocessing stages for different tasks in machine vision and image processing. Their use has been partially motivated by findings that two dimensional Gabor filters can be used to model receptive fields of orientation selective neurons in the visual cortex (Daugman, 1980) and three dimensional spatiotemporal Gabor filters can be used to model biological image motion analysis (Adelson, 1985). A Gabor filter has a complex valued impulse response which is a complex exponential modulated by a Gaussian function.


Use of a Multi-Layer Perceptron to Predict Malignancy in Ovarian Tumors

Neural Information Processing Systems

Here we sudy the continuous time, continuous state-spacestochastic case, which covers a wide variety of control problems including target, viability, optimization problems (see [FS93], [KP95])}or which a formalism is the following.