Reinforcement learning addresses the problem of learning to select actions in order to maximize one's performance in unknown environments. To scale reinforcement learning to complex real-world tasks, such as typically studied in AI, one must ultimately be able to discover the structure in the world, in order to abstract away the myriad of details and to operate in more tractable problem spaces. This paper presents the SKILLS algorithm. SKILLS discovers skills, which are partially defined action policies that arise in the context of multiple, related tasks.

An application of reinforcement learning to a linear-quadratic, differential game is presented. The reinforcement learning system uses a recently developed algorithm, the residual gradient form of advantage updating. The game is a Markov Decision Process (MDP) with continuous time, states, and actions, linear dynamics, and a quadratic cost function. The game consists of two players, a missile and a plane; the missile pursues the plane and the plane evades the missile. The reinforcement learning algorithm for optimal control is modified for differential games in order to find the minimax point, rather than the maximum. Simulation results are compared to the optimal solution, demonstrating that the simulated reinforcement learning system converges to the optimal answer. The performance of both the residual gradient and non-residual gradient forms of advantage updating and Q-learning are compared. The results show that advantage updating converges faster than Q-learning in all simulations.

The macaque lateral geniculate nucleus (LGN) exhibits an intricate lamination pattern, which changes midway through the nucleus at a point coincident with small gaps due to the blind spot in the retina. We present a three-dimensional model of morphogenesis in which local cell interactions cause a wave of development of neuronal receptive fieldsto propagate through the nucleus and establish two distinct lamination patterns. We examine the interactions between the wave and the localized singularities due to the gaps, and find that the gaps induce the change in lamination pattern. We explore critical factors which determine general LGN organization.

It is well known that a combination of many different predictors can improve predictions. Inthe neural networks community "ensembles" of neural networks has been investigated by several authors, see for instance [1, 2, 3]. Most often the networks in the ensemble are trained individually and then their predictions are combined. This combination is usually done by majority (in classification) or by simple averaging (inregression), but one can also use a weighted combination of the networks.

We investigate the computational power of a formal model for networks ofspiking neurons, both for the assumption of an unlimited timing precision, and for the case of a limited timing precision. We also prove upper and lower bounds for the number of examples that are needed to train such networks.

Understanding knowledge representations in neural nets has been a difficult problem. Principal components analysis (PCA) of contributions (products of sending activations and connection weights) has yielded valuable insights into knowledge representations, but much of this work has focused on the correlation matrix of contributions. The present work shows that analyzing the variance-covariance matrix of contributions yields more valid insights by taking account of weights.

Deformable models are an attractive approach to recognizing nonrigid objects which have considerable within class variability. However, there are severe search problems associated with fitting the to data. We show that by using neural networks to providemodels better starting points, the search time can be significantly reduced. The method is demonstrated on a character recognition task.

A fundamental feature of cortical architecture is its columnar organization, manifested in the tendency of neurons with similar properties to be organized in columns that run perpendicular to the cortical surface. This organization of the cortex was initially discovered by physiological experiments (Mouncastle, 1957; Hubel and Wiesel, 1962), and subsequently confirmed with the demonstration of histologically defined that axonal projections throughout thecolumns. Tracing experiments have shown tend to be organized in vertically aligned clusters or patches.

"How We Know Universals: The Perception Their first paper made many intellectual waves--which are still spreading, 50 years later. They had claimed that the truth or falsity of any (computable) proposition could, in with AI, for it's difficult to say just principle, be computed by a simple type of The future of psychology, they good a date as any, however, is 1943--almost said, consisted of the design of various sorts exactly half a century ago. This In that year, Warren McCulloch (a psychiatrist, novel methodology, and the nascent technology cybernetician, philosopher, and poet) associated with it, promised to show just and Walter Pitts (a research student in mathematics) how mind is grounded in mechanism. Much of this was "logical" in nature result was a heady brew, which explicitly and developed into what's known as classical, promised to revolutionize psychology and or symbolic, AI. But some was what is nowadays philosophy--and which, in the event, revolutionized called connectionist, studying networks technology too. In the late 1980s, however, it McCulloch and Pitts' paper ("A Logical Calculus blossomed--hitting the newsstands with of the Ideas Immanent in Nervous rash promises of "brainlike" computers just Activity") concentrated on how propositions around the corner. But both these forms of AI expressible in logic could be computed by share the same historical roots. Those nets consisted of So much for pedigree. But does a mere halfcentury cells passing inhibitory and excitatory messages of work count as a pedigree? Might it between them and acting as what computer rather be a mere blip, an unfortunate academic scientists (soon afterwards) called "and-mutation with no real intellectual fitness?

Daewoo Shipbuilding Company, one of the largest shipbuilders in the world, has experienced great deal of trouble with the planning and scheduling of its production process. To solve the problems, from 1991 to 1993, Korea Advanced Institute of Science and Technology (KAIST) and Daewoo jointly conducted the Daewoo Shipbuilding Scheduling (das) Project. To integrate the scheduling expert systems for shipbuilding, we used a hierarchical scheduling architecture. To automate the dynamic spatial layout of objects in various areas of the shipyard, we developed spatial scheduling expert systems. For reliable estimation of person-hour requirements, we implemented the neural network-based person-hour estimator. In addition, we developed the paneled-block assembly shop scheduler and the long-range production planner. For this large-scale project, we devised a phased development strategy consisting of three phases: (1) vision revelation, (2) data-dependent realization, and (3) prospective enhancement. The DAS systems were successfully launched in January 1994 and are actively being used as indispensable systems in the shipyard, resulting in significant improvement in productivity and visible and positive effects in many areas.