The Computational Linguistics of Biological Sequences

Classics (Collection 2)

Shortly after Watson and Crick's discovery of the structure of DNA, and at about the same time that the genetic code and the essential facts of gene expression were being elucidated, the field of linguistics was being similarly revolutionized by the work of Noam Chomsky [Chomsky, 1955, 1957, 1959, 1963, 1965]. Observing that a seemingly infinite variety of language was available to individual human beings based on clearly finite resources and experience, he proposed a formal representation of the rules or syntax of language, called generative grammar, that could provide finite--indeed, concise--characterizations of such infinite languages. Just as the breakthroughs in molecular biology in that era served to anchor genetic concepts in physical structures and opened up entirely novel experimental paradigms, so did Chomsky's insight serve to energize the field of linguistics, with putative correlates of cognitive processes that could for the first time be reasoned about 48 ARTIFICIAL INTELLIGENCE & MOLECULAR BIOLOGY While Chomsky and his followers built extensively upon this foundation in the field of linguistics, generative grammars were also soon integrated into the framework of the theory of computation, and in addition now form the basis for efforts of computational linguists to automate the processing and understanding of human language. Since it is quite commonly asserted that DNA is a richly-expressive language for specifying the structures and processes of life, also with the potential for a seemingly infinite variety, it is surprising that relatively little has been done to apply to biological sequences the extensive results and methods developed over the intervening decades in the field of formal language theory. While such an approach has been proposed [Brendel and Busse, 1984], most investigations along these lines have used grammar formalisms as tools for what are essentially information-theoretic studies [Ebeling and Jimenez-Montano, 1980; Jimenez-Montano, 1984], or have involved statistical analyses at the level of vocabularies (reflecting a more traditional notion of comparative linguistics) [Brendel et al., 1986; Pevzner et al., 1989a,b; Pietrokovski et al., 1990].

Learning and executing generalized robot plans


"In this paper we describe some major new additions to the STRIPS robot problem-solving system. The first addition is a process for generalizing a plan produced by STRIPS so that problem-specific constants appearing in the plan are replaced by problem-independent parameters.The generalized plan, stored in a convenient format called a triangle table, has two important functions. The more obvious function is as a single macro action that can be used by STRIPS—either in whole or in part—during the solution of a subsequent problem. Perhaps less obviously, the generalized plan also plays a central part in the process that monitors the real-world execution of a plan, and allows the robot to react "intelligently" to unexpected consequences of actions.We conclude with a discussion of experiments with the system on several example problems."Artificial Intelligence 3:251-288

Strategy building with the graph traverser


I shall discuss automatic methods of search for solutions in problems susceptible of a particular formal representation, namely that on which the Graph Traverser program (Doran & Michie 1966, and see Doran p. 105) has been based. One approach, based on state-evaluation, generates all the states of the problem which can be reached in a small number of moves from the current state, and then seeks by some process of evaluation to decide which state shall form the next point of departure. In the classical studies of Newell, Shaw & Simon (1960) selection is applied by going down a priority sequence of operators, applying to each in turn a number of tests, first of applicability to the current state and then of whether the operator conduces towards one or another of various desirable intermediate states, or subgoals.

A formal theory of inductive inference


In Part II these models are applied to the solution of three problems--prediction of the Bernoulli sequence, extrapolation of a certain kind of Markov chain, and the use of phrase structure grammars for induction. The solution to the second problem uses less certain approximations, but the properties of the solution that are discussed, are fairly independent of these approximations. The third application, using phrase structure grammars, is least exact of the three. This formal solution is then applied in an approximate way to the determination of the "optimum" phrase structure grammar for a given set of strings.

Semantic Message Detection for Machine Translation, Using an Interlingua


What is needed is a discipline which will study semantic message-connection in a way analogous to that in which metamathematics studies mathematical connection, and to that in which mathematical linguistics now studies syntactic connection. Research Used as Data for the Construction of T (a) Conceptual Dictionary for English The uses of the main words and phrases of English are mapped on to a classificatory system of about 750 descriptors, or heads, these heads being streamlined from Roget's Thesaurus. For Instance, a single card covers Disappoint, Disappointed, Disappointing, Disappointment. The two connectives, / ("slash") and: ("colon") and a word-order rule are used as in T to replace R.H. Richens' three subscripts, and every two pairs of elements are bracketted together, two bracketted pairs of elements counting as a single pair for the purpose of forming 2nd order brackets.

Steps Toward Artificial Intelligence


... The literature does not include any general discussion of the outstanding problems of this field. In this article, an attempt will be made to separate out, analyze, and find the relations between some of these problems. Analysis will be supported with enough examples from the literature to serve the introductory function of a review article, but there remains much relevant work not described here.Proc. Institute of Radio Engineers 49, p. 8-30

Pattern recognition by machine


Even the earliest computers could do arithmetic superbly, but only very recently have they begun to read the written digits that a child recognizes before he learns to add them. Understanding speech and reading print are examples of a basic intellectual skill that can variously be called cognition, abstraction or perception; perhaps the best general term for it is pattern reecognition. Except for their inability to recognize patterns, machines (or, more accurately, the programs that tell machines what to do) have now met most of the classic criteria of intelligence that skeptics have proposed.Scientific American [August, 1960] 203: 60-68. Computers and Thought, Section 6 (1963).

Hierarchies in Pattern Recognition


Particular attention is given to processes involving pattern recognition, learning, planning ahead, and the use of analogies or?models!. Second, we can often find simple machines which in certain situations do exhibit performances which would be called intelligent If done by a man. In attempting to design intelligent machines we are, in effect, concerned with the problems of "creativity". Usually the problem is not so much to find the basic structure (or the domain of things to try) as to find ways of reducing this structure to reasonable size.

Empirical Explorations with the Logic Theory Machine: A Case Study in Heuristics


This is a case study in problem-solving, representing part of a program of research on complex information-processing systems. We have specifieda system for finding proofs of theorems in elementary symbolic logic, and by programming a computer to these specifications, have obtained empirical data on the problem-solving process in elementary logic. The program is called the Logic Theory Machine (LT); it was devised to learn how it is possible to solve difficult problems such as proving mathematical theorems, discovering scientific laws from data, playing chess, or understanding the meaning of English prose.The research reported here is aimed at understanding the complexp rocesses (heuristics) that are effective in problem-solving. Hence, we are not interested in methods that guarantee solutions, but which require vastamounts of computation. Rather, we wish to understand how a mathematician, for example, is able to prove a theorem even though he does not know when he starts how, or if, he is going to succeed.Proceedings of the Western Joint Computer Conference, 15:218-239. Reprinted in Feigenbaum and Feldman, Computers and Thought (1963).

Intelligent machinery


I propose to investigate the question as to whether it is possible for machinery to show intelligent behaviour. On the other hand the human intelligence seems to be able to find methods of ever-increasing power for dealing with such problems'transcending' the methods available to machines. It is already possible to produce machines where this sort of situation arises in a small degree. The states of'continuous' machinery on the other hand form a continuous manifold, and the behaviour of the machine is described by a curve on this manifold.