The paper investigates a phenomenon of free word order through the analysis by reduction. It exploits its formal background and data types and studies the word order freedom by means of the minimal number of word order shifts (word order changes preserving syntactic correctness, individual word forms, their morphological characteristics and/or their surface dependency relations). The investigation focuses upon an interplay of two phenomena related to word order: (non-)projectivity of a sentence and number of word order shifts within the analysis by reduction. This interplay is exemplified on a sample of Czech sentences with clitics.

In NL2OCL project, we aim to translate English specification of constraints to formal constraints such as OCL (Object Constraint Language). In English to OCL translation, our contribution is a semantic analyzer that uses the output of the Stanford parser for shallow and deep semantic parsing. Our analysis of the output of shallow semantic parsing showed that semantic roles were mis-identified for a few English constraints due to semantic ambiguity. Similarly, in deep semantic parsing, it is difficult to resolve scope of quantifier operators due to scope ambiguity that is another sub-type of semantic ambiguity. In this paper, we highlight the identified cases of semantic ambiguities in English constraints. We also present a novel approach to automatically resolve the identified cases of the semantic ambiguities. The presented approach is also evaluated to show that by addressing the identified cases of semantic ambiguities, we can generate more accurate and complete formal (OCL) specifications.

Domain knowledge is crucial for effective performance in autonomous control systems. Typically, human effort is required to encode this knowledge into a control algorithm. In this paper, we present an approach to language grounding which automatically interprets text in the context of a complex control application, such as a game, and uses domain knowledge extracted from the text to improve control performance. Both text analysis and control strategies are learned jointly using only a feedback signal inherent to the application. To effectively leverage textual information, our method automatically extracts the text segment most relevant to the current game state, and labels it with a task-centric predicate structure. This labeled text is then used to bias an action selection policy for the game, guiding it towards promising regions of the action space. We encode our model for text analysis and game playing in a multi-layer neural network, representing linguistic decisions via latent variables in the hidden layers, and game action quality via the output layer. Operating within the Monte-Carlo Search framework, we estimate model parameters using feedback from simulated games. We apply our approach to the complex strategy game Civilization II using the official game manual as the text guide. Our results show that a linguistically-informed game-playing agent significantly outperforms its language-unaware counterpart, yielding a 34% absolute improvement and winning over 65% of games when playing against the built-in AI of Civilization.

We present a system that produces sentential descriptions of video: who did what to whom, and where and how they did it. Action class is rendered as a verb, participant objects as noun phrases, properties of those objects as adjectival modifiers in those noun phrases,spatial relations between those participants as prepositional phrases, and characteristics of the event as prepositional-phrase adjuncts and adverbial modifiers. Extracting the information needed to render these linguistic entities requires an approach to event recognition that recovers object tracks, the track-to-role assignments, and changing body posture.

Many human social exchanges and coordinated activities critically involve dialogue interactions. Hence, we need to develop natural humanlike dialogue processing mechanisms for future robots if they are to interact with humans in natural ways. In this article we discuss the challenges of designing such flexible dialogue-based robotic systems. We report results from data we collected in human interaction experiments in the context of a search task and show how we can use these results to build more flexible robotic architectures that are starting to address the challenges of task-based humanlike natural language dialogues on robots.

This paper proposes a parsing algorithm for scene understanding which includes four aspects: computing 3D scene layout, detecting 3D objects (e.g. furniture), detecting 2D faces (windows, doors etc.), and segmenting background. In contrast to previous scene labeling work that applied discriminative classifiers to pixels (or super-pixels), we use a generative Stochastic Scene Grammar (SSG). This grammar represents the compositional structures of visual entities from scene categories, 3D foreground/background, 2D faces, to 1D lines. The grammar includes three types of production rules and two types of contextual relations. Production rules: (i) AND rules represent the decomposition of an entity into sub-parts; (ii) OR rules represent the switching among sub-types of an entity; (iii) SET rules rep- resent an ensemble of visual entities. Contextual relations: (i) Cooperative “+” relations represent positive links between binding entities, such as hinged faces of a object or aligned boxes; (ii) Competitive “-” relations represents negative links between competing entities, such as mutually exclusive boxes. We design an efficient MCMC inference algorithm, namely Hierarchical cluster sampling, to search in the large solution space of scene configurations. The algorithm has two stages: (i) Clustering: It forms all possible higher-level structures (clusters) from lower-level entities by production rules and contextual relations. (ii) Sampling: It jumps between alternative structures (clusters) in each layer of the hierarchy to find the most probable configuration (represented by a parse tree). In our experiment, we demonstrate the superiority of our algorithm over existing methods on public dataset. In addition, our approach achieves richer structures in the parse tree.

According to Baddeley, "The episodic buffer is assumed to be a limitedcapacity Our research team has been working on the development of a language analysis model (Ball, 2011; Ball, Heiberg & temporary storage system that is capable of Silber, 2007) within the ACT-R cognitive architecture integrating information from a variety of sources…the (Anderson, 2007) since 2002 (Ball, 2004). The focus is on buffer provides not only a mechanism for modeling the development of a general-purpose, large-scale, functional environment, but also for creating new cognitive model (Ball, 2008; Ball et al., 2010) that adheres to well representations" (ibid, p. 421). A key empirical result which established cognitive constraints on human language motivated Baddeley to introduce the episodic buffer after 25 processing (HLP) as realized by ACT-R.

I ask why humans are smarter than other primates, and I hypothesize that an important part of the answer lies in what I call the Strong Story Hypothesis, which holds that story telling and understanding have a central role in human intelligence. Next, I introduce another hypothesis, the Driven Perception Hypothesis, which holds that we derive much of our commonsense, including the commonsense required in story understanding, by deploying our perceptual apparatus on real and imagined events. Then, after discussing methodology, I describe the representations and methods embodied in the Genesis system, a story-understanding system that analyzes stories ranging from precis of Shakespeare's plots to descriptions of conflicts in cyberspace. The Genesis system works with short story summaries, provided in English, together with low-level commonsense rules and higher-level reflection patterns, likewise expressed in English. Using only a small collection of commonsense rules and reflection patterns, Genesis demonstrates several story-understanding capabilities, such as determining that both Macbeth and the 2007 Russia-Estonia Cyberwar involve revenge, even though neither the word revenge nor any of its synonyms are mentioned. Finally, I describe Rao's Visio-Spatial Reasoning System, a system that recognizes activities such as approaching, jumping, and giving, and answers commonsense questions posed by Genesis.

Language processing architectures today are rarely designed to provide psychologically plausible accounts of their representations and algorithms. Engineering decisions dominate. This has led to words being seen as an incidental part of the architecture: the repository of all of language’s idiosyncratic aspects. Drawing on a body of past and ongoing research by myself and others I have concluded that this view of words is wrong. Words are actually present at the most abstract, pre-linguistic levels of the NLP architecture and that there are phenomena in language use that are best accounted for by assuming that concepts are words.

This paper discusses the implementation and evaluation of automatically generated cloze questions in the style of the definitions found in Collins COBUILD English language learner’s dictionary. The definitions and the cloze questions are used in an automated reading tutor to help second and third grade students learn new vocabulary. A parser provides syntactic phrase structure trees for the definitions. With these parse trees as input, a pattern matching program uses a set of syntactic patterns to extract the phrases that make up the cloze question answers and distracters.