We aim at finding the minimal set of fragments that achieves maximal parse accuracy in Data Oriented Parsing (DOP). Experiments with the Penn Wall Street Journal (WSJ) treebank show that counts of almost arbitrary fragments within parse trees are important, leading to improved parse accuracy over previous models tested on this treebank. We isolate a number of dependency relations which previous models neglect but which contribute to higher accuracy. We show that the history of statistical parsing models displays a tendency towards using more and larger fragments from training data.

Most information systems that deal with natural language texts do not tolerate much deviation from their idealized and simplified model of language. Spoken dialog is notoriously ungrammatical, however. Because the MAREDI project focuses in particular on the automatic analysis of scripted dialogs, we needed to develop a robust capacity to analyze transcribed spoken language. This paper summarizes the current state of our work. It presents the main elements of our approach, which is based on exploiting surface markers as the best route to the semantics of the conversation modelled. We highlight the foundations of our particular conversational model, and give an overview of the MAREDI system. We then discuss its three key modules, a connectionist network to recognise speech acts, a robust syntactic analyzer, and a semantic analyzer.

The GEIG metric for quantifying the accuracy of parsing became influential through the Parseval programme, but many researchers have seen it as unsatisfactory. The Leaf-Ancestor (LA) metric, first developed in the 1980s, arguably comes closer to formalizing our intuitive concept of relative parse accuracy. We support this claim via an experiment that contrasts the performance of alternative metrics on the same body of automatically-parsed examples. The LA metric has the further virtue of providing straightforward indications of the location of parsing errors.

We present an Augmented Template-Based approach to text realization that addresses the requirements of real-time, interactive systems such as a dialog system or an intelligent tutoring system. Template-based approaches are easier to implement and use than traditional approaches to text realization. They can also generate texts more quickly. However traditional template-based approaches with rigid templates are inflexible and difficult to reuse. Our approach augments traditional template-based approaches by adding several types of declarative control expressions and an attribute grammar-based mechanism for processing missing or inconsistent slot fillers. Therefore, augmented templates can be made more general than traditional ones, yielding templates that are more flexible and reusable across applications.

A principal aim of this journal is to bridge the gap between traditional computational linguistics research and the implementation of practical applications with potential real-world use. This new column aims to help with that bridge-building, by providing a window on to what is happening with speech and language technologies in the industrial and commercial worlds, and a discussion forum for topics of interest in the technology transfer space.