Morpho-syntactic annotations: the theory

Since I keep receiving questions concerning the relation between the TEI guidelines and POS annotations, let me start a series of posts on the issue. As a first step, I present here the basic concepts of the underlying ISO standard: ISO 24611:2012 Language resource management — Morpho-syntactic annotation framework (MAF).

ISO 24611 provides a framework for the representation of morpho-syntactic (also referred to as part-of-speech) annotations. Such annotations correspond to a first lexical abstraction over language data (textual or spoken) and, depending on the language to be annotated, as well as the characteristics of the annotation tool or annotation scheme that is being used, can vary enormously in structure and complexity.

In order to deal with such complex issues as ambiguity and determinism in morpho-syntactic annotation, ISO 24611 introduces a meta-model that draws a clear distinction between the two levels of tokens (representing the surface segmentation of the source) and word-forms (identifying lexical abstractions associated with groups of tokens). Both levels can be represented as simple sequences and as local graphs such as multiple segmentations and ambiguous compounds. Besides, any n-to-n combination can stand between word forms and tokens.

As linguistic segments (sometimes called ‘markables’ in the literature, see for instance, Carletta et al. 1997), tokens may be embedded in the source document as inline mark-up, or they may point remotely to it by means of so-called stand-off annotations.

As linguistic abstractions, word-forms can be qualified by various linguistic features characterising the morpho-syntactic properties that are instantiated in the realisation of the lexical entry within the annotated text. Such properties may range from the simple indication of a lemma up to an explicit reference to a lexical entry in a dictionary. In most existing applications of morpho-syntactic annotation, linguistic properties are expressed by means of so-called tags; these codes refer to basic feature structures (see early examples in Monachini and Calzolari, 1994). Such codes may also provide morphological information, including its part of speech (e.g. noun, adjective or verb), and features such as number, gender, person, mood and verbal tense.

In keeping with the general modelling strategy of ISO/TC 37, MAF provides means of relating morpho-syntactic tags expressed as feature structures (compliant with ISO 24610) to the data categories available in ISOCat.org. A normative annex of this International Standard elicits a core set of data categories that can be used as reference for most current morpho-syntactic annotation tasks in a multilingual context. However, when implementers of the standard find these categories inappropriate in either coverage, scope or semantics, they are encouraged to use ISOCat to define their own categories.

Associated to the meta-model, MAF also provides a default XML serialisation. Still, as we shall see in a further post, it is possible to combine a TEI based representation with MAF compliant annotations (See also Romary and Witt, 2013).

 

ISO 24610-1:2006 Language resource management — Feature structures — Part 1: Feature structure representation (see also the corresponding chapter in the TEI guidelines: http://www.tei-c.org/release/doc/tei-p5-doc/en/html/FS.html)

Jean Carletta, Nils Dahlbäck, Norbert Reithinger and Marilyn A. Walker (Eds.) (1997) Standards for Dialogue Coding in Natural Language Processing, Dagsthul-Seminar Report 167; 03.02.-07.02.97 (9706)

Monachini, Monica and Nicoletta Calzolari (1994). Synopsis and Comparison of Morpho-syntactic Phenomena Encoded in Lexicon and Corpora. A Common Proposal and Applications to European Languages. Internal Document, EAGLES Lexicon Group, ILC, Università Pisa, Oct. 1994

Romary, Laurent and Andreas Witt (2013), “Data formats for phonological corpora”, in Handbook of Corpus Phonology Oxford University Press (Ed.). http://hal.inria.fr/inria-00630289

TBX goes TEI

As a first start into this blogging adventure, let me focus on a current activity which I started in the context of the recent ISO/TC 37 general meetings in Pretoria in June. The context is the ongoing revision process for ISO 300421.

Since its P4 edition, the TEI guidelines did not have a chapter on terminology any more. This is indeed missing since the current TEI dictionary chapter is covering semasiological views on lexical data (word to sense) and many applications require an onomasiological perspective (concept to term). For instance, translators, technical writers or language learners often manipulate terminologies, i.e. lexical structures are organised according to concepts, which in turn are subdivided in language sections (all words or expressions used to express the corresponding concept for this language) and then in term sections (all types of information needed to represent and qualify a given term). The underlying model is standardized  in ISO 16642 TMF2. TBX is thus a serialization of TMF.

I have now started to create an TEI/ODD specification that takes up the basic TBX structure, starting at <termEntry> level and allowing these to occur anywhere a traditional dictionary entry (<entry>) would in the TEI architecture.

To illustrate this, let me just show a typical  (yet awfully simple) example. The TBX namespace is a fake one (it is under discussion and we may want to get all this in the TEI name space).

<TEI xmlns=”http://www.tei-c.org/ns/1.0″ xmlns:tbx=”http://www.tbx.org”
xmlns:tei=”http://www.tei-c.org/ns/1.0″>
<teiHeader>…</teiHeader>
<text>
<body>
<div>

<termEntry xml:id=”c5″ xmlns=”http://www.tbx.org”>
<langSet xml:lang=”en”>
<tig>
<term>e-mail</term>
</tig>
</langSet>
<langSet xml:lang=”fr”>
<tig>
<term>courriel</term>
</tig>
</langSet>
</termEntry>

</div>
</body>
</text>
</TEI>

  1. ISO 30042:2008 Systems to manage terminology, knowledge and content — TermBase eXchange (TBX) []
  2. ISO 16642:2003 Computer applications in terminology — Terminological markup framework []