From print to code – multiple forms in one

An interesting discussion started on the TEI list ( about dictionary encoding, which I would like to reflect here to point to a couple of issues related to forms in dictionaries.

The post was sent by Susanna Allés, who is currently working on a Latin medieval dictionary. Here’s the initial question:

“we can’t find a satisfactory solution to encode a term (<term>) that corresponds to several languages. Here is the text:

arrencare  [cat. arrencar, arrancar (cf. oc., esp., port. arrancar), d’origen discutit | de origen discutido | of debated origin]  rompre roturar to break up: 1038 ACUrgell, Cart. I 357, f. 118, col. 2: terra quae fuit arrenchada qui est in Subuineas quae escamiauit Ermengauda a me Mirone. De una parte affrontat … in terra quae arrencauit … de IIII uero parte in ipsa fonnada de terra de Riamball que arrencauit.

The part in (square) brackets corresponds to the etymology (encoded with <etym>) and in red (cf. oc., esp., port. arrancar) corresponds to a term (the Romanic result); we want to relate this term with several languages (that it is, to Occitan, Spanish, Portuguese, in this case). Usually, if we are dealing with just a language, we encode it as <term xml:lang=”es”>arrancar</term> , but since we need to add other languages and the element term just allow one @xml:lang, we really don’t know how to handle.”

For sake of place, the original editor of the dictionary has used the same character sequence to represent three “words” from three different languages, which just happen, by chance (and linguistic proximity…), to be written in the same way.

The first line of thought, which considers the source text at face value may lead (like stated in the question) to  one could want to refer to three languages in relation to the sequence “arrancar”. Possibilities that have been alluded to on the list are:

  • putting more that one language codes as values for xml:lang, theoretically feasible, given the recent evolution of the corresponding W3C specification, not in the TEI though and probably an “interoperability nightmare” (quoting Stuart Yeates)
  • tagging the language references with <lang>, as suggested by Sabine Thuillier, in keeping to the basic constructs associated to <etym> (etymology) in the TEI guidelines (i.e. ​cf. ​<lang>oc., esp., port.</lang> <mentionned>arrancar</mentionned>​)​; Sabine rightly points to the fact that this solution is not optimal from a processing point of view.

Still, a more lexicographic analysis seems to lead to another path which consists in reconstructing the underlying logical representation, as nicely stated by Toma Tasovac:

“what we have here is typical lexicographic shorthand —  we know how lexicographers used to be stingy when it comes to space —  but that, if we really think about it, we are not (on the conceptual, modeling, even philosophical level) talking about one word being “the same” in three languages, but rather about three “different” words from three different languages that happen to have the same graphical representation.

What one could therefore also do is encode three instances of “arrancar” and each with an xml:lang for a different language.”

It is interesting to observe that several contributors, even being in line with this analysis, have actually suggested mechanisms to retain the “sameness” in the encoding (using @sameas, @exclude or <alt>; Syd Baumann actually provided quite a comprehensive set of possibilities)). No, Toma (supported by David Birnbaum) put it right:

“if one wanted to preserve the economic display in the end output, one would have to use XSLT or whatever one uses to check whether subsequent terms have the same graphical form and then display only one term, but display language ids from all of them.”

Which means that we need to produce an encoding where nothing may lead a processor to think that we have the “same” form, but it just happened that the three were printed out as one accidentally.

Finally, let me add that there is an issue with the encoding of “arrancar” as <term>. It is obviously a <form> (even better a <form>/<orth> if we anticipate that we could provide additional pronunciation information.

So my take at the encoding of this example would be as follows:

(cf. <lang>oc.</lang>, <lang>esp.</lang>, <lang>port.</lang> <form xml:lang=”oc”><orth>arrancar</orth></form><form xml:lang=”sp” ><orth>arrancar</orth></form><form xml:lang=”pt”><orth>arrancar</orth></form>)

But there is a hack here… <form> is not allowed there. Shall we change the content model of <etym>?

5 thoughts on “From print to code – multiple forms in one

  1. Toma Tasovac

    It’s interesting how one thing leads to another. I fully agree that <form> and a nested <orth> would be more appropriate. <term> should be used for “technical terms” — although, to be honest, that’s another can of worms, because what’s technical for one person is daily bread for somebody else. But that’s not the issue here. The really interesting question is what to do with the content model of <etym>. And the problem doesn’t end with <form> not being allowed within <etym>.

    <etym> relies on elements such as <lang> and <mentionned> to express word forms from different languages, sometimes also with <gloss> if translations are provided.

    I always thought that this was slightly bizarre — if for no other reason than at least because we got rid of <tr> in P5 and replaced it with <cit type=”translation></cit>. The argument went, as far as I can remember, that <cit> was providing a more abstract and more coherent approach to citing all sorts of things — in and beyond dictionaries.

    Now, it seems to me that the Guidelines treat <etym> in a very “old-fashioned” way, if we consider P4 old-fashioned. I don’t see why we really need <mentionned> and <gloss> when we could equally well use something like <cit> instead and type them appropriately. (<cit> is allowed within an <etym> but I don’t think anybody uses it there). After all, all <mentionned> and <gloss> do is cite word forms and their translations. Oder?

    So, if we were going to go down the road of changing the content model of <etym> to accomodate <form>, we should also think about recommending <cit> instead of <mentionned> and <gloss>, i.e. we would have to turn <etym> into a mini-entry itself.

    Obviously, I am making this up as I go along and this would require some serious thinking, but it could be worth it. At the same time, I would also be interested in hearing counter-arguments as to why it would be a better idea to keep <etym> special and dependent only on <mentionned> and <gloss>.

    1. Lou

      The historical explanation for why is the way it currently is is that the workgroup originally charged with coming up with some candidate subelements for it said “it’s complicated” and none of those who have subsequently tried revising the dictionary spec have really been historical lexicographers, or interested in etymology. So has been left as a relatively unstructured soup containing more or less the same general purpose phrase-level elements as elsewhere. Of these, clearly <term>, <mentioned>, <gloss> etc. are all quite useful (one might also think of <distinct> being used here too) though they don’t have the specific semantics “form cited in an etymology”. No reason why a suitably motivated group shouldn’t come up with better proposals, of course. I would just note that etymologies really do vary a very great deal in their content and organization.

  2. Piotr

    My first remark is that, if you’re taking this half-half approach, preserving explicit language labels and at the same time applying a lexical-database-like approach to the forms, you might want to intertwine them, for surface beauty, i.e. go for

    <lang>oc.</lang> <form xml:lang=”oc”><orth>arrancar</orth></form>, <lang>esp.</lang> …

    Another, side remark is that I can see the need for many more occurrences of xml:lang in that multilingual entry.

    And the final remark refers to the issue at hand alone, rather than the TEI itself: especially in etymological references, I would be happy to see dates on the quoted forms — contemporariness seems a somewhat relative notion… </rant>

    1. Lou

      Just to comment in response to Piotr: <lang> is for indicating the language label as it appears in the entry, xml:lang for indicating what the encoder believes to be the language of the content of the element. Splitting the label up in the way suggested seems to me to be false to the source in a way that is potentially very confusing.

      1. Piotr

        Thanks Lou, but I suggested that exactly because I know the difference, and, as far as I can see, the assumption of “truth to the source” has been abandoned here at the very beginning. What has been suggested was changing the source:

        1) <composite label> “arrancar”,

        and instead going for

        2) <composite label> “arrancar”, “arrancar”, “arrancar”

        (with extra data transparently added to each of the three identical forms). That is no different, and, I would argue, actually worse, than doing:

        3) <label 1> “arrancar”, <label 2> “arrancar”, <label 3> “arrancar”

        because, if the modified source is post-processed by XSLT or XQuery or whatever, both two “new” orderings, (2) and (3), can be handled just as easily, and in cases where it is not postprocessed but e.g. merely adorned by CSS, the latter ordering at least looks good, as opposed to the former, which looks erroneously repetitive and thus bad bad bad. 🙂


Leave a Reply to Toma Tasovac Cancel reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.