Proposal: Metadata syntax

[SimonClark]

There has been discussion lately about how and where to add string metadata/attributes/context. Some options have it placed as part of the string as an attribute. Others place it outside the string in the resource structure that is used to deliver a group of strings.

It seems to me that there are three classes of ancillary string information, that have distinct uses and requirements...

• a: attributes of the string - eg: gender
  • Has a potential impact on runtime, but not when rendering in isolation.
  • potentially altered by translation (a noun may not have the same gender in all target languages)
• b: specifiers about the string variants - eg: plural form, inflection, capitalization, etc.
  • Often it is necessary to deliver a number of different variants for a particular string.
  • Useful when choosing the most appropriate variant of a string. eg:
    • this is the many-plural form.
    • This is the the sentence-start capitalization
    • This is the variation to use if the subject is Canadian
• c: context metadata about the string - eg: tone, context, URL of string in situ
  • no runtime impact
  • valuable input to the translation process (ie: what homonym of "State" is this)

Types A and B are important to deliver to the runtime. Type C is probably important to removed before delivery to production.

Type A should follow the string around. If the string is embedded as a parameter in another string, it should be included as well. If the string is sent to a translation service, Type A should be considered part of the output of the service.

@gender=female
employee

if a string with type A metadata is passed into a different MF2.0 string as a parameter, then the metadata about that string can be used in a match statement:

.match {$record :attribute type=gender} 
male {{le {$record}}}
female {{la {$record}}}

Type B has served its purpose once the string has been selected. It is appropriate to specify type B at the resource level, and not pass it around further.

Type C is important and valuable to track during development and release, but should be redacted before shipping to production. It is possible that this is never appropriate in the resource, and the right choice here is for the content management system to generate XLIFF, etc, withough MF2Resource as in intermediate.

Related Discussion on MF2 Resource repo

Old content...

There are aspects of a string that would be valuable to embed in the string. If this is a noun, what noun class is it? Is it masculine, feminine, or ungendered?

Imagine a scenario where we are delivering the string "Your search returned 5 laptops." The quantity and the item are passed in as parameters. If we need to react to gendered nouns in the parameter, it would be very beneficial to not require that the gender of the noun be passed in as a separate parameter. Developers working in English may not consider that this is required, and the code would have to be sent back for further changes, once the translators start to work with it.

If we could add metadata key:value pairs to MF2.0 "[gender:masculine]{laptop}", then the string that consumes it could then call a function to retrieve the gender for a select statement:

match {$itemName :gender} when masculine {Le} when accusative {La} {$itemName}

This would also require that MF2.0 formatted messages can be passed unprocessed into the parameters of another MF2.0 string.

[eemeli]

One way to express this with the current syntax would be to use markup elements, i.e. something like

{{+meta gender=masculine}laptop{-meta}}

with an implementation that can format this to some kind of parts structure that the :gender selector also understands.

[mihnita]

Note: I don't think this was mentioned before, ever.
I don't say that it is right, or wrong.
Only that it is something that was not discussed, so there is no agreement one way or another.

What was discussed was using the gender in placeholders, as a parameter for functions.

As Eemeli (accurately) said "one way to express this", not "the way" or "the recommended way" :-)

We might end up recommending something, or not, tbd.

But TLDR: it's possible, and there might be more than one way

[SimonClark]

Fair enough.

Is there value, in light of the upcoming hardening of the spec, in allowing message-level attributes, so as to not close the door on something like the proposed solution?

[grhoten]

Let me add some information about this topic. You shouldn't really call it a plain string. It oversimplifies the discussion. You can start with a string that can be easily turned into a container of attributes. As you expand to a list of things and apply the same logic to each item, the string metaphor breaks down.

For our use case, there are 3 main things to consider, which is almost the same as the proposal:

1. The input metadata that provide the initial source context. This can either be looked up in a dictionary to derive the word properties or provided by the developer. For example, I may provide a name, like "Robin", and I may have additional data provided by some other subsystem that notes that the gender of "Robin" is feminine, masculine, neuter, common (gendered) or epicene.
2. The constraint metadata that provide target desired output. In a given context within a message, I may want to make a phrase definite, indefinite, construct, genitive or match the context from item 1 (e.g. refer to them as an actor or actress).
3. Additional metadata for the current state. That can contain how to pronounce a string instead of only print a string. A stereotypical string may look like "1", but I may want it to be pronounced as "one" or "first" depending on the context. If you had visual markup involved, that could be used instead of the speakable form. If you're selecting words, there are times that you don't want to deal with the visual or aural markup that can be involved, and you want to deal with just the words themselves for interpretation.

Text transformations, like capitalization, quoting and stuff like that are separate from inflection, which is primarily the focus in my list above. It's similar to what is mentioned in this proposal, but the reinflection through constraints is a separate topic than other text transformations.

In my previous proposal I gave some examples of how to deal with some of this. I believe some people would call it, constraint logic programming. You can have a set of initial constraints set by the developer's code or database, and a set of output constraints defined by the translator. The rest of the markup goes along for the ride or is omitted when necessary.

As you expand your work to other languages, you will also have to consider words that take on grammatical gender for more than one bound morpheme. This make simple string handling hard to leverage. As I recall in Arabic, "your employee" turns into one word that takes on the gender of the person in the second person with the gender of the object in the third person. To contrast this with Spanish, the gender of the possessive pronoun is deduced from the object instead of the audience in the second person.

On a side note, this proposal highlights a great need for consistency in grammeme handling (the names for grammatical category values). Would you use "male" or "masculine" for checking or specifying the gender? How can you validate such values? We settled on masculine instead of male, since male implies physical gender, while masculine implies both grammatical gender and physical gender.

I hope that this information helps.

[SimonClark]

Yes, helps a great deal, and has me rethinking a number of my preconceptions. You mention a previous proposal, but I couldn't find an issue or discussion. Do you have a link handy?

[mihnita]

"one" or "first" depending on the context

+1
And that might also be gendered:
Spanish "first": primera (feminine) / primero (masculine)

Romanian: prima (fem) / primul (masc)
Worse, add case in the mix: primei (fem) / primulul (masc) ("of the first")

[grhoten]

The proposal that I gave to the group was given around March 2022. It covered linguistic issues and example syntax. It was a presentation and not written here. It would take some time to figure out how to publish it and make the presentation available publicly. It was meant just for this group.

[SimonClark]

This (and other discussions) also brings up a topic on string variants. I can see a need, at least in my use cases, where there are two distinct types of variants of a string:

• explicit variants (structural variants? functional variants) where the string is rendered differently based on the options passed in. These are well understood, and foreseen at the time the code is written and the string is created (either by the developer or the content manager)
• adhoc variants, where the string may need to change based on criteria unknown at development time. An example of this is documented here

To further complicate this, I could see that some variants (like plural form) could be done either way, based on workflow requirements of the content management tool. If someone creates a new string, and declares it to be a noun, then the authoring tool would want to encourage or enforce that the various plural forms for a given target language are provided. Asking a translator for five different plural forms is easier than asking for an MF2 string that can generate the five different plural forms.

Of course, these could be consolidated into a single explicit variant MF2 string at build time.