Representing literal values in SSSOM

[matentzn]

While SSSOM is clearly focused on mappings between controlled vocabulary terms (ids which correspond to some entity in the world), there is a huge amount of work going on that is concerned with "literal" mappings -> mapping a string to a concept in an ontology/controlled vocabulary. These are byproducts for example of pipelines for named entity recognition tasks, or manual mapping efforts of curators that read papers and map strings they read to an ontology concept. I believe that the same concerns that we have for normal terminological mappings also apply here - (provenance, curation rules metadata), and therefore propose to develop a scheme by which we can represent literals in the subject_id field. One example would be to use URL encoded strings in a standard namespace:

SSSOM_LITERAL:Antinukle%C3%A4re%20Antik%C3%B6rper%20%28Kernantik%C3%B6rper%29

I believe this would take care of quite a few problems, and users can easily handle or ignore these kinds of subjects. Note that strictly speaking, this is already permitted by the spec of SSSOM - so we are not discussing here whether this should be allowed or not. The question is more a matter of discussing whether we should as a community recommend a "standard" way to handle literals.

I invite everyone to voice their support, ideas or concerns here.

see related: mapping-commons/sssom-py#28 (@cmungall also has some big plans to use these for mapping reconciliation).

[cmungall]

Agree with all the above

1. The subject_id MUST be a valid CURIE
2. the CURIE prefix MUST be declared in the header to supplementary metadata file
3. the CURIE prefix SHOULD be registered on bioregistry.io
   • corollary: no SSSOM_LITERAL
4. the CURIE SHOULD NOT inject IDs into someone else vocabulary/database (except for below)
5. we will create a new prefix that is intended for CURIEs-for-strings, e.g. sssom.literal
6. anyone MAY inject into sssom.literal, but if they do they MUST use agreed upon literal to curie translation
7. a standard literal to CURIE translation SHOULD be used. Candidates:
   • md5 hash
   • urlencode (I favor this)
8. Different communities MAY create their own extension standards that provide consistent and potentially stricter guidelines than these

An example of an extension would be one for users of LinkML. LinkML allows for the specification of enums, the values of which may be associated with CURIEs within a LinkML schema. To expose these as SSSOM there would be a deterministic mapping that would create a CURIE that has the schema CURIE/URL as base, suffixed with the URL encoding. linkml/linkml#272

[matentzn]

Here is the concrete non-normative proposal for now:

Motivation: In much the same way as entity-entity mappings which are the primary concern of SSSOM, literals are often mapped to entities during curation activities, such as Named Entity Recognition or manual scientific publication curation. Usually, a string that "represents" an entity is linked to an ontology term or a database entity. We usually refer to these associations as synonyms, and curate them as part of our ontologies.

While not entirely obvious through the SSSOM conceptual model, we want to support this use case by offering a syntactic convention:

A literal L (for example, "Alzheimer’s disease") can be represented like this (using f-string syntax in the example):

f"sssom.literal:{urlencode(L)}"

For example, the literal Alzheimer’s disease will get encoded as the CURIE sssom.literal:Alzheimer%E2%80%99s%20disease by applying a URL encoding method to the literal Alzheimer’s disease and prepending the sssom.literal: prefix to the value. The actual literal should be added as subject_label or object_label.

subject_id	subject_label	predicate_id	object_id	object_label	mapping_justification
sssom.literal:Alzheimer%E2%80%99s%20disease	Alzheimer’s disease	skos:exactMatch	MONDO:0004975	Alzheimer disease	sempav:HumanCuration

Note that there is currently no support in sssom tools for these kinds of mappings, but feel free to request.
This is not normative - just a convention to accommodate for a slightly adjacent use case to SSSOM.

[cthoyt]

I am irked by this example. URL encoding gets out of hand pretty fast, and using dummy prefixes sort of defeats the purposes of using validatable, controlled vocabularies

Perhaps we could alternatively consider generating a similar standard to SSSOM specifically for properties where the subject always has to be a CURIE, the predicate always a "property" (in the OBO file format sense) and the object a literal (e.g., like synonyms, SMILES strings, numbers). It could share most of the same stuff as SSSOM with the addition of an optional object data type column (which would default to XSD's string, like most people expect)

[cthoyt]

If you want to stick this in SSSOM, then do we start adding special cases for validation based on the prefix?

[matentzn]

Re validity: variance across url encode is a problem. Didnt know that!
I guess a trivial qc check would be to test if the decoded value corresponds to the _label column literal in sssom validate.

Re special case for prefixes: yes, you are right, it will add a precedent for special case handling for prefixes. But this is not the only one we have in mind, hopefully 90% of the use cases can be dealt with by using less than 3 special prefixes.

[cthoyt]

if you have any special prefixes, the we will inevitably have to tie the standard to the reference implementation. I think this is quite bad

[matentzn]

I think I need a bit of elaboration on this point..

[cmungall]

consider generating a similar standard to SSSOM

This is reasonable but none of the tooling built for sssom should be expected to work for it

IMO if we are going to do this then we may as well also generalize to allow for also location of substring matches:
#155

[matentzn]

Just had a longer discussion about this with @udp and we are now convinced that we should not overload the current SSSOM profile, but instead provide another "class/profile" specifically for literal mappings. I will make a stab at this.

[nichtich]

This is not about named entity recognition but about entity linking. There is a working group to standardize Reconciliation API for linking literal values to identifiers from terminologies.

[matentzn]

Wow thank you for bringing that to our attention! Again, I have never even heard of it!