Validate ARCs against specification by default

[Brilator]

I'd suggest to validate all ARCs pushed to the DataHUB against the specification.

The annoying bit and reason to decide to let all ARCs "pass", was the default-validation against invenio (which lead to failed pipelines for missing or incomplete investigation contacts).
From my point of view it's perfectly fine to get a nasty email, if what you upload is not at all an ARC.

Originally posted in nfdi4plants/nfdi4plants.knowledgebase#519 (comment)

Opinions?

@kMutagene @j-bauer @HLWeil @muehlhaus @SabrinaZander @shiltemann @Mestizia

[HLWeil]

I agree, given the validation package is stable and does not throw false negatives. But I think that requirement is met?

[kMutagene]

I think this is only a good idea in theory, and leads to a lot of hassle in practice. We have different versions of the ARC specs, outdated tools (and with outdated i mean old installations of maintained tools) that might create ARCs targeting arc spec versions that are old and/or versions that have no validation package (yet).

Would such a mandatory validation step always target the latest spec? That 'invalidates' any old ARC that follows old spec versions.

A first step IMO would be moving to a world where at least every tool adds arc_spec_version information to the .arc directory, as well as the version information about the tool. Once that is propagated across all tools we can think about how to implement this, e.g. by pulling the arc_spec_version and validating against that.

[HLWeil]

Having the tools document the expected ARC specification version would probably be good in any case.

But I see another possible solution here:

• We could have the validation on by default, always validating against the latest version
• To validate against older version, the user could explicitly state an ARC validation package with a fixed version

This would make following the newest version an opt-out operation, hopefully nudging users to regularly update their ARCs.

[kMutagene]

This would make following the newest version an opt-out operation, hopefully nudging users to regularly update their ARCs.

Or annoy users that publish their arc and assume to be done afterwards.

I think my general problem is enforcing of a data hub wide opt-out model. I think it would be better to enforce this on the organization level, e.g. for CEPLAS. But i do not think that we have the tools for that. Just spitballing here, but if an organization would provide an ARC template that users must use, this would be solved without opting in everybody.

[j-bauer]

While there is an option to create group-level project template, it is sadly behind the paywall...

https://docs.gitlab.com/ee/user/group/custom_project_templates.html