Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DMN16-130/DMN16-130 Use fixed Namespace URIs and xsi:schemaLocation #33

Draft
wants to merge 9 commits into
base: 1.6-ballot05
Choose a base branch
from

Conversation

falko
Copy link
Contributor

@falko falko commented Jan 5, 2024

Currently, we identify new versions of DMN by changing the XML namespace with every revision. This breaks backwards compatibility, DMN files using older versions become invalid and have to be migrated to the new namespace to be validated with the new XML schema.

However, the DMN Revision Task Force makes only backwards-compatible changes to the language. So there is no need for such an aggressive versioning approach

The work on SCE made me review the OMG Policy for Versioning of Specification URIs, File URIs, and XML Namespaces and I discovered some interesting options...

Proposal

From DMN 1.6 and all future 1.x revisions, DMN uses fixed XML namespace URIs:

In addition to that, a version attribute on the root element indicates which concrete DMN version is used in a file:
<dmn:definitions version="1.6" ...

To indicates which exact DMN version is used and to make it easier for tools to find the right schema an xsi:schemaLocation attribute is added which points to the correct schema file for that version which is hosted by OMG at a fixed location:

xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="
https://www.omg.org/spec/DMN/
https://www.omg.org/spec/DMN/20240322/DMN16.xsd
https://www.omg.org/spec/DMN/DI/
https://www.omg.org/spec/DMN/20240322/DMNDI16.xsd
http://www.omg.org/spec/DMN/20180521/DI/
https://www.omg.org/spec/DMN/20180521/DI.xsd
http://www.omg.org/spec/DMN/20180521/DC/
https://www.omg.org/spec/DMN/20180521/DC.xsd
">

Tools like VS Code will immediately use that xsi:schemaLocation to download the XML schema and perform validation, which would be a great service to implementers and users who look at the XML. With xsi:schemaLocation in every DMN file there is no need to manually download the schema files and configure an XML catalog or otherwise instruct the XML validator, where to find the schema.

The OMG policy does not apply for the FEEL language URI because is not a real XML namespace, i.e. there is no XML schema for it and it is basically just a single string value. I propose to use the following URI including a semantic version number that would still be updated with every DMN revision:

<xsd:attribute name="expressionLanguage" type="xsd:anyURI" use="optional" default="https://www.omg.org/spec/DMN/FEEL/1.6"/>
<xsd:attribute name="typeLanguage" type="xsd:anyURI" use="optional" default="https://www.omg.org/spec/DMN/FEEL/1.6"/>

References

https://www.xfront.com/Versioning.pdf?page=3
https://xml.coverpages.org/HP-StephensonSchemaBestPractices.pdf?page=8

@falko falko changed the title Define new versioning scheme for DMN New versioning scheme for DMN Jan 5, 2024
@falko falko changed the title New versioning scheme for DMN New Versioning Scheme for DMN Jan 5, 2024
@falko
Copy link
Contributor Author

falko commented Jan 5, 2024

@sringuette @BPMAdvantage I'd be interested in your feedback on this idea. If it works for DMN, I'd also make that the approach for SCE and SDMN. This approach could also be a gentle way to improve BPMN's XML schema without immediately fragmenting the tool market.

@falko falko requested a review from barmac January 5, 2024 11:02
@SimonRinguette
Copy link
Contributor

I have no problem with that, it would be a marginal improvements from my point of view on code generation.

@BPMAdvantage
Copy link
Contributor

I have no problem with this if it doesn't delay any other issue resolutions

@barmac
Copy link

barmac commented Jan 8, 2024

So this means that there is a shared namespace for DMN 1.6 and future versions, but the dmn:Definitions#version attribute defines the specification version. How should a tool handle mismatch between the version attribute and the given schema location? My understanding would be that the version attribute has priority, and schema location is optional.

@nikku
Copy link

nikku commented Jan 8, 2024

From our (Camunda Modeling) perspective it would not make a big difference as we're forward migrating diagrams to the latest version we support anyway.

The good thing about the old (existing) approach is that it prevents issues like #33 (comment).

To me it feels a little bit as we work around the intention of XML schemata + namespaces with the new approach.


Also, schema location is optional and we should not use it to derive the used DMN version (but instead use xmlns + version attribute as the source of truth).

So to your point @barmac: Tools that play nicely with their users may update the schema location, as they migrate diagrams to a newer version.

@barmac
Copy link

barmac commented Jan 9, 2024

Potentially we could fix the version attribute for given schema, but then we end up with the previous situation of "new schema shows old file as invalid":

- <xsd:attribute name="version" type="xsd:decimal" use="required"/>
+ <xsd:attribute name="version" type="xsd:decimal" use="required" fixed="1.6"/>

@falko
Copy link
Contributor Author

falko commented Jan 9, 2024

Notes from today's SDMN meeting:

  • The spec text will enforce xsi:schemaLocation to be mandatory for vendors to put in every DMN file
  • We should make sure that this doesn't cause issues for vendors, e.g. their parsers insisting on using the online version over a locally provided one
  • version can be left out and xsi:schemaLocation is enough to identify the version of all dependencies
  • Does this technically prevent BC breaks if necessary?

@falko
Copy link
Contributor Author

falko commented Jan 9, 2024

  • I could re-run my tests for previous DMN versions to validate if this approach would have worked say since DMN 1.1 or to see how BC breaks if any would have affected this
  • A PR to add xsi:schemaLocation to the models of the DMN TCK would show if it affects the schema validation there
  • the BPMN MIWG tools could also be tried for impacts of xsi:schemaLocation

@nikku
Copy link

nikku commented Jan 9, 2024

Will you/we guarantee the following:

  • An editor A openes a DMN v1.8 diagram as an editor that can only read DMN v1.6
  • A makes changes to the diagram (knowing only 1.6 semantics)
  • A saves the diagram
  • Another editor B knowing v1.8 semantics will find a completely consistent v1.8 diagram

Otherwise I don't understand how this approach improves the existing version scheme if all editors and engines have to do (right now) is to:

  • Check if the DMN diagram they want to open is in a known version they support (for reading)
  • Transform it (one way operation) to a DMN version they support for opening (and writing for editors)
  • Open the diagram.
  • As an editor, make changes + write in the new diagram version

@falko
Copy link
Contributor Author

falko commented Feb 5, 2024

@nikku thank you for your feedback and questions. To protect the investments of vendors and users, DMN has been following the OMG's policy of backwards compatibility since version 1.1 (1.0 was unimplementably broken):

A machine readable file is backwards compatible with an earlier version if:

  • Any implementation designed to work with the earlier version of the file will continue to interoperate with implementations designed to work with the newer file version.
  • In general, this implies that no productions specified in the file will be removed in a backwards compatible version with that file alias
  • Additional productions defining optional elements and types may be specified in the backwards compatible file, as long as an implementations not using these new optional features will interoperate with implementations designed to
    use the new features.
  • Names of elements and types in the earlier version of the file MUST NOT be
    changed in a backwards compatible revision of that file

We even have a tool to test for BC breaks before a new release.

So in summary, DMN files created with newer versions are perfectly fine to open and edit with older tools unless new language features are used. I estimate that only small percentage of models actually use new features.

So the majority of models, could still be opened and edited in older tools if it weren't for the namespace to completely prevent that. With the new mechanism tool vendors have more options to handle older & newer versions more gracefully:

  • It is still possible to read the mandatory schemaLocation to identify the DMN version indicated by the exporting tool
    • This can be used to display a warning if an older tool opens a newer file
  • In addition, a validation using the XML schema of the version that the tool supports can be used to check if any unsupported features are used
    • Similarly, the FEEL parser can check for unsupported syntax from newer FEEL versions

In hindsight, it was foolish to just rely on a namespace URI to assert if a file is valid DMN. XML schema validation should be performed as often as possible to check files before reading and after writing. Over the years, we have seen a lot of interchange problems in the BPMN Model Interchange Working Group (MIWG) simply because vendors have not performed schema validations. So this proposal would further incentivize validation and could lead to better interchange in the long-run.

@falko
Copy link
Contributor Author

falko commented Feb 5, 2024

The habit to always change the namespace has somewhat accidentally grown:

With DMN shipping bugfixes on a yearly pace, the fragmentation of the tool market increases, while the number of changes to the XML schema decreases. DMN 1.6 has no changes to the XML schema at all but it if we follow the "tradition" would declare itself totally incompatible with DMN 1.5 and all prior versions.

There is an even more specific OMG definition for backwards compatibility of XML schemas:

The following are examples of backwards compatible changes that
would not require assignment of a new namespace URI:

  • addition of new global element, attribute, complexType or simpleType
    definitions
  • modifications to the cardinality of elements for which the value-space of possible instance documents conformant to the previous revision of the schema would still be valid with regards to the revised cardinality rule

By that definition DMN 1.6 should not change its namespace URI at all, which shows that is was a bad design choice to use it as a version marker.

@nikku
Copy link

nikku commented Feb 5, 2024

Thanks for elaborating in depth on this @falko! If there is clear indication from other tool vendors and users that this is an issue I have no objections to change course. It should not cause many issues on our end (other than adapting our current migration strategy).

The only thing I'd as is to define a clear mechanism to identify which DMN version a diagram adheres to so that I as an editor can handle the miss-match in one way or the other.

@falko
Copy link
Contributor Author

falko commented Feb 5, 2024

We wouldn't be the first ones to do xsi:schemaLocation sniffing

@falko
Copy link
Contributor Author

falko commented Feb 5, 2024

@opatrascoiu @baldimir @tarilabs @etirelli @agilepro @brsilver @garyhallmark do you have any opinion on this new DMN namespace & versioning scheme?

@SimonRinguette
Copy link
Contributor

Personal 2 cents,

I would be ok with keeping the xsd:schema to the same URI but using the xsi:schemaLocation would not be my preferred way of handling the version. I would prefer a version attribute on the root element.

@falko
Copy link
Contributor Author

falko commented Feb 5, 2024

@SimonRinguette the version property was criticized for being redundant with the xsi:schemaLocation, which would force tools to deal with inconsistencies. Also the URL in the xsi:schemaLocation would be quite similar to the current namespace URIs.

@SimonRinguette
Copy link
Contributor

@falko Not going to die on that hill. xsi:schemaLocation could also be used to refer to shemas defined locally on your disk, just giving you my opinion.

@falko
Copy link
Contributor Author

falko commented Feb 5, 2024

@SimonRinguette no worries. Your feedback is welcome. What also made me lean towards schemaLocation-only is its capability to identify exact versions for all dependencies, e.g.

<?xml version="1.0" encoding="UTF-8"?>
<sdmn:sharedDataModel id="HelloWorldDataModel"
   targetNamespace="https://example.org/hello-world/shared-data-model"
   xmlns="https://example.org/hello-world/shared-data-model"
   xmlns:sdmn="https://www.omg.org/spec/SDMN/"
   xmlns:sce="https://www.omg.org/spec/SCE/"
   xmlns:scedi="https://www.omg.org/spec/SCE/SCEDI/"
   xmlns:di="https://www.omg.org/spec/SCE/DI/"
   xmlns:dc="https://www.omg.org/spec/SCE/DC/"
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xsi:schemaLocation="
     https://www.omg.org/spec/SDMN/
     https://www.omg.org/spec/SDMN/20240210/SDMN.xsd
     https://www.omg.org/spec/SCE/
     https://www.omg.org/spec/SCE/20240210/SCE.xsd
     https://www.omg.org/spec/SCE/SCEDI/
     https://www.omg.org/spec/SCE/20240210/SCEDI.xsd
     https://www.omg.org/spec/SCE/DI/
     https://www.omg.org/spec/SCE/20240210/DI.xsd
     https://www.omg.org/spec/SCE/DC/
     https://www.omg.org/spec/SCE/20240210/DC.xsd
   ">
 <!-- ... -->
</sdmn:sharedDataModel>

I'm not saying that I want to open the door for arbitrary combinations of those but the backwards compatibility would in theory even allow it.

@baldimir
Copy link

baldimir commented Feb 9, 2024

+1 from me to this.

@falko falko changed the base branch from master to 1.6-ballot05 April 23, 2024 02:11
@falko falko changed the title New Versioning Scheme for DMN DMN16-130/DMN16-130 Use fixed Namespace URIs and xsi:schemaLocation Apr 23, 2024
@falko falko marked this pull request as draft April 26, 2024 23:39
@falko
Copy link
Contributor Author

falko commented Apr 26, 2024

I hate to say it, but this proposal will have to wait until DMN 1.7 because it would otherwise conflict with the adoption of some other SCE features, which I would like to propose in DMN 1.7. So, for this release, I'll prepared yet another switch to a dated namespace URI in #36

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants