JSON Schema
Aligning views on stability guarantees, their implications, and community perception #295
Replies: 15 comments 65 replies
-
|
As a (the?) proponent of 2, I'm going to slightly rephrase the argument, though I don't terribly disagree with it as-is. But if I were to strengthen it without elaborating too much beyond a simple initial paragraph:
|
Beta Was this translation helpful? Give feedback.
-
|
The OpenAPI 3.1 situation is a little more nuanced that. Although the default JSON Schema dialect is compatible with 2020-12, OpenAPI 3.1 allows users to use whatever dialect they choose. So, 3.1 users aren't stuck with 2020-12 the same way 3.0 users are stuck with draft-04(-ish). However, the argument is still valid because most people will use the default unless they have reason not to and some tooling may not support other dialects anyway. My hope is that a stable JSON Schema version is ready and made the default for OpenAPI 4. 3.1 has very low adoption right now, so hopefully most people end up skipping 3.1 and going straight for 4. |
Beta Was this translation helpful? Give feedback.
-
|
OpenAPI 3.0 had a very low adoption rate when we were doing 3.1. Now 3.0 is reasonably broadly adopted, and 3.1 is just getting started. I expect that dynamic to repeat with 3.1 and 4.0. It took a very long time for 3.0 to pick up steam compared to 2.0, and 4.0 seems likely to be an even more dramatic change. |
Beta Was this translation helpful? Give feedback.
-
|
wrt OpenAPI 3.1; I agree that the extensibility nuance gives us pause; but for me the real question is "what do tool/library vendors do when faced with different JSON Schema dialects in OpenApi?" Most, I would guess [research required!], support whatever "the default" might be - and, with luck, whatever the defaults "used to be" in a backwards compatibility kind of way (the first thing I can think of is Kiota, for example). It is not that they can't supprot other dialects, but "bring your own" is necessarily an exercise for the reader. |
Beta Was this translation helpful? Give feedback.
-
I can't agree with this. It's not a contradiction to say "we're making some breaking changes between 2020-12 and this version" and "from here on out, no more breaking changes except in places where we explicitly say something is unstable". That said, I'm not sure if we can/should promise to make no more breaking changes after draft-next, but we can certainly state that we're aware of the burden involved in doing so and therefore will keep it to a minimum, and only when clearly necessary and beneficial. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks, @karenetheridge, yes we need to "officially declare" our definition somewhere. @jdesrosiers has made a couple efforts toward that, and I think we could use with some sort of "policy" document (in addition to the forthcoming charter) that declares our intentions for the specification. |
Beta Was this translation helpful? Give feedback.
-
|
Using semver semantics, the way I think about it is that we've been operating as releasing major-versions for each draft and patch-versions for clarification/bugfix updates. So, draft-07 is essentially 7.0.0 and the draft-07 update was 7.0.1. What we're talking about here is shifting from releasing major-versions to releasing minor-versions. So 2023 would be something like 10.0.0, 2024 would be 10.1.0, and clarification/bugfix updates to 2024 would be 10.1.1. So, for option 1, it's like making one last major version release and option 2 is like starting minor-version updates from the last release. The question is, do we want to release 9.1.0 or 10.0.0. Personally, I think it makes perfect sense to go with a new major-version as we start a new way of operating, but mostly I just don't want to start off with the baggage of things we never would have included yet if we knew it would end up getting set in stone. There are known problematic issues. The biggest problem being collecting unknown keywords being incompatible with the forward compatibility guarantees we want to provide. That has to change, which means we need a new major-version anyway (option 1). So, the question isn't "do we change things", but rather, "how much should we change things". My position is that we change anything that needs to be changed. Anything with known issues should be changed, or marked as experimental, or removed. That should be a relatively small set of changes. |
Beta Was this translation helpful? Give feedback.
-
Vocabularies are here to stay. This is the mechanism by which JSON Schema can be extended. A lot of work has gone into the feature by multiple people over the past two specifications, and while we're not quite done with it, it's definitely the direction we want to go. |
Beta Was this translation helpful? Give feedback.
-
That was my fear from the very beggining in my quest to decide what I should do next. I'm not a big fan of vocabularies to be honest, but don't understand that I don't appreciate your work. For my small projects, I hope not to be forced to use the vocabularies. If they existed somewhere in the background without me knowing that they exist, they wouldn't be a reason to bother me. |
Beta Was this translation helpful? Give feedback.
-
In practice you should only need to "use" vocabularies if you're defining custom keywords. If you're just writing schemas using what's defined in the spec, then vocabs don't impact you at all, really. |
Beta Was this translation helpful? Give feedback.
-
|
That's nice. I see the potential of vocabularies, basically they transform a bidimensional schema into a multidimensional schema. Having backword compatibility guaranteed, through vocabularies you basically bring the forward compatibility because any added vocabulary will be basically a new version. The versioning system will basically become useless this way. The scary part is the merging system, with schemas having vocabularies and referencing other schemas which also may have vocabularies that may colide for some keywords, I didn't quite understand that part, but as you say the work is still in progress... |
Beta Was this translation helpful? Give feedback.
-
|
Haven't looked. Re: ~ external vocabularies with XML: https://cheatsheetseries.owasp.org/cheatsheets/XML_External_Entity_Prevention_Cheat_Sheet.html :
(Edit) https://en.wikipedia.org/wiki/XML_external_entity_attack :
|
Beta Was this translation helpful? Give feedback.
-
Do you mean "let's not discuss this here" or "this is not in question and therefore true"? |
Beta Was this translation helpful? Give feedback.
-
My understanding is that it has. I haven't seen anyone oppose the idea of guaranteeing stability (in some form). |
Beta Was this translation helpful? Give feedback.
-
|
It may not have been totally clear, but I objected providing guarantees for the interim release, and would rather we make signals of what we THINK is extremely likely to be for the following release. I don't feel we are getting wide enough feedback, and this would allow that without compromising on releasing sooner. If others feel strongly opposed, I'd like to hear why, but I'd bow if that's truly the consensus. |
Beta Was this translation helpful? Give feedback.
-
|
I feel that we have consensus that we want to provide stability guarantees, even at a high level what that means (no more breaking changes), but the crux of the disagreement is when to apply them. Do we promise what we have now won't break with future iterations, or do we publish once more and promise from that point forward? It sounds to me like you're more in the "maybe break it one last time" camp, but you also want wider feedback from the community. |
Beta Was this translation helpful? Give feedback.
-
|
I think we're making the most educated decisions possible in terms of what we believe to be stable or require further work. I feel the team here is eminently qualified! But, I think that when we disallow additional keywords (which I think we do with the more immediate release), it's going to force people to come out of the woodwork so to speak, and then share their opinions, and that might shed light on previously unseen considerations or inperfections. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the mention @Relequestual. I'll prioritize accordingly. |
Beta Was this translation helpful? Give feedback.
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as off-topic.
-
|
Are the stability guarantees expected to disallow the kind of changes that were made in previous draft releases, or merely to formalize the kind of guarantees that were already implicitly being followed? I.e. does 1) mean "the next publication will contain a comparable amount of breakage to 2020-12 and previous drafts, and we commit to doing less breakage in subsequent publications", or does it mean "the next publication will contain unusually large breaking changes compared to 2020-12 and previous drafts; subsequent publications will revert to the normal level of backward and forward compatibility, and augment this with an explicit formal definition of what that means"? |
Beta Was this translation helpful? Give feedback.
-
Well that's always the aim. If course 1 was taken and the next publication was 'our last chance to "fix something by breaking it."', would that (in the worst case) mean bigger breaking changes than previous drafts? (I think @Julian 's point about the implicit stability gurantee is a good one; my view is that making the same kind of breaking changes as in previous draft publications does not break people's implicit expectation (or at least, should not break reasonable people's implicit expectation), but allowing bigger-than-normal breaking changes to "get in before the freeze" would undermine the point of making a stability guarantee in the next publication somewhat)
Is this a substantial change compared to what 2020-12 and previous drafts did? Just trying to get a sense of how much this is a change versus a codification of existing practice (e.g. committing to never introducing new keywords feels substantial, but in practice how often were new keywords actually being introduced?) |
Beta Was this translation helpful? Give feedback.
-
Currently, the only breaking change that we've identified that we would absolutely need to make is removing "unknown keyword" support, however we have several ideas that will make the transition easier.
This point of view is a good one to have. Thank you for articulating it. |
Beta Was this translation helpful? Give feedback.
-
Man, in not even one sentence I went from making a good point to not being a reasonable person :) |
Beta Was this translation helpful? Give feedback.
-
Nothing is likely to change drastically or without very good reason. We're not looking at this as a chance to fix all the little things that annoy us. There will be some necessary changes to unblock us from evolving the spec without introducing breaking changes. The only impactful change that I think is likely is the one @gregsdennis mentioned, disallowing unknown keywords. But, I don't think that those necessary changes are what this discussion is really about ... In the past, we haven't been concerned about compatibility between releases and we've included things in the specification that are experimental. The expectation was that we would get feedback from implementation and usage and adjust (including breaking changes) as necessary in later releases. These include dynamic references, the vocabulary system, annotations, and output formats. What we are trying to decide now is really about what to do with those experiments. Should they be considered stable as of their state in 2020-12, or can we continue to treat them as experimental and allow breaking changes to those experimental features in future releases until they are considered stable. I only expect breakage in these experimental features which should be very low impact. |
Beta Was this translation helpful? Give feedback.
-
No, this discussion isn't about any particular feature or what to do with experiments. It's about whether we can avoid breaking changes for the next release and promise no breaking changes moving forward. Is it possible for us to promise nothing will break moving forward without including any breaking changes from 2020-12 now? I'm not concerned about identifying features as experimental or whatever. That discussion and decision can come after this one is resolved. |
Beta Was this translation helpful? Give feedback.
-
|
Speaking as a user, please just take whatever you currently have right now and stick with it. The value of JSON Schema is the ability to use tools and libraries that have been designed around it. Every time a new version with breaking changes is released, this causes significant inconvenience for people who have built production systems around the spec and third-party libraries. At this point there are no further changes that could be made to the spec whose benefits would outweigh the costs to those who are already using the spec and the ecosystem of software built around it. Declaring JSON Schema to be "complete" and promising not to make any future backwards-incompatible changes is the single most valuable thing that could be done right now. |
Beta Was this translation helpful? Give feedback.
-
|
We hear you. This sentiment aligns with @Julian's comment. However, we need to be cognizant of the impact current features have on our ability to promise compatibility between future releases (my comment). |
Beta Was this translation helpful? Give feedback.
-
|
We hear you loud and clear on this! |
Beta Was this translation helpful? Give feedback.
-
@peterkelly it may be looking like we will be unable to make that promise without breaking some (one or very few) things now. I know it's not ideal, but it may be something we're forced into. How do you feel understanding that if we include these changes now we will be able to make that promise with subsequent releases? What if we define a migration path and provide tooling to assist? |
Beta Was this translation helpful? Give feedback.
-
|
Thank you for asking this question. As I see it, any breaking changes at this point, will offset adoption of any new standards years and years into the future. If you make breaking changes now, I would reconsider how I write parsers and validators - should it be one code base or multiple code bases? One major game-changer, would be to have official schema parsers and validators, in enough languages to cover the majority of popular programming languages. Performance should of course be considered, but they wouldn't need to be the absolutely most performant libraries out there. Personal note: being forgiven if needed would probably be necessary. I think that looking at how the Protocol Buffers and Docker Compose specifications works is a good place to start. Besides that, having clear semver adherence makes you trust that things "just works". I am certain that you could get a lot of contributors to such a project. Sorry for the rant, I guess the conclusion is, that I hope you go with one of 2 options:
I personally hope you go with the last option, as I think this would help the adoption of JSON Schema in geneal as well. |
Beta Was this translation helpful? Give feedback.
-
|
Part of the challenge around this is finances. Protobuff has huge resources. JSON Schema has more than it ever has, but it's minuscule by compariosn. And, there has been question over if we would even want to have "official" implementations. What we ARE trying to do, is make finding and determining compliance of implementations easier: https://bowtie-json-schema.github.io/bowtie/ While neither of those two projects are "official" as of right now, I hope they will be. We really really appreciate you sharing your thoughts. It's a huge challenge to get this level of feedback usually. Please, stay, continue to provide feedback and insight where you can add value =] |
Beta Was this translation helpful? Give feedback.
-
|
I'm not going to be able to write a reply to each of the posts for the next week at least, but I will quickly try to inspire an appreciation for how nuanced this problem is. The purpose of proposing a media type through the "Internet-Draft" system is so that implementations can be written, experimented with, and used to produce iterations of the specifications. Although it is common to formalize protocols that are de-facto standards in production, it's not necessarily the case that every protocol (or format/media type) is safe to use in production. While a feature being used in production "in the wild" generally shouldn't be changed, the only guarantee it will remain stable is the publication of a media type that defines it to work that way. In the absence of a published media type, a specification can include all the explicit stability guarantees it wants... but if a subsequent publication can change these, then these guarantees aren't meaningful. You have to have a published media type definition. A published media type can still evolve, though. Evolution is made possible by incorporating "interoperability requirements" — in the form of BCP 14 language that says what implementations "SHOULD" or "MUST" do. When specifications are published that describe de-facto standards in the wild, it's usually made possible by having sufficient interoperability requirements from the start, so the specification can still evolve in the draft process while still being adopted in production. There's many mechanisms to enable this—CSS has vendor prefixes, HTTP/2 had ALPS identifiers, and HTML and ECMAScript are each powerful enough you can support all sorts of "graceful degradation" techniques. JSON Schema does not yet define requirements like this. So, the current goal for the specifications is to add interoperability requirements that permit evolution without disrupting older implementations. The essence of this is to prohibit unknown keywords, since unknown keywords may represent an intent by the schema author to reject some set of JSON documents from validation. It is correct that the addition of this requirement could break some implementations, but (1) it doesn't have to be this way—not all applications actually rely on this guarantee, so it doesn't have to be a requirement a such. What can happen instead is applications that require "strict" handling can specify a different default for unknown properties—instead of unknown keywords accepting by default, they can reject by default, or produce an error by default ("indeterminate result"). I wrote a whitepaper describing this technique, and I encourage everyone here to read it and incorporate the ideas there into your comments. And (2) when a schema author adds arbitrary keywords, this implies an assumption that the validator they are using will not change. But if you are expecting to upgrade your validator and writing custom keyword names, then unfortunately you've already committed yourself to breakage at some time in the future, and there's nothing that can be done about that—what if we define a new keyword that matches what you've written? We can minimize breakage, but schemas that use nonstandard keywords are flawed from the start and there's no changes to the spec that can be made that can fix that. Now, what do those interoperability requirements look like? Here's the top priorities, and then I have several more edits to circulate after these... but first things come first:
I would appreciate comments on these issues. |
Beta Was this translation helpful? Give feedback.
-
|
And to speak to your point @peterkelly, how JSON Schema got to this point is people took the draft and implemented it as a stand-alone library that wasn't intended to evolve. This means that stand-alone applications begun using JSON Schema, but Web/hypermedia/Internet applications that rely on a media type definition have not come unto widespread use yet (though some are in limited aspects, e.g. OpenAPI). While it's understandable to want to lock compatibility for stand-alone uses, this doesn't satisfy the needs of the Internet community at large, as there's no way to safely exchange JSON Schema documents between validators (by safely, I mean, a schema author can be confident that two validators won't disagree on the validity of a given JSON document). |
Beta Was this translation helpful? Give feedback.
-
|
I think this comment from the posting on Hacker News supports an idea that @awwright previously proposed, specifically allowing for additional keywords outside of those defined ONLY if they include a specific prefix. |
Beta Was this translation helpful? Give feedback.
-
|
@Relequestual {
"type": "string",
"@foo": "bar"
}to mark keywords acording what they are, like this: {
"@type": "string",
"foo": "bar"
}or even better, to invent the term {
"validate": {
"type": "string",
"format": "email"
},
"foo": "bar"
}... which after validation process would become: {
"type": "string",
"format": "email",
"foo": "bar"
}Btw, I would prefer the term |
Beta Was this translation helpful? Give feedback.
-
|
The idea of a prefix for unknown keywords was presented in https://github.com/orgs/json-schema-org/discussions/241, and I'd prefer that discussion stayed there. We need to focus on whether we are able to promise no more breaking changes without first breaking something. |
Beta Was this translation helpful? Give feedback.
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment has been hidden.
This comment has been hidden.
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment has been hidden.
This comment has been hidden.
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment has been hidden.
This comment has been hidden.
-
|
This comment on HN (and some others) suggest that people are less concerned with breaking changes if there is tooling to easily upgrade their schemas. @jviotti created and @gregsdennis collaborated on AlterSchema, which does just that. If we break things one last time, providing an upgrade path for schema authors will ease that pain. |
Beta Was this translation helpful? Give feedback.
-
|
Alterschema has some edgy limitations (like not fixing up references to parts of the schema that do not exist anymore) but I'm working on a revamp that rethinks some of the approach to make it even more robust. |
Beta Was this translation helpful? Give feedback.
-
|
Also, while I agree that users of JSON Schema are fine as long as we provide an automated upgrade path, it's tooling makers that suffer the most. Updating non-trivial JSON Schema tooling to support another version of the specification is often not an easy ride. |
Beta Was this translation helpful? Give feedback.
-
This strikes at the heart of the issue. By promising that there will be no more breaking changes, we make the process to support newer versions exponentially simpler. The question for this discussion, then, is can we (is it possible to) make that promise without first including one last set of breaking changes? |
Beta Was this translation helpful? Give feedback.
-
|
I think we're coming (I know I am) to the conclusion that we cannot. I think the only way we can is by making one last set of breaking changes. |
Beta Was this translation helpful? Give feedback.
-
|
On Tue, Jan 31, 2023, 3:32 PM Greg Dennis ***@***.***> wrote:
Updating non-trivial JSON Schema tooling to support another version of the
specification is often not an easy ride.
This strikes at the heart of the issue. By promising that there will be no
more breaking changes, we make the process to support newer versions
exponentially simpler.
The question for this discussion, then, is *can* we (is it possible to)
make that promise without first including one last set of breaking changes?
No, that's a pipe dream. Specs tend to change over time. When you have
breaking changes, there's a deprecation schedule for the
unfortunately-namespace-versioned API, and you increment the MAJOR field of
a.n. e.g. SemVer MAJOR.MINOR.PATCH so that people can tell *from the
version number* that there could be breaking changes which requires fixing
their implementation.
For example, SPARQL 1.2 etc. are revised to support RDFstar and SPARQLstar
years later; after having been thought frozen.
w3c/rdf-star-wg#4 (comment)
"Call for breaking changes before this funded major revision" would be more
realistic IMHO
Because e.g. https://github.com/lexiq-legal/pydantic_schemaorg isn't
sufficient to describe all of the Preconditions and Postconditions, it's
probably safe to say that developers will need unspecified custom keywords
or that they need yet another schema document with the same data shapes.
|
Beta Was this translation helpful? Give feedback.
-
|
Are you saying "No, there must be one more set of breaking changes", or "No, you can't make forever compatability promises"? |
Beta Was this translation helpful? Give feedback.
-
|
Having a deprecation schedule is one of the changes we're looking at making. But it doesn't make sense to do so outside of some sort of stability guarantee. Yes, we want to never have a breaking change between releases again. |
Beta Was this translation helpful? Give feedback.
-
|
Something relevant but not covered here: How frequently are new drafts going to be published? If I were a spec author, I would be concerned about any future-compatible promise which I am not, as an author, ever allowed to break. Especially because it can be hard to define what a compatible change is. What about making the future-compatibility guarantee only ever one or two steps into the future?
|
Beta Was this translation helpful? Give feedback.
-
|
By the way, this suggestion is informed by my experience working/living with CPython, which is currently being very good, IMO, about having a clear deprecation schedule which is legible to developers. I always know when things are being removed from the stdlib at least two releases ahead of when it happens. Of course, that's software and this is a spec. So there are major and important differences. But one thing I've noticed is that if you promise me compatibility with tomorrow's version, it's still worth quite a lot. |
Beta Was this translation helpful? Give feedback.
-
We, as spec authors, are aware of the seriousness of what we're trying to promise. Making this promise will force us to very carefully consider when a feature is considered stable. Please have a read through json-schema-org/json-schema-spec#1368 and json-schema-org/json-schema-spec#1348, which ultimately led to this discussion. We're definitely considering as much as possible. There are many examples of technologies which progress without breaking changes, and we don't think that this is an unreachable goal. |
Beta Was this translation helpful? Give feedback.
-
|
Unless I missed it, the definition of "breaking change" hasn't been clearly defined. Is it a breaking change if Is it a breaking change if draft-next's base metaschema includes a new vocabulary that defines a keyword that might overlap with a keyword that appears in someone's schema where they were assuming that it's purely annotative and otherwise ignored? After all, if they're requesting evaluation under We're already making breaking changes in draft-next: for example, the introduction of Extending that thought some more -- I don't think we can possibly consider the addition of new keywords as "breaking", even if/when we declare that unknown keywords are prohibited by default. The way to handle unknown keywords in a future draft would be to declare a vocabulary that defines them (with a schema of Therefore - we need to be clear about what constitutes a "breaking change". And after that, I think I am in favour of declaring that we'll (try to) avoid making such changes, but I want to know exactly what I'm agreeing to, first. |
Beta Was this translation helpful? Give feedback.
-
The complaints that we've received is that features don't work the same across JSON Schema releases. This means that users generally do consider them to be the same thing. (We have a deeper understanding of how JSON Schema works, so we see the technical nuance of them being different, but most users don't.) So for a keyword to change how it works from one version of the vocabulary to the next version of the vocabulary is indeed considered a breaking change. What we're defining a breaking change to be is reasonable compatiblity between specification releases. If an instance validates against a schema under version X rules, then it should validate against that same (unchanged) schema under version X+1 rules. Historically, this hasn't been the case. We do have some ideas for feature deprecation that we need to work out (elsewhere), and there are a few caveats around supporting unforeseen or edge-case scenarios, but on the whole we want this instance-validation stability across spec versions. As an example of the "reasonable compatibility" mentioned above,
Yes, this has been addressed by the "unknown keyword" arguments in the discussion above.
Yes, this is the idea. Conversely, if we don't prohibit unknown keywords, then we absolutely must consider the addition of new keywords as "breaking." The discussion of how to prohibit them without causing too much chaos is happening here. |
Beta Was this translation helpful? Give feedback.
-
The following was my attempt to define compatibility. (json-schema-org/json-schema-spec#1368)
Agreed. Technically, no changes have been breaking changes because every release is a distinct new version, not a change to the previous version. You can't have breaking changes if nothing ever changes. The idea is that going forward instead of releasing a whole new version with every release, releases are an update to the long-lived stable version. A keyword in the 2024 specification is the same keyword in the same vocabulary in the same dialect as it was in 2023. That's why any changes need to be compatible. The question being posed in this discussion is, do we start doing updates starting from 2020-12 or do we introduce one last new version (2023) and start doing updates from there.
Yes, I think they should. I also don't think dialect URI should change with spec updates, only new versions. If changes are always compatible, it shouldn't be a problem to update schemas rather than release a whole new set with each release. I believe it's possible for us to never have another breaking change release after 2023, but if we absolutely have to break something, that's the only time I think the dialect URI should need to change. |
Beta Was this translation helpful? Give feedback.
-
Actually, I think users do know that features aren't necessarily the same across versions. They know that upgrading from one version to the next is at best a chore and and at worst liable to break their code. Moving to releases that update in a compatible way rather than create a new potentially breaking version allows for a safe and painless way for users to make use of new features and stay up-to-date. |
Beta Was this translation helpful? Give feedback.
-
I agree that they know they're not the same, and that's the source of the complaints. They expect them to be the same across versions, and they're not. |
Beta Was this translation helpful? Give feedback.
-
DECISION: After getting feedback from the community (thanks to everyone!), we have decided to move forward with option 1, which is including minimal breaking changes that allow us to promise that there will be none in subsequent versions, with the caveat that we provide as easy an upgrade path as possible, including instruction and possibly tooling.
Further discussion about dropping support for unknown keywords will continue in this discussion, which already contains several proposed solutions.
Recently I asked the core members to provide their thoughts on the stability status of the features we have in the spec. The collated results focused on keywords and revealed two primary opinions:
Until this conversation, I was firmly of opinion 1. However, now I'm on the fence. This discussion aims to resolve this difference.
Not in question:
Beta Was this translation helpful? Give feedback.
All reactions