Versioning for Definitions

[simonharrer]

Definitions should have a version as well.

[mschloesser]

In our scenario, we have a bunch of Parquet files in a S3 bucket. The schema evolved over time in terms of data type changes, newly added columns etc. From my understanding I don't have the ability to reflect those changes, neither in the server.location property, nor the model definition itself.

Is this what you mean when saying

Definitions should have a version as well.

?

Even if I have the ability to tag a model with a certain version (e.g. by introducing a property), I also need to be able to to tell the checker which S3 objects match the which model/schema version. As stated above, in our case we might have n files that match version 1.0.0 and m files that match version 1.1.0 all stored under the same S3 path. However, we don't have the ability to specify exactly which of these files match the 1.0.0 definition and which the 1.1.0.

I hope this issue was the right one for this discussion and my description makes sense.