Add fragmentation strategies

[KasperZutterman]

Possible fragmentation strategies that can be added:

• chronological:
  • normal
  • reverse
  • day per day (range e.g. per week)
• lexicographical (alphabetical):
  • normal
  • reverse
  • per substring
• geographical
• object version
• filter

[pietercolpaert]

We’re working on bucketizers which simply fragment an LDES streamingly and add a ldes:bucket predicate to the member with one or more buckets this member can be stored at. I would love a generic Bucketizer fragmentation strategy in here, in which you can configure the name of an NPM bucketizer package (such as the substring bucketizer) and the LDES action will work. This way, no new release of the LDES action would be needed for another fragmentation strategy to work :-)

[KasperZutterman]

This may be possible using the Dynamic import() expression.

[pietercolpaert]

I don’t think we’d need dynamic import, as we can just add a step in our workflow to also install the right bucketizer, no?

[ddvlanck]

@pietercolpaert : installing a bucketizer package through configuration is not (yet) possible because GitHub Action requires the build result (dist folder) to be pushed onto GitHub. It uses that folder to run the action and the npm packages are already bundled in that folder, and thus restricting a bucketizer to be added at runtime.

[pietercolpaert]

Ok, then we need to fallback to just having a bucketizer repo that keeps a package.json with all known bucketizers as dependencies and exports them?

[ddvlanck]

We already have a repository with our bucketizers: https://github.com/TREEcg/bucketizers. So, instead of creating another repo, I suggest to add a new package (e.g. bucketizer-repo) with that functionality.

[pietercolpaert]

Or just bucketizers?