Allow to add/switch components of a published local repository

[cfiehe]

Fixes #1355, #1338

Description of the Change

This commit modifies the behavior of the publish switch method in the way, that it can also be used to add/update components of published local repositories. Furthermore, the commit ensures that the sources returned by a published repository are ordered by their component names.

Checklist

• functional test added/updated (if change is functional)
• man page updated (if applicable)
• documentation updated
• author name in AUTHORS
• make sure slash can be used in prefix (Update Publish does not suport slash in prefix #1338)
• swagger documentation

[neolynx]

changing the logging is a bit tricky, since the aptly output is used extensively in the testing. but I like the cleanup !

if you wanna update the tests with the new logs automatically, uncomment CAPTURE in the Makefile locally and run the system-tests (it will write new gold files) ;-)

[cfiehe]

Thanks a lot. I was not aware of the consequences. Unfortunately, we must change the output of the switch command and make it more generic, so that it can be also used for published local repositories. It requires some time for fixing the test cases.

[neolynx]

amazing work :)

I sent you an invitation as maintainer, it would be great to have more of your contribution as well as reviewing some PRs (i.e. #1352) to move things forward a bit faster !

[neolynx]

I dont have the rights to push to your branch, but here would be the golang-lint fix: eb15a65 :-)

[neolynx]

there is one more log update in a S3 test (where your i does not have creds): https://github.com/aptly-dev/aptly/actions/runs/11153050817/job/30999844378#step:9:1456

[cfiehe]

@neolynx
Have you got an idea, why the test execution is failing?

--- PASS: Test (0.01s)
PASS
coverage: 74.9% of statements
ok  	github.com/aptly-dev/aptly/utils	0.011s	coverage: 74.9% of statements
FAIL

Stopping etcd ...

Unit Tests FAILED

[cfiehe]

I dont have the rights to push to your branch, but here would be the golang-lint fix: eb15a65 :-)

Ok, that is strange. The flag that the maintainer has editing rights is set.

[neolynx]

coverage 76.85% of diff hit (target 74.90%)

nice !

[cfiehe]

It finally turns into green 😅. Thanks a lot for your support.

[cfiehe]

That would be an alternative REST-API but it is not so RESTful, because we actually model actions and it has the problem, that switch is currently done by via PUT on /api/publish/{prefix}/{distribution}.

/api/publish/{prefix}/{distribution}/add [post]
/api/publish/{prefix}/{distribution}/switch [post]
/api/publish/{prefix}/{distribution}/remove [post]
/api/publish/{prefix}/{distribution}/update [post]

or

/api/publish/{prefix}/{distribution}/sources [post, get] - Single source creation, List sources
/api/publish/{prefix}/{distribution}/sources-bulk [post] - Multiple operations (Creates, Updates, Removes)
/api/publish/{prefix}/{distribution}/update [post]

We model /sources-bulk as bulk operation resource allowing to execute multiple operations on more than one source in a single request. It would be enough to only implement /sources-bulk.

Inspired by:

• https://github.com/NationalBankBelgium/REST-API-Design-Guide/wiki/Bulk-operations-Resources-naming-convention
• https://www.mscharhag.com/api-design/bulk-and-batch-operations)

What do you think?

Maybe we can keep the old way to switch a snapshot of a published snapshot repository for some time by using PUT on /api/publish/{prefix}/{distribution} and give a deprecation warning, we could do the same for triggering a package update for a published local repository. We could add MultiDist and make it modifiable like SkipContents and Co.

[cfiehe]

I have implemented the new publish add command and used the same CLI style, which is used for publish switch. add succeeds only when the component does not already exist in the published repository. I have modified the publish switch command, so that you can also switch the local repositories of a published local repository. The manual might need an extension.

I have simplified the REST API and introduced the new Sources attribute. Now a simple PUT with the entire source list triggers the update of the published repository. The only drawback is that in most cases the client needs to query the resource first in order to get the full source list. With the new published repository resource, which allows you to query a single published repository, the overhead becomes much smaller. In the next step, the source list is modified, so that it represents the entire target state. Sources, which are not in the list, but exist in the published repository, are removed. I think this approach is more RESTful and fits best in Aptly's REST API. There is still the possibility to add something in the future, which is smarter and may avoid the GET request.

In the next day, I will have a look at the (Swagger) documentation and add/update what is missing.

[neolynx]

typo