OpenDAL S3 only with 50Mbit/s Upload

[bebehei]

I'm currently testing S3 remote repositories and I'm comparing restic with rustic.

I'm doing tests with the initial backup of about 10GB and different compression schemes. After inital backup, I also do a test restore.

Now after checking my data, rustic seems to be exactly half as fast as restic.

When checking my prometheus data, it becomes clear:

• I ran 3 tests with restic (from 15:50 to 16:35)
• I ran 4 tests with rustic (from 16:45 to 18:20)
• The dataset for restic and rustic is the same. The destination is also the same.
• As you can see, I've got 500Mbit/s down and 100Mbit/s up.
• restic uses the full 100Mbit/s for the inital backup, but rustic uses only about 50Mbit/s
• no memory/CPU pressure

Is there anything I can configure for OpenDAL, which might have limited it that way? I have searched the S3 service documentation, but I cannot find anything for limit/max connections.

Is there something else, I'm overlooking?

[simonsan]

Hey, thanks for opening that topic. To understand it right, you are comparing restic+rclone vs rustic w/opendal? Would you be able to do another measurement with rustic+rclone to understand on which side the issue might be?

[simonsan]

I have done the tests with rclone, I'm stuck with 50Mbit/s.

This implies rustic+rclone, right? Because it seems it is an issue with rustic then if either rclone and opendal don't change anything.

[bebehei]

I might miss rclone settings (AFAIK there is a connection parameter or something else).

Also during backup, there is every few seconds a short CPU spike from rustic doing read+compression+encryption and then going to sleep again. It's visible in htop and also in prometheus (if you set the scrape interval to ridiculous 1s).

The limit is clearly in the arbitrary network threshold.

Are there any connection parameters I can tune in rustic itself?

[bebehei]

Oh, I've overlooked your answer.

This implies rustic+rclone, right? Because it seems it is an issue with rustic then if either rclone and opendal don't change anything.

Yes, this is right.

For me, it seems to be in the communication from rustic -> storage abstraction.

In NixOS 24.05, there is still version 0.7.0 included. I'm going to rule this out, too. I'm currently building v0.9.1 inside the provided docker container.

[bebehei]

Ok, I had been successfully now with a docker container and v0.9.1, yields full speed for about 100Mbit/s.

After building the docker container again with v0.7.0, I got the old result of 50Mbit/s. Seems like a real version problem there.

[simonsan]

Thanks for investigating this, though!