Feature suggestion: lzham for lzma-like ratios and up to 8x faster decompression

[M-Gonzalo]

Quoting its readme:

Public Domain

LZHAM is a lossless data compression codec..., with a compression ratio similar to LZMA but with 1.5x-8x faster decompression speed. It officially supports Linux x86/x64

Pros:
. Strong compression
. Fast decompression (still slower than zstd AFAICT)

Cons:
. Comparatively slow compression (although not really impractical, except for the 'extreme' setting, which really lives up to its name and should never be used)

dwarfs being a read-only filesystem designed for a create-once, read-often use case, slow compression shouldn't be that much of an issue.

[Phantop]

Just some quick testing (on a single archive to be fair) and it compresses a fair bit faster than zstd using the default options, ended up only slightly bigger, but did have only slightly slower decompression. Might be some merit to it but it does seem rather interchangeable with zstd.

[M-Gonzalo]

Just some quick testing (on a single archive to be fair) and it compresses a fair bit faster than zstd using the default options, ended up only slightly bigger, but did have only slightly slower decompression. Might be some merit to it but it does seem rather interchangeable with zstd.

Maybe it's a good idea to try the higher settings, since its intended target is actually lzma. But I see how it can be competitive with zstd 22, which is a fairly extreme setting that spends a lot of time trying to optimize the compression without compromising on d-speed. In that sense, they are very similar, yes. Both zstd-...20,21,22 and lzham trade compression speed for decompression speed, only zstd trades a bit more ratio (you don't get any better than that) and lzham trades more c-speed (it can get extremely slow on the highest setting of all).
So maybe there is some sort of continuity to be arranged from zstd->lzham. According to your tests, we can replace the last zstd mode with the default lzham, and from there try higher settings

[Phantop]

My build says that the default is the maximum...

[M-Gonzalo]

Right! It follows 7-zip naming conventions, where maximum is actually not the maximum because after it comes "ultra"
Anyway, on GUI, the ultra level is pathologically slow, but I think the command line options allow for finer settings for better compression than "maximum", without getting into the "extreme" mode of ultra

[M-Gonzalo]

Ok I've been tinkering around with the command line options and it seems it doesn't get any better than the "maximum" level without dropping to 4x slower compression. And like you said, marginally better compression than zstd, with slower decompression speed. Maybe the "ultra" -mx=9 option is not too slow on modern machines?

[mhx]

I'll move this to discussions as it's not really in issue.

[mhx]

I've actually played around with lzham a little and while it compresses well and is certainly fast, it's not that it blows any of the other algorithms out of the water. The biggest issue for me is that it's pretty much abandoned by the author and thus unmaintained. There doesn't seem to be much interest in maintaining it either. So at this point it's highly unlikely that it'll be included.