confighttp: Including lz4 as supported compression encoding

[MovieStoreGuy]

Description

Adding in support for lz4 compression into the project, I have seen it perform amazing well in other projects that send a lot of traffic and were originally using gzip or flate.

It would be really great to have for environments that are cpu, memory sensitive.

Link to tracking issue

Fixes 9128

Testing

I've updated the existing test frameworks to include lz4 in their suite

Documentation

I've updated the readmes to include lz4 but I suspect there might be more places to update.

[MovieStoreGuy]

Fixes #9128

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 91.92%. Comparing base (8bc3eae) to head (e529edf).
Report is 3 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main   #11108   +/-   ##
=======================================
  Coverage   91.91%   91.92%           
=======================================
  Files         430      430           
  Lines       20313    20327   +14     
=======================================
+ Hits        18671    18685   +14     
  Misses       1268     1268           
  Partials      374      374           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[jpkrohling]

This looks good. I reviewed the library's code as well, and I don't think it would be affected by a situation similar to the CVE we had a couple of months ago. That said, would you be able to do a positive confirmation? The test would be to encode a huge blob that compressed to a tiny payload, and see if the decoding would explode the collector.

[MovieStoreGuy]

This looks good. I reviewed the library's code as well, and I don't think it would be affected by a situation similar to the CVE we had a couple of months ago. That said, would you be able to do a positive confirmation? The test would be to encode a huge blob that compressed to a tiny payload, and see if the decoding would explode the collector.

So this somewhat blew out the original PR intent but, I also noticed that snappy would also be subject to a similar attack considering the steps it would take:

1. Create a buffer to write uncompress information to
2. Read the compressed body and write to uncompressed buffer
3. return uncompressed buffer

To work around this, I wrote a small wrapper that effectively delays the read until it is actually needed however, since we are no longer reading the entire compressed body, that buffer is also coupled with the compressed reader. This allows the wrapper on close to discard any remaining compressed values and return the result of the original buffer's closed state.

[MovieStoreGuy]

Turns out I don't actually need the wrapper, I could move the logic into the ServeHTTP method.

I will clean up the commit history once the pipeline is happy.

[jpkrohling]

Looks great now!