Support multiple proxy servers in Forwarded header parsing

[lukasbindreiter]

Related Issue(s):
(none, since I directly created the Pull request for the fix)

Description:
According to the Forwarded HTTP header spec, multiple proxy servers can add values to the Forwarded header by creating a comma separated list:

If there are multiple proxy servers between the client and server, they may each specify their own forwarding information. This can be done by adding a new Forwarded header to the end of the header block, or by appending the information to the end of the last Forwarded header in a comma-separated list.

The existing ProxyHeaderMiddleware however doesn't account for this, and instead raises a ValueError in such cases, because it only splits the string on ; but not on , before that.

Therefore, a valid Forwarded header of b"proto=https;host=test:1234,proto=https;host=second-server:1111" produces a: ValueError: too many values to unpack (expected 2) because it essentially runs:

part = "host=test:1234,proto=https"  # because it doesn't split on `,` first
key, value = part.split("=")  # ValueError: too many values to unpack (expected 2)

I've encountered this when deploying our STAC API server on GCP Cloud run, behind a GCP Load balancer.
GCP doesn't even actually set the host but the middleware in place still causes all requests to fail with a ValueError.

(The header on GCP that we receive is b'for="85.193.181.55";proto=https,for="85.193.181.55";proto=https')

PR Checklist:

• pre-commit hooks pass locally
• Tests pass (run make test)
• Documentation has been updated to reflect changes, if applicable, and docs build successfully (run make docs)
• Changes are added to the CHANGELOG.

[vincentsarago]

@lukasbindreiter hope you don't mind, I've pushed some change directly in your branch

Let me know how it looks for you and then I could merge and release this tomorrow

[lukasbindreiter]

@lukasbindreiter hope you don't mind, I've pushed some change directly in your branch

Let me know how it looks for you and then I could merge and release this tomorrow

@vincentsarago
Awesome, thanks a lot!

I've made some small changes to it:

• use compiled regexes
• avoid groupdict and instead call group directly to avoid unnecessary dict creations

(worth it I think in this case, its an easy optimization thats twice as fast)

[lukasbindreiter]

@vincentsarago everything looks ready from my side - would be awesome if you could take another look, and then merge it if everything is fine. Thanks a lot!

(I also just rebased it, there was a conflicting changelog entry already upstream)

[vincentsarago]

This is great @lukasbindreiter 🙏

I'll merge this to main and maybe backport it to a 3.0.6 release if you absolutely need this in a 3.0.* version (next version in main has breaking change and will be released in a 3.1 or 4.0)

[lukasbindreiter]

Awesome, thanks a lot! 🙏

It's fine if its in 3.1 or 4.0 - I already have the fixed version of the middleware inplace locally in my project and can just use that until this is released.