-
Notifications
You must be signed in to change notification settings - Fork 883
Performance Tuning
BDP (byte) = Link speed (bit/s) * RTT (s) / 8 (bit/byte). Under the default effect of net.ipv4.tcp_adv_win_scale, BDP * 2 should be a good setting for maximum receive buffer size.
Example: Assuming 1Gbps link with 256ms RTT, it's a 32MiB maximum window size requiring 64MiB maximum buffer size. Add to sysctl.conf:
- (Client only)
net.ipv4.tcp_rmem = 4096 131072 67108864 - (Server only)
net.ipv4.tcp_wmem = 4096 131072 67108864.
BBR should be able to auto-tune the window size to reduce bufferbloat. Assuming large download and small upload, client-side net.ipv4.tcp_wmem and server-side net.ipv4.tcp_rmem can be left as default. net.core.rmem_max and net.core.wmem_max are not used in window size auto-tuning and can be left as default.
- https://blog.cloudflare.com/the-story-of-one-latency-spike/ tells us large buffer sizes are not always better.
- https://blog.cloudflare.com/optimizing-tcp-for-high-throughput-and-low-latency
sudo sysctl -w net.ipv4.tcp_congestion_control=bbr
HTTP/2 uses a single connection per host. TCP tuning is no longer needed to reduce bufferbloat.
sudo sysctl -w net.ipv4.tcp_slow_start_after_idle=0
This setting can improve persistent single connection performance slightly.
- https://hpbn.co/building-blocks-of-tcp/
- https://github.com/http2/http2-spec/wiki/Ops
- https://dropbox.tech/infrastructure/optimizing-web-servers-for-high-throughput-and-low-latency
sudo sysctl -w net.ipv4.tcp_notsent_lowat=131072
This setting can improve interactive latency by optimizing send buffer handling. Note that this kernel setting is useful for HTTP/2 and may be detrimental to other applications, as it applies to all applications on the server.
Be cautious with this setting. Small values tend to increase server CPU usage and negatively affect throughput. You need to benchmark it to determine good settings. You can check the HTTP/2 load time in https://http2.akamai.com/demo while also downloading a large file with and without this setting.
Common values range from 16KB to 128KB.
- https://blog.cloudflare.com/http-2-prioritization-with-nginx/
- https://web.archive.org/web/20170924184144/https://insouciant.org/tech/prioritization-only-works-when-theres-pending-data-to-prioritize/
- (Benchmarking HTTP/2 Priorities) https://docs.google.com/document/d/1oLhNg1skaWD4_DtaoCxdSRN5erEXrH-KnLrMwEpOtFY/preview
Its Linux implementation is too conservative to be useful, and its usage is rare in practice thus creating a distinct traffic feature.
- https://blog.donatas.net/blog/2017/03/09/tfo/
- https://squeeze.isobar.com/2019/04/11/the-sad-story-of-tcp-fast-open/
Chromium limits the number of connections per proxy to 32. New connections that exceed this limit will be stalled, but multi-tab browsing often needs more than 32 connections. Create a policy to override this limit.
Note: This is a browser setting, not applicable to any proxies.
- For Chromium on Linux:
sudo mkdir -p /etc/chromium/policies/managed
echo '{ "MaxConnectionsPerProxy": 99 }' | sudo tee /etc/chromium/policies/managed/proxy.json
- For Chrome on macOS:
defaults write com.google.Chrome MaxConnectionsPerProxy -int 99
and restart Chrome to update the policy.
You should be able to see it in chrome://policy once set up.
99 is the maximum value for MaxConnectionsPerProxy allowed by Chromium. It is still too low. You are recommended to use an ad-blocker to save on connections.
For Chrome or other OSes, see: