Adaptive System Protection
Adaptive system protection maintains high system throughput under the premise of reliability of the system.
The idea of TCP BBR gives us inspiration. We should balance the requests that the system can handle and the requests that are allowed to pass, rather than relying on a single metric (system load). Our ultimate goal is to increase the throughput of the system within appropriate system load, rather than the load must be restricted below a threshold.
Sentinel's approach to system load protection is to use load1 as the metric to initiate traffic control, and the traffic allowed to pass is determined by the ability to process the request, including the response time and current QPS.
There are several kinds of global protection item:
- System load
- Global QPS
- Global average response time
- Global max thread count
The request will be blocked under the condition:
- Current system load (
load1) exceeds the threshold (highestSystemLoad); - Current concurrent requests exceeds the capacity (
thread count > minRt * maxQps)
We have a global statistic node ENTRY_NODE which records global metrics (e.g. QPS, average RT and thread count).