buffer replicas / resource request based on label

[harpratap]

Spot Nodes are very cheap but they have high risk of getting evicted at very short notice.

In order to use Spot VMs efficiently we can always have much higher replica count compared to running on OnDemand Nodes, this will solve 2 problems -

• Spread pods on more nodes so less risk of eviction of 100% of replicas at same time
• The eviction period is quite short 120seconds at best, but new pods take long time to become ready (sometimes more than 5minutes), so existing replicas should be able to handle traffic even when lot of replicas are down. Meaning CPU utilization per pod will be lower than that of OnDemand Nodes

So suggestion is to run 25% (or even 50%) more replicas and ideal scenario if Deployment is running completely on Spot Nodes

[sanposhiho]

It sounds like related to --upper-target-resource-utilization that we already have for the admin configuration:
https://github.com/mercari/tortoise/blob/main/docs/flag-configuration.md#upper-target-resource-utilization

So, we can extend this for the usecase so that the admin can specify which kind of Pods should get XXX % target at max.
Like:

upperTargetResourceUtilization:
- labelSelector:
     matchExpressions:
     - key: nodepool
        operator: In
        values:
         - spot
   upperTargetResourceUtilization: 50%
- labelSelector:
     matchExpressions:
     - key: nodepool
        operator: NotIn
        values:
         - spot
   upperTargetResourceUtilization: 80% 

(after #98)

[sanposhiho]

Another usecase is some workloads, which have strict SLO, need to have additional buffer on the top of VPA recommendation.

[sanposhiho]

So, we need to have a new configuration for that:

additionalBuffer:
- labelSelector:
     matchExpressions:
     - key: slo
        operator: In
        values:
         - "super-high"
   # either resources or resourceRatio should be configured.
   resources:
      cpu:        1         # 1 CPU is always added on the top of VPA recommendation.
      memory: 1GB    # 1GB memory is always added on the top of VPA recommendation.
   resourcesRatio:
      cpu:        10%     # 10% CPU is always added on the top of VPA recommendation.
      memory: 10%    # 10% memory is always added on the top of VPA recommendation.