Avoid evicting queues below their fair share #2611
Conversation
Codecov ReportPatch coverage:
Additional details and impacted files@@ Coverage Diff @@
## master #2611 +/- ##
==========================================
- Coverage 58.85% 58.80% -0.06%
==========================================
Files 238 238
Lines 30551 30535 -16
==========================================
- Hits 17982 17957 -25
- Misses 11200 11213 +13
+ Partials 1369 1365 -4
Flags with carried forward coverage won't be shown. Click here to find out more.
☔ View full report in Codecov by Sentry. |
internal/armada/server/lease.go
Outdated
| @@ -469,7 +469,8 @@ func (q *AggregatedQueueServer) getJobs(ctx context.Context, req *api.StreamingL | |||
| schedulerobjects.ResourceList{Resources: totalCapacity}, | |||
| ) | |||
| for queue, priorityFactor := range priorityFactorByQueue { | |||
| if err := sctx.AddQueueSchedulingContext(queue, priorityFactor, allocatedByQueueAndPriorityClassForPool[queue]); err != nil { | |||
| weight := 1 / priorityFactor | |||
There was a problem hiding this comment.
I've changed it so that the scheduling contexts store the weight instead of the priority factor.
| @@ -490,6 +496,19 @@ func (qctx *QueueSchedulingContext) ClearJobSpecs() { | |||
| } | |||
| } | |||
|
|
|||
| // FractionOfFairShare returns a number in [0, 1] indicating what fraction of its fair share this queue is allocated. | |||
| func (qctx *QueueSchedulingContext) FractionOfFairShare() float64 { | |||
There was a problem hiding this comment.
The fair share computation is now done in the context. This is necessary now that several components (the eviction and the queue multiplexing) now need it.
| return false | ||
| } | ||
| if qctx, ok := sch.schedulingContext.QueueSchedulingContexts[job.GetQueue()]; ok { | ||
| if qctx.FractionOfFairShare() <= sch.protectedFractionOfFairShare { |
There was a problem hiding this comment.
This is the line of code preventing queues at or below their fair share from being preempted.
This PR introduces a new config
protectedFractionOfFairShare. Only queues allocated more than this fraction of their fair share are considered for preemption.┆Issue is synchronized with this Jira Task by Unito