Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove calls to GetRequirements where possible #2608

Merged

Conversation

zuqq
Copy link
Collaborator

@zuqq zuqq commented Jun 26, 2023

The implementation of GetRequirements for api.Job allocates new structs, which we'd like to avoid:

armada/pkg/api/util.go

Lines 109 to 145 in fac9411

func (job *Job) GetRequirements(priorityClasses map[string]configuration.PriorityClass) *schedulerobjects.JobSchedulingInfo {
podSpec := job.GetMainPodSpec()
priority, ok := PriorityFromPodSpec(podSpec, priorityClasses)
if priorityClasses != nil && !ok {
// Ignore this error if priorityByPriorityClassName is explicitly set to nil.
// We assume that in this case the caller is sure the priority does not need to be set.
err := errors.Errorf("unknown priorityClassName %s", podSpec.PriorityClassName)
logging.WithStacktrace(logrus.NewEntry(logrus.New()), err).Error("failed to get priority from priorityClassName")
}
preemptionPolicy := string(v1.PreemptLowerPriority)
if podSpec.PreemptionPolicy != nil {
preemptionPolicy = string(*podSpec.PreemptionPolicy)
}
podRequirements := &schedulerobjects.PodRequirements{
NodeSelector: podSpec.NodeSelector,
Affinity: podSpec.Affinity,
Tolerations: podSpec.Tolerations,
Annotations: maps.Clone(job.Annotations),
Priority: priority,
PreemptionPolicy: preemptionPolicy,
ResourceRequirements: job.GetResourceRequirements(),
}
return &schedulerobjects.JobSchedulingInfo{
PriorityClassName: podSpec.PriorityClassName,
Priority: LogSubmitPriorityFromApiPriority(job.GetPriority()),
SubmitTime: job.GetCreated(),
ObjectRequirements: []*schedulerobjects.ObjectRequirements{
{
Requirements: &schedulerobjects.ObjectRequirements_PodRequirements{
PodRequirements: podRequirements,
},
},
},
}
}

Many of the calls to GetRequirements (or PodRequirementFromLegacySchedulerJob) don't actually need the entire JobSchedulingInfo; I've replaced them by more granular access. There are still a couple of calls to PodRequirementFromLegacySchedulerJob in parts of the code that deal with the node database; I'd like to get rid of those separately.

I've also renamed GetRequirements to GetJobSchedulingInfo.

@codecov
Copy link

codecov bot commented Jun 26, 2023

Codecov Report

Patch coverage: 61.11% and project coverage change: +0.03 🎉

Comparison is base (f2cdeab) 58.78% compared to head (3b8d2d6) 58.81%.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2608      +/-   ##
==========================================
+ Coverage   58.78%   58.81%   +0.03%     
==========================================
  Files         238      238              
  Lines       30568    30542      -26     
==========================================
- Hits        17970    17964       -6     
+ Misses      11223    11209      -14     
+ Partials     1375     1369       -6     
Flag Coverage Δ
armada-server 58.81% <61.11%> (+0.03%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
internal/armada/server/lease.go 7.56% <0.00%> (ø)
internal/scheduler/jobdb/job.go 78.70% <12.50%> (-2.64%) ⬇️
internal/scheduler/preempting_queue_scheduler.go 65.66% <50.00%> (+0.52%) ⬆️
internal/scheduler/common.go 86.23% <100.00%> (+10.45%) ⬆️
internal/scheduler/context/context.go 33.67% <100.00%> (-0.59%) ⬇️
internal/scheduler/jobiteration.go 91.17% <100.00%> (ø)
internal/scheduler/scheduling_algo.go 72.47% <100.00%> (ø)

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@zuqq zuqq merged commit df9ad30 into master Jun 26, 2023
@zuqq zuqq deleted the zuqq/remove-calls-to-PodRequirementFromLegacySchedulerJob branch June 26, 2023 14:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants