Ensure priority weight is capped at 32-bit integer to prevent roll-over #43611
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Besides some adjustments because of rework of classes in regards of TeskSDK I think this can also be back-ported to Ariflow 2.10 line to prevent errors in roll-over. Therefore tagging as 2.10.4 milestone... but backport would most likely a re-write of this PR. |
|
should we have a newsfragment (just in case) just explaining what happened? |
potiuk
approved these changes
Nov 12, 2024
eladkal
approved these changes
Nov 12, 2024
jscheffl
force-pushed
the
bugfix/out-of-range-priority-weight
branch
from
c18bfe7
to
e9649e6
Compare
pierrejeambrun
approved these changes
Nov 13, 2024
ashb
reviewed
Nov 13, 2024
Comment on lines
+878
to
+880
| self.priority_weight = max( | ||
| MINIMUM_PRIORITY_WEIGHT, min(MAXIMUM_PRIORITY_WEIGHT, self.priority_weight) | ||
| ) |
There was a problem hiding this comment.
Shoud we use db_safe_priority here instead of re-impl?
There was a problem hiding this comment.
Would like to but the implementaiton currently is in airflow-utils - I asusme we want to have task SDK split apart from core?
Still what I can do is implement the same in task-SDK as utility... the old abstract/base operator in core will go away once AIP-72 is complete, correct?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is an alternative PR to #42410 via not converting the priority_weight to float but ensure no roll-over in int is happening - by capping INT values to database limits.
There are several attempts to fix this problem: