Rollout dashboard fixes. #21
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
**Fix 1** When rollout dashboard asks for a rollout's task instances, rollout dashboard attempts to retrieve the tasks in a paged way to avoid overcoming Airflow's baked-in 100-item limit. For some reason, when tasks are retrieved, the result is unordered, so when page 2, 3, 4 of task instances are fetched, some task instances are missing and some task instances are duplicated. apache/airflow#41283 explains the issue in detail. The impact to us is that, sometimes, subnets have no associated state, despite the fact that Airflow clearly has progressed on that subnet. To mitigate this, we transgress the limitation and ask for 500 task instances. This seems to work fine, much to my surprise! With that, the issue of rollout subnets sometimes not having any associated state is fixed. **Fix 2** The URL is always present in the rollout data structure, so stop pretending it is not, and remove unnecessary conditional. **Fix 3** Subnet state icons gain a hyperlink to the pertinent running, failed, or last completed Airflow task instance (specifically, to the log tab). This way, one can directly access the log by clicking on the subnet state icon. This improves usability of the dashboard and speed to diagnose an issue that may be occurring during a rollout.
LittleChimera
approved these changes
Aug 6, 2024
|
Thanks @LittleChimera -- stand by for rollout PR please. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fix 1
When rollout dashboard asks for a rollout's task instances, rollout dashboard attempts to retrieve the tasks in a paged way to avoid overcoming Airflow's baked-in 100-item limit. For some reason, when tasks are retrieved, the result is unordered, so when page 2, 3, 4 of task instances are fetched, some task instances are missing and some task instances are duplicated. apache/airflow#41283 explains the issue in detail. The impact to us is that, sometimes, subnets have no associated state, despite the fact that Airflow clearly has progressed on that subnet.
To mitigate this, we transgress the limitation and ask for 500 task instances. This seems to work fine, much to my surprise! With that, the issue of rollout subnets sometimes not having any associated state is fixed.
Fix 2
The URL is always present in the rollout data structure, so stop pretending it is not, and remove unnecessary conditional.
Fix 3
Subnet state icons gain a hyperlink to the pertinent running, failed, or last completed Airflow task instance (specifically, to the log tab). This way, one can directly access the log by clicking on the subnet state icon.
This improves usability of the dashboard and speed to diagnose an issue that may be occurring during a rollout.