Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Adds retries to ECS task run creation for ECS worker #303

Merged
merged 3 commits into from
Aug 7, 2023

Conversation

desertaxle
Copy link
Member

@desertaxle desertaxle commented Aug 7, 2023

Adds tenacity to retry failed attempts to create a task run for the ECS worker.

Currently, if an ECS task for a flow run cannot be placed in the specified cluster, then the flow run will be marked as crashed. This change will enable the ECS worker to retry the failed task run placement to reduce the likely hood of flow runs crashing due to failed task placement.

Closes #301

Checklist

  • References any related issue by including "Closes #" or "Closes ".
    • If no issue exists and your change is not a small fix, please create an issue first.
  • Includes tests or only affects documentation.
  • Passes pre-commit checks.
    • Run pre-commit install && pre-commit run --all locally for formatting and linting.
  • Includes screenshots of documentation updates.
    • Run mkdocs serve view documentation locally.
  • Summarizes PR's changes in CHANGELOG.md

@desertaxle desertaxle requested a review from a team as a code owner August 7, 2023 18:59
Copy link
Contributor

@chrisguidry chrisguidry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Woot, never seen tenacity before, I like it

@desertaxle desertaxle merged commit 2e17708 into main Aug 7, 2023
6 checks passed
@desertaxle desertaxle deleted the retry-create-task-run branch August 7, 2023 19:22
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ECSWorker fails to submit tasks to cluster
2 participants