Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Retry on 503 #5938

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Conversation

TheRealFalcon
Copy link
Member

@TheRealFalcon TheRealFalcon commented Dec 20, 2024

Follow-up to #5578

Proposed Commit Message

feat(url_helper): Retry on 503 error

If the server is busy, no need to fail.
Add type hints to adjascent code paths
and necessary refactors.

Fixes GH-5577

Additional Context

Test Steps

Merge type

  • Squash merge using "Proposed Commit Message"
  • Rebase and merge unique commits. Requires commit messages per-commit each referencing the pull request number (#<PR_NUM>)

@@ -33,6 +43,7 @@
LOG = logging.getLogger(__name__)

REDACTED = "REDACTED"
ExceptionCallback = Optional[Callable[["UrlError"], bool]]
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My goal here is to unify the exception_cb interface used in readurl and wait_for_url as they are currently different.

None of the pre-existing callbacks used the "request_args" argument, and it can also be retrieved from an HTTPError if needed in the future, and calculated it would be harder in wait_for_url(), so I removed it from the callback interface and updated the call points accordingly.

return to_wait


def _handle_error(
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm hoping this wouldn't be hard to extend to also modify future request arguments if ever needed in the future...but...YAGNI

except exceptions.SSLError as e:
# ssl exceptions are not going to get fixed by waiting a
# few seconds
raise UrlError(e, url=url) from e
except exceptions.HTTPError as e:
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The isinstance() check is silly now that we can use multiple except arms

@@ -639,61 +724,60 @@ def dual_stack(
return (returned_address, return_result)


class HandledResponse(NamedTuple):
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't love this honestly, but it provides a more structured way of return stuff from the nested functions in wait_for_url()

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants