Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Allow combining non base-derived objects for retries #485

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
features:
- |
Allow for callables to be combined as retry values. This will only
work when used combined with their corresponding implementation
retry objects, e.g. only async functions will work when used together
with async retry strategies.
6 changes: 4 additions & 2 deletions tenacity/asyncio/retry.py
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@ def __init__(self, *retries: typing.Union[retry_base, async_retry_base]) -> None
async def __call__(self, retry_state: "RetryCallState") -> bool: # type: ignore[override]
result = False
for r in self.retries:
result = result or await _utils.wrap_to_async_func(r)(retry_state)
result = result or (await _utils.wrap_to_async_func(r)(retry_state) is True)
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see you added is True everywhere. This seems like a different change not required. Are we sure we need this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While adding the tests I noticed that, if a wrong combination of strategies is added (combination of async strategies in a sync context), then we'd end up trying to and/or a coroutine, which hangs forever as it's never going to be resolved in the sync context. By adding the is True we directly check if the result is exactly that and avoid endlessly waiting for the coroutine.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, thinking about it a bit more... Maybe we'd rather let it hang? Not sure what's best they, without the changes it hangs forever, which is not very intuitive, and with these changes it finishes without failures but it does not run the strategy at all, giving an invalid outcome 😕

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it feasible to trap this invalid call pattern and raise an exception to make it obvious that there's a programming error?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I came up with this, it is more coupled to the actual implementations, but it now fails on invalid scenarios so it's more explicit. I used a similar check to the one already existing in __init__.py, let me know what you think @jd f8725d6

if result:
break
return result
Expand All @@ -119,7 +119,9 @@ def __init__(self, *retries: typing.Union[retry_base, async_retry_base]) -> None
async def __call__(self, retry_state: "RetryCallState") -> bool: # type: ignore[override]
result = True
for r in self.retries:
result = result and await _utils.wrap_to_async_func(r)(retry_state)
result = result and (
await _utils.wrap_to_async_func(r)(retry_state) is True
)
if not result:
break
return result
22 changes: 18 additions & 4 deletions tenacity/retry.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,15 +30,29 @@ def __call__(self, retry_state: "RetryCallState") -> bool:
pass

def __and__(self, other: "retry_base") -> "retry_all":
return other.__rand__(self)
if isinstance(other, retry_base):
# Delegate to the other object to allow for specific
# implementations, such as asyncio
return other.__rand__(self)
return retry_all(other, self)

def __rand__(self, other: "retry_base") -> "retry_all":
# This is automatically invoked for inheriting classes,
# so it helps to keep the abstraction and delegate specific
# implementations, such as asyncio
return retry_all(other, self)

def __or__(self, other: "retry_base") -> "retry_any":
return other.__ror__(self)
if isinstance(other, retry_base):
# Delegate to the other object to allow for specific
# implementations, such as asyncio
return other.__ror__(self)
return retry_any(other, self)

def __ror__(self, other: "retry_base") -> "retry_any":
# This is automatically invoked for inheriting classes,
# so it helps to keep the abstraction and delegate specific
# implementations, such as asyncio
return retry_any(other, self)


Expand Down Expand Up @@ -269,7 +283,7 @@ def __init__(self, *retries: retry_base) -> None:
self.retries = retries

def __call__(self, retry_state: "RetryCallState") -> bool:
return any(r(retry_state) for r in self.retries)
return any(r(retry_state) is True for r in self.retries)


class retry_all(retry_base):
Expand All @@ -279,4 +293,4 @@ def __init__(self, *retries: retry_base) -> None:
self.retries = retries

def __call__(self, retry_state: "RetryCallState") -> bool:
return all(r(retry_state) for r in self.retries)
return all(r(retry_state) is True for r in self.retries)
Loading