Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: max recursion depth exceeded #335

Merged
merged 1 commit into from
Jun 5, 2024

Conversation

zermelo-wisen
Copy link
Collaborator

Fixes #321.

  • Disables recording while executingevent.py::describe_value.
  • Prevents "Fatal Python error: Cannot recover from stack overflow." in sympy project while running tests with AppMap. This problem occurs while describing the return_value since describing Params are already protected by recording_disabled here.
  • Adds a test to ensure that calling instrumented methods as a side effect of describing the return value does not produce additional events.

@zermelo-wisen zermelo-wisen force-pushed the fix/max-recursion-depth-exceeded branch 3 times, most recently from 5a4c3a3 to e3c213a Compare June 2, 2024 14:31
Copy link
Collaborator

@apotterri apotterri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this. I think it would cleaner to fix recording_disabled. Have a look and let me know if that makes sense.

_appmap/event.py Outdated
ret.update(_describe_schema(name, val, 0, max_depth))
if any(_is_list_or_dict(type(val))):
ret["size"] = len(val)
def _describe_value():
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of doing this, let's fix recording_disabled:

@contextmanager
def recording_disabled():
    tls = appmap_tls()
    orig = tls.get("instrumentation_disabled")
    tls["instrumentation_disabled"] = True
    try:
        yield
    finally:
        tls["instrumentation_disabled"] = orig

Then you can just change describe_value to start with

def describe_value(name, val, max_depth=5):
    with recording_disabled():

Copy link
Collaborator Author

@zermelo-wisen zermelo-wisen Jun 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, I thought about this but wasn't sure if it wasn't intended to be that way. I'll change it now.

@zermelo-wisen zermelo-wisen force-pushed the fix/max-recursion-depth-exceeded branch from e3c213a to 1c4d0fa Compare June 3, 2024 11:54
Copy link
Collaborator

@apotterri apotterri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need to consider the performance impact of this change. Please see my suggestions.

@@ -15,11 +15,12 @@
@contextmanager
def recording_disabled():
tls = appmap_tls()
original_value = is_instrumentation_disabled()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seeing this led me to think a bit more about these changes.

In the work we've been going to generate large numbers of AppMaps, it's become clear that there's a significant performance penalty to running with the agent. We'll certainly need to take other steps to address this, but we it's something we should start thinking about.

Here, rather than calling is_instrumentation_disabled(), which will incur 4 or 5 extra function calls, let's just do

Suggested change
original_value = is_instrumentation_disabled()
original_value = tls.get("instrumentation_disabled")

Let's also remove with recording_disabled(): from describe_value, and add it to FuncReturnEvent.__init__. It's already here, so there's no need to call it again when creating call event.

@zermelo-wisen zermelo-wisen force-pushed the fix/max-recursion-depth-exceeded branch from 1c4d0fa to 4223079 Compare June 3, 2024 12:55
Copy link
Collaborator

@apotterri apotterri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

@apotterri apotterri merged commit 467477d into master Jun 5, 2024
2 checks passed
@apotterri apotterri deleted the fix/max-recursion-depth-exceeded branch June 5, 2024 10:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

RecursionError: maximum recursion depth exceeded while calling a Python object
2 participants