[errors-] options.debug_stacktrace_full for more callstack

[midichef]

This PR is an aid to debugging. It adds more levels to the usual exception traceback, when an option is set.

Why this is useful: when viewing an exception trace, Python only shows the stack trace to the level of the function that caught the exception. Often such an exception trace will top out with a common wrapper function like wrapply() or callNoExceptions(). And that's too ambiguous to identify the line causing a bug.

Here is some sample output to show the effect. I've split it into two parts. The top part is the new extra output, the bottom is the usual exception trace. The full trace doubles the length of the traceback, as well as having a different format. So it's good to default off for most users.

File "/home/midichef/.venv/bin/vd", line 6, in <module>
    visidata.main.vd_cli()
  File "/home/midichef/.venv/lib/python3.12/site-packages/visidata/main.py", line 382, in vd_cli
    rc = main_vd()
  File "/home/midichef/.venv/lib/python3.12/site-packages/visidata/main.py", line 344, in main_vd
    run(vd.sheets[0])
  File "/home/midichef/.venv/lib/python3.12/site-packages/visidata/vdobj.py", line 34, in _vdfunc
    return getattr(visidata.vd, func.__name__)(*args, **kwargs)
  File "/home/midichef/.venv/lib/python3.12/site-packages/visidata/extensible.py", line 65, in wrappedfunc
    return oldfunc(*args, **kwargs)
  File "/home/midichef/.venv/lib/python3.12/site-packages/visidata/extensible.py", line 65, in wrappedfunc
    return oldfunc(*args, **kwargs)
  File "/home/midichef/.venv/lib/python3.12/site-packages/visidata/extensible.py", line 65, in wrappedfunc
    return oldfunc(*args, **kwargs)
  File "/home/midichef/.venv/lib/python3.12/site-packages/visidata/mainloop.py", line 322, in run
    ret = vd.mainloop(scr)
  File "/home/midichef/.venv/lib/python3.12/site-packages/visidata/extensible.py", line 65, in wrappedfunc
    return oldfunc(*args, **kwargs)
  File "/home/midichef/.venv/lib/python3.12/site-packages/visidata/mainloop.py", line 252, in mainloop
    vd.callNoExceptions(sheet.checkCursor)

Traceback (most recent call last):
  File "/home/midichef/.venv/lib/python3.12/site-packages/visidata/mainloop.py", line 28, in callNoExceptions
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/midichef/.venv/lib/python3.12/site-packages/visidata/extensible.py", line 79, in wrappedfunc
    r = oldfunc(*args, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/midichef/.venv/lib/python3.12/site-packages/visidata/sheets.py", line 643, in checkCursor
    elif self.bottomRowIndex < self.cursorRowIndex:
         ^^^^^^^^^^^^^^^^^^^
  File "/home/midichef/.venv/lib/python3.12/site-packages/visidata/sheets.py", line 370, in bottomRowIndex
    return self.topRowIndex+self.nScreenRows-1
                            ^^^^^^^^^^^^^^^^
  File "/home/midichef/.venv/lib/python3.12/site-packages/visidata/sheets.py", line 414, in nScreenRows
    n = (self.windowHeight-self.nHeaderRows-self.nFooterRows)
                                            ^^^^^^^^^^^^^^^^
  File "/home/midichef/.venv/lib/python3.12/site-packages/visidata/sheets.py", line 427, in nFooterRows
    return len(self.allAggregators) + 1
               ^^^^^^^^^^^^^^^^^^^
  File "/home/midichef/.venv/lib/python3.12/site-packages/visidata/extensible.py", line 149, in call_if_not
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/midichef/.venv/lib/python3.12/site-packages/visidata/sheets.py", line 780, in allAggregators
    col = self.availCols[vcolidx]
          ~~~~~~~~~~~~~~^^^^^^^^^
IndexError: list index out of range

[saulpw]

Do we need an option for this? It seems like it would be fine to include the extra levels always.

[midichef]

In my own use, I'd want the extra levels always. And I'd rather not create another unnecessary option.

My concerns would be, will long stack traces be annoying or intimidating for newer users? And will they find them difficult to copy/paste for bug reports?

[saulpw]

I like the top format better, without the ^^^^. Can they both be in the same format? Also can we have the more relevant stacktrace first? It's usually the inner one, which is the second one in the example you showed.

I don't mind if these errors messages are annoying for newer users; this is one of the reasons that error sheets are bound to Ctrl+E and not Shift+E (Ctrl being more "system" focused). But I agree that longer stack traces are less ideal for copy/pasting into bug reports. As long as it's only 2-3x as large I hope it's not too onerous. Maybe we could use options.disp_expert > 1 or something? In any event let's omit the extra option.

[midichef]

Yes, we can match the format to get rid of the carets.

Since stacktraces sometimes show up on stdout of visidata, I would worry about that output being overwhelming. And a bit confusing, because if users copy the text there, they will likely not expect the unusual stack trace ordering.

What if we just print one long stacktrace (the full output of format_stack()) for all exceptions? And handle one case specially: the most common case, where errors occur in mainloop.py? We can get rid of the line ret = vd.mainloop(scr) and everything above it. It's quite a dirty solution, but it would get rid of 16 lines of stack trace.

[midichef]

To demonstrate, I've implemented these features in the commit I just pushed: caret stripping, converting the two separate stack traces to one long one, and trimming lines starting at ret = vd.mainloop(scr).

I worry about long tracebacks in particular because they're not always tucked away inside error sheets, sometimes the user sees them on stdout at exit.