Awkward version 1.x has a conflict (just a warning?) with Numba 0.53

[jpivarski]

Version of Awkward Array

1.x

Description and code to reproduce

In scikit-hep/vector#388, a test based on Awkward 1.x is failing:

FAILED tests/backends/test_awkward_numba.py::test - numba.core.errors.LoweringError: Failed in nopython mode pipeline (step: native lowering)
Failed in nopython mode pipeline (step: nopython frontend)
Code using Numba extension API maybe depending on 'old_style' error-capturing, which is deprecated and will be replaced by 'new_style' in a future release. See details at https://numba.readthedocs.io/en/latest/reference/deprecation.html#deprecation-of-old-style-numba-captured-errors
Exception origin:
  File "/opt/hostedtoolcache/Python/3.11.5/x64/lib/python3.11/site-packages/awkward/_connect/_numba/arrayview.py", line 877, in generic
    raise TypeError(


File "tests/backends/test_awkward_numba.py", line 23:
    def extract(x):
        return x[2][0]
        ^

During: lowering "$20binary_subscr.4 = static_getitem(value=$8binary_subscr.2, index=0, index_var=$const18.3, fn=<built-in function getitem>)" at /home/runner/work/vector/vector/tests/backends/test_awkward_numba.py (23)

Awkward 1.x should either fix this (Coffea 0.7 users are using it, and Coffea uses Numba) or put an upper cap on Numba versions, which would be difficult because Numba is a soft/runtime dependency.

The warning about old-style and new-style Numba error capturing might be like #2659, which fixed the issue for Awkward 2.x. Maybe all this needs is for the Numba-handling in Awkward 1.x to get the same kind of fix.

It will require a new Awkward 1.x release, which will take some thinking to remember how to do.

[Saransh-cpp]

There is another failing test in the same CI run related to awkward v1, but I'm not sure if the failure is a vector problem or an awkward problem -

FAILED tests/test_issues.py::test_issue_161 - numba.core.errors.LoweringError: Failed in nopython mode pipeline (step: native lowering)
Failed in nopython mode pipeline (step: nopython frontend)
Code using Numba extension API maybe depending on 'old_style' error-capturing, which is deprecated and will be replaced by 'new_style' in a future release. See details at https://numba.readthedocs.io/en/latest/reference/deprecation.html#deprecation-of-old-style-numba-captured-errors
Exception origin:
  File "/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/awkward/_connect/_numba/arrayview.py", line 877, in generic
    raise TypeError(


File "tests/test_issues.py", line 38:
    def repro(generator_like_jet_constituents):
        <source elided>
            s = 0
            for generator_like_constituent in sublist:
            ^

During: lowering "$18for_iter.2 = iternext(value=$phi18.1)" at /home/runner/work/vector/vector/tests/test_issues.py (38)
.
.
.
E                 File "/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/awkward/_connect/_numba/arrayview.py", line 877, in generic
E                   raise TypeError(
E               
E               
E               File "tests/test_issues.py", line 38:
E                   def repro(generator_like_jet_constituents):
E                       <source elided>
E                           s = 0
E                           for generator_like_constituent in sublist:
E                           ^
E               
E               During: lowering "$18for_iter.2 = iternext(value=$phi18.1)" at /home/runner/work/vector/vector/tests/test_issues.py (38)

/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/numba/core/errors.py:846: LoweringError
.
.
.
>       raise NotImplementedError("No definition for lowering %s%s" % (key, sig))
E       NotImplementedError: No definition for lowering static_getitem(ak.ArrayView(ak.RecordArrayType((ak.NumpyArrayType(array(int64, 1d, A), none, {}), ak.NumpyArrayType(array(int64, 1d, A), none, {})), (('x', 'y')), none, {"__record__": "Vector2D"}), None, ()), Literal[int](0)) -> VectorObject2DType(AzimuthalObjectXY(int64 x 2))

[ianna]

@Saransh-cpp - I hope PR #2735 fixes the issue, but I haven't tested it with vector yet

[Saransh-cpp]

Thanks, @ianna!! Your branch fixed everything in vector; the CI is all green (red for Python 3.12 because of distutils, but should be fine once awkward has a new release) - https://github.com/scikit-hep/vector/actions/runs/6406416975

[jpivarski]

It's good to know that the Numba changes fixed the Vector tests, but just running the tests again after so much time revealed other breaking changes from third-party libraries. I've taken over the PR and have found one change in NumExpr and one change in pyarrow that break our test suite. I'll only merge it and make an Awkward 1.x release after our own tests pass.

I know that that's blocking this PR on vector; I don't know if there's time pressure to get this out. Maybe not—the original reason for the issue was to support Python 3.12, and third-party libraries like Numba don't yet. (I hear from the Numba team that supporting Python 3.12 is likely to take less time than Python 3.11 took. Currently, some 94% of Numba's test suite is passing in Python 3.12, which is a good sign. Still, I think I can fix these Awkward 1.x issues on a shorter time-scale than that.)