Slicing with Awkward Arrays versus slicing with NumPy arrays

[dr-stringfellow]

Version of Awkward Array

1.8.0rc3

Version of numpy

1.21.2

Description and code to reproduce

Casting first as numpy array makes the code crash even though there is an outer cast to ak.Array in both cases

import numpy as np
import awkward as ak

arr = ak.Array([[[1,2,3],[4,5,6],[0,0,0]],[[10,11,12],[0,0,0],[0,0,0]]])
print("First attempt -- works")
print(arr[ak.num(arr[arr > 0], axis=2) > 0])

arr = ak.Array(np.array([[[1,2,3],[4,5,6],[0,0,0]],[[10,11,12],[0,0,0],[0,0,0]]]))
print("Second attempt -- crashes")
print(arr[ak.num(arr[arr > 0], axis=2) > 0])

Error

  File "/local/usr/lib/python3.9/site-packages/awkward/operations/structure.py", line 256, in num
    out = layout.num(axis=axis)
ValueError: 'axis' out of range for 'num'

(https://github.com/scikit-hep/awkward-1.0/blob/1.8.0rc3/src/libawkward/array/NumpyArray.cpp#L1598)

[agoose77]

Hi @dr-stringfellow, thanks for filing a bug report!

This is actually a design decision, and not a bug, but I can appreciate that it very much looks like a bug at first glance! You did help me discover a bug in our next version v2, though!

Awkward Array builds upon the advanced indexing of NumPy. In NumPy, if you index with a boolean array, the result is a flat array. Because your second arr is NumPy-like (totally regular) layout, Awkward uses the NumPy indexing behavior, not Awkward's extension.

If you were to convert your array to a jagged array, then it would index properly. A jagged array is any array with var in the type, even if every sublist has the same length.

There are some discussions about making this nicer, but for now you need to ensure that arr > 0 is jagged. You can do this with multiple calls to ak.from_regular on arr, or on arr > 0. In either case, the result will be a jagged array.

I think this does demonstrate that we need to improve the docs here. I was looking for any mention of this behavior, but it is currently missing (at least, in an obvious sense)!

https://awkward-array.readthedocs.io/en/latest/_auto/ak.Array.html#nested-indexing