Revert unproject_layout's action; use shape_touched in column optimization.
#317
Conversation
| @@ -95,6 +95,7 @@ def test_json_bytes_no_delim_defined(ndjson_points_file: str) -> None: | |||
|
|
|||
|
|
|||
| def test_to_and_from_dask_array(daa: dak.Array) -> None: | |||
| daa = dak.from_awkward(daa.compute(), npartitions=3) | |||
There was a problem hiding this comment.
the normal daa fixture is an array that comes from parquet, and the form ends up getting a mask when we do column selection. We avoid the mask by going ahead and computing the parquet-originating collection and creating a new collection from the concrete result via from_awkward. A bit roundabout but we still test what we mean to test here.
| @@ -273,6 +274,8 @@ def test_from_lists(caa_p1: ak.Array) -> None: | |||
| def test_to_dask_array(daa: dak.Array, caa: dak.Array) -> None: | |||
| from dask.array.utils import assert_eq as da_assert_eq | |||
|
|
|||
| daa = dak.from_awkward(daa.compute(), npartitions=4) | |||
There was a problem hiding this comment.
same as above
|
Busy with other things today - but I can give it a shot later. |
of course always no rush :) |
douglasdavis
force-pushed
the
revert-unproject-layout
branch
from
1ab3212
to
b7f3b69
Compare
Codecov Report
❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more. @@ Coverage Diff @@
## main #317 +/- ##
==========================================
- Coverage 92.31% 90.82% -1.49%
==========================================
Files 21 21
Lines 2564 2562 -2
==========================================
- Hits 2367 2327 -40
- Misses 197 235 +38
📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
douglasdavis
force-pushed
the
revert-unproject-layout
branch
from
b7f3b69
to
1482d37
Compare
|
@douglasdavis so this gets coffea tests to pass again but overtouching is has run rampant again in my usual torture tests, compared to dask-awkward 2023.7.0 + uproot 5.0.10 (where in my test with this branch I'm running this branch with uproot 5.0.10). |
|
Closing in favor of #319 |
Should fix #314 and #313.
This reverts the action that unproject_layout and plugs the hole left behind by using the
shape_touchedattribute of the typetracer report. The only negative side-effect we have is converting todask.array.Arrayis failing in tests because of a mismatch between a regular numpy array and a masked numpy array.