You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
deanm0000 opened this issue
Sep 23, 2024
· 1 comment
Labels
A-ioArea: reading and writing databugSomething isn't workingP-mediumPriority: mediumperformancePerformance issues or improvementspythonRelated to Python PolarsrustRelated to Rust Polars
parquet file must be read, statistics not sufficient for predicate.
parquet file must be read, statistics not sufficient for predicate.
parquet file must be read, statistics not sufficient for predicate.
Issue description
This is separate from but highly related to #18867. Even when using a file written by pyarrow where the statistics are correct, predicate pushdown doesn't work.
If I try to explicitly make the rhs a Categorical then I simply don't get a verbose message at all so I'm not sure if it's silently working or not working.
deanm0000
changed the title
Parquet scanner doesn't do predicate pushdown for categoricals
Parquet scanner doesn't do predicate pushdown for categoricals/enums
Sep 23, 2024
A-ioArea: reading and writing databugSomething isn't workingP-mediumPriority: mediumperformancePerformance issues or improvementspythonRelated to Python PolarsrustRelated to Rust Polars
Checks
Reproducible example
Log output
Issue description
This is separate from but highly related to #18867. Even when using a file written by pyarrow where the statistics are correct, predicate pushdown doesn't work.
If I try to explicitly make the rhs a Categorical then I simply don't get a verbose message at all so I'm not sure if it's silently working or not working.
Even with a StringCache still no verbosity.
Expected behavior
Partition pruning should work
Installed versions
The text was updated successfully, but these errors were encountered: