Consider removing unnecessary projection for single key join followed by .select(pl.len())

[henryharbeck]

Description

I'm unsure why the optimised plan projects 2/3 columns instead of one for this single key join followed by .select(pl.len())

This example shows a case where I believe only a single column needs to be projected. If I am missing something, or there is some other reason why this needs to happen, then feel free to close this.

a = pl.LazyFrame({"i": range(100)})
b = pl.LazyFrame({"i": range(100), "j": range(100), "k": range(100)})
print(
    a.join(b, on="i")
    .select(pl.len())
    .explain()
)

 SELECT [len()] FROM
  INNER JOIN:
  LEFT PLAN ON: [col("i")]
    DF ["i"]; PROJECT 1/1 COLUMNS; SELECTION: None
  RIGHT PLAN ON: [col("i")]
    DF ["i", "j", "k"]; PROJECT 2/3 COLUMNS; SELECTION: None    <---- can 1/3 columns be projected?
  END INNER JOIN

DuckDB only projects a single column (the i column) from both tables

query = """
CREATE OR REPLACE TABLE A AS SELECT range as i, range as j FROM range(100);
CREATE OR REPLACE TABLE B AS SELECT range as i, range as j, range as k FROM range(100);
EXPLAIN SELECT COUNT(*) FROM A JOIN B USING(i);
"""
print(duckdb.sql(query).pl()["explain_value"].item())

┌───────────────────────────┐
│    UNGROUPED_AGGREGATE    │
│    ────────────────────   │
│        Aggregates:        │
│        count_star()       │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│         HASH_JOIN         │
│    ────────────────────   │
│      Join Type: INNER     │
│     Conditions: i = i     │
│        Build Min: 0       ├──────────────┐
│       Build Max: 99       │              │
│                           │              │
│         ~102 Rows         │              │
└─────────────┬─────────────┘              │
┌─────────────┴─────────────┐┌─────────────┴─────────────┐
│         SEQ_SCAN          ││         SEQ_SCAN          │
│    ────────────────────   ││    ────────────────────   │
│             A             ││             B             │
│                           ││                           │
│       Projections: i      ││       Projections: i      │
│                           ││                           │
│         ~100 Rows         ││         ~100 Rows         │
└───────────────────────────┘└───────────────────────────┘

[ritchie46]

I think we can improve that. Currently the len triggers a read of at least the first column of the schema. We need somehow to backtrack later if that column is really required or not.