Code

evgenii/fix limit rewrite#100

Code

Closed

ewgenius wants to merge 49 commits intodatafusion-contrib:maindatafusion-contrib/datafusion-federation:mainfrom spiceai:evgenii/fix-limit-rewritespiceai/datafusion-federation:evgenii/fix-limit-rewriteCopy head branch name to clipboard

+5,168-619

Commits

Commits on May 9, 2024

• Update to DataFusion 37.1

  phillipleblanc

  committed
  a25a523

  Copy full SHA for a25a523

Commits on May 10, 2024

• Allow TableScans of non-federated sources

  phillipleblanc

  committed
  a852cd3

  Copy full SHA for a852cd3

• .clone() -> Arc::clone(&

  phillipleblanc

  committed
  c81a766

  Copy full SHA for c81a766

• Update Cargo.toml license to Apache-2.0

  phillipleblanc

  committed
  c6c9542

  Copy full SHA for c6c9542

• Merge branch 'main' into spiceai

  phillipleblanc

  committed
  4808d74

  Copy full SHA for 4808d74

Commits on May 13, 2024

• Upgrade to DataFusion 38

  phillipleblanc

  committed
  5d9b98f

  Copy full SHA for 5d9b98f

Commits on May 14, 2024

• Add fallback TableProvider to FederatedTableProviderAdaptor

  phillipleblanc

  committed
  3de50a9

  Copy full SHA for 3de50a9

• Make connectorx optional

  phillipleblanc

  committed
  0fadae1

  Copy full SHA for 0fadae1

Commits on May 16, 2024

• Rewrite all table scans to use federated table name (#4)

  Show description for 8ba03bd
  phillipleblanc

  authored
  8ba03bd

  Copy full SHA for 8ba03bd

Commits on May 20, 2024

• Rewrite subquery table scans to point to remote table (#5)

  Show description for 8fcda49
  phillipleblanc

  authored
  8fcda49

  Copy full SHA for 8fcda49

Commits on May 24, 2024

• Fix the fallback to the table provider for supports_filters_pushdown

  phillipleblanc

  committed
  1c77d8c

  Copy full SHA for 1c77d8c

• Use SpiceAI datafusion fork

  phillipleblanc

  committed
  b221f4e

  Copy full SHA for b221f4e

Commits on May 27, 2024

• Fix the table scan rewrite to properly rewrite column relations (#3)

  Show description for f5d4797
  phillipleblanc

  authored
  f5d4797

  Copy full SHA for f5d4797

• Handle table rewrites for aggregation functions

  phillipleblanc

  committed
  532ecd0

  Copy full SHA for 532ecd0

Commits on May 29, 2024

• Update datafusion version

  phillipleblanc

  committed
  f5d20bb

  Copy full SHA for f5d20bb

• Use Unparser Dialect for plan_to_sql

  phillipleblanc

  committed
  65d4be2

  Copy full SHA for 65d4be2

• Use unparser Dialect

  phillipleblanc

  committed
  6d714cb

  Copy full SHA for 6d714cb

• bump datafusion fork

  phillipleblanc

  committed
  9dc7404

  Copy full SHA for 9dc7404

Commits on May 30, 2024

• Add layer to convert actual schema to expected schema

  phillipleblanc

  committed
  bf5fac6

  Copy full SHA for bf5fac6

Commits on Jun 11, 2024

• Update datafusion-federation to datafusion 39 (#6)

  phillipleblanc

  authored
  4ea973e

  Copy full SHA for 4ea973e

Commits on Jun 13, 2024

• Improve rewrite logic for expressions

  Show description for e85aa96
  

  sgrebnov

  authored and

  phillipleblanc

  committed
  e85aa96

  Copy full SHA for e85aa96

Commits on Jun 24, 2024

• update spiceai datafusion to include new unparsers logic (#8)

  y-f-u

  authored
  15d4bd9

  Copy full SHA for 15d4bd9

Commits on Jun 26, 2024

• Update datafusion to include time predicate fix (#9)

  sgrebnov

  authored
  776930f

  Copy full SHA for 776930f

Commits on Jun 27, 2024

• Remove table reference mutating during logical planning, converting query when executing (#10)

  Show description for c3f696a
  y-f-u

  authored
  c3f696a

  Copy full SHA for c3f696a

Commits on Jul 8, 2024

• Update SpiceAI Datafusion fork

  phillipleblanc

  committed
  1becf0c

  Copy full SHA for 1becf0c

Commits on Jul 15, 2024

• Upgrade to DF 40

  phillipleblanc

  committed
  77548d3

  Copy full SHA for 77548d3

• Bump datafusion-federation version to 0.1.5

  phillipleblanc

  committed
  f430401

  Copy full SHA for f430401

• do not over eager rewrite column when col relation is there (#12)

  Show description for eeb9b9c
  

  y-f-u

  and

  phillipleblanc

  authored
  eeb9b9c

  Copy full SHA for eeb9b9c

Commits on Aug 14, 2024

• Upgrade to DataFusion v41

  phillipleblanc

  committed
  9b4392f

  Copy full SHA for 9b4392f

Commits on Aug 15, 2024

• Use Display for displaying federated plans

  phillipleblanc

  committed
  ae9da45

  Copy full SHA for ae9da45

Commits on Aug 22, 2024

• Make the FederatedPlanner public for use in custom query planners (#14)

  phillipleblanc

  authored
  21cfcc8

  Copy full SHA for 21cfcc8

Commits on Aug 25, 2024

• Add List types parsing support to schema cast (#13)

  Show description for 2cb3330
  sgrebnov

  authored
  2cb3330

  Copy full SHA for 2cb3330

• Add interval cast given original schema (#15)

  Show description for e0a4eae
  

  3 peopleauthored
  e0a4eae

  Copy full SHA for e0a4eae

Commits on Aug 26, 2024

• Add Structs parsing support to schema cast (#16)

  Show description for 21f07be
  sgrebnov

  authored
  21f07be

  Copy full SHA for 21f07be

Commits on Aug 30, 2024

• feat: Add AST analyzer middleware (#19)

  Show description for b668294
  peasee

  authored
  b668294

  Copy full SHA for b668294

Commits on Sep 18, 2024

• Upgrade to DF 42 & Arrow 53

  phillipleblanc

  committed
  7c69f7b

  Copy full SHA for 7c69f7b

Commits on Oct 9, 2024

• Preserve records batch order when SchemaCastScanExec is involved (#20)

  sgrebnov

  authored
  f1e7b17

  Copy full SHA for f1e7b17

Commits on Oct 17, 2024

• Support for rewriting plans with UNNEST (#21)

  sgrebnov

  authored
  06b9000

  Copy full SHA for 06b9000

Commits on Oct 18, 2024

• Correctly handle unnest by FederationAnalyzerRule (#22)

  sgrebnov

  authored
  ce1ce61

  Copy full SHA for ce1ce61

Commits on Oct 21, 2024

• Filters push down optimization for federation queries (#23)

  sgrebnov

  authored
  0aabd42

  Copy full SHA for 0aabd42

Commits on Nov 4, 2024

• Run optimize_projections as part of federated plan optimization (#24)

  sgrebnov

  authored
  914bd08

  Copy full SHA for 914bd08

Commits on Nov 12, 2024

• Upgrade to DF 43

  phillipleblanc

  committed
  5af0df8

  Copy full SHA for 5af0df8

Commits on Nov 29, 2024

• Add own modified version of Datafusion optimize_projections rule (#25)

  sgrebnov

  authored
  bbc0f89

  Copy full SHA for bbc0f89

Commits on Dec 23, 2024

• Federation analyzer should federate the subquery when possible (#26)

  Show description for 04487ba
  Sevenannn

  authored
  04487ba

  Copy full SHA for 04487ba

Commits on Dec 24, 2024

• Fix unnest rewriting logic (#28)

  sgrebnov

  authored
  72374f6

  Copy full SHA for 72374f6

Commits on Dec 26, 2024

• fix: clean up outdated dependencies & modules and fix tests (#32)

  Show description for 884b2a5
  Sevenannn

  authored
  884b2a5

  Copy full SHA for 884b2a5

Commits on Dec 27, 2024

• fix: rewrite shouldn't be performed on a column name same as the table name (#33)

  Show description for ce91d5a
  Sevenannn

  authored
  ce91d5a

  Copy full SHA for ce91d5a

Commits on Dec 30, 2024

• fix: don't rewrite table names inside subquery when engine doesn't support it (#29)

  Show description for 4c0ff79
  Sevenannn

  authored
  4c0ff79

  Copy full SHA for 4c0ff79

Commits on Jan 2, 2025

• Handle LogicalPlan::Limit separately to preserve skip and offset in rewrite_table_scans

  ewgenius

  committed
  5832471

  Copy full SHA for 5832471