add duckdb support

[ahuang11]

Closes #1397

import hvplot.duckdb

import pandas as pd
import duckdb

df = pd.DataFrame({
    'x': [1, 2, 3],
    'y': [4, 5, 6]
})

df.to_parquet('test.parquet')

connection = duckdb.connect(database=':memory:', read_only=False)
connection.from_parquet("test.parquet").hvplot(
    x='x', y='y', kind='scatter'
)

[codecov]

Codecov Report

Attention: Patch coverage is 93.20388% with 7 lines in your changes missing coverage. Please review.

Project coverage is 88.86%. Comparing base (efeda78) to head (9ed44f2).

Files with missing lines	Patch %	Lines
hvplot/converter.py	33.33%	2 Missing ⚠️
hvplot/duckdb.py	87.50%	2 Missing ⚠️
hvplot/tests/testpatch.py	89.47%	2 Missing ⚠️
hvplot/plotting/core.py	96.87%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1398      +/-   ##
==========================================
+ Coverage   88.73%   88.86%   +0.12%     
==========================================
  Files          51       52       +1     
  Lines        7592     7694     +102     
==========================================
+ Hits         6737     6837     +100     
- Misses        855      857       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[MarcSkovMadsen]

Is the long term goal to implement a real duckdb backend for more efficient usage?

[ahuang11]

Can you elaborate on real duckdb backend?

This PR doesn't change the converter at all; only patches duckdb so DuckDBPyRelation.hvplot() is available.

It does not do DuckDBPyRelation.fetchdf().hvplot() AFAIK.

[MarcSkovMadsen]

Yes.

Is the plan to implement a real HoloViz backend such that you only query the data needed for the plot? For example when using groupby.

[ahuang11]

I'm not sure I follow; this PR allows hvplot off of a DuckDBRelation, which is already the result of a query.

connection > relation (select * from table where ...) > hvplot

connection.execute("SELECT * FROM table WHERE col = 'A'").hvplot()

Ah I see the confusion:

connection.from_parquet("test.parquet").hvplot(
    x='x', y='y', kind='scatter'
)

Can be drilled down further

connection.from_parquet("test.parquet").execute(...).hvplot(
    x='x', y='y', kind='scatter'
)

[ahuang11]

Was slightly mistaken, but a little modification works:
https://github.com/holoviz/hvplot/pull/1398/files#diff-a47979cba5da76fbc1aec07d45b4403241da5cfbbbf6b36652f75c52fb644824R360

Not sure if we should manually add the optimization to subset x/y columns (e.g. relation.select(x, y), or let the user do it themselves.

[ahuang11]

Oh I guess it is converting to pandas object first:
lambda self: hvPlotTabular(self.df())

https://github.com/holoviz/hvplot/pull/1398/files#diff-be8c4d86a0e1601a53aca277191c3a5d8e160fc39c2b4d75396c83b6a38ae610R15

Edit: Now it does what you proposed

[maximlt]

I'm ok on adding DuckDB support. It looks like you should update the diagram on the landing page too, unless you consider this experimental first.

[maximlt]

@ahuang11 can you also update this PR based on #1359 now it has been merged?

[ahuang11]

Okay I think this is ready. I added the diagrams + added changes from #1359

[maximlt]

Oh have you just replaced Intake by DuckDB? Have you recently removed Intake support?

[hoxbro]

I think he has moved intake up.

Some notes:

• Arrow are missing
• Matplotlib is not straight.

[maximlt]

Gosh I'm blind!

[maximlt]

dask.gif shows bokeh.sampledata.penguins.

Suggested change

	```{image} ./_static/home/dask.gif
	```{image} ./_static/home/pandas.gif