Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Interactive downsampling large vector fields (example gallery) #1372

Merged
merged 6 commits into from
Sep 13, 2024

Conversation

iuryt
Copy link
Contributor

@iuryt iuryt commented Jul 18, 2024

I made a promise almost a year ago to create this example and here I am.
The idea came from those discussions from discourse.
https://discourse.holoviz.org/t/highres-quiver-plot-returning-memory-error/6261/26
https://discourse.holoviz.org/t/dealing-high-resolution-vector-plots-xarray-holoviz/6310

Direct plotting large vector fields can quickly consume excessive memory, leading to MemoryError. To address this issue, we introduce a dynamic downsampling technique that enables interactive exploration of vector fields without sacrificing performance.

fd3a258f5d11b3ca780f190ed4a1af01d2f67f19

@maximlt
Copy link
Member

maximlt commented Jul 18, 2024

Hi @iuryt, it looks like pre-commit has caught that the notebook hasn't been cleaned up before being pushed to Github. You need to install pre-commit as described in the developer guide (and run it manually with pre-commit run --all in this case, the next times you commit locally won't to do that).

@iuryt
Copy link
Contributor Author

iuryt commented Jul 19, 2024

Sorry. Gonna fix that.

Copy link

codecov bot commented Jul 22, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 88.80%. Comparing base (6c96c7e) to head (9708d55).
Report is 25 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1372      +/-   ##
==========================================
+ Coverage   87.39%   88.80%   +1.40%     
==========================================
  Files          50       51       +1     
  Lines        7490     7510      +20     
==========================================
+ Hits         6546     6669     +123     
+ Misses        944      841     -103     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@iuryt
Copy link
Contributor Author

iuryt commented Jul 23, 2024

@maximlt
Let me know if this is fine now. Thanks!

Copy link
Member

@maximlt maximlt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a few suggestions, very nice addition!

doc/reference/xarray/vectorfield.ipynb Outdated Show resolved Hide resolved
doc/reference/xarray/vectorfield.ipynb Show resolved Hide resolved
doc/reference/xarray/vectorfield.ipynb Outdated Show resolved Hide resolved
"\n",
"# Create interactive plot components\n",
"range_xy = hv.streams.RangeXY() # Stream to capture range changes\n",
"filtered = hv.DynamicMap(create_quiver, streams=[range_xy]) # Dynamic plot\n",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Name it perhaps downsampled_quiver?

doc/reference/xarray/vectorfield.ipynb Outdated Show resolved Hide resolved
"cell_type": "markdown",
"metadata": {},
"source": [
"If we just try to call `ds.hvplot.vectorfield` this is probably returning `MemoryError`. The alternative is to dynamically downsample the data based on the visible range. This helps manage memory consumption when dealing with large datasets, especially when plotting vector fields."
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would be nice to link to a HoloViews user guide and explain that we're going to use some HoloViews concepts to work around the memory error, while stile using hvPlot to create the quiver.

iuryt and others added 2 commits July 26, 2024 10:34
Co-authored-by: Maxime Liquet <35924738+maximlt@users.noreply.github.com>
@maximlt maximlt added this to the 0.11.0 milestone Sep 13, 2024
@maximlt maximlt merged commit efeda78 into holoviz:main Sep 13, 2024
9 checks passed
@iuryt
Copy link
Contributor Author

iuryt commented Sep 17, 2024

Thanks @maximlt !
Sorry for not wrapping this up before.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants