Improve x-limit, y-limit, and rasterize support for geo features #1135

hoxbro · 2023-09-14T15:32:00Z

Fixes #1093
Fixes #1090
Fixes https://discourse.holoviz.org/t/any-smart-approach-for-determining-what-holoviz-package-broke-my-notebook/5944

If geo is true, we will use GeoViews tile sources, and if false, we will use HoloViews. GeoViews features and tiles are projected to the respective projection.

1093 code:

import cartopy.crs as ccrs
import geoviews as gv
import hvplot.xarray  # noqa
import numpy as np
import xarray as xr


def sample_data(shape=(20, 30)):
    """
    Return ``(x, y, u, v, crs)`` of some vector data
    computed mathematically. The returned crs will be a rotated
    pole CRS, meaning that the vectors will be unevenly spaced in
    regular PlateCarree space.

    """
    crs = ccrs.RotatedPole(pole_longitude=177.5, pole_latitude=37.5)

    x = np.linspace(311.9, 391.1, shape[1])
    y = np.linspace(-23.6, 24.8, shape[0])

    x2d, y2d = np.meshgrid(x, y)
    u = 10 * (2 * np.cos(2 * np.deg2rad(x2d) + 3 * np.deg2rad(y2d + 30)) ** 2)
    v = 20 * np.cos(6 * np.deg2rad(x2d))

    return x, y, u, v, crs


xs, ys, U, V, crs = sample_data()

mag = np.sqrt(U**2 + V**2)
angle = (np.pi / 2.0) - np.arctan2(U / mag, V / mag)

ds = xr.Dataset(
    {
        "mag": xr.DataArray(mag, dims=("y", "x"), coords={"y": ys, "x": xs}),
        "angle": xr.DataArray(angle, dims=("y", "x"), coords={"y": ys, "x": xs}),
    },
    attrs={"crs": crs},
)
ds.hvplot.vectorfield(
    x="x",
    y="y",
    angle="angle",
    mag="mag",
    hover=False,
    geo=True,
    tiles="CartoLight",
)

1090 code:

import hvplot.pandas
import pandas as pd

df = pd.DataFrame({"lon": [0, 10], "lat": [40, 50], "v": [0, 1]})

(
    df.hvplot.points(x="lon", y="lat", s=200, c="y", geo=True, tiles="CartoLight")
    * df.hvplot.points(x="lon", y="lat", c="v", geo=True)
)

Code from #1053 (Comments are from original PR; it seems that every case works)

import xarray as xr
import hvplot.xarray

dz = xr.tutorial.open_dataset('air_temperature')
dz.hvplot('lon', 'lat', coastline=True).opts(ylim=(20, 30), xlim=(-160, -140))

Case 1:

import hvplot.pandas  # noqa
import cartopy.crs as ccrs
import geoviews as gv
from bokeh.sampledata.airport_routes import airports

# Case 1: This example works
airports.hvplot.points(
    "Longitude",
    "Latitude",
    color="red",
    alpha=0.2,
    xlim=(-180, -30),
    ylim=(0, 72),
    geo=True,
    tiles="OSM",
)

# Case 2: xlim/ylim are not obeyed when the map tile is removed
airports.hvplot.points(
    "Longitude",
    "Latitude",
    geo=True,
    color="red",
    alpha=0.2,
    xlim=(-180, -30),
    ylim=(0, 72),
)

# Case 3: Setting ymin > 0 raises errors
# Change ymin from 0 to 10
airports.hvplot.points(
    "Longitude",
    "Latitude",
    geo=True,
    color="red",
    alpha=0.2,
    xlim=(-180, -30),
    ylim=(10, 72),
)

# Case 4: Setting ymax < 0 also raises errors
airports.hvplot.points(
    "Longitude",
    "Latitude",
    geo=True,
    color="red",
    alpha=0.2,
    xlim=(-180, -30),
    ylim=(-20, -5),
)

It works but gives a long empty plot, with one red dot.

# Works if geo=False
airports.hvplot.points(
    "Longitude",
    "Latitude",
    geo=False,
    color="red",
    alpha=0.2,
    xlim=(-180, -30),
    ylim=(10, 72),
)

hvplot/converter.py

ahuang11 · 2023-10-02T15:16:31Z

This looks good. I just wanted to ask whether this should this work (xlim/ylim in hvplot call), or out of scope in this PR?

import hvplot.xarray
import xarray as xr
import cartopy.crs as ccrs

ds = xr.tutorial.open_dataset("air_temperature")
ds.hvplot.points(
    "lon", "lat", crs=ccrs.PlateCarree(), rasterize=True, features=["coastline", "land"],
    xlim=(-140, -80), ylim=(20, 45)
)

This works:

import hvplot.xarray
import xarray as xr
import cartopy.crs as ccrs

ds = xr.tutorial.open_dataset("air_temperature")
ds.hvplot.points(
    "lon", "lat", crs=ccrs.PlateCarree(), rasterize=True, features=["coastline", "land"],
).opts(xlim=(-140, -80), ylim=(20, 45))

ahuang11 · 2023-10-02T15:20:58Z

Okay, confusingly sometimes xlim & ylim should be in kwargs, sometimes in opts.

This doesn't work (xlim, ylim in opts with tiles=True)

import hvplot.xarray
import xarray as xr
import cartopy.crs as ccrs

ds = xr.tutorial.open_dataset("air_temperature")
ds.hvplot.points(
    "lon", "lat", crs=ccrs.PlateCarree(), tiles=True
).opts(xlim=(-140, -80), ylim=(20, 45))

But this works:

import hvplot.xarray
import xarray as xr
import cartopy.crs as ccrs

ds = xr.tutorial.open_dataset("air_temperature")
ds.hvplot.points(
    "lon", "lat", crs=ccrs.PlateCarree(), tiles=True, xlim=(-140, -80), ylim=(20, 45)
)

hvplot/converter.py

ahuang11 · 2023-10-02T15:28:44Z

Not sure if this is out of scope of this PR, but this isn't correct: (with rasterize=True + projection)

import hvplot.xarray
import xarray as xr
import cartopy.crs as ccrs

ds = xr.tutorial.open_dataset("air_temperature")
ds.hvplot.points(
    "lon", "lat", crs=ccrs.PlateCarree(), features=["coastline", "land"],
    xlim=(-140, -80), ylim=(20, 45), projection=ccrs.LambertConformal(), rasterize=True
)

Dropping rasterize=True works:

import hvplot.xarray
import xarray as xr
import cartopy.crs as ccrs

ds = xr.tutorial.open_dataset("air_temperature")
ds.hvplot.points(
    "lon", "lat", crs=ccrs.PlateCarree(), features=["coastline", "land"],
    projection=ccrs.LambertConformal()
)

hoxbro · 2023-10-02T15:52:06Z

rasterize and datashade with geographic operations are out of scope for this PR. Though I will give it a try, but I won't spend too long on it.

At some point, I want it only to be needed to pass limits directly into hvplot like hvplot(..., xlim=(...) ylim=(...).

hoxbro · 2023-10-02T16:48:17Z

@ahuang11, can you try to play around with the latest commit?

import hvplot.xarray
import xarray as xr
import cartopy.crs as ccrs

ds = xr.tutorial.open_dataset("air_temperature")
ds.hvplot.points(
    "lon",
    "lat",
    crs=ccrs.PlateCarree(),
    features=["coastline", "land"],
    xlim=(-140, -80),
    ylim=(20, 45),
    projection=ccrs.LambertConformal(),
    rasterize=True,
    dynspread=True,
)

ahuang11 · 2023-10-02T16:55:14Z

Thanks! Most of the things I tried worked. This still doesn't work:

import hvplot.xarray
import xarray as xr
import cartopy.crs as ccrs

ds = xr.tutorial.open_dataset("air_temperature")
ds.hvplot.points(
    "lon", "lat", crs=ccrs.PlateCarree(), projection=ccrs.GOOGLE_MERCATOR, tiles=True
).opts(xlim=(-140, -80), ylim=(20, 45))

Dropping tiles works.

hoxbro · 2023-10-02T17:06:39Z

But putting xlim and ylim inside the hvplot call works?

ahuang11 · 2023-10-02T17:07:25Z

Yes

hoxbro · 2023-10-02T17:18:27Z

I think this is more of a problem with GeoViews with projected overlays as this will also zoom in to small numbers:

import cartopy.crs as ccrs
import geoviews as gv

ds = xr.tutorial.open_dataset("air_temperature")
plot = gv.Points(ds.isel(time=0), kdims=["lon", "lat"]).opts(projection=ccrs.GOOGLE_MERCATOR)
(plot * plot).opts(xlim=(-140, -80), ylim=(20, 45))

ahuang11 · 2023-10-02T17:20:07Z

Right, I'm not sure how it gets handled without overlays, but xlim/ylim is not being projected properly in opts when there's an overlay.

hoxbro · 2023-10-02T17:22:19Z

My example above works fine if there is no overlaid: plot.opts(xlim=(-140, -80), ylim=(20, 45)).

I don't think we can do anything on the hvplot side of things.

ahuang11 · 2023-10-02T17:24:24Z

It might be related to this:
holoviz/geoviews#340

ahuang11 · 2023-10-02T17:25:21Z

One last comment is can we rename proj_crs to be more descriptive (like feature_projection)?

hvplot/converter.py

hoxbro · 2023-10-02T17:31:42Z

I don't mind a better name, but I'm not sure if it's correct to call it feature_projection. It is used for the features but also for other calculations like finding limits and the geo projection.

ahuang11 · 2023-10-02T17:33:21Z

maybe output_projection?

ahuang11 · 2023-10-02T17:57:43Z

So I'm using the latest released version of hvplot:

I wonder how image is handled under the hood (i.e. why does this work)

import hvplot.xarray
import xarray as xr
import cartopy.crs as ccrs

ds = xr.tutorial.open_dataset("air_temperature")
ds.hvplot.image(
    "lon", "lat", crs=ccrs.PlateCarree(), features=["coastline", "land"],
    projection=ccrs.LambertConformal(), rasterize=True
)

vs points (but not this?):

import hvplot.xarray
import xarray as xr
import cartopy.crs as ccrs

ds = xr.tutorial.open_dataset("air_temperature")
ds.hvplot.points(
    "lon", "lat", crs=ccrs.PlateCarree(), features=["coastline", "land"],
    projection=ccrs.LambertConformal(), rasterize=True
)

hoxbro · 2023-10-02T18:00:32Z

The latest release of hvplot has problems with xlimit and ylimit, which is what this PR is trying to fix.

ahuang11 · 2023-10-02T18:00:43Z

All in all, I think this PR generally improves geo support!

Do you think a test should be added for rasterize + points?

hoxbro · 2023-10-02T18:01:48Z

Do you think a test should be added for rasterize + points?

Yes! But I didn't want to do it before we agreed this was the right approach.

ahuang11 · 2023-10-02T18:01:53Z

The latest release of hvplot has problems with xlimit and ylimit, which is what this PR is trying to fix.

Right; I suppose I should have commented under obj = gv.project(obj, projection=self.proj_crs); my question is why does a rasterized image project properly but not rasterized points.

hoxbro · 2023-10-02T18:12:44Z

why does a rasterized image project properly but not rasterized points

I don't know. But if I move around the image on the latest release it disappears, which does not happen after this PR.

ahuang11 · 2023-10-02T18:16:07Z

Okay; I think after tests are added, this can be merged.

hoxbro added 4 commits September 14, 2023 17:27

Improve xlimit and ylimit support for geo features

a0dddc9

Update tests

c049c7b

Don't use projection on tiles

f2ce55b

Update notebook

a273b5e

maximlt requested a review from ahuang11 October 2, 2023 08:23

Merge branch 'main' into geo_lim_improvements

c3785cf

hoxbro force-pushed the geo_lim_improvements branch from 29174a7 to c3785cf Compare October 2, 2023 10:11

maximlt added this to the 0.9.0 milestone Oct 2, 2023

ahuang11 reviewed Oct 2, 2023

View reviewed changes

hvplot/converter.py Outdated Show resolved Hide resolved

Apply suggestion

2f8cc55

ahuang11 reviewed Oct 2, 2023

View reviewed changes

hvplot/converter.py Outdated Show resolved Hide resolved

hoxbro added 2 commits October 2, 2023 18:10

Add regression test

57d6197

Try to support geo + rasterize

a9a6dff

ahuang11 reviewed Oct 2, 2023

View reviewed changes

hvplot/converter.py Outdated Show resolved Hide resolved

Rename to output_projection

d3b1dbb

Add more if statement

ad396ce

Add geo + rasterize test

c6ade58

ahuang11 approved these changes Oct 2, 2023

View reviewed changes

hoxbro changed the title ~~Improve x-limit and y-limit support for geo features~~ Improve x-limit, y-limit, and rasterize support for geo features Oct 3, 2023

hoxbro merged commit bf34dd1 into main Oct 3, 2023
7 checks passed

hoxbro deleted the geo_lim_improvements branch October 3, 2023 06:19

hoxbro mentioned this pull request Oct 4, 2023

Revert geo + rasterize changes introduced in 1135 #1163

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve x-limit, y-limit, and rasterize support for geo features #1135

Improve x-limit, y-limit, and rasterize support for geo features #1135

hoxbro commented Sep 14, 2023 •

edited

Loading

ahuang11 commented Oct 2, 2023 •

edited

Loading

ahuang11 commented Oct 2, 2023 •

edited

Loading

ahuang11 commented Oct 2, 2023

hoxbro commented Oct 2, 2023

hoxbro commented Oct 2, 2023

ahuang11 commented Oct 2, 2023

hoxbro commented Oct 2, 2023

ahuang11 commented Oct 2, 2023

hoxbro commented Oct 2, 2023

ahuang11 commented Oct 2, 2023

hoxbro commented Oct 2, 2023

ahuang11 commented Oct 2, 2023

ahuang11 commented Oct 2, 2023

hoxbro commented Oct 2, 2023

ahuang11 commented Oct 2, 2023

ahuang11 commented Oct 2, 2023 •

edited

Loading

hoxbro commented Oct 2, 2023

ahuang11 commented Oct 2, 2023

hoxbro commented Oct 2, 2023

ahuang11 commented Oct 2, 2023

hoxbro commented Oct 2, 2023

ahuang11 commented Oct 2, 2023

Improve x-limit, y-limit, and rasterize support for geo features #1135

Improve x-limit, y-limit, and rasterize support for geo features #1135

Conversation

hoxbro commented Sep 14, 2023 • edited Loading

ahuang11 commented Oct 2, 2023 • edited Loading

ahuang11 commented Oct 2, 2023 • edited Loading

ahuang11 commented Oct 2, 2023

hoxbro commented Oct 2, 2023

hoxbro commented Oct 2, 2023

ahuang11 commented Oct 2, 2023

hoxbro commented Oct 2, 2023

ahuang11 commented Oct 2, 2023

hoxbro commented Oct 2, 2023

ahuang11 commented Oct 2, 2023

hoxbro commented Oct 2, 2023

ahuang11 commented Oct 2, 2023

ahuang11 commented Oct 2, 2023

hoxbro commented Oct 2, 2023

ahuang11 commented Oct 2, 2023

ahuang11 commented Oct 2, 2023 • edited Loading

hoxbro commented Oct 2, 2023

ahuang11 commented Oct 2, 2023

hoxbro commented Oct 2, 2023

ahuang11 commented Oct 2, 2023

hoxbro commented Oct 2, 2023

ahuang11 commented Oct 2, 2023

hoxbro commented Sep 14, 2023 •

edited

Loading

ahuang11 commented Oct 2, 2023 •

edited

Loading

ahuang11 commented Oct 2, 2023 •

edited

Loading

ahuang11 commented Oct 2, 2023 •

edited

Loading