Plotting with UXarray: Design, Discussion, & Community Requests

[philipc2]

Plotting with UXarray: Design, Discussion, & Community Requests

This post will serve as a place to discuss all-things related to visualizing unstructured grids with UXarray, but specifically to discuss the design and implementation of plotting functionalities. Similar to the Redesign Discussion, it meant to showcase our plans and to engage with the community before committing to a final implementation. Critiques and requests are encouraged.

TL;DR

UXarray internally represents all Unstructured Grids in the UGRID conventions, which means we can write a standardized set of plotting routines to use these conventions. The geometry of an Unstructured Grid can be easily interpreted from these conventions, allowing us to construct geometric primitives for visualization (Polygons, Edges, Nodes). By abstracting helper functions and creating wrappers around libraries such as Datashader and GeoViews, similar plotting routines to Xarray can be achieved, but written for Unstructured Grids. See the Example API section for intended functionality.

Previous Discussion

Redesign: #185 #216

Visualization: #214 #105

My SIParCS 2022 Summer Project: PolyMesh

Background

Plotting with Xarray is done through a thin wrapper around the matplotlib library. Functionality is centered around xarray.DataArray and xarray.Dataset objects. Our intention with Uxarray is to follow a similar approach, wrapping a visualization library around our custom uxarray.DataArray and uxarray.Dataset objects. See here for a more detailed explanation of how plotting works in Xarray

Data Representation

There are many formats and conventions used for representing unstructured grids (UGRID, MPAS, SCRIP, etc). Since UXarray internally encodes all input unstructured grids in the UGRID conventions at the data loading step, only one set of plotting functions needs to be written. This means that energy can be redirected to adding support for converting different grid formats to UGRID instead of writing separate plotting routines.

Example

Say a user is attempting to visualize unstructured grid data represented in the SCRIP format. By using UXarray, the user does not need to worry about how the original data is encoded, since UXarray creates a 1:1 conversion to the UGRID conventions. They can go straight from data loading to visualization, similar to Xarray.

Unstructured Grid Geometry

Topology & Dimensions

Since the representation of an Unstructured Grid is inherently a geometric problem, the UGRID conventions outline standardized topologies for grids of different dimensions. The following table summarizes the geometries that make up an Unstructured Grid. For a more detailed description, see the UGRID conventions documentation here.

Dimensionality	Proposed Name	Comments
0	node	A point, a coordinate pair or triplet: the most basic element of the topology. The word "node" seems to be more commonly used than the alternative "vertex".
1	edge	A line or curve bounded by two nodes.
2	face	A plane or surface enclosed by a set of edges. In a 2D horizontal application one may consider the word "polygon", but in the hierarchy of elements the word "face" is most common.
3	volume	A volume enclosed by a set of faces. The alternative word "cell" was considered, but "cell" is often used in the community to describe 2 dimensional structures.

This post will focus around a pair of 2 dimensional topologies: 2D Triangular Mesh (Only Triangles) and 2D Flexible Mesh
(Mixed Triangles, Quadrilaterals, etc).

2D Topology

For both the Triangular and Flexible Mesh Topologies, there are only two variables needed to construct the geometry of our grid.

node_coordinates (x, y): Coordinates representing the location of each node

face_node_connectivity: Indices used to construct a face from nodes

By using these variables, we can create geometric primitives (Points, Edges, Polygons) that can be used to represent our Unstructured Grid. More advanced visualizations could utilize other optional UGRID variables, but this discussion will highlight the essential routines that can be derived from nodes and connectivity.

Expected Visualizations

Since Unstructured Grids can be broken down into their geometric primitives, our visualization functions will be centered around them. The following descriptions are for the core set of visualizations functions

Nodes

Each node is plotted using it's coordinates. Equivalent to a 2D scatter-plot of all node locations.

Edges

Nodes are connected using the face node connectivity, but they are not filled with any data.

Faces

Same as edges, but each face is filled with it's corresponding data

Visualization Challenges

The most complex step in visualizing Unstructured Grids is converting from raw data (Coordinates and Connectivity) into a format that is interpretable by visualization libraries. Additionally, for spatial applications handling edges cases where a 2D geometry breaks or slows down is also needed (Antimeridian Crossings and Coordinate Transforms). Our goal is to abstract most of this under the hood with UXarray. The user should not need to worry about the underling data wrangling, cleaning, or processing.

Core Libraries

While Xarray is built around Matplotlib, we believe that building UXarray around the HoloViz stack of tools would be more appropriate, due to the need to visualize often large amounts of geometries (Polygons, Edges). For large Unstructured Grids, the rasterization support from Datashader allows us to quickly render large datasets. Listed below are the libraries required to make this visualization workflow possible

Visualization

HoloViews & Datashader: Core visualization libraries that will be wrapped. Can be configured to create plots using matplotlib or bokeh extensions.

GeoViews: Provides similar functionality to Cartopy, but built for the HoloViz stack of tools.

Geometry

Shapely 2.0: Core geometry library. Provides high-performance functions for creating geometric primitives.

Spatialpandas: Pandas and Dask extension for vectorized geometric applications used for visualizing data with HoloViz tools. Build around the spatialpandas.GeoDataFrame, which holds our geometry.

Example API

Plotting will be done directly through uxarray.DataArray and uxarray.Dataset objects. Visualizations that do not depend on any data variables will be done through the uxarray.Dataset object and similarly visualizations that use data will be done by indexing a uxarray.Dataset to obtain a uxarray.DataArray

Setup

import uxarray as ux
import cartopy.crs as ccrs
import geoviews.feature as gf

# grid and data file paths
grid_path = "/path/to/grid.nc"
data_path = "/path/to/data.nc"

# grid and data files stored in a ux.dataset
uxds = ux.open_dataset(grid_path, data_path)

Grid Exclusive Plotting Routines (Dataset)

# outline of the edges that construct a face
uxds.plot.edges()

# 2D scatter of node coordinates
uxds.plot.nodes()

# defaults to plot.edges()
uxds.plot()

Grid & Data Plotting Routines (DataArray)

# filled nodes if data lies on nodes
uxds['v1'].plot.nodes()

# faces are represented as filled polygons
uxds['v1'].plot.faces()

# same as above with geographic projection
uxds['v1'].plot.faces(projection=ccrs.Robinson())

# same as above, but rasterized
uxds['v1'].plot.faces(raster=True, projection=ccrs.Robinson())

# defaults to plot.faces()
uxds['v1'].plot()

Example Visualizations

Faces

uxds['v1'].plot.faces(projection=ccrs.Robinson()) * gf.borders
# could also do uxds['v1'].plot(projection=ccrs.Robinson()) *  gf.borders

uxds['v1'].plot.faces(projection=ccrs.Geostationary()) * gf.borders

Edges

uxds.plot.edges()
# could also do uxds.plot()

Nodes

uxds.plot.nodes()

Rasterization

Datashader provides support for rasterization, which maps our vector image (Polygons) onto a structured (Rasterized) grid/image, trading detail for performance.

uxds['v1'].plots(raster=False) * gf.borders

uxds['v1'].plots(raster=True) * gf.borders

Future Work

The work discussed here will represent the initial plotting functionality supported by UXarray. Support for more routines and methods of visualization (Triangulation, Voronoi Diagrams, etc.) are other possibilities that could be implemented in the future.