Expanded geo testing

docs/source/whatsnew/releases/v0.4.0.rst

@@ -13,15 +13,17 @@ Scenarios
 * Added geospatial bounding box support. Clip unwanted data with rectangular bounding boxes. See: ``geospatial.apply_bounding_box``
 * Added stochastic non-uniform density downselection function to preferentially select for mountains (higher point density with changes in elevation, lower density when flatter.) See:  ``geospatial.elevation_stochastic_downselection``
 * Updated non-uniform downselection with thresholding and non-linear normalization (support for logarithmic and exponential normalization) See: ``geospatial.identify_mountains_weights``
-* Added Scenario and GeospatialScenario methods for quick plotting and downselection. See: ``GeospatialScenario.plot_coords``, ``GeospatialScenario.plot_meta_classification``, ``GeospatialScenario.plot_USA``, ``Scenario.extract``, ``Scenario.plot``, ``GeospatialScenario.classify_mountain_weights``, ``GeospatialScenario.classify_mountain_radii``, ``GeospatialScenario.downselect_elevation_stochastic``
+* Added Scenario and GeospatialScenario methods for quick plotting and downselection. See: ``GeospatialScenario.plot_coords``, ``GeospatialScenario.plot_meta_classification``, ``GeospatialScenario.plot_USA``, ``Scenario.extract``, ``Scenario.plot``, ``GeospatialScenario.classify_mountain_weights``, ``GeospatialScenario.classify_mountain_radii``, ``GeospatialScenario.downselect_elevation_stochastic``.  
+* Added a convenience method ``GepspatialScenario.geospatial_data`` to quickly pull the geospatial weather and metadata from a scenario. Matches the API for ``pvdeg.weather.get``.  
 
 Geospatial Improvements  
 
-* Autotemplating system for geospatial analysis using `pvdeg.geospatial.autotemplate`
+* Autotemplating system for geospatial analysis using `pvdeg.geospatial.autotemplate`.  
 * New module `pvdeg.decorators` that contains `pvdeg` specific decorator functions.
 * Implemented `geospatial_result_type` decorator to update functions and preform runtime introspection to determine if a function is autotemplate-able.
 * `Geospatial Templates.ipynb` notebook to showcase new and old templating functionality for users.
-* Implemented testing for geospatial analysis
+* Implemented testing for geospatial analysis.  
+* Added chunked and unchunked testing.  
 
 Symbolic Evaluation  
 
@@ -40,6 +42,7 @@ Bug Fixes
 * Replaced deprecated numba `jit(nopython=True)` calls with `njit`
 * Fix incorrect keyword arguments in `pvdeg.standards.T98_estimate`
 * Fixed ``pvdeg.temperature.module`` and ``pvdeg.temperature.cell`` docstring return type. Correct return type ``pd.Series``
+* Fixed broken Geospatial Analysis, when using chunked (dask) xarrays for weather data
 
 Dependencies
 ------------
@@ -52,4 +55,4 @@ Dependencies
 Contributors
 ~~~~~~~~~~~~
 * Tobin Ford (:ghuser:`tobin-ford`)
-* Mike Kempe (:ghuser:`MDKempe`)
+* Mike Kempe (:ghuser:`MDKempe`)

pvdeg/geospatial.py

@@ -27,7 +27,6 @@
 from typing import Tuple
 from shapely import LineString, MultiLineString
 
-
 def start_dask(hpc=None):
     """
     Starts a dask cluster for parallel processing.
@@ -269,17 +268,17 @@ def output_template(
     dims = set([d for dim in shapes.values() for d in dim])
     dims_size = dict(ds_gids.sizes) | add_dims
 
-    # if len(ds_gids.chunks) == 0:
-    #     raise ValueError(f"argument ds_gids must contain chunks")
-
     output_template = xr.Dataset(
         data_vars={
-            var: (dim, da.empty([dims_size[d] for d in dim]), attrs.get(var))
+            var: (dim, da.empty([dims_size[d] for d in dim]), attrs.get(var)) # this will produce a dask array with 1 chunk of the same size as the input
             for var, dim in shapes.items()
         },
         coords={dim: ds_gids[dim] for dim in dims},
         attrs=global_attrs,
-    )  # .chunk({dim: ds_gids.chunks[dim] for dim in dims})
+    ) 
+
+    if ds_gids.chunks: # chunk to match input
+        output_template = output_template.chunk({dim: ds_gids.chunks[dim] for dim in dims})
 
     return output_template
 

pvdeg/scenario.py

@@ -1206,7 +1206,9 @@ def addLocation(
 
             geo_meta = geo_meta[geo_meta["county"].isin(county)]
 
-        geo_meta, geo_gids = pvdeg.utilities.gid_downsampling(
+        # we don't downsample weather data until this runs 
+        # because on NSRDB we are storing weather OUT of MEMORY with dask
+        geo_meta, geo_gids = pvdeg.utilities.gid_downsampling( 
             geo_meta, downsample_factor
         )
 
@@ -1525,6 +1527,29 @@ def coords_tonumpy(self) -> np.array:
 
         return coords
 
+    def geospatial_data(self) -> tuple[xr.Dataset, pd.DataFrame]:
+        """
+        Extract the geospatial weather dataset and metadata dataframe from the scenario object
+
+        Example Use:
+        >>> geo_weather, geo_meta = GeospatialScenario.geospatial_data()
+
+        This gets us the result we would use in the traditional pvdeg geospatial approach.
+
+        Parameters:
+        -----------
+        None
+
+        Returns:
+        --------
+        (weather_data, meta_data): (xr.Dataset, pd.DataFrame)        
+            A tuple of weather data as an `xarray.Dataset` and the corresponding meta data as a dataframe.
+        """
+        # downsample here, not done already happens at pipeline runtime
+        geo_weather_sub = self.weather_data.sel(gid=self.meta_data.index) 
+        return geo_weather_sub, self.meta_data
+
+
     def addJob(
         self,
         func: Callable = None,