Handling old HRRRv2 files

[Hedrick-ARS]

Hello,

I'm a researcher with USDA Agricultural Research Service in Boise and our group has been running windninja operationally for a little over 4 years in support of our snow modeling work for water supply forecasting. Windninja is an awesome product and we're thrilled with how it's been working out. We wrap windninja using a repo we call katana (link to wrapper function: https://github.com/USDA-ARS-NWRC/katana/blob/main/katana/wind_ninja.py).

I'm now working with some researchers to run our model historically for water year 2017 (10/1/16 - 9/30/17) but I'm getting an error when trying to run windninja that I don't get for more recent dates. The grib2 files for HRRRv2 do look somewhat different from v3 and v4, and I'm wondering if this is the root cause. We store historical HRRR forecasts starting in 2015 locally on our server, so I've made sure they are identical to what AWS hosts using grib_compare.

I don't want to take too much of your time, but do you have any pointers for working with old HRRRv2 grib2 files? The only error logs I have are from our Katana wrapper, but I'm including that here to see if you have any insight as to what's crashing windninja.

Cheers!
Andrew Hedrick

log.txt

[latwood]

Looks like I'll have to get working on the archived HRRR project a bit sooner than expected. If I understand correctly, WindNinja currently is updated to work with later versions of HRRR data, both in download and in storage. So in theory since your files are likely predownloaded, it should just be reverting back the HRRR storage/initialization from before the HRRR version changes, while still holding onto any newer stuff not related to the direct changes in HRRR version in the HRRR data storage/initialization. So in theory reverting back to older versions of https://github.com/firelab/windninja/blob/3.8/src/ninja/ncepHrrrSurfInitialization.h and https://github.com/firelab/windninja/blob/3.8/src/ninja/ncepHrrrSurfInitialization.cpp. In practice it might get more complicated than that, I know that when we discussed this project there was an extra work issue of needing to trim the downloaded data to just the area of interest, but that might just be a download problem. Could you upload one of the HRRR files that you are using? Then I can work on this next Monday after getting back from the weekend. Probably the smaller/lighter the better. Not sure if there's a specific way to upload bigger files on this, I'm still pretty new to addressing WindNinja issues like this. Thanks! Loren Atwood

…

On Fri, Apr 28, 2023 at 4:30 PM Andrew Hedrick ***@***.***> wrote: Hello, I'm a researcher with USDA Agricultural Research Service in Boise and our group has been running windninja operationally for a little over 4 years in support of our snow modeling work for water supply forecasting. Windninja is an awesome product and we're thrilled with how it's been working out. We wrap windninja using a repo we call katana (link to wrapper function: https://github.com/USDA-ARS-NWRC/katana/blob/main/katana/wind_ninja.py). I'm now working with some researchers to run our model historically for water year 2017 (10/1/16 - 9/30/17) but I'm getting an error when trying to run windninja that I don't get for more recent dates. The grib2 files for HRRRv2 do look somewhat different from v3 and v4, and I'm wondering if this is the root cause. We store historical HRRR forecasts starting in 2015 locally on our server, so I've made sure they are identical to what AWS hosts using grib_compare. I don't want to take too much of your time, but do you have any pointers for working with old HRRRv2 grib2 files? The only error logs I have are from our Katana wrapper, but I'm including that here to see if you have any insight as to what's crashing windninja. Cheers! Andrew Hedrick log.txt <https://github.com/firelab/windninja/files/11357631/log.txt> — Reply to this email directly, view it on GitHub <#487>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEZOWJIFZXJ73GDNDRZHPADXDRAJNANCNFSM6AAAAAAXPYUY6I> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[Hedrick-ARS]

Thanks for getting back to me. I'm including two zipped folders. One is for 10/1/2016 and extends over a bounding box around Grand Mesa, CO. The files inside are the HRRRv2 files and only have the four variables needed by WindNinja. The other zip file is similar but for 10/1/2019 over a smaller area in Northern Colorado outside Fort Collins. I believe that date is HRRRv3, but I'm not certain.

Currently our Docker image is calling version 3.5.0 and all of its older dependencies. I can update or roll back the version as needed by just updating the Dockerfile.

Thanks!
-Andrew

hrrr.20161001.zip
hrrr.20191001.zip

[nwagenbrenner]

@Hedrick-ARS We have support for some of the older HRRR versions and try to detect the needed bands in all formats. Yes, if you can send us the HRRR file you're using we can take a look. Shouldn't be too much effort to get something working for you. Also, which version of WindNinja are you using? The latest support for reading HRRR is in the master branch.

[nwagenbrenner]

Hi @Hedrick-ARS, I'd first upgrade your docker build to either 3.8.1 or the master branch. It's likely this will solve your problem. I'll download your files and take a look to confirm this afternoon.

[Hedrick-ARS]

Thanks so much! I'll update the version and give it a try.

[nwagenbrenner]

Just a quick update. I just tested on the master branch and it looks like there is an issue still. I'll work on this and report back once I've identified the issue.

[nwagenbrenner]

@Hedrick-ARS Disregard my last message. The master branch works with your HRRR file. I was originally using a DEM outside of the HRRR grid (I didn't realize at first that it was a clipped HRRR file and expected that it would cover all of CONUS). Let me know if you have issues using these HRRR grids in the master branch -- I think you should be good. We should include a check that reports a useful error message if the DEM is not completely covered by the weather model forecast.

[Hedrick-ARS]

That's great to know. I'm working on the finishing touches of updating to v3.8.1 in our Katana docker container. Will let you know when I get everything running. Thanks so much for your help.