-
Notifications
You must be signed in to change notification settings - Fork 39
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BUG] Lack of Availability of HRRR Weather Models Means 3-Date Azimuth Grid Interpolation Fails for GUNW workflow #584
Comments
@cmarshak just also documenting the relevant issue here. |
Using the above GUNW, I get past the first bug but see a new one:
|
It's clear to me that this occurs when not enough weather files are available. This is not an edge case issue. Here is the relevant output:
The GUNW I am using is: https://hyp3-tibet-jpl-contentbucket-81rn23hp7ppf.s3.us-west-2.amazonaws.com/fe0ef2be-262b-4031-8e22-99b93df661f8/S1-GUNW-A-R-091-tops-20171230_20161223-221939-00069W_00045N-PP-c9b7-v3_0_0.nc I can easily just weight whatever weather models are available... however, this could yield unexpected results. Not sure if all the weather model files (and the total used) are recorded. Would be important here. @dbekaert @bbuzz31 At some point it will be helpful to understand the distribution of available HRRR model files that are available over Sentinel-1 catalog. It's producing the bulk of our errors for this step. There are other options too - we could go through download of say 6 closest dates, download them and then just use the 3 that are closest. This has implications related to how long it takes to download data. |
You can recreate the error running |
@jlmaurer and @dbekaert just reiterating this is about weather model availability for particular datetimes. We can solve this easily, but given we are in the midst of processing want to utilize something with long term utility. Previously, for what is now called Here are some possible solutions to the error reported:
There may be other solutions too. Again, not sure what these fixes means in terms of interpolating the tropo correction layers save for the ease of stitching. |
@jlmaurer - thank you for offer. Based on conversation with @dbekaert, @bbuzz31 and @sssangha, all the proposed solutions are not going to be used. 😆 We want to do the following:
Given our extended deadline - there is not the same urgency - our processing deadline is likely going to be extended passed FY. In other words, we will record processing issues and make sure we can remedy them in the next month or so. |
After this Thursday evening (so beginning Friday), we can also merge PRs that you have postponed thanks to the extension of processing deadline. Thank you for your patience @jlmaurer. |
@cmarshak sounds like a good plan!! |
I wanted to provide small update thanks to @asjohnston-asf and @jhkennedy. The ingest of GUNWs occurs here: https://github.com/asfadmin/grfn-ingest/blob/test/echo10-construction/src/echo10_construction.py#L134 Currently, we only write to the json here: https://github.com/dbekaert/RAiDER/blob/dev/tools/RAiDER/cli/raider.py#L609 (love the use of Therefore, as we move towards adding control flow that does not include tropo layer if HRRR datetimes are not available, we would have to ensure these are properly synced up via tests. Furthermore, want to higlight a few more pieces:
Again, should have tests for all 3. |
Resolved by #597 |
is raised from these inputs:
As usual, here is the input GUNW to the above step:
https://hyp3-tibet-jpl-contentbucket-81rn23hp7ppf.s3.us-west-2.amazonaws.com/00312fde-8f89-4c5c-a0d6-ec2df678f316/S1-GUNW-A-R-106-tops-20170210_20170129-225948-00078W_00042N-PP-cde3-v3_0_0.nc
Hoping this can be quickly modified together.
The text was updated successfully, but these errors were encountered: