invalid day number using ERA5

[DoMatte]

Hi,

I am using ERA5 as a reference dataset for the ClimWIP recipe.
I am trying to use a seasonal std on daily psl, which look like

psl_STD:
<<: *performance_settings
preprocessor: climatological_std_reg1
short_name: psl
mip: day

where
climatological_std_reg1:
<<: *region1
climate_statistics:
seasons: DJF
operator: std_dev

Note that I have produced ERA5 daily from hourly data and modified the time:units accordingly (from hours to day) and shift the time by 30 min (but I do not think it was necessary..).

So,in the preproc, psl_STD is cumputed form CMIP6 model, but then got stuck with ERA5 with something like this

File "src/cftime/_cftime.pyx", line 1321, in cftime._cftime.datetime.replace
File "src/cftime/_cftime.pyx", line 2114, in cftime._cftime.DatetimeGregorian.init
File "src/cftime/_cftime.pyx", line 1158, in cftime._cftime.datetime.init
File "src/cftime/_cftime.pyx", line 1755, in cftime._cftime.assert_valid_date
ValueError: invalid day number provided in cftime.DatetimeGregorian(1981, 2, 29, 12, 0, 0, 0, has_year_zero=False)

So, I guess... I am not sure what is goinf, there is no February 29 in 1981 (and the of course in ERA5).
I am not sure why he is complaining about that.
Any solution would be appreciated

main_log_debug.zip

recipe_climwip_GC_third_attempt.txt

[DoMatte]

Dear @remi-kazeroni , @valeriupredoi,

I am not sure if you have seen this, should I tag you whenI open a discussion?
Meanwhile, I wonder if I can not used ERA5 hourly and specifically mention to resample only era5?

Something like this,
climatological_std_reg1:
<<: *region1
daily_statistics:
operator: mean
exclude: *model_data
climate_statistics:
seasons: DJF
operator: std_dev

but then
psl_STD:
<<: *performance_settings
preprocessor: climatological_std_reg1
short_name: psl
mip: day
tas_TREND:

the mip is day for the model and the 1hr for ERA5, can I specify it here?
Thanks in advance

[DoMatte]

Hi, I am now beginning to think that daily data using native6 project is not possible, since it is not in the _fixes/native6/era5.py. Anyway I can mange to fix that? I am not sure that if I just change this, I will screw up something else in the process..

def get_frequency(cube):
"""Determine time frequency of input cube."""
try:
time = cube.coord(axis='T')
except iris.exceptions.CoordinateNotFoundError:
return 'fx'

time.convert_units('days since 1850-1-1 00:00:00.0')
if len(time.points) == 1:
    if cube.long_name != 'Geopotential':
        raise ValueError('Unable to infer frequency of cube '
                         f'with length 1 time dimension: {cube}')
    return 'fx'
interval = time.points[1] - time.points[0]
if interval - 1 / 24 < 1e-4:
    return 'hourly'

return 'monthly'

[remi-kazeroni]

Hi @DoMatte, thanks for your question. The subtlety here is that the on-the-fly CMORization of ERA5 variables is only supported for raw data, which are given with hourly and monthly frequencies. In these case, you can use the native6 project for the ERA5 entries in your recipe. See our doc here.

For daily ERA5 data, a CMORization needs to be done before running the recipe. Daily means can be generated from raw hourly data (stored in your RAWOBS directory) by using the recipe cmorizers/recipe_daily_era5.yml in which you can select the variables that need to be CMORized. See documentation about the cmorization recipe and the handling of different input frequencies for ERA5 data. You will obtain the CMORized OBS6*.nc files in the recipe run dir.

Once the daily data are obtained, you need to move them to the directory containing your CMORized obs data (OBS6/Tier3/ERA5/). Entries for daily ERA5 data in recipes should use the OBS6 project. A good example recipe is hydrology/recipe_globwat.yml which contains entries for monthly ERA5 (native6) and daily ERA5 (OBS6) data.

[DoMatte]

Excellent, thanks @remi-kazeroni