forked from NOAA-GFDL/FUDGE
-
Notifications
You must be signed in to change notification settings - Fork 0
/
release-notes
116 lines (98 loc) · 5.71 KB
/
release-notes
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
05/07/2015
(a1r)
1. submit jobs from $HOME directory
05/06/2015
(cew)
1. Cleanup/removal of no-longer-used code
2. Explicit documentation directory
04/01/2015
(a1r)
1. fudgeList redundant MMDD fix
03/30/2015
(a1r)
1. Reduced number of echo statements, warnings in expergen, postProc where possible.
2. Corrected: PP appears to be checking in the wrong location for pre-existing concatenated output
3. Corrected: <ifpreexist> statements.
03/20/2015
(a1r)
1. remove timestamps from runscripts,master scripts,runcode and pp scripts as per esd team's request (informal dev option floats around for later clean up and implementations)
2. submit_job changed according to 1.
3. Moab standard out logs moved to experiment scripts log directory as per esd team's request.
03/17/2015
(a1r)
1. fudgelist outputs script.base and also prints script directory to feed clobber.
(cew)
1. utils/bin/cleanup_script.csh cleans up script directories based off of project_ID,
as determined by the script directory
03/14/2015
(a1r)
1. bugfix, makedirs and scripts directory to use project_ID (and not spat_mask_ID)
03/13/2014
(cew)
1. bugfix for using a time trimming mask in the R code
02/26/2015
(a1r)
1. checker count bugfix on 1 and 2d with varname param. Plus postProc/ppFudge fixes to output masks
dir relative to out_dir
2. postProc template updates <BRANCH> inclusion.
3. expergen uses NEMSemail if available to print msub email notification syntax along with postProc_command
02/24/2015
(a1r)
1. expergen call fudgeList outputting summary log in scripts directory. (postProc - fudgeList calls unchanged )
2. postProc parser switched, whitespaces parsable from -v.
3. postProc exits if variable not found in OneD.
4. postProc_command is now a msub friendly script. postProc_command structure is different and is saved in scripts directory.
5. workflow can be run on PP nodes as well.
1/19/2015
(a1r,cew)
1. Support for updated input and output directory structure (based on document provided by MJN and KD, sent Jan 16)
- Related expergen,postProc changes.
- Disabled features that existed within expergen to construct output path (commented); most of FUDGE experiment name construction and now uses dumb param.
2. XML1.9 support for the following new/updated attributes and tags
(listing just for future reference)
- target_ID,in_root,dim,file_start_time/end_time/train_start_time/end_time with MMDD included.
- project_ID, absolutely out_dir
3. Make optional spatial masks and temporal masks
4. Support for sourcing ppscript and msub ppscript
- If mask is relevant, that variable gets added.
5. Tested with synthetic dataset "example", PM "example", RR "example". More tests to follow.
6. ppFudge uses unique temp directory.
7. cleanup-script with XML as input
- Cleanup script accepts 2 possible options: 'e' for erase, and 'm' for move, which moves the experiment experiment-name to experiment-name~n, where n is the number of experiments by that name in the current directory
- Cleanup script no longer throws an error when trying to remove nonexistant directory
8. R code changes
- Added method BCQM and EDQM for formal Red River runs (other DS methods exist, but are in the process of QC)
- Added new option for precipitation pre- and post- downscaling adjustment to downscale only days with precipitation and apply the mask of days with 0 pr from the future predictor dataset to the downscaled output at the end; option can be turned on or off via the XML
- Units and calendar are optional for the input datasets, with the exeption of pr with the us_trace or global thresholds, which both require unit conversion
- FudgeIO's ReadNC functions can directly clone dimensions and variables that are not the vars of interest, and write them to the output file
- Support for downscaling any variable
--
01/13/2015
Aparna Radhakrishnan
1.Changes to support absence of spatial and temporal masks in XML (tested only in expergen)
2. Previous changes to support PM merged
3. This version not to be used with postProc (updates to follow)
11/15/2014
postProc changes:
Aparna Radhakrishnan
1. Ability to post post-process two fields at a time (e.g. tasmin, tasmin_qcmask)
2. BasicChecker before post post-processing begins (this is coded in a way it can be a potential generic stand-alone tool)
-- Quits if there are extraneous files in OneD directory
-- Quits if there are extraneous files in twoD catted directory
-- Quits if the total number of minifiles does not match the expected number (lon range)
-- Quits if atleast one of the files in the OneD or existing twoD directory is Zero bytes.
3. Color coding Error messages.
4. Fix predictand_calibration_dataset global attribute (item from QC results from ESD call 11/12/14)----- removed if the variable is a mask.
expergen changes:
Aparna Radhakrishnan
1. Changes to use <ifprefixexist>tags for anti-clobber feature.
-- If <ifprefixexist> has exit: if either OneD output OR scripts already exist, expergen throws errors and exits.
-- If <ifprefixexist> has erase: if either OneD output OR scripts already exist, expergen throws warning messages, then executes the external cleaner/scrubber script (from cew). After this expergen continues it's regular workflow of creating runscripts, etc.
--- If platform is anything other than PAN, expergen quits with an error at this time.
--- Errors/warnings color coding
2. "automatic msub" option added to expergen. This is a command-line option --msub. When you pass --msub to expergen, expergen not only creates master runscripts for you, but also submits them to MOAB automatically. By default, this is turned off. One can always opt to use the existing submit_job script to submit jobs as well.
--
incomplete release notes
--
08/07/2014
a1r, Placing set of FudgeIO functions