Developing Code

Making code changes using a forking workflow

If developers would like to make code changes, they need to make a personal fork, set up upstream remote (for merging with the original ufs-weather-model), and create a branch for ufs-weather-model and each of the sub component repositories they want to change. They can then make code changes, do testing and commit the changes to the branch in their personal fork. It is suggested that they merge their branch with the develop branch of the original repositories periodically to get the latest updates and bug fixes. To merge with the develop branch from upstream (original repository), they can do:

% cd github_branch (branch of ufs_weather_model or sub-component repository)
% git pull upstream develop

If developers would like to get the code committed back to the original repository, it is suggested to follow the steps below:

1. Create an issue in the authoritative repository. For example to commit code changes to fv3atm, please go to https://github.com/NOAA-EMC/fv3atm, under NOAA-EMC/fv3atm and find the “issue” button next to “code” button. Click on “issue” and a new page will come out. On the right side of the page, there is a green “new issue” button, clicking on that will lead to a new issue page. Fill out the title, comments to describe the code changes, also please provide personal fork and branch information. Lastly, click on “submit new issue” button, an issue is created.
   
2. When the development is mature, tests have been conducted, and the developer is satisfied with the results, create a pull request to commit the code changes.

• Merge developer’s branch to the latest ufs-weather-model develop branch in authoritative repository. If changes are made in model sub-components, developers need to merge their branches to branches with the corresponding authoritative repository (or original repository for some components). For this, code management practices of the subcomponents need to be followed.
• Regression test needs to pass on at least one supported platform.
• For each component branch where developers make changes, developers need to go to their personal fork on GitHub and click on the “new pull request” button. When a new page “compare changes” shows up, Developers will choose the branch in their fork with code changes to commit and the branch in upstream repository that the changes will be committed to. The code differences between the two branches will be displayed. Developers can review the differences and click on “submit pull request” to make pull request. After code changes are committed to component repository, developers will make pull requests to ufs-weather-model repository.

It is suggested that the developers inform all the related code managers as the hierarchy structure of the ufs-weather-model repository may require the collaboration among the code managers.

Engaging in the code review process

When code managers receive a pull request to commit the code changes, it is recommended that they add at least two code reviewers to review the code. The reviewers will write comments about the code changes and give recommendation if the code changes can be committed. What kind of code changes will be accepted in the repository is beyond the scope of this document, future ufs-weather-model code management may have detailed answer for that.

Reviewers may suggest some code changes during the review process, developers need to respond to these comments in order to get code changes committed. If developers make further changes to their branch, reviewers need to check the code changes again. When both reviewers give recommendation to commit the code, code managers will merge the changes into the repository.

Conducting regression tests

Developers can compile and run tests using the ufs-weather-model same way as NEMSfv3gfs.

• To run regression test using rt.sh

% cd ufs-weather-model/tests
% ./rt.sh -f

Regression test log files (ufs-weather-model/tests/Compile_$(MACHINE_ID).log and ufs-weather-model/tests/RegressionTests_$(MACHINE_ID).log ) will be updated.

To create new baseline:

% cd ufs-weather-model/tests
% ./rt.sh -f -c

• To run regression test using NEMSCompsetRun

% cd ufs-weather-model
% ./NEMS/NEMSCompsetRun -f 

Regression test log files (ufs-weather-model/log/$MACHINE_ID/* ) will be updated.

To create new baseline:

% cd ufs-weather-model
% ./NEMS/NEMSCompsetRun--baseline fv3 --platform=${PLATFORM}

The value of ${PLATFORM} can be found in ufs-weather-model/compsets/platforms.input.

Developers need to commit the regression test log files to their branch before making pull request.

Compiling the code and running a test

Developers can compile and run tests using the ufs-weather-model same way as NEMSfv3gfs. In the document below, ufs-weather-model directory points to a branch in the developer’s personal fork.

• Compile and run a test using rt.sh

  i) compile the code

Developers can compile the ufs-weather-model same as NEMSfv3gfs. The compile script is ufs_weather_model/tests/compile.sh. To compile the code:

 % cd ufs-weather-model/tests
 % ./compile.sh PATHTR MACHINE_ID MAKE_OPT BUILD_NR

Where:

PATHTR: the full path of FV3 directory under ufs-weather-model
MACHINE_ID: machine ID, e.g: wcoss_cray, hera.intel
MAKE_OPT: compile options, default (‘’, empty string) is for 64-bit OpenMP non-hydrostatic build using AVX2, other options are:
-       ‘DEBUG=Y’: turn on debug option
-       ‘VERBOSE=Y’: turn on VERBOSE mode to get additional details on compile
-       ‘OPENMP=Y’: use openmp
-       ‘AVX2=Y’: use AVX2 in Intel Haswell for better performance.
-       ‘HYDRO=Y’: hydrostatic mode
-       ‘CCPP=Y’: using ccpp framework and physics
-       ‘STATIC=Y’: when CCPP=Y, using static build instead of dynamic build
-       ‘SUITE=xxx': CCPP physics suite, e.g. “FV3_GFS_2017_gfdlmp”
BUILD_NR: the number of build (there might be several copies of the executable). The final executable would be fv3_${BUILD_NR}.exe

Example:

 %  ./compile.sh /gpfs/hps/emc/global/noscrub/First.Last/ufs-weather-model/FV3 wcoss_cray '' 1

The executable generated from the compile.sh can be used in global workflow to run experiments.

ii) run a regression test

Developer can run a single regression test from the regression test suite. The full list of the regression tests can be found at:

us-weather-model/tests/fv3_conf

To run a specific regression test:

 %cd ufs_weather_model/tests
 % cp rt.conf rt.conf1

Edit the rt.conf1 file, keep the test developers intended to run and remove all the rest tests.

COMPILE | fv3                           | standard| wcoss_cray  |             |
RUN     | fv3_control                   | standard|             |             |

To compile and run the test, do

 % ./rt.sh -l rt.conf1

The code will be compiled and run on wcoss_cray. A log directory will be shown at: ufs-weather-model/tests /log_$MACHINE_ID. If there are compile errors, please check file: compile_${BUILD_NR}.log under above log directory. If the code is successfully compiled, but the job failed, please go to above log directory to look for rt_${test_number}_${test_id}.log for details.

• Compile and run a test using NEMSCompsetRun

  i) Compile the code

% cd ufs-weather-model
% ./NEMS/NEMSAppBuilder app=coupledFV3_WW3

ii) Run a regression test

NEMSCompsetRun runs same tests as rt.sh. The two share the same baseline results. To run a single test using NEMSCompsetRun:

% cd ufs-weather-model
% ./NEMS/NEMSCompsetRun “{test_name}”

The test name can be found in compsets/all.input.

---