-
Notifications
You must be signed in to change notification settings - Fork 625
FDS Daily Workflow
In this wiki we will describe how to work within "our system" in the firemodels group at NIST. Some of this guidance is specific to the NIST working environment, but some is of general usefulness. As a prerequisite, read the Git Notes Getting Started wiki. Here we will assume you have forked the firemodels/fds repository to <your GitHub username>/fds and have cloned your repo to your local account on our linux cluster "blaze" or "burn".
The purpose of this workflow is to keep your repository up-to-date with the latest changes to the development version of FDS. We expect developers in our group and all collaborators doing research to work with the latest development version of the code. The build status for the latest version is given here: Firebot_Build_Staus. If the build status is "Build Failure!", you should skip updating for that particular day.
If you are brand new to working on unix/linux systems, it will be very helpful if you do a little homework and get up to speed on some basic commands. Note: Be very careful with rm. And, unless you really know what you are doing, do not type rm *. You will be very disappointed. There is no "undo" from this operation in linux.
It will be handy to have a couple of aliases established for running different versions of FDS. If you have not already, open a shell (Terminal on Mac, Putty on Windows) and ssh into blaze. Type
$ ssh <hostname>
Enter password
Now you are in your home directory. Open your .bashrc file with vim (here is a vi cheat sheet). Type
$ vi .bashrc
and make sure your have the following lines.
export FDS_Repo=/<PATTH TO YOUR REPO>/<YOUR REPO NAME>
alias fds_db="$FDS_Repo/Build/impi_intel_linux_db/fds_impi_intel_linux_db"
alias fds_dv="$FDS_Repo/Build/impi_intel_linux_dv/fds_impi_intel_linux_dv"
alias qfds.sh="$FDS_Repo/Utilities/Scripts/qfds.sh"
If you do not have any of these lines, type i to enter "insert mode" and copy these lines into your file. Then type :wq to "write" and "quit" the file.
Close your terminal and reopen it so that new aliases will become active. Now you will be able to simply type, for example, fds_db to run the "debug" version of FDS (once it is compiled, see below).
Setting aliases is a one-time thing. In this section, we will describe what we do on a daily basis.
For this example, I will be the user with an account on blaze. If you have not already, open a shell (Terminal on Mac, Putty on Windows) and ssh into blaze. Type
$ ssh <hostname>
Enter password
Change directories ("cd") into your repo's directory. I keep my forked repo in ~/GitHub/FireModels_rmcdermo/fds/. So, I type
$ cd GitHub/FireModels_rmcdermo/fds
First, make sure you are on the master branch of your repo. Type
$ git branch -a
* master
remotes/firemodels/master
remotes/origin/HEAD -> origin/master
remotes/origin/master
If you do not see the output shown above -- most importantly the asterisk next to master -- then something went wrong in your repo setup; revisit Git Notes Getting Started.
Next, fetch changes from the firemodels central repository. Again, this assumes you have correctly setup the development branch firemodels/fds for remote tracking (see Git Notes Getting Started).
$ git remote update
Fetching origin
...
Fetching firemodels
...
Merge any changes from firemodels into your local repo.
$ git merge firemodels/master
...
Push these changes to your forked repo on GitHub so that this repo also stays up-to-date.
$ git push origin master
...
This is an important step because if and when you find a problem with the code or documentation we would like YOUR help in fixing it. This involves your making changes, committing those changes, pushing the changes to your forked repo, and sending a pull request (see Git Notes Topic Branches), which the project maintainers will then evaluate. If your forked repo is not up-to-date with the firemodels central repo, your pull request will be impossible to accept.
If all has gone well at this point you can check the status of your repo and see the following:
$ git status -uno
# On branch master
nothing to commit (use -u to show untracked files)
Let's now move on to compiling the latest code.
CD into Build and do an ls. You will see several choices or "platforms" for compiling FDS.
$ cd Build
$ ls
...
If you are not doing development work you probably just want to compile the release target that works by default with qfds.sh. That target is impi_intel_linux.
$ cd impi_intel_linux
Now do an ls of this directory. The first time you enter into this directory you will only see this
$ ls
make_fds.sh
The make_fds.sh script is what you run to compile the code. Once the code is compiled this directory will be populated with object and module files (*.o and *.mod). To do a "clean" compile of the code on a daily basis you will need to remove these files before compiling. So, usually you will now type the following two commands. The rm command can be omitted the first time you compile. But generally it is better to do a clean compile.
$ rm *.o *.mod
$ ./make_fds.sh
...
Note the ./ before make_fds.sh. This says you are running the script from this specific directory.
If you need to do any development or debug work, you will have to compile the debug (_db) or development (_dv) versions of the code. CD up one level, cd .., and back down into whichever target you want to compile. I usually work with the (_dv) version because this version compiles reasonably fast and yet still runs reasonably fast. If I am debugging pesky errors that I cannot find with fds_dv then I will resort to the _db version. The process for compiling each of these versions is the same:
$ cd impi_intel_linux_dv
$ rm *.o *.mod
$ ./make_fds.sh
...
Now, because of the alias we set up, you can run fds_dv in the foreground:
$ fds_dv
Fire Dynamics Simulator
Current Date : April 19, 2024 14:13:11
Revision : FDS-6.9.1-55-g307c7a9-master
Revision Date : Sun Apr 14 07:19:02 2024 -0400
Compiler : Intel(R) Fortran Intel(R) 64 Compiler Classic for applications running on Intel(R) 64, Version 2021.7.1 Build 20221019_000000
Compilation Date : Apr 19, 2024 14:09:54
MPI version: 3.1
MPI library version: Intel(R) MPI Library 2021.7 for Linux* OS
Consult FDS Users Guide Chapter, Running FDS, for further instructions.
The above example is run without OpenMP enabled. We discovered that the overhead to evaluate OpenMP directives is actually not trivial, costing roughly 20 % slowdown of the code for single OMP thread case. Since many of our use cases involve MPI with a single cpu core per mesh, we do not want to pay this penalty. Hence, we now compile two versions of the code, one with OpenMP enabled and one without. If you want to use the OpenMP version of the code, look in the build directory and select a target with "openmp" in its name. Further, you need to explicitly set the number of OpenMP threads in your startup script. For example, to use 4 cup cores for OpenMP set the following in your .bashrc file:
export OMP_NUM_THREADS=4
One both blaze and burn you can submit jobs to the queueing system via
$ qfds.sh [options] job_name.fds
The options needed to run on multiple cores, etc., can be found by simply typing qfds.sh without any arguments at the command line.
All of the above may look daunting at first, but it will quickly become routine. You are simply going to do this:
- git remote update
- git merge firemodels/master
- compile the new code