Building on Lassen

Quick and dirty build example on Lassen

• [One time only], make sure git lfs is enabled by running the following prior to cloning the TPS repo:

git-lfs install

• setup gcc/MPI environment:

ml swap xl gcc/8.3.1

• setup modules environment to access installs in our shared space:

# shared s/w location where missing 3rd party libs are installed
export SW_HOME=/usr/workspace/utaustin/software/lassen
export MODULEPATH=$SW_HOME/modules/gcc:$MODULEPATH

• load 3rd party libs necessary for TPS build (this assumes GPU variant)

module load grvy mfem-cuda hypre metis cuda/11.2.0

Now, you should be able to build from cloned repo as follows:

./bootstrap
./configure CUDA_ARCH=sm_70 --enable-gpu-cuda --with-cuda=$CUDA_HOME --disable-valgrind
make -j 4

If all went well, you have a libtps.a library and tps executable in the src/ subdirectory. You can now run a quick check on a compute node. You can access a compute node interactively on Lassen as follows (30 minute example):

$ bsub -nnodes 1 -ISs  -W 30 /bin/bash

[now on compute node, run test suite...]
make check
...

PASS: cyl3d.gpu.test 1 [cyl3d.cuda] check for input file inputs/input.dtconst.cyl
PASS: cyl3d.gpu.test 2 [cyl3d.cuda] run tps with input -> inputs/input.dtconst.cyl
PASS: cyl3d.gpu.test 3 [cyl3d.cuda] verify tps output with input -> inputs/input.dtconst.cyl
PASS: cyl3d.gpu.test 4 [cyl3d.cuda] verify exit code if restart files missing
PASS: cyl3d.gpu.test 5 [cyl3d.cuda] verify consistent solution with restart from 2 iters
PASS: cyl3d.gpu.test 6 [cyl3d.cuda] verify tps output after variable p restart
PASS: cyl3d.mflow.gpu.test 1 [cyl3d.gpu.mflow.outlet] check for input file inputs/input.2iters.mflow.dtconst.cyl
PASS: cyl3d.mflow.gpu.test 2 [cyl3d.gpu.mflow.outlet] verify outlet area calculation with input -> inputs/input.2iters.mflow.dtconst.cyl
PASS: cyl3d.mflow.gpu.test 3 [cyl3d.gpu.mflow.outlet] verify tps output with input -> inputs/input.2iters.mflow.dtconst.cyl
PASS: wedge.gpu.test 1 [2d/wedge] check for input file inputs/input.dtconst.wedge
PASS: wedge.gpu.test 2 [2d/wedge] run tps with input -> inputs/input.dtconst.wedge
PASS: wedge.gpu.test 3 [2d/wedge] verify computed inlet area in output -> wedge.log
PASS: wedge.gpu.test 4 [2d/wedge] verify # of inlet faces detected -> wedge.log
PASS: wedge.gpu.test 5 [2d/wedge] verify computed inlet outlet in output -> wedge.log
PASS: wedge.gpu.test 6 [2d/wedge] verify # of outlet faces detected -> wedge.log