Exploring online compression with the Lorenz96 model  

---

This project was developed during the ESiWACE3 hackathon at CSC Finland on 18.10. - 20.10.2023 by Victor Azizi from the Netherlands eScience Center and Juniper Tyree from the University of Helsinki.

Compiling ZFP for LUMI-G

This sections details the process to compile ZFP on LUMI-G. Note that the Makefile should take care of the entire process - the following are Victor Azizi's notes on how he got ZFP to compile:

• First, check out the zfp submodule:

  git submodule init
  git submodule update

  Alternatively, clone the repository and check out the staging branch (checked with hash e935a81):

  git clone https://github.com/LLNL/zfp.git
  cd zfp
  git checkout staging

• Next, load the right modules on LUMI:

  module load LUMI
  module load buildtools

  They are also available in the modules file, which can be sourced.
• Now, configure the cmake build of ZFP:

  mkdir -p build
  cd build
  cmake -DZFP_WITH_HIP=ON -DZFP_WITH_OPENMP=OFF -DHIP_PATH=/opt/rocm/hip -DCMAKE_C_COMPILER=hipcc -DCMAKE_CXX_COMPILER=hipcc -DBUILD_TESTING=OFF ../

• Next, build with VERBOSE, as you will manually have to fix some commands

  make VERBOSE=1

• You now have a library, but the compilation step of zfp.dir/hip/interface.cpp.o shows that the wrong offload target was used. Therefore, you have to manually fix the offload target to gfx90a:

  cd build
  CC -DZFP_ROUNDING_MODE=ZFP_ROUND_NEVER -DZFP_WITH_HIP -Dzfp_EXPORTS -I../../include -O3 -DNDEBUG --offload-arch=gfx90a -fPIC -std=gnu++14 -o CMakeFiles/zfp.dir/hip/interface.cpp.o -x hip -c ../../src/hip/interface.cpp

• Finally, rebuild the library:

  cd build
  make

• Success! The AMD GPU enabled library can now be found in zfp/build/lib64/libzfp.so

Reproducing the plot notebooks from the command line

• First, clear the previous output:

  jupyter nbconvert --clear-output --inplace plot.ipynb

• Second, use nbconvert or papermill to execute the notebook, e.g.:

  papermill --kernel python3 --autosave-cell-every 5 --progress-bar plot.ipynb plot.nbconvert.ipynb

• Optionally, remove execution metadata:

  jupyter nbconvert --ClearMetadataPreprocessor.enabled=True --to=notebook --inplace plot.nbconvert.ipynb

• Finally, replace the old notebook with the new:

  mv plot.nbconvert.ipynb plot.ipynb

License

Unless specified otherwise below, this repository is licensed under either of

• Apache License, Version 2.0 (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
• MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)

at your option.

---

• The cmdparser.hpp file, which is part of Florian Rappl's C++ CmdParser utility, is licensed under the MIT license (FlorianRappl/CmdParser/LICENSE).
• The output/ folder, which contains the experiment output files, is licensed under the CC0 1.0 license (output/LICENSE or https://creativecommons.org/publicdomain/zero/1.0/).
• The performance/ folder, which contains the experiment performance files, is licensed under the CC0 1.0 license (performance/LICENSE or https://creativecommons.org/publicdomain/zero/1.0/).
• The zfp/ submodule, is licensed under the BSD 3-Clause license (zfp/LICENSE and zfp/NOTICE).

Citation

Please refer to the CITATION.cff file and refer to https://citation-file-format.github.io to extract the citation in a format of your choice.

Funding

This project has been developed as part of ESiWACE3, the third phase of the Centre of Excellence in Simulation of Weather and Climate in Europe.

Funded by the European Union. This work has received funding from the European High Performance Computing Joint Undertaking (JU) under grant agreement No 101093054.