Skip to content

Conference call notes 20240703

Jump to bottom
Kenneth Hoste edited this page Jul 3, 2024 · 1 revision

(back to Conference calls)

Notes on the 249th EasyBuild conference call, Wednesday 3 July 2024 (15:00 UTC)

Attendees

List of attendees (10):

  • Kenneth Hoste (HPC-UGent, Belgium)
  • Lars Viklund (Umeå University, Sweden)
  • Simon Branford (Univ. of Birmingham, UK)
  • Jasper Grimm (University of York, UK)
  • Alexandre Strube (JSC, Germany)
  • Alan O'Cais (CECAM)
  • Kurt Lust (UAntwerpen, Belgium + LUMI User Support Team)
  • Sébastien Moretti (SIB, Switzerland)
  • Mikael Öhman (Chalmers University of Technology, Sweden)
  • Cintia Willemyns (Vrije Universiteit Brussel, Belgium)

Agenda

  • overview of recent developments
  • outlook to EasyBuild 5.0 release
  • 2024a update of common toolchains
  • Q&A

Recent developments

  • latest EasyBuild release: 4.9.2 (12 June 2024)
  • next EasyBuild release: hoping to make that EasyBuild 5.0.0 (but not before Aug'24, at best)
  • EasyBuild v5.0.0
    • detailed notes on latest developments in https://hackmd.io/tZtz6vMmRfGL6N9Rf__OEA
    • to test development version of EasyBuild 5.0: 
      # set up Python virtual environment, and jump into it
python3 -m venv eb5
source eb5/bin/activate

# install EasyBuild 5.0 development version into it
pip install https://github.com/easybuilders/easybuild-framework/archive/5.0.x.tar.gz
pip install https://github.com/easybuilders/easybuild-easyblocks/archive/5.0.x.tar.gz
pip install https://github.com/easybuilders/easybuild-easyconfigs/archive/5.0.x.tar.gz

# go!
eb --version

Merged PRs

  • docs (merged PRs)

    • ...

  • framework (merged PRs)

    • bug fixes
      • Correctly evaluate result for --dep-graph (PR #4554)
      • fix fetch progress bar showing too many files (PR #4568)
      • Resolve internal imkl>=2021 version subdir via "latest" symlink (PR #4570)
    • enhancements
    • changes
      • ...
    • code cleanup
      • ...
    • EasyBuild 5.0 (5.0.x branch)
      • check presence of current working directory at the end of run_shell_cmd and try to return to original working directory if non-existent (PR #4390)

  • easyblocks (merged PRs)

    • bug fixes
      • Remove dummy license server info from ANSYS (PR #3377)
    • enhancements
      • make a project environment and manage LOAD_PATH for JuliaPackage (PR #3239)
      • set $ESMFMKFILE environment variable in environment module for ESMF (PR #3368)
      • Use build dir for big files/folders while building TensorFlow (PR #3371)
      • add RISC-V support to Boost easyblock (PR #3376)
    • updates
      • support >= 2024a for MCR (PR #3369)
      • update custom easyblock for Bazel for versions >= 7.0 (PR #3370)
    • changes
      • ...
    • new easyblocks
      • ...
    • EasyBuild 5.0
      • ...

  • easyconfigs (merged PRs)

    • over 60 easyconfig PRs were merged since last conf call
    • bug fixes/reports
      • add wrapper for Julia with linking safeguards and delegate environment setup to JuliaPackage (PR #20103)
    • enhancements
      • ...
    • (noteworthy) new software
      • ...
    • noteworthy software updates
      • ...
    • changes
      • ...

Open (active) PRs

  • docs (open PRs + issues)

    • ...

  • framework (open PRs + issues)

    • bug fixes
      • ...
    • enhancements
      • reuse pre-computed checksums (PR #4569)
    • changes
      • use Intel's oneAPI Fortran compiler by default for version 2024.0.0 and newrer (oneapi_fortran toolchain option set to True) (PR #4567)
    • EasyBuild 5.0
      • Make generic EasyBlock usable (PR #4531)
        • not a blocker for EasyBuild 5.0, can be introduced at any time without breaking changes (though would be nice to have in place by the time EasyBuild 5.0 is ready to be released)
        • some concerns due to pre_configure_cmds and configure_cmd not being run in same shell session, which will lead to surprises (like export command in pre_configure_cmds not having an effect on environment for configure_cmd)
        • pre_configure_cmds + configure_cmd (+ equivalent for build, test, install) are introduced as general easyconfig parameters, but will actually only be picked up when EasyBlock class is used as easyblock instead of an actual generic/custom easyblock;

  • easyblocks (open PRs + issues)

    • bug fixes
      • Patch ctypes for Python installations when filtering LD_LIBRARY_PATH (PR #3352)
        • not ready to merge yet, needs more work
        • would be good to get feedback from Bart & co once rework is done
        • Python's test suite is not as good as we hoped, cfr. test result for PR #3355
    • enhancements
      • enhance CMakeMake easyblock to check whether correct Python installation was picked up by cmake (PR #3233)
      • Set CMake hints for path to Python instalation if it's included as a dependency (when using CMake >= 3.12) (PR #3282)
      • Set build type for PyTorch explicitely (PR #3332)
      • update PythonPackage easyblock to allow installation of Python packages with $PIP_REQUIRE_VIRTUALENV set + move temporary pip folder into build dir (PR #3374)
      • add testing by default for recent Python versions (3.10+) (PR #3355)
    • updates
      • update CP2K easyblock to use do_regtest.py script for recent versions (WIP) (PR #2984)
      • update custom easyblock for Tensorflow for versions 2.14 + 2.15 (PR #3303)
      • Don't consider lib/release for impi >= 2021.11 (PR #3375)
    • changes
      • Merge of the ConfigureMake and CMakeMake versions of the easyblock for QuantumESPRESSO (PR #3338)
        • test suite fails because of multiple class definitions in same easyblock file...
        • QE 7.1 with intel/2022a proves to be horribly broken with lots of failrues in the test suite
          • not worth fixing, probably best to remove that easyconfig in EB 5.0
          • in a dedicated PR, so we have a clear entry in release notes
    • new
      • LLVMcore easyblock for compilation of clang/flang + other llvm-projects (PR #3373)
    • EasyBuild 5.0
      • rename custom easyblock parameters to adher to naming policy and make things more consistent (issue #3361)

  • easyconfigs (open PRs + issues)

    • bug fixes/reports
      • add patch to SciPy-bundle 2024.05 that fixes numpy test failures on RISC-V (PR #20847)
      • add patch to fix implicit function declaration in OpenMPI 4.1.4 (PR #20949)
      • disable auto-vectorizer (-ftree-vectorize) for OpenFOAM v10 + v11 when using toolchain that with GCC >= 11 (PR #20958)
    • enhancements
      • ...
    • (noteworthy) new software
      • ...
    • software updates
      • jax 0.4.25 w/ CUDA (PR #20119)
        • do we just need to accept that the jax test suite requires a substantial amount of GPU memory?
        • we could emit a build info message that mentions that test suite may fail (and suggest --ignore-test-failure)
      • TensorFlow v2.15.1 w/ CUDA 12.1.1 (PR #20358)
        • Alexander says "should be fine"
      • R-bundle-CRAN w/ 2023b (PR #20756)
      • GROMACS v2024.2 w/ CUDA 12.5.0 (PR #20809)
        • increased timeout for tests, which helps is required on HPC-UGent GPU clusters to pass test suite
      • R v4.4.1 + R-bundle-CRAN v2024.06 w/ foss/2023b (PR #20898)
      • intel/2024.06 (candidate for intel/2024a) (PR #20926)
    • changes
      • ...

2024a common toolchains

  • GCC 13.3 as a base
  • latest version of all components (except GCC, not using 14.1):
    • foss/2024a (candidate toolchain foss/2024.05 merged: PR #20646)
      • GCC 13.3 (20240521) + binutils 2.42 (20240129)
      • OpenMPI 5.0.3 (20240408)
        • PMIx 5.0.2 (20240321), UCX 1.16.0 (20240415), libfabric 1.21.0 (20240329)
      • FlexiBLAS 3.4.3 (20240521) + OpenBLAS 0.3.27 (20240404) + BLIS 1.0 (20240506)
      • FFTW 3.3.10 (20210914)
      • ScaLAPACK 2.2.0 (20220202)
    • intel/2024a
      • GCCcore 13.3 + binutils 2.42
      • Intel C/C++/Fortran compilers 2024.1.0 (20240327)
        • some trouble arose when combining C/C++ 2024.1.2 with Fortran 2024.1.0
        • bundle provided by Intel sticks to 2024.1.0 for both
        • we should take a look at the release notes
      • Intel MPI 2021.12.1 (20240424)
      • Intel MKL 2024.1.0 (20240327)
      • easyconfig PRs towards candidate for intel/2024a: PR #20919 + PR #20926

Q&A / others

  • Mikael: looking into adding GitHub Actions workflow
    • for auto-tagging PRs with new/update/bugfix-or-enhancement-or-change
    • automatically find closest easyconfig to compare with
      • use output of eb --review-pr --review-pr-max=1?
    • maybe we should consider adopting https://github.com/actions/stale too?
  • Kenneth's bot to post comments when CI is broken isn't working anymore due to GitHub API changes
    • there doesn't seem an easy way to fix it
  • Alan: we could/should be leveraging EESSI bot to go test PRs automatically across a range of systems/configurations
    • boegelbot implementation is a bit crappy, just a cron
    • EESSI bot is event based
    • we should set up EESSI bot in dedicated bot account on jsc-zen3, and properly configure it
Clone this wiki locally