Skip to content

Conference call notes 20240814

Jump to bottom
Kenneth Hoste edited this page Aug 14, 2024 · 1 revision

(back to Conference calls)

Notes on the 252nd EasyBuild conference call, Wednesday 14 Aug 2024 (08:00 UTC)

Attendees

List of attendees (19):

  • Simon Branford (Univ. of Birmingham, UK)
  • Luc Cardoso (University of Luxembourg)
  • Bibek Chapagain (Barcelona Supercomputing Centre, Spain)
  • Davide Grassano (CECAM, Switzerland)
  • Jasper Grimm (University of York, UK)
  • Alexander Grund (ZIH, Dresden, Germany)
  • Thomas Hayward-Schneider (Max Planck Institute for Plasma Physics (IPP))
  • Leonardo Honfi Camilo (Univ. of Wageningen, Netherlands)
  • Kenneth Hoste (HPC-UGent, Belgium)
  • Adam Huffman (Big Data Institute, Oxford, UK)
  • Georgios Kafanas (University of Luxembourg)
  • Lev Lafayette (Univ. of Melbourne, Australia
  • Kurt Lust (UAntwerpen, Belgium + LUMI User Support Team)
  • Sam Moors (Vrije Universiteit Brussel, Belgium)
  • Sébastien Moretti (SIB, Switzerland)
  • Jan Reuter (JSC, Germany)
  • Ãke Sandgren (Umeå University, Sweden)
  • Cintia Willemyns (Vrije Universiteit Brussel, Belgium)
  • Jure Pečar (EMBL, Heidelberg, Germany)

Agenda

  • overview of recent developments
  • outlook to EasyBuild 5.0 release
  • 2024a/2024b update of common toolchains
  • Q&A

Recent developments

  • latest EasyBuild release: 4.9.2 (12 June 2024)
  • next EasyBuild release:
    • likely EasyBuild 4.9.3 quite soon (days/weeks, as EasyBuild 5.0.0 is still weeks/months away at best...
    • We'll probably
  • EasyBuild v5.0.0
    • detailed notes on latest developments in https://hackmd.io/tZtz6vMmRfGL6N9Rf__OEA
    • to test development version of EasyBuild 5.0: 
      # set up Python virtual environment, and jump into it
python3 -m venv eb5
source eb5/bin/activate

# install EasyBuild 5.0 development version into it
pip install https://github.com/easybuilders/easybuild-framework/archive/5.0.x.tar.gz
pip install https://github.com/easybuilders/easybuild-easyblocks/archive/5.0.x.tar.gz
pip install https://github.com/easybuilders/easybuild-easyconfigs/archive/5.0.x.tar.gz

# go!
eb --version

Merged PRs

  • docs (merged PRs)

    • ...

  • framework (merged PRs)

    • bug fixes
      • handle post-install patches in check_checksums_for (PR #4605)
      • fix copy_file with a folder as the target (PR #4609)
    • enhancements
      • Add --skip-sanity-check option (PR #4590)
      • add GNU_FTP_SOURCE template constant (PR #4597)
      • improve error messages for empty EasyConfigs (PR #4603)
    • changes
      • ...
    • code cleanup
      • ...
    • EasyBuild 5.0 (5.0.x branch)
      • add GNU_FTP_SOURCE template constant (PR #4598)
      • limit maxparallel to 16 by default (PR #4606)
      • Add support for --software-commit and an associated template %(software_commit)s (PR #4608)

  • easyblocks (merged PRs)

    • bug fixes
      • fix --sanity-check-only for impi (PR #3403)
      • change extra $PATH entry for OCaml to use opam/default rather than opam/system (PR #3390)
    • enhancements
      • support revisions in crates extraction procedure in Cargo easyblock (PR #3405)
      • set $R_LIBS_USER in RPackage easyblock to avoid picking up on R packages installed in home directory (PR #3407)
        • see also easyconfigs PR #21175 to disable installing R packages in home dir with RStudio-Server
    • updates
      • update ORCA easyblock for version 6 (PR #3395)
        • many people seem to be interested in having ORCA 6 installed...
    • changes
      • ...
    • new easyblocks
      • ...
    • code cleanup
      • ...
    • EasyBuild 5.0
      • ...

  • easyconfigs (merged PRs)

    • ~50 easyconfig PRs were merged since last conf call
    • bug fixes/reports
      • Highway 1.0.4 fix for failing test on zen4 (PR #20942)
      • Ensure that recent BioPerl easyconfigs use Bundle easyblock (PR #21136)
      • Fix download URLs for old Intel software (2018-23) (PR #21108)
    • enhancements
      • ...
    • (noteworthy) new software
      • ...
    • noteworthy software updates
    • changes
      • rename intel/2024.06 toolchain to intel/2024a (PR #21101)

Open (active) PRs

  • docs (open PRs + issues)

    • ...

  • framework (open PRs + issues)

    • bug fixes
      • fix help message for --http-headers-fields-urlpat configuration option (PR #4594)
    • enhancements
      • add support for --sources-url to fetch sources from additional URLs (PR #4079)
      • Copy build log and artifacts to a permanent location after failures (WIP PR #4601)
    • changes
      • ...
    • EasyBuild 5.0
      • tracker issue with overview of effort for improving consistency in naming of options/parameters: issue #4563
      • Fix the checksum type check (PR #4578)
      • Fix to_checksums with None values in dicts and recursion (PR #4579)
      • Deprecate use of ec['parallel'] and fix updating the template value (PR #4580)
      • avoid suggesting that toolchainopts are supported for SYSTEM compiler, since they're not (PR #4585)
      • add support for alternative/deprecated toolchain options (PR #4587)
      • Use default value $XDG_CONFIG_DIRS from XDG basedir spec: /etc/xdg (instead of /etc) (PR #4591)
        • we should probably also still consider /etc/easybuild.d/*.cfg
      • convert template constant lists to dicts and export the constants by name (PR #4595)
        • Python + PythonPackage will need to be updated too because they use TEMPLATE_CONSTANTS
      • Add capability to export RPATH wrappers for a toolchain (PR #4596)
      • Don't try to overwrite an existing file if the contents are identical (PR #4604)

  • easyblocks (open PRs + issues)

    • bug fixes
      • fix GCC-system without Intel license (PR #3402)
      • fix crash in Cargo easyblock when no crates are specified (PR #3404)
      • fix double initialization of Cargo by CargoPythonPackage (PR #3406)
    • enhancements
      • Set CMake hints for path to Python installation if it's direct or indirect dependency (when using CMake >= 3.12) (PR #3282)
      • Add sanity_check_test_inputs, add mapping for zen4, and update python command for LAMMPS 2Aug2023 and older versions (PR #3336)
      • enhance custom easyblock for GCC to use with-arch option for nvptx with 13.1+ (PR #3396)
      • add conda-lock support to conda easyblock (PR #3397)
    • updates
      • update custom easyblock for NAMD for version >= 3.0 (PR #3248)
      • Update NCL easyblock for intel (PR #3409)
    • changes
      • ...
    • new
      • add custom easyblock for BLIS (PR #3385)
    • EasyBuild 5.0
      • enhance CMakeMake easyblock to check whether correct Python installation was picked up by CMake (PR #3399)

  • easyconfigs (open PRs + issues)

    • bug fixes/reports
      • ...
    • enhancements
      • ...
    • (noteworthy) new software
      • ...
    • software updates
      • TensorFlow v2.15.1 w/ CUDA 12.1.1 (PR #20358)
      • define foss/2024a toolchain (copy of foss/2024.05) (PR #21100)
    • changes
      • make pytest v7.4.2 independent of Python-bundle-PyPI (PR #21004)
      • reorganize Flax/JAX stack in 2023a: scvi-tools, Flax, Optax, jax, absl-py (PRs #21038, #21039)

2024b common toolchains

  • GCC 14.2 as a base (see easyconfigs PR #21114)
  • we should define candidate toolchains using latest version of all components
    • are newer versions of Intel components compared to intel/2024a already available?
  • should we consider replacing intel with Clang-based toolchain as 2nd common toolchain next to foss?
    • Fortran frontend in Clang is becoming mature enough?

Q&A / others

  • should we set $HOME to a subdirectory of /tmp after configuring EasyBuild?
    • may need to restore original $HOME before submitting jobs, and before running sanity check
    • we need to be aware of HOME constant that can be used in easyconfigs
  • Åke: cstd toolchain option shouldn't affect $CXXFLAGS, should be via new cxx_std toolchain option
Clone this wiki locally