Skip to content

Conference call notes 20220511

Jump to bottom
Kenneth Hoste edited this page Jul 6, 2022 · 7 revisions

(back to Conference calls)

Notes on the 197th EasyBuild conference call, Wednesday 11 May 2022 (15:00 UTC)

Attendees

Alphabetical list of attendees (12):

  • Sebastian Achilles (JSC, Germany)
  • Simon Branford (Univ. of Birmingham, UK)
  • Fotis Georgatos (EPFL, Switzerland)
  • Jasper Grimm (University of York, UK)
  • Kenneth Hoste (HPC-UGent, Belgium)
  • Adam Huffman (Big Data Institute, Oxford, UK)
  • Kurt Lust (Univ. of Antwerpen, Belgium + LUMI User Support Team)
  • Sebastien Moretti (SIB, Switzerland)
  • Jurij Pečar (EMBL, Germany)
  • Bart Oldeman (ComputeCanada)
  • Davide Vanzo (Microsoft)
  • Lars Viklund (Umeå University, Sweden)

Agenda

  • overview of recent developments + outlook to next EasyBuild release
  • progress on 2022a update of common toolchains
  • Q&A

Recent developments

  • release timeline
    • latest release: EasyBuild v4.5.4 (31 March 2022)
    • ETA next release: mid May 2022
  • recent changes
    • framework
      • bug fixes
        • remove system-compiled binutils dirs from $LDFLAGS in binutils easyblock (PR #2712)
        • fix printing of list of attempted download URLs when url-encoded characters are used in URL (PR #4005)
          • fix for regression (included in EasyBuild v4.5.4) introduced in PR #3976
      • enhancements
        • add definition for gfbf toolchain (GCC, FlexiBLAS, FFTW) (PR #4006)
      • changes
        • change 'eb' command to import easybuild.framework to check if EasyBuild framework is available (PR #3998)
        • checking with import of easybuild.main is too expensive, but usign easybuild is not specific enough (see PR #3998)
    • easyblocks
      • bug fixes
        • remove system-compiled binutils dirs from $LDFLAGS in binutils easyblock (PR #2712)
        • fix for FreeSurfer easyblock: define $FREESURFER needed by recon_al (PR #2717)
      • enhancements
        • ...
      • updates
        • update sanity check in R easyblock for versions >= 4.2.0, since S.h is not included anymore (PR #2713)
        • update LLVM easyblock for LLVM v14.0.x (PR #2718)
        • update Mesa easyblock to remove swr driver configopts for versions 22+ (PR #2719)
      • changes
        • don't use gold linker by default for GCC >= 11.3 (PR #2711
          • ld.gold is no longer actively developed, so back to classic ld.bfd linker (by default)
      • new software
        • ...
    • easyconfigs
  • to merge/fix/tackle soon
    • framework
      • reported bugs / bug fixes
        • ...
      • enhancements
        • add Generation module naming scheme (PR #3547)
        • improve error finding (draft PR #4001)
          • copies ctest_log_parser.py from Spack, contains patterns and filters that an be used to recognize error messages
        • update prepare_rpath_wrappers to enable wrapper shipping with a module (WIP) PR #4003
          • see also companion PR for GCC easyblock (PR #2638)
        • tarball update when working with cloned repositories (issue #4002)
        • crash when using string value for 'sources' in extension (issue #4004)
        • testoutput option is ignored (issue #4007)
      • changes
        • ...
    • easyblocks
      • bug reports/fixes
        • fix extension filter for Perl packages (PR #2699)
        • make Amber easyblock aware of FlexiBLAS (PR #2720)
        • ScaLAPACK header files no longer installed in include/ (PR #2715)
      • enhancements
        • enhance Clang easyblock to add support for building with AMDGPU offload (PR #2684)
        • enhance Clang easyblock to support also installing Python bindings (PR #2721)
        • filter out unsupported compute capabilities from environment when loading CUDA? (PR #2714)
      • updates
        • update sanity check in OpenMPI easyblock to support OpenMPI v5.0.0 (PR #2709)
      • new software
        • ...
      • changes
        • ...
    • easyconfigs
      • now over 600 open easyconfig PRs, we're way overdue a significant cleanup round...
      • bug fixes
        • fix installation with setuptools>=61 by explicitly declaring there are no Python packages (PR #15206)
        • add hwloc dependency to recent tbb easyconfigs (PR #15294)
        • fix RepeatMasker-4.1.2-p1 easyconfig by moving the database configure step to be after installation (PR #15280)
      • enhancements
        • ...
      • new software
      • noteworthy software updates
        • RoseTTAFold v1.0.0 (PR #13795)
        • TensorFlow v2.7.1 (WIP) (PR #14990)
          • failing tests are for new features of TensorFlow
          • ignore these failing tests for now, report upstream
        • hipSYCL v0.9.2 (PR #15074)
        • Qt6 (PR #15096)
        • AlphaFold v2.2.0 (PR #15129)
          • failing jax tests on A100?
          • Sam: easy way out, just use older jax version that is still supported
        • PyTorch v1.11.0 (PR #15137)
        • CP2K v9.1 with foss/2021a (PR #15146) and intel/2021a (PR #15147)
          • call for testing by others, and keep an eye on result of test suite...
      • changes
        • ...

2022a update of common toolchains

  • see (2022a project](https://github.com/easybuilders/easybuild-easyconfigs/projects/4) in easyconfigs repo
  • GCC 11.3.0 as base (see merged PR #15311)
    • with ld.bfd as default linker (rather than ld.gold, which is no longer actively maintained)
    • see easyblocks PR #2711
  • foss/2022a:
    • GCC 11.3.0 (vs 11.2.0 in foss/2021b)
    • latest OpenMPI 4.x: v4.1.4 (PR #15426)
      • not going for OpenMPI 5.0.0, too big of a change
      • awaiting final release of OpenMPI v4.1.4
      • (vs 4.1.1 in foss/2021b)
      • on top of UCX-1.12.1 (~PR #15451) + PMIx v4.1.2 + libfabric v1.15.0 (PR #15456)
    • latest FlexiBLAS/OpenBLAS(/BLIS) (see PR #15399)
      • FlexiBLAS v3.1.3 (vs 3.0.4 in foss/2021b)
        • with OpenBLAS + BLIS backends
        • OpenBLAS v0.3.20 (vs 0.3.18 in foss/2021b)
        • BLIS v0.9.0 (vs 0.8.1 in foss/2021b)
    • FFTW: 3.3.10 (same as in foss/2021b)
      • split up in "serial" FFTW and FFTW.MPI
      • requires small change in framework
    • ScaLAPACK: 2.2.0 (update from 2.1.0 vs foss/2021b)
  • intel/2022a
  • other software versions
    • Perl 5.34.1
    • Python 3.10.4
    • LLVM 14.0.3
    • Mesa 22.0.3
  • software to test on top of candidate versions for foss/2022a and intel/2022a:
    • SciPy-bundle, CP2K, OpenFOAM, ...

Q&A

  • Bart has been looking into the OpenMPI + CUDA mess
    • alternative approach to Åke's PR #14919
    • could use CUDA as a build dependency to OpenMPI, but then we "break" the policy to only have FOSS in foss toolchain
    • alternative can be to include a small CUDA header file to build OpenMPI GPU-aware
    • also fixes the problematic case reported by Caspar (cfr. issue #14801)
    • some performance measurements were also done, shows that this approach introduces very little overhead; an overhead of max 10% is visible on very small messages (<1K) on MPI_Alltoall for 64 cores on shared memory but not on small point-to-point messages or MPI_Allreduce for example. We are not aware of real-world problematic overheads.
    • PR by Bart coming up soon...
Clone this wiki locally