Skip to content

Conference call notes 20210317

Jump to bottom
Kenneth Hoste edited this page Mar 17, 2021 · 5 revisions

(back to Conference calls)

Notes on the 168th EasyBuild conference call, Wednesday March 17th 2021 (09:00 UTC)

Attendees

Alphabetical list of attendees (7):

  • Alan O'Cais (Jülich Supercomputing Centre, Germany)
  • Jörg Saßmannshausen (NIHR Biomedical Research Centre, UK)
  • Mikael Öhman (Chalmers University of Technology, Sweden)
  • Simon Branford (University of Birmingham, UK)
  • ‪André Gemünd (Fraunhofer Institute, Germany)
  • Robert Mijakovic (LuxProvide)
  • Fotis Georgatos (SDSC, Switzerland)

Agenda

  • update on recent developments
  • revamp of EasyBuild documentation
  • Q&A

Recent developments

  • next release: some time in the coming weeks
  • recent changes
    • framework
      • bug fixes
        • fix for --merge-pr: ignore undismissed 'changes requested' review if there is an 'approved' review by the same user (PR #3607)
      • enhancements
        • add support for using fallback 'arch=*' key in dependency version specified as arch->version mapping (PR #3600)
        • enhancement for --merge-pr: also check for pending change requests and mergeable_state in check_pr_eligible_to_merge (PR #3604)
        • enhance 'eb' command to ensure that easybuild.main can be imported before settling on python* command to use (PR #3610)
        • sort output of eb --search in natural order (respecting numbers) (PR #3609)
      • changes
        • (none)
    • easyblocks
      • bug fixes
        • adjust call to python-config for Python >= 3.8 in VMD easyblock (PR #2355)
          • Change in Python will probably break more software
        • pass down compiler flags provided by EasyBuild in g2clib easyblock (PR #2357)
        • update LAPACK easyblock to keep control of compiler options for versions >= 3.9.0 (PR #2358)
      • enhancements
        • update GROMACS easyblock to support GROMACS v2021 with CUDA (PR #2353)
      • changes
        • (none)
    • easyconfigs
      • over 100 merged easyconfig PRs since last conf call
      • bug fixes
        • fix name of source file for GDRCopy v2.1 (PR #11887)
        • bump pip to 20.3.4 in Python 2.7.18 easyconfig to fix unicode error (PR #12293)
        • add new sources from download.qt.io/new_archive and add missing checksums in Qt5 easyconfigs (PR #)
          • Qt sources are disappearing (e.g. Qt 5.11.2, 5.10.1), some are only 2 years old and already gone
          • If we are allowed according to the licence, we can add missing sources to http://sources.easybuild.io/
          • What's the best way to get software added there?
            • Only Kenneth can upload so I guess we need an issue in the easyconfigs?
        • fix source URL and add alternate checksum for Hypre 2.14.0 (PR #12337)
        • add patch for recent netCDF easyconfigs to fix MPI_Info_f2c issue with OpenMPI (PR #12340)
        • add SourceForge fallback source URL for recent freetype easyconfigs (PR #12341)
        • enable -fPIC for g2clib (PR #12349)
      • enhancements
        • (nothing major)
      • new software
        • (nothing major)
      • noteworthy software updates
        • CP2K 8.1 (PR #12302)
          • There are always some failing tests with CP2K, we are doing pretty well though, down to about 5 failures (out of >3000)
      • noteworthy changes
        • revert to Seaborn 0.10.1 as dependency for NanoPlot 1.33.0 (PR #12345)
  • to merge/fix/tackle soon
    • framework
      • bug fixes
        • downloading of large files fails with urllib.request with recent Python 3.x (issue #3455)
        • performance improvements for easyconfig parsing (PR #3555)
        • avoid module use in Lmod if possible to allow faster execution (PR #3557)
      • enhancements
        • support additional features in easystack files
        • added configuration option to define the env command to use for shebangs (PR #3613)
          • Problem was found in the case of CentOS 7 where we can't pass -S to env, but that is required in some cases
          • No way to use flags with perl shebang if we modify to use env on CentOS7 (would have to drop flags)
            • How common is use of -w flag
    • easyblocks
      • bug fixes
        • (nothing new worth highlighting?)
      • enhancements
        • enhance CUDA support in CP2K easyblock (WIP) (PR #2349)
        • enhance cuDNN, CUDA, and Java easyblocks to support aarch64 (PR #2356)
          • Do we care about different Java versions in this case?
          • We've been using different suffixes for the actual Java versions
            • These can be completely different
            • Java 1.8 had to be built from source on Power
          • This might need some special considerations, additional templating required?
            • Wrapper allows us to have different underlying versions which is ok for Java but not for the cuDNN or CUDA
            • For Power, we're going to have to keep using the wrapper trick
              • And has the added bonus of being able to update the underlying version
          • Comment in PR if you have concerns here
        • make WRF and WPS easyblocks aware of (pre)configopts (PR #2361)
        • add Java wrapper support to OpenMPI (PR #2360)
          • Should open a PR with an easyconfig just to register the support
          • Might be possible to make this multideps to support multiple Java versions
          • Probably not many use cases, might not be worth the effort right now
      • changes
        • (none)
      • new software
        • new easyblock for NCCL (built from source) (PR #2337)
          • Working, default builds for all GPU archs
          • No pressing need, probably best suited to 2021a releases
    • easyconfigs
      • bug fixes
        • (nothing major?)
      • enhancements
        • (nothing major?)
      • new software
      • software updates
        • PyTorch 1.8.0 (PR #12347)
          • PyTorch only test against MKL which is probably connected to failures in their testsuite

Revamp of EasyBuild documentation

  • revamp of EasyBuild documentation to mkdocs is work-in-progress
  • TODO:
    • flesh out general outline + navigation menu on the left
      • getting started: terminology, installation, configuration, basic usage (typical workflow)
      • community: contributing, common toolchains, GitHub integration, getting help
      • supported software: list of supported software, writing easyconfigs, implementing easyblocks
      • advanced (?) topics: ...
      • release notes
      • tutorial
    • fix hanging build in readthedocs
    • port existing docs pages from .rst to .md
    • set up search in docs (mkdocs plugin?)
    • how to avoid broken links...?

Q&A

  • What is the plan regarding the Intel compilers?
    • Renamed iccifort to intel-compilers
    • Probably because compilers in OneAPI will change to icx, icpx and ifx
    • Will only change within bundle for intel
    • Also requires a change for anyone using a hierarchical scheme
      • People with custom schemes will need to take note
  • Status of AMD toolchains?
    • AOCC 3.0 in the works
  • 2021a will be interesting
    • UCX with CUDA support, so that we can add CUDA to foss and potentially be able to drop fosscuda
      • will drop back to version suffixes for CUDA variants
      • should reduce a lot of unnecessary duplication
      • rpath-ing things makes this a little tricky
      • Mikael might do some experiments to see how this would pan out if we had done this in 2020b
  • Fotis: Why move docs from rst to markdown
    • GitLab has some features that support rst, would that be an option?
    • Alan: I don't think so, the rendering there is pure rst, whereas our linking is mostly via sphinx
  • Fotis: Is anyone relying on Lmod on Bright Computing HPC clusters?
    • Have the RPM production workflow for Lmod for that if anyone is interested
  • PMIx
    • We are currently following 3.1.5, should we have 3.2.3 and 4.0?
      • Will be updated along with OpenMPI
      • We can ask the OpenMPI people what we should be using, to know if 4.0 is really production ready
Clone this wiki locally