Skip to content

Conference call notes 20190515

Kenneth Hoste edited this page May 15, 2019 · 3 revisions

(back to Conference calls)

Notes on the 124th EasyBuild conference call, Wednesday May 15th 2019 (17:00 - 18:00 CEST)

Attendees

Alphabetical list of attendees (7):

  • Damian Alvarez (Jülich Supercomputing Centre)
  • Markus Geimer (Jülich Supercomputing Centre)
  • Fotis Georgatos (SDSC, Switzerland)
  • Kenneth Hoste (HPC-UGent)
  • Adam Huffman (Big Data Institute, Oxford)
  • Bart Oldeman (ComputeCanada)
  • Davide Vanzo (Vanderbilt University

Agenda

  • update on EasyBuild 3.9.1
  • update on road to EasyBuild 4.0
  • dealing with the OpenBLAS problems in foss/2018b on Intel Skylake (not discussed)
  • Q&A

Outlook to EasyBuild v3.9.1

The road to EasyBuild v4.0

  • ETA for EasyBuild 4.0 is (still) by summer 2019...

  • rename of dummy to system https://github.com/easybuilders/easybuild-framework/pull/2877 MERGED

    • a bit more work is done on top to add support for "eb --fix-deprecated"
    • Fotis: should be renamed to --fix-deprecated-easyconfigs (but that's a bit long?)
  • Bart: can we distinguish between toolchain for binary installations vs build done with system compilers?

    • toolchain = None vs toolchain = SYSTEM?
      • or more explicit like toolchain = BINDIST
      • implies making toolchain optional?
    • why does using dummy/system toolchain even work when EB f*cks up build env?
    • Damian: can we clean up the use of True in dependency specs, and use SYSTEM instead
      • example: ('Java', '11', '', SYSTEM) or ('Java', '11', SYSTEM) or ('Java', '11', '', toolchain=SYSTEM)
  • deprecation of runtest in favor of testopts

  • TODO: OpenMPI easyblock

Q&A

  • Intel MPI 2019 update 3 that occurred today at JSC

    • libfabric (pre-compiled with impi) wasn't working properly (slow, always using TCP rather than IB)
      • not an issue on Omnipath systems...
    • fixed by untarring included libfabric and building that on own system
    • Bart: other option is to use libfabric release
      • Damian: libfabric shipped with impi has additional patches
    • 'tricky' bit: configuration of libfabric
      • MLX (Mellanox) provider configuration failed (even if UCX is not installed)
        • Bart: MLX provider is not that good (and low priority anyway)
      • if both IB & Omnipath libraries are available on system, build works but runtime fails because Omnipath bits are missing in system
    • should be integrated in impi easyblock, ideally
    • but not trivial to get configuration right based on what's available on the system
      • introduce libfabric_configopts for impi easyblock so you can push in custom configure options if needed
      • separate parameter to control rebuilding of shipped libfabric (rebuild_libfabric=True by default)
    • Damian: please open issue in easyblocks repo on this with more details
    • are the scaling issues fixed?
      • Damian: not sure, will check
      • previous issues with collectives got fixed in impi 2019.2, much better now
  • Fotis: binaries = [...] to list names of expected binaries

    • also run ldd on them and capture the output in the EB log
    • could also be done for (dynamic) libraries
    • default logic could be to use (lowercase?) software name in binaries (which implies checking for bin/<name>)?
      • current default sanity check is to require non-empty bin & lib directories
    • Davide: checking ldd output could help with catching unintended linking to system libraries
  • Markus: using Clang as a library doesn't really work right now, because it's registered as a compiler

    • KH: related to recent discussions with Alan & Mikael?
Clone this wiki locally