[MacOS] legacy_async_memcpy test appears to hang ( homebrew build, gcc-10)

[bjoo]

Ctest test 23: legacy_async_memcpy appears to hang on MacOS when built with gcc-10 from homebrew -- MacOS Catalina (10.15.7) - similar behaviour is seen on MacOS CI where this test times out along with test 27: legacy_memcpy

[AlexVlx]

@bjoo I'm momentarily Mac-challenged, would it be too much to ask for a few extra details e.g. machine configuration and a stack-trace (across all threads) for the live-locked state? Thank you. Cheers!

[bjoo]

No probs, I’ll get this as soon as I can. :)

…

-- Balint Joo, Oak Ridge Leadership Computing Facility, Oak Ridge National Laboratory P.O. Box 2008, 1 Bethel Valley Road, Oak Ridge, TN 37831, USA email: joob AT ornl.gov. Tel: +1-757-912-0566 (cell, remote) From: Alex Voicu <notifications@github.com> Reply-To: ROCm-Developer-Tools/HIP-CPU <reply@reply.github.com> Date: Thursday, January 7, 2021 at 11:53 AM To: ROCm-Developer-Tools/HIP-CPU <HIP-CPU@noreply.github.com> Cc: Balint Joo <bjoo@jlab.org>, Mention <mention@noreply.github.com> Subject: [EXTERNAL] Re: [ROCm-Developer-Tools/HIP-CPU] [MacOS] legacy_async_memcpy test appears to hang ( homebrew build, gcc-10) (#6) Resent-From: <bjoo@jlab.org> Resent-Date: Thursday, January 7, 2021 at 11:53 AM @bjoo<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_bjoo&d=DwMCaQ&c=CJqEzB1piLOyyvZjb8YUQw&r=WG01dcOd8ZFLFFNkrpbj6A&m=RsQH12cU8BuFdldWz2WYgLPmZ1FrVQBsGDlkTrLdxJI&s=ZSmNo1MyLEP_xHh9TEFQ-GWV6vtpcFf-EKXuvUZ_BCQ&e=> I'm momentarily Mac-challenged, would it be too much to ask for a few extra details e.g. machine configuration and a stack-trace (across all threads) for the live-locked state? Thank you. Cheers! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_ROCm-2DDeveloper-2DTools_HIP-2DCPU_issues_6-23issuecomment-2D756238830&d=DwMCaQ&c=CJqEzB1piLOyyvZjb8YUQw&r=WG01dcOd8ZFLFFNkrpbj6A&m=RsQH12cU8BuFdldWz2WYgLPmZ1FrVQBsGDlkTrLdxJI&s=OujZOuvE-27zuGVQLmP8ckTwtndVtln_dFpfbqwTtts&e=>, or unsubscribe<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AAEPL2EH2D55LQY4DIQ7CRTSYXRI5ANCNFSM4VWLKADQ&d=DwMCaQ&c=CJqEzB1piLOyyvZjb8YUQw&r=WG01dcOd8ZFLFFNkrpbj6A&m=RsQH12cU8BuFdldWz2WYgLPmZ1FrVQBsGDlkTrLdxJI&s=5ZdTSbBS46Vd6dcEWZ2uv7uo5jdFMAGhd36IJXvXU9A&e=>.

[Noerr]

This legacy_async_memcpy test continues to hang. (MacOS Ventura 13.2.1, brew gcc 12.2.0, M1 Max aarch64)

Open Files & Ports:
HIP-CPU/build/tests
HIP-CPU/build/tests/legacy_tests
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/libgcc_s.1.1.dylib
/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/libstdc++.6.dylib
/dev/ttys004
->0x6498cb799eca4a97
->0x6498cb799eca4a97

Process sample attached showing two threads spinning in _pthread_cond_wait and vectorAdd()
Sample of legacy_tests.txt

[Noerr]

Contrary to the HIP-CPU macOS install instructions I did not install Intel tbb. (figuring gcc has supported STL parallel algorithms for several major versions now)

[Noerr]

The legacy_memcpy test hangs in similar way. Process sample attached. Sample of legacy_tests legacy_memcpy.txt

We can disable these two tests and all others pass.

ctest --output-on-failure --exclude-regex '(legacy_async_memcpy|legacy_memcpy)'

100% tests passed, 0 tests failed out of 31

Why are nearly all test names prefixed with "legacy_" ?

[Noerr]

I did brew install tbb then redid the HIP-CPU install. I don't see any evidence that tbb is utilized. It feels like something is missing from the instructions to tell cmake to use tbb.

build-tbb % otool -L tests/legacy_tests
tests/legacy_tests:
	/opt/homebrew/opt/gcc/lib/gcc/current/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.30.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.0.0)

I'm not seeing any logic in /CMakeLists.txt that would reference TBB for CMAKE_HOST_APPLE.

[Noerr]

I patched line 48 of /CMakeLists.txt to naively force enable TBB. I find the built tests are linked to TBB, but the same legacy_memcpy and legacy_async_memcpy tests hang.

-    if (STL_DEPENDS_ON_TBB0 OR STL_DEPENDS_ON_TBB1)
+    if (CMAKE_HOST_APPLE OR STL_DEPENDS_ON_TBB0 OR STL_DEPENDS_ON_TBB1)
        find_package(TBB REQUIRED)
        target_link_libraries(
            ${PROJECT_NAME} INTERFACE TBB::tbb)
    endif()

build-tbb % otool -L tests/legacy_tests
tests/legacy_tests:
	/opt/homebrew/opt/tbb/lib/libtbb.12.dylib (compatibility version 12.0.0, current version 12.8.0)
	/opt/homebrew/opt/gcc/lib/gcc/current/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.30.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.0.0)