multi-threaded software gets stuck #889

jpablolorenzetti · 2023-06-23T22:05:27Z

jpablolorenzetti
Jun 23, 2023

Hi, I have a software that starts 1 thread per CPU, each thread has one io_uring, each thread waits for something to happen using io_uring_wait_cqes, there are different sources of events such as eventfd that are submitted before entering the wait and so other processes trigger these event sources to wake up these threads to take action on them. The software run as expected when is run normally/directly but when is run using valgrind it just gets stuck on io_uring_wait_cqes when creating the very first thread and the program does not continue. i have tried to debug the issue using gdb and was able to get the following trace:

#0 __sys_io_uring_enter2 (sz=, sig=, flags=, min_complete=,
to_submit=, fd=) at queue.c:117
#1 _io_uring_get_cqe (ring=ring@entry=0x4ab6a40, cqe_ptr=cqe_ptr@entry=0x5671ce8, data=data@entry=0x5671c80) at queue.c:113
#2 0x000000000011090d in __io_uring_get_cqe (sigmask=0x0, wait_nr=1, submit=, cqe_ptr=0x5671ce8, ring=0x4ab6a40) at queue.c:146
#3 io_uring_wait_cqes (ring=0x4ab6a40, cqe_ptr=0x5671ce8, wait_nr=1, ts=, sigmask=0x0) at queue.c:314
#4 0x000000000010e1da in frt_smp_reactor_wait (r=0x4ab6a40) at /home/jpablo/focsrt/src/posix/reactor.c:65
#5 0x000000000010da6f in vcore_sleep (vcore=0x123950 <gl_config+912>) at /home/jpablo/focsrt/src/vcore.c:176
#6 0x000000000010d881 in frt_vcore_main_loop (arg=0x1fff000160) at /home/jpablo/focsrt/src/vcore.c:111
#7 0x0000000004854ea7 in start_thread (arg=) at pthread_create.c:477
#8 0x0000000004981a2f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95