Skip to content

Switch to using Python SharedMemory

Latest
Compare
Choose a tag to compare
@tskisner tskisner released this 17 Mar 21:54
02cd641

In order to resolve portability challenges with use of shared memory, switch to using multiprocessing.shared_memory.SharedMemory from the standard library. This fixes tests on MacOS. One workaround is still needed to manually disable resource tracking, but that monkey patch will not be needed after python-3.13.0. One possible concern with this approach is that the resource_tracker code used by this python package spawns a helper process. Historically, some MPI implementations were very unhappy with this. However, I tested this with OpenMPI on Linux and MacOS, and also with Cray MPICH, running across multiple nodes. No problems observed so far.