Replies: 1 comment
-
All valid points to consider, but I'll touch this first:
The way To synchronize, you could follow up a Alternatively, senders could agree to target a distinct stream ID on their respective receivers in the |
Beta Was this translation helpful? Give feedback.
-
Currently, all ACCL calls like send/recv require a reference to a BaseBuffer object as mandatory argument. When streaming is used as input or output, this requires to create and pass a BaseBuffer which is not used at all.
CCLO *ACCL::send(BaseBuffer &srcbuf, unsigned int count, unsigned int dst, unsigned int tag, communicatorId comm_id, bool from_fpga, streamFlags stream_flags, dataType compress_dtype, bool run_async, std::vector<CCLO *> waitfor)
There are different options to change/extend the API to overcome this issue:
ACCL::prepare_call
but it would change the signatures for all calls.ACCL::stream_send
,ACCL::stream_recv
... that explicitly support streaming and only take the required arguments as input.I personally would favor the last option because it is most explicit. However, I am not sure if it is flexible enough to support further enhancements of ACCL in the future since we would basically remove control over the streaming flags by the user.
I would also argue for a
stream_recv
call because the current way of handling communication via streams as one-sided communication is ambiguous. What happens in cases, where two ranks send data to the same destination rank? The order the messages will arrive is not defined and there is no way to find out the current sender on the receiving side. With that, we need to use barriers to enforce the correct order. With thestream_recv
calls we would be able to define the order how the messages are received and handled by the user kernel.I think all this needs further discussion to find the best solution. Maybe there are also technical restrictions i am not aware of.
Beta Was this translation helpful? Give feedback.
All reactions