Module all_to_all 

Typed AllToAll / AllToAllv requests.

cudarc 0.19.4 does not safely expose ncclAllToAll / ncclSend / ncclRecv against a raw comm: ncclComm_t (the field is private). Implementations therefore decompose AllToAll into a group_start / paired send + recv / group_end sequence using cudarc’s safe Comm::send and Comm::recv — which is the standard NCCL idiom for AllToAll regardless. AllToAllv adds per-peer (count, offset) pairs.

Structs

AllToAllRequest

Symmetric AllToAll: each rank sends an equal-sized shard of length count to every peer. send carries world_size * count elements; recv likewise.

AllToAllvRequest

AllToAllv: per-peer (count, offset) shards in send and recv.