Scaling-Up Contrastive Learning over NVLink Multicast

Contrastive losses built on PyTorch, JAX, or even NCCL and NVSHMEM directly leave much of the hardware on the table. Want 5x larger batches at 2x lower latency, with collectives designed for CUDA-graph replay? Get ready to write some PTX!