🚀 Looking forward to our tutorial on GPU Communication at IEEE Hot Interconnects. This is a fantastic opportunity to deepen your expertise in NCCL and NVSHMEM—two essential libraries powering large-scale AI and HPC. Hands-on learning with top experts awaits, and registration is now open. Don’t miss your chance to join on August 22, 2025. 🔗 Register here: https://lnkd.in/g43JjsVc
This year NVIDIA is giving a tutorial on GPU Communications at IEEE Hot Interconnects! The tutorial will cover NVIDIA Collective Communication Library (NCCL) and NVSHMEM, the technologies behind cutting-edge large-scale AI and HPC infrastructures. The resources will be provided by Forschungszentrum Jülich. Save the date (Aug 22, 2025) Presenters: Arnav Goel, Benjamin G., Pouya Kousha, PhD, Andreas Herten Abstract: This tutorial provides a comprehensive introduction to two advanced GPU communication libraries: NVIDIA Collective Communication Library (NCCL) and NVSHMEM. Designed for researchers, engineers, and developers working on HPC and deep learning, the session explores the core concepts, programming models, and practical use cases of both libraries. NCCL is optimized for collective and point-to-point communication among GPUs, offering high throughput and low latency by leveraging hardware features such as NVLink, PCIe, and RDMA networking (InfiniBand, RoCE, etc). It is widely used for distributed deep learning and scientific computing, enabling efficient data movement and synchronization across multiple GPUs within and across nodes. NVSHMEM, on the other hand, implements a Partitioned Global Address Space (PGAS) model for NVIDIA GPUs, supporting one-sided communication and fine-grained data exchange. It enables both host- and device-initiated operations, allowing for asynchronous, in-kernel communication and efficient overlap of computation and communication. NVSHMEM is particularly suited for irregular communication patterns and workloads requiring low-latency, fine-grained GPU-initiated operations. Through hands-on examples we elaborate on the new features in both libraries, participants will gain practical skills in leveraging NCCL and NVSHMEM to accelerate multi-GPU applications, understand their interoperability, and select the right tool for diverse HPC and DL scenarios. IEEE Computer Society NVIDIA Networking #NCCL #NVSHMEM #NVIDIA #AI #HPC #hoti #hoti25