3 - Warps and Cooperative Groups

Summary

Chapter 3 contains a comprehensive description of CUDA cooperative groups including the powerful features for explicit warp-level programming. Warp-level programming is becoming more prominent with recent CUDA developments such as the warp matrix functions introduced to support tensor core hardware. The various types of thread groupings are discussed and illustrated in examples.We show a revised reduce kernel which uses warp-level intrinsic functions instead of shared memory.This kernel is further improved by using 128-bit vector loading of data from global GPU memory to kernel local register-based variables. A variation of this example using coalesced thread groups is also shown.The conditions for avoiding deadlock when working with warp-level thread divergence are explored with differences between the older and newer generations of GPU explained. An example using the new the cg::reduce function is shown; this function has hardware support for CC ≥ 8 devices.