Sometimes the number of possible dimensions of a CUDA block and grid can get confusing. This is not helped by the fact that both of their dimensions is specified by the type dim3.
I try to distinguish their dimensions by the meaning of their words:
Block: Think of a block of wood. It has all 3 dimensions: length, width and height.
Grid: A grid is a lattice of horizontal and vertical lines. Thus it has only 2 dimensions.
Thus, a block has a maximum of 3 dimensions and a grid a maximum of 2 dimensions.
A block has a minimum of 1 dimension and a grid too has a minimum of 1 dimension. This seems obvious, but in the CUDA 1.x days, the minimum dimension of a block was 2. Needless to say that many programmers must have swore at NVIDIA for that weird restriction! 😁
Similarly, I also find the global variables blockDim
and gridDim
very confusing. Instead, I mentally replace blockDim
by threads-per-block and gridDim
by blocks-per-grid.