Sometimes the number of possible dimensions of a CUDA block and grid can get confusing. This is not helped by the fact that both of their dimensions is specified by the type dim3.
I try to distinguish their dimensions by the meaning of their words:
- Block: Think of a block of wood. It has all 3 dimensions: length, width and height.
- Grid: A grid is a lattice of horizontal and vertical lines. Thus it has only 2 dimensions.
- Thus, a block has a maximum of 3 dimensions and a grid a maximum of 2 dimensions.
- A block has a minimum of 1 dimension and a grid too has a minimum of 1 dimension. This seems obvious, but in the CUDA 1.x days, the minimum dimension of a block was 2. Needless to say that many programmers must have swore at NVIDIA for that weird restriction! 😀
Similarly, I also find the global variables
gridDim very confusing. Instead, I mentally replace
blockDim by threads-per-block and
gridDim by blocks-per-grid.