CUDA: Block and Grid Dimensions

Sometimes the number of possible dimensions of a CUDA block and grid can get confusing. This is not helped by the fact that both of their dimensions is specified by the type dim3.

I try to distinguish their dimensions by the meaning of their words:

  • Block: Think of a block of wood. It has all 3 dimensions: length, width and height.
  • Grid: A grid is a lattice of horizontal and vertical lines. Thus it has only 2 dimensions.
  • Thus, a block has a maximum of 3 dimensions and a grid a maximum of 2 dimensions.
  • A block has a minimum of 1 dimension and a grid too has a minimum of 1 dimension. This seems obvious, but in the CUDA 1.x days, the minimum dimension of a block was 2. Needless to say that many programmers must have swore at NVIDIA for that weird restriction! πŸ˜€

Similarly, I also find the global variables blockDim and gridDim very confusing. Instead, I mentally replace blockDim by threads-per-block and gridDim by blocks-per-grid.

3 thoughts on “CUDA: Block and Grid Dimensions

  1. Between grid, block, and thread…one terminology too many! numBlock and numThread would have sufficed!!


    1. Chiang: This is an old blog post. In current version of CUDA both block and grid are three dimensional. So, my analogy goes out the window πŸ™‚


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.