The NVIDIA System Management Interface tool is the easiest way to explore the GPU topology on your system. This tool is available as nvidia-smi and is installed as part of the NVIDIA display driver. GPU topology describes how one or more GPUs in the system are connected to each other and to the CPU and other devices in the system. The topology is important to know how data is copied between GPUs or between a GPU and CPU or other device.
$ nvidia-smi topo -h
$ nvidia-smi topo -m
The output of this command shows a matrix of the connections between your GPUs with interconnect info, CPU affinity and NUMA affinity.
Some examples of GPU topologies:
$ nvidia-smi topo -m GPU0 CPU Affinity GPU0 X 0-7 $ nvidia-smi topo -m GPU0 GPU1 GPU2 GPU3 CPU Affinity GPU0 X PHB SOC SOC 0-9,20-29 GPU1 PHB X SOC SOC 0-9,20-29 GPU2 SOC SOC X PHB 10-19,30-39 GPU3 SOC SOC PHB X 10-19,30-39 $ nvidia-smi topo -m GPU0 GPU1 GPU2 GPU3 GPU4 GPU5 GPU6 GPU7 mlx5_0 mlx5_2 mlx5_1 mlx5_3 CPU Affinity GPU0 X NV1 NV1 NV2 NV2 SOC SOC SOC PIX SOC PHB SOC 0-19,40-59 GPU1 NV1 X NV2 NV1 SOC NV2 SOC SOC PIX SOC PHB SOC 0-19,40-59 GPU2 NV1 NV2 X NV2 SOC SOC NV1 SOC PHB SOC PIX SOC 0-19,40-59 GPU3 NV2 NV1 NV2 X SOC SOC SOC NV1 PHB SOC PIX SOC 0-19,40-59 GPU4 NV2 SOC SOC SOC X NV1 NV1 NV2 SOC PIX SOC PHB 20-39,60-79 GPU5 SOC NV2 SOC SOC NV1 X NV2 NV1 SOC PIX SOC PHB 20-39,60-79 GPU6 SOC SOC NV1 SOC NV1 NV2 X NV2 SOC PHB SOC PIX 20-39,60-79 GPU7 SOC SOC SOC NV1 NV2 NV1 NV2 X SOC PHB SOC PIX 20-39,60-79 mlx5_0 PIX PIX PHB PHB SOC SOC SOC SOC X SOC PHB SOC mlx5_2 SOC SOC SOC SOC PIX PIX PHB PHB SOC X SOC PHB mlx5_1 PHB PHB PIX PIX SOC SOC SOC SOC PHB SOC X SOC mlx5_3 SOC SOC SOC SOC PHB PHB PIX PIX SOC PHB SOC X
The command also prints this legend as reference:
Legend: X = Self SYS = Connection traversing PCIe as well as the SMP interconnect between NUMA nodes (e.g., QPI/UPI) NODE = Connection traversing PCIe as well as the interconnect between PCIe Host Bridges within a NUMA node PHB = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU) PXB = Connection traversing multiple PCIe bridges (without traversing the PCIe Host Bridge) PIX = Connection traversing at most a single PCIe bridge NV# = Connection traversing a bonded set of # NVLinks