ECC is the generic name used for error correcting codes that can be used to detect and possibly correct read errors from RAM. Supporting it requires extra hardware and thus increases the cost of the hardware. This is why it is not generally used or available on desktop computers. The GTX and similar lines of consumer graphics cards from NVIDIA do not have ECC support in their hardware.
On the other hand, the Quadro and Tesla line of hardware are aimed at workstations and servers used for scientific computing. These applications rely on ECC and NVIDIA added ECC support for these hardware with the Fermi architecture.
If you are using Quadro or Tesla, do note that ECC is turned on by default on these hardware! It provides reliability, but delivers lower memory bandwidth and thus affects performance.
If your application does not rely on ECC, it might be wise to turn it off. To do this open the NVIDIA Control Panel and go to Workstation → Change ECC State to toggle it off. You will need to restart the computer for this to be effected. The performance gains can be quite substantial, I observed a performance gain of 25% by just turning off the ECC state.
Tried with: Tesla C2050 and CUDA 4.0