target_include_directories is an useful CMake directive to specify the include directories for building a particular target, as described here. The FindCUDA module for CMake which handles CUDA compilation seems to completely ignore this directive. The include directories specified for the target are not passed to CUDA compilation by nvcc.
This will most commonly result in errors of the form: someheader.h: No such file or directory.
This is a well known limitation of the CUDA module of CMake, as documented here. There seems to be no plan currently to support target_include_directories for CUDA compilation.
The only solution is to switch to include_directories to add these directories for all the targets in the CMakeLists.txt file.
Tried with: CMake 220.127.116.11, CUDA 6.5 and Ubuntu 14.04
$ sudo apt-get install cuda
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:
The following packages have unmet dependencies:
cuda : Depends: cuda-6-5 (= 6.5-14) but it is not going to be installed
E: Unable to correct problems, you have held broken packages.
On searching, a lot of folks had this error. Their errors were solved when they removed old NVIDIA packages they had installed. But, I did not have any NVIDIA package installed!
What were these broken packages I had on the system, I didn’t know! I followed the dependency chain by trying to install cuda-6-5, which complained on another package and so on. In the end, I found that I had a lot of unnecessary packages, all of which had :i386 at the end of their name.
The strangest part was that running sudo apt-get autoremove had no effect on these packages, they were not removed. So, I manually removed all of them using sudo apt-get remove.
When I tried to install CUDA after this, it worked fine! 🙂
Installing CUDA is becoming increasingly easier on Ubuntu. I think I keep hitting problems because I am usually updating from an older NVIDIA graphics driver or CUDA version. NVIDIA continues to be quite bad at providing error-free upgrades. Anyway, this is what worked for me:
Do not try to install any of the NVIDIA drivers or CUDA packages that are in the Ubuntu repositories. I wasted a day with the errors these operations threw up!
Uninstall all CUDA packages and NVIDIA drivers you may have on your Ubuntu system.
Download the CUDA .deb package for Ubuntu 14.04 from here. For me, it was a cuda-repo-ubuntu1404_6.5-14_amd64.deb file.
The .deb file just adds a CUDA repository maintained by NVIDIA. Install this .deb file and update:
PyCUDA enables a Python program to pass data to and call CUDA kernels from a Python program. Getting it to install and work correctly on Ubuntu took a bit of work.
I tried installing the nvidia-343 drivers for Ubuntu. But, it turns out that only 340.x or earlier drivers support the GTS 250 I use for display (not for compute). So, I reverted back and installed nvidia-331 drivers. Note that the nvidia-331-updates driver will not work with CUDA either. Do not ask me why! 🙂
Installing the CUDA toolkit was easier, just install nvidia-cuda-toolkit package.
PyCUDA is available in Ubuntu as a python-pycuda package. But, that is the very old 2013.1.1 version. Instead I installed it from the Python Package Index (PyPI):
$ sudo pip install pycuda
The install script kept complaining about the absence of a configure.py script, but it seemed to end with success.
I have seen this error while compiling a CUDA program. I have also seen this error on running a PyCUDA program:
#error -- unsupported GNU version! gcc 4.9 and up are not supported!
I was using GCC 4.9.2 on this system, but I did not know why CUDA had a problem with these newer versions. Turns out that it does not have any problem, this warning just needs to be choked in the relevant header file.
For CUDA, this header file was /usr/local/cuda/include/host_config.h. For PyCUDA, this header file was /usr/include/host_config.h.
After installing PyCUDA, I tried to run a sample PyCUDA program and got this error:
$ python hello_cuda.py
Traceback (most recent call last):
File "hello_cuda.py", line 1, in <module>
File "/usr/local/lib/python2.7/dist-packages/pycuda/autoinit.py", line 4, in <module>
pycuda._driver.Error: cuInit failed: unknown
The error refers to line 4 in /usr/local/lib/python2.7/dist-packages/pycuda/autoinit.py. The lines relevant to that error in the file are:
import pycuda.driver as cuda
So, CUDA initialization is failing for some reason. After a bit of Googling, I found that this means that the necessary (CUDA? NVIDIA?) modules were not being loaded into the kernel. NVIDIA provides a modprobe package that can auto-load modules into the kernel as needed.
I installed this package:
$ sudo apt-get install nvidia-modprobe
CUDA was able to initialized immediately after this package was installed, no reboot was needed.
Tried with: PyCUDA 2014.1, CUDA 5.5, NVIDIA 331 drivers, GTX Titan and Ubuntu 14.04
The Ubuntu repositories now host CUDA packages. These may not be latest and greatest, but I find them to work well with the NVIDIA drivers from the same place. Also, they are very easy to install compared to the packages from the NVIDIA website.
First, install the latest NVIDIA driver available from the repositories. NVIDIA driver package is named as nvidia-xyz, where xyz is the version. Pick the largest number version that is available from the repositories. For example:
$ sudo apt install nvidia-331
The installation process compiles the driver for your particular Linux kernel and deploys that module. Restart the computer once the install is done.
You must be able to see a NVIDIA module when you list the kernel modules. For example, on my computer:
$ dkms status
nvidia-331, 331.38, 3.13.0-24-generic, x86_64: installed
Now you are ready to install CUDA. This is really easy since installing the package nvidia-cuda-toolkit will pull in all the hundred other CUDA packages and tools that are needed:
$ sudo apt install nvidia-cuda-toolkit
That is it, enjoy your CUDA programming! 🙂
Tried with: Linux kernel 3.13.0-24-generic, CUDA 5.5, NVIDIA driver 331, NVIDIA GTX Titan and Ubuntu 14.04
You have a GCC pragma in your CUDA code to ignore a specific warning produced by your code. While the GCC compiler uses this for the C++ section of the code, the CUDA compiler frontend complains about it with a warning:
foo.cu(44): warning: unrecognized GCC pragma
To request the frontend of the CUDA compiler to ignore or hide this warning, pass this option during compilation:
If you work with CUDA programs, you will use the Visual Profiler regularly. Another tool that can be useful is the commandline profiler, named nvprof. This does not have as many features of the Visual Profiler, but is very easy and quick to use.
Pass the name of your program to nvprof:
$ nvprof ./my-cuda-program
nvprof runs the program and gives a summary of results that is similar to the default output in Visual Profiler. It shows the kernels sorted in decreasing order of total execution times. The columns show the percentage of execution time, the actual time, the number of calls, the average-min-max of a single call for every kernel.