How to print in a Makefile

Makefiles and especially recursive Makefiles can be hard to understand and debug. But the good old debugging method of printing out values from certain locations in the source code can be employed here too.

  • To print a message when make parses past that location in the Makefile:
$(info Hello World Make just parsed past this line)

Note that make typically parses a file twice before it executes what is needed to satisfy its targets. So, you will see the message you are printing twice.

  • To print the value of a make variable at a certain location in the Makefile:
$(info $(FOOBAR_VAR))

How to debug make using remake

In big projects that use Make, it can be hard to examine how a make build is working and to diagnose a build problem. Imagine my joy when I finally discovered ReMake. This is a debugger, you heard it right, built off a version of GNU Make. It essentially allows you to step through a make build just like you would step through a program using GDB and examine state at any point in time.

  • Installing remake is easy on Ubuntu:
$ sudo apt install remake
  • Using remake is as easy as replacing your make command with it:
$ remake
$ remake some_target
  • If you want a verbose output of what is happening with your build use -x option:
$ remake -x
  • To actually debug a build with its debugger:
$ remake -X

The debugger jumps and waits at the line with the target that it needs to execute first. Use the help command to get a list of commands that you can use. They are all similar to GDB: step to step into, next to execute next line, print to print values of make variables (this is a lifesaver!) and so on.

Tried with: remake 3.82 and Ubuntu 16.04

NVCC argument redefinition error

Problem

I was trying to build a CUDA project that worked on one computer on a second computer. Though this project built without problems on the first computer, nvcc gave an argument redefinition error on the second computer:

nvcc fatal   : redefinition of argument 'std'

Solution

At first, I thought that the compilation error was arising from the source code. Actually, the error is being reported about a compilation argument to nvcc. It is saying that a std argument has been provided more than once.

This project was using CMake, so the actual compilation commands invoked by make are hidden. I used the VERBOSE trick, described here, to view the actual commands issued to the C++ and CUDA compilers. Not surprisingly, I found that the -std=c++11 argument was being passed twice. But why?

I checked the CMakeLists.txt file and found that indeed the C++11 argument was being passed to the compiler twice by setting it in both CMAKE_CXX_FLAGS and CUDA_NVCC_FLAGS. How and why was this working fine on the first computer? Turns out that the older CMake used on the first computer would not pass the C++ flag to NVCC, so it had to be specifically redefined for CUDA compiler flags.

The second computer was using a newer version of Ubuntu, with newer version of CMake. This was intelligent enough to pass a C++ flag to the CUDA NVCC compiler too. But since the NVCC flag of the same name also existed, it was causing the argument redefinition error. Removing the flag from the CUDA flags made the problem go away.

Clock skew detected during make

Problem

I was trying to build source files on a remote server using make. The make command would sometimes throw up a clock skew warning like this:

make[2]: Warning: File `foobar.cpp' has modification time 36 s in the future
make[2]: warning:  Clock skew detected.  Your build may be incomplete

Solution

It turns out that these source files were actually shared from another computer to this remote server using NFS. Using the date command at the local computer and the remote server, I found that there was a difference of a few seconds between the two.

More investigation showed that only the local computer was using NTP and so its time was correct. The remote server was not using NTP and so its time was wrong.

To install and enable NTP on the remote server, I used these commands:

$ sudo apt install ntp
$ sudo service ntp reload

Note that it takes a few minutes for the synchronized correct time to show up on the computer after these commands. After the sync, I no longer get the clock skew warnings from make.

Reference: Time Synchronisation with NTP

Tried with: Ubuntu 16.04

How to make in parallel

Building the source code of big projects can take a lot of time. By default, the make command builds one target at a time. Since most of our computers have multiple cores, this process can be sped up by building multiple targets in parallel. This can be done easily by using the --jobs or -j parameter.

  • make -j will try to build all targets in parallel whose dependencies are available. This will swamp your machine with build tasks.
  • make -j 6 will try to build a maximum of 6 (or any non-zero positive integer you specify) targets in parallel. The actual number of builds done in parallel might be less depending on how many targets have their dependencies ready.
  • If you use parallel make frequently, then it is a good idea to turn it into an alias in your shell.
  • The -j option can be set in the MAKEFLAGS environment variable, if you want parallel make to be used whenever make is invoked. This is better than an alias since parallel make is used even if make is called inside programs which are not using (or not aware of) your alias.
  • You may want to use the same alias or MAKEFLAGS variable across machines with varying number of CPU cores. One way to solve this problem is to use the output of the nproc --all command to set the number of parallel jobs.

How to uninstall a make install

Many libraries can only be installed by building from source and installing using make install. Removing these installations can be quite painful.

In some rare cases, these Makefiles are written to support uninstall. In that case, you can go the same directory from where you did make install and try make uninstall. The only library I’ve seen supporting this is OpenCV.

For other installs which do not support uninstall, I highly recommend installing using checkinstall as described here.

Tried with: Ubuntu 14.04

How to set makeprg based on filetype in Vim

The command string set in makeprg in Vim is applied to compile a file or a project when you invoke the :make command.

For most types of files, this usually just invokes the make program. You can see what makeprg is set to for a certain filetype, by opening a file of that type and trying :set makeprg?

You can set any program to run on the current file or any other file when :make is invoked, by setting makeprg yourself.

To set it based on a certain filetype, say for filetype foo, add this to your Vimrc:

autocmd Filetype foo setlocal makeprg=/bin/foo_compiler %

The percent character denotes the path to the currently open file.

Tried with: Vim 7.4 and Ubuntu 14.04

Dispatch plugin for Vim

20150328_vim_dispatch

Vim can be used to build your code by using the :make command. It runs the command appropriate for your compiler, parses the compile errors or warnings and provides them in the Quickfix window. The only problem with this is that the build process is synchronous: the build command takes over the terminal and until it is over you can neither view nor interact with Vim.

The Dispatch plugin by Tim Pope enables you to run the build command asynchronously. The plugin can be installed from source here. You have two choices: run the build command asynchronously in the foreground or in the background.

  • Run :Make to run the build asynchronously in the foreground. This is possible only if you are using a terminal multiplexer like Tmux. A split terminal opens up at the bottom and you can see the output of your build there. The focus remains in Vim and you can view and interact with the editor while the build carries on. If you are running Vim in a normal terminal, this command just runs the plain old :make synchronously.

  • Run :Make! to run the build asynchronously in the background. This can be run on Vim running in any setup. The build command runs in the background invisibly and you can view and interact with Vim while this is happening.

  • Run :Copen to view the build output either during the build or after it is complete in foreground or background.

I highly recommend this plugin for anyone who is already using :make since with no change you can now build asynchronously.

Tried with: Vim 7.4 and Ubuntu 14.04

errorformat in Vim

Vim can parse compilation errors when it is passed as an error file using the argument -q. It can also parse the errors produced by running :make. In both these cases, the file path, line and column number of errors can be parsed and traversed using :cnext or displayed in the QuickFix window.

The errorformat is the list of error formats that Vim uses to parse and recognize errors from input text. The default error format in Vim can be seen by trying the command :set errorformat?. The items in the list of error formats are separated by a comma. When separated, these are the components of the list:

This default error format is set in Vim as DFLT_EFM based on the platform as seen in the source code here. Other error formats can be set at runtime based on the compiler flag. The different options for compiler and their corresponding error formats can be seen listed in the files in the directory /usr/share/vim/vim74/compiler. In fact, the default error format string set on Linux is the same as that in /usr/share/vim/vim74/compiler/gcc.vim

For more on error format see :help errorformat in Vim.

Tried with: Vim 7.4 and Ubuntu 14.04

How to open Quickfix window automatically after make

The Quickfix feature of Vim can be used to view errors or warnings produced by a compilation in a window and jump to those locations in source files. The Quickfix window has to be manually opened using :copen. Instead, I like the Quickfix window to be automatically opened after running a :make.

This can be done by adding these line to .vimrc:

" Open Quickfix window automatically after running :make
augroup OpenQuickfixWindowAfterMake
    autocmd QuickFixCmdPost [^l]* nested cwindow
    autocmd QuickFixCmdPost    l* nested lwindow
augroup END

For more info, see this Vim Wiki entry.

Tried with: Vim 7.4 and Ubuntu 14.04