How to disable Nouveau driver

Problem

Nouveau is the open-source graphics driver for NVIDIA graphics cards. When I tried to install a driver downloaded from the NVIDIA website on a computer, the installer complained like this:

The Nouveau kernel driver is currently in use by your system. This driver is incompatible with the NVIDIA driver, and must be disabled before proceeding. Please consult the NVIDIA driver README and your Linux distribution's documentation for details on how to correctly disable the Nouveau kernel driver.

Solution

To disable the Nouveau kernel driver:

Remove the installed NVIDIA drivers:

$ sudo apt-get purge "nvidia*"

Create a new file named /etc/modprobe.d/disable-nouveau.conf with the following lines:

blacklist nouveau
options nouveau modeset=0

Update the boot environment for your kernel:

$ sudo update-initramfs -u

Now reboot and you should get a low resolution GUI which indicates that Nouveau graphics driver is not being used.

Tried with: Ubuntu 14.04

The strange case of varying floats in Protobuf

Problem

I was using Google Protobuf in a Python program to read some text format Protobuf messages, merge them and write them out. Surprisingly, for the same set of input text format message files, I was getting different outputs on two computers! The values that were different were float values. The float values were generally correct, but varied slightly in precision between the two computers.

Solution

This strange observation took quite a long investigation. I initially assumed that maybe the Protobuf library (libprotobuf.so) or the Python Protobuf package were of different versions on these two computers. Surprisingly, they were exactly the same.

The mystery finally turned out to be the Protobuf implementation type. There are currently two possible types: cpp and python. By default, the cpp implementation is used. However, on one of the computers, the python implementation had been chosen by an engineer during the PIP package installation. The way to pick the engine is by setting an environment variable named PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION to either cpp or python. The engineer had set this environment variable in his shell when playing around with Protobuf and had later installed the PIP package.

Once I explicitly set the PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION environment variable manually in my Python code before importing protobuf, the float values were the same on both computers!

Now why should the engine affect the float value? Because Python’s float is actually double precision. On the other hand, when a 32-bit float moved between Python code and the C++ engine and back to Python code, it was sometimes changing precision. By using the same engine on all computers, we ensured that at least the float values did not vary between machines.

Tried with: Python Protobuf 3.3.0 and Ubuntu 14.04

Undefined reference to Boost copy_file

Problem

C++ code that uses Boost Filesystem was compiling fine with Boost 1.49. However, when I switched to Boost 1.55, the same code gave this error:

foobar.cpp.o: In function `do_something()':
foobar.cpp:(.text+0xb78): undefined reference to `boost::filesystem::detail::copy_file(boost::filesystem::path const&, boost::filesystem::path const&, boost::filesystem::copy_option, boost::system::error_code*)'

This was surprising since the declaration of the copy_file method was present in filesystem.hpp and libboost_filesystem.so was linked during compilation.

Solution

Turns out this is a known bug with Boost Filesystem as described here. Apparently, this happens only if the C++ code is compiled with the -std=c++11 option. I was indeed using that option.

The current fix for this bug is to temporarily disable the C++ scoped enums when the Boost Filesystem header file is included, like this:

#define BOOST_NO_CXX11_SCOPED_ENUMS
#include <boost/filesystem.hpp>
#undef BOOST_NO_CXX11_SCOPED_ENUMS

Tried with: Boost 1.55, GCC 4.8.4 and Ubuntu 14.04

How to create GIF from images using ImageMagick

ImageMagick is truly a magical tool for images. It makes it super-easy to create an animated GIF file from a series of image files.

To create an animated GIF from 3 input images with a delay of 2 seconds between each frame and loop infinitely:

$ convert -delay 200 -loop 0 1.png 2.png 3.png out.gif

Tried with: ImageMagick 6.8.9.9 and Kubuntu 16.04

How to grep in Vim

When you are editing files in Vim, you might feel like finding out which other files have a certain text. Vim has such grep features built into it to support this usage. There are many popular Vim plugins that offer similar features, but you might want to first examine what Vim grep offers. In my opinion, most users should find the grep features built inside Vim to be quite adequate for their use.

Right off the bat, you need to know that there are two types of grep that are available in Vim: internal grep and external grep.

And you should remember that all grep operations operate from the present working directory (PWD) of Vim. You can check what is the PWD using the command :pwd. So, it is to your advantage to invoke Vim from the base directory of a code base.

Internal grep

This is the grep feature that is implemented inside Vim. Not surprisingly, this internal grep is far easier to use compared to Unix grep. However, it is also noticeably slower compared to Unix grep, for obvious reasons. However, the ease of use of internal grep is so high that I tend to prefer it over external grep.

The command to invoke internal grep is :vimgrep. But, you can use the short form :vim. Yeah this is the one command that won the Vim lottery to have the same name as the editor! 😄

This command in its most common form takes two arguments: the pattern to search for and which files to search for.

To search for Xanadu in all the files in the PWD: :vim Xanadu * This shows all the search results in a Quickfix window at the bottom and Vim jumps to the first match right away. You can navigate through this window and press Enter on search results to jump to them.
To recurse the directories or files in Vim, you have this strange syntax called star-star. For example, to search recursively in all files and directories in PWD: :vim Xanadu **
Once you are comfortable with the star-star syntax, you will find it very convenient to use: :vim Xanadu ../include/**/*.h
One trick I like to use a lot is to search for the current word under the cursor in the files. I first press * so that the word is selected. Then I use this empty pattern to search for it: :vim // *.cpp

External grep

External grep is nothing but Vim invoking the commandline grep in a shell child process. This is super dumb: Vim just invokes whatever you pass it verbatim and grabs the grep output, parses it and fills it in the Quickfix window.

To search in all files in the PWD: :grep Xanadu *
To search recursively in all files and directory under the PWD: grep -R Xanadu * Note how the recursive search is far easier to invoke in the internal grep.

Quickfix window trick to browse code

No matter whether you use internal or external grep, its results are filled in the Quickfix window. As you do new searches, new Quickfix windows are created and the latest one is displayed at the bottom. You can move through this list of Quickfix windows using the :colder and :cnewer commands.

With these grep features and Quickfix window history Vim can be used as an rudimentary IDE to examine a code base: use the grep commands to search for text (variables, methods or classes) and jump to them. And you can move back and forth in this investigation using the Quickfix window commands described above.

Tried with: Vim 7.4 and Kubuntu 16.04

Grep cheatsheet

Grep is the quintessential text search tool of the Unix shell. Many text search tools like ack and ag are popular now for searching in source code. But, for most common scenarios grep can still be a fast and good enough solution. It is available on every Linux machine you will be working at, so it is a huge bonus to be aware of its usage and capabilities.

Trivia: If you type :help grep in the Vim editor, you will see the origin of the name grep:

[Unix trivia: The name for the Unix "grep" command comes from ":g/re/p", where
"re" stands for Regular Expression.]

The most important use of grep is to search for text from standard input. This is usually done by piping output of another command to grep and searching in that. For example:

$ ls -1 | grep Xanadu

Note that the search text can be a regular expression. You can find more info about regular expressions from other detailed sources.

The other common use of grep is to search for text in files. For example:

$ grep Xanadu *.txt

Note that the *.txt is expanded by the shell and those filenames are passed to grep. Grep does not lookup the filenames.

To ignore case of the search text (make it case insensitive), use the -i option:

$ grep -i Xanadu *.txt

Since the filename lookup is actually done by the shell, it is not recursive by default. To search recursively in all files and directories starting from current directory:

$ grep -R Xanadu *

Note again, that the wildcard is expanded by the shell and passed to grep. Grep then takes each of those directories and it recurses through them on its own.

To exclude certain files, say save files, from grep use the --exclude option:

$ grep --exclude="*~" Xanadu *

To exclude multiple types of files, specify multiple --exclude options.

To exclude certain directories, say Git directories, from grep use the --exclude-dir option:

$ grep --exclude-dir=".git" Xanadu *

To exclude files based on glob patterns, you can also put those patterns in a file and pass it to the --exclude-from option. However, this did not seem to work for me.
By default, grep prints the file paths which have the search text. To print the line number, along with the file paths, use the --line-number option:

$ grep --line-number Xanadu *

Grep throws out a wall of text where the searched text can be hard to find. Asking it to use color using --color can be an useful option:

$ grep --color Xanadu *

On my computer, this shows the file path in purple, the line number in green and the searched text in red.

When searching text in source code files, it is helpful to get some context of where the text is found. Using the -C option you can ask grep to show you some lines of context around the result lines. For example, to get 3 lines of context above and 3 lines of context below every result line:

$ grep -C 3 Xanadu *.cpp

Your shell expands both matching filenames and directories and passes them as input to grep. When grep gets a directory as input, it warns that it Is a directory. For other files it might also give warnings such as No such device or address or Permission denied. To hide such messages use the --no-messages option:

$ grep --no-messages Xanadu *

Grep does look through binary files by default. If the searched text was found, it will just report that the binary file has that text, it does not try to print the matching line, for obvious reasons (no line separators in binary file). However, if you do not want grep poking around in binary files, use the -I or --binary-file=without-match option:

$ grep --binary-file=without-match Xanadu *

Tried with: Grep 2.25 and Ubuntu 16.04

How to debug Caffe

Caffe is written in C++ while its GPU accelerated portions are written in CUDA. It also ships with PyCaffe, a Python wrapper to the C++ code.

To debug Python code of PyCaffe

You might have written a Python script to train or load and use models for inference. The Python code of Caffe can be debugged as usual using the PDB Python debugger as described here.

To debug C++ code from Caffe binary

We can use GDB to debug the C++ code in Caffe. First, remember to build Caffe with debugging information. This can be done as described here, but remember to indicate Debug mode to CMake, like this:

$ cmake -DCMAKE_BUILD_TYPE=Debug ..

If you are using binary Caffe, then debugging the C++ code is straightforward using GDB:

$ gdb --args ./caffe --your-usual-caffe-arguments-go-here

You can set breakpoints in any Caffe C++ code and debug using GDB as usual. For more info see my GDB cheatsheet.

To debug C++ code from PyCaffe

If you are using PyCaffe and need to debug the C++ parts of Caffe code that is still possible using GDB! Remember to first build Caffe in Debug mode as shown above. Note that after make install, you might have to rename the _caffe-d.so file to _caffe.so to be able to import caffe in your Python script.

To be able to debug the C++ code, we invoke GDB and pass it the python interpreter and its arguments, which is our Python script that calls PyCaffe:

$ gdb --args python my_script_calls_pycaffe.py --some_input_arguments

GDB first begins in the Python interpreter binary. Python will later load all the required shared libraries including _caffe.so and more importantly libcaffe-nv.so which is the compiled version of the Caffe C++ code. The problem is that these libraries are not yet loaded at the beginning. What we can do is set one or more breakpoints at the places we want right away. GDB will complain that these breakpoint locations are as yet not known to it. That is okay, it will turn them into pending breakpoints that will be enabled when the shared library having that location is loaded.

For example, to set up a breakpoint right away in convolution layer code:

(gdb) b base_conv_layer.cpp:15

After you have set your breakpoints, press c to continue and GDB will load the shared libraries and stop when your breakpoint locations are hit. After this point it is debugging as usual in GDB.

If you are adventurous, you can even connect GDB to the Python process that is running PyCaffe/Caffe by using its PID. For example, after I find out that my Python script is running as PID 589:

$ gdb -p 589

You can set breakpoints in Caffe C++ code and GDB will stop at those locations.

I hope you have fun stepping through and exploring Caffe code! 🙂

Tried with: GDB 7.7.1 and Ubuntu 14.04

How to group windows as tabs in KDE

KDE/Plasma/KWin 4.x that ships in Ubuntu 14.04 has one amazing feature that I discovered today. I keep many XTerm, RXVT and related file explorer windows open. These were related to many servers I was SSHed to and I was looking for a way to group windows related to each server together. Guess what, KDE 4.x has a fantastic solution!

KDE 4.x allows you to group any set of open application windows together as tabs. Just right-click on the titlebar of an application and choose Attach as tab and pick the other application with which you want to group this one. These two application windows will now appear in a single window as tabs! You can repeat this operation and group together apps as tabs inside super-windows. (In older KDE, this option appears as Move window to group.) This works for any application window, they can be from different applications. What a great way to organize your windows on the desktop!

Sadly, this feature is missing in the recent KDE/Plasma/KWin 5.x series that ships in Ubuntu 16.04. It lies on the missing features list and I really hope they will reimplement this great feature.

How to view CMake cache variables

Running CMake for a project creates many variables that are required for its own operation and for the project being built. If you use the CMake GUI, then these are the variables which you see populated after you perform Configure and then Generate. When you want to diagnose a CMake build problem at the shell, it is useful to be able to view these variables.

To perform a CMake build and then print the cache variables:

$ cmake -L ..

Note that in the above command, I am in the build directory and the CMakeLists.txt is in the parent directory. This command builds as usual and at the end prints the CMake variables and their values.

To perform a CMake build and list all the cache variables, including advanced variables:

$ cmake -LA ..

Note that the number of variables that are printed can be quite high for big projects. I have seen it run into the hundreds.

To perform a CMake build and list all the cache variables and print the help documentation about each variable too:

$ cmake -LAH ..

If you have already done a CMake build and just need to print out the cache variables, add the -N option:

$ cmake -L -N

This command assumes that the current directory is the build directory containing the cache files, so no directory needs to be specified. This command is quick because it does not do a CMake build. Additionally, you can also add the -AH options as before if you want.

Tried with: CMake 2.8.12.2 and Ubuntu 16.04

Code Yarns

How to disable Nouveau driver

Problem

Solution

Firefox tooltip with terrible colors

Problem

Solution

The strange case of varying floats in Protobuf

Problem

Solution

Undefined reference to Boost copy_file

Problem

Solution

How to create GIF from images using ImageMagick

How to grep in Vim

Internal grep

External grep

Quickfix window trick to browse code

Grep cheatsheet

How to debug Caffe

To debug Python code of PyCaffe

To debug C++ code from Caffe binary

To debug C++ code from PyCaffe

How to group windows as tabs in KDE

How to view CMake cache variables

Problem

Solution

Share this:

Like this:

Problem

Solution

Share this:

Like this:

Problem

Solution

Share this:

Like this:

Problem

Solution

Share this:

Like this:

Share this:

Like this:

Internal grep

External grep

Quickfix window trick to browse code

Share this:

Like this:

Share this:

Like this:

To debug Python code of PyCaffe

To debug C++ code from Caffe binary

To debug C++ code from PyCaffe

Share this:

Like this:

Share this:

Like this:

Share this:

Like this: