How to list environment variables used by program

Problem

Many programs fail or behave mysteriously because they are looking for or reading environment variables that you may not be aware of. It would be very useful if you could see what environment variables they are probing for.

Solution

If the program is using the getenv standard library call, then you can list all those calls, the environment variable requested for and what value was returned using ltrace:

$ ltrace -e getenv ./your_program

You will be surprised how many environment variables you had no idea about are probed by common programs that you use. I find this usage to be immensely useful in increasing my understanding of any program I use.

Advertisements

How to use ltrace

ltrace is an immensely useful tool to know what a program is doing. It displays the library calls made by a program that you execute. Just like strace, it will display the function called, list its input arguments and also display the return value from the function. Note that only calls made to dynamically linked libraries are reported. Not those to libraries that have been statically linked into the executable.

  • Installing ltrace is easy:
$ sudo apt install ltrace
  • Provide a program as argument when you invoke it:
$ ltrace ./foobar
$ ltrace touch

Note that it writes its output to stderr.

  • To redirect output of ltrace to a file:
$ ltrace -o foobar.log ./foobar
  • To filter and display only calls to a certain library function:
$ ltrace -e foobar_function_name ./foobar
$ ltrace -e malloc ./foobar

Note that you need to specify the exact name of the function. You can also use some method name and library name globs and regular expressions. For more info, please see the manual.

Tried with: ltrace 0.7.3

How to change Firefox double-click selection behavior

When you double-click text in Firefox, what is selected depends on the punctuation characters that are inside or around the text. By default, punctuation characters like dashes are not part of the word. So, you might find that a word-like-this is not fully selected on double-click.

This selection behavior is controlled by the preference layout.word_select.stop_at_punctuation. You can go to about:config and change it if you would prefer that punctuation characters be ignored.

How to configure local computer for FastAI course

I wanted to check out the Practical Deep Learning for Coders course by FastAI. However, I noticed that the course provided configuration instructions mainly for cloud GPU instance providers like Paperspace. I have a notebook and a desktop computer with powerful NVIDIA GPUs and wanted to try the course on my local machines. The course material is also provided in the form of Jupyter notebooks, while I intended to turn those into Python programs to run locally.

Here are the steps I followed to get my local computer setup for the FastAI course:

  • The local computer was running Ubuntu 16.04 and NVIDIA drivers were already installed on it and working.
  • CUDA 9.0 was installed using the online instructions from NVIDIA.
  • The latest release of CuDNN was installed as described here.
  • Conda was installed and configured as described here. You should be able to run conda info from the shell. I wanted to try the course without resorting to Anaconda, but that seems unnecessarily complicated.
  • Clone the fastai Github repo:
$ git clone git@github.com:fastai/fastai.git
  • In the fastai directory, use this command to create a Conda environment named fastai and install all the required Python packages (including PyTorch) under that:
$ conda env update
  • Activate the fastai environment:
$ conda activate fastai
  • Add the path to fastai to your PYTHONPATH environment variable, so that you can import it from Python.

  • Open a Python interpreter and check if you can import PyTorch and it has CUDA and CuDNN support:

$ python
>>> import torch
>>> torch.cuda.is_available()
True
>>> torch.backends.cudnn.enabled
True
  • Open a Python interpreter and check if you can import the fastai package:
$ from fastai.imports import *

You are now ready to execute any of the code shown in the course at a Python interpreter or inside Python scripts. Note that you will still need to download any additional datasets needed by the course. You will find these instructions in the Jupyter notebooks or course material.

byobu status bar shifting under ConEmu

Problem

I installed the latest stable release of ConEmu. I opened a Cygwin shell and used SSH to connect to a remote computer. I ran Byobu on the remote computer. I noticed that its status bar would shift upward, overwriting the entire screen after some time.

Solution

This problem is documented here. The solution suggested by the ConEmu creator is to use the ConEmu/msys terminal connector plugin. Using this connector is however a bit complicated. But, according to this post, it has been rolled into ConEmu starting with the 170705 release. I checked and found that the stable release I had installed was 161206. The stable release was years behind!

Seeing no recent stable version available, I opted to download a preview release from here. This release was dated later than 170705. byobu worked fine through SSH from Cygwin after that!

How to use Cygwin X server for local and remote

Cygwin not only provides most of the Linux programs and shells, it also comes with a builtin X server called Cygwin/X. This is great if you want to run Cygwin GUI applications locally or over SSH from a remote computer.

Install Cygwin/X

  • Install Cygwin, if you do not already have it.
  • Use the Cygwin setup program and install the xorg-server and xinit packages. The first one is the X server and the second installs a lot of scripts and programs that enable you to start the server and use it.
  • Install the xeyes package. This is a small X application that we will use for testing X.

Use Cygwin/X for local apps

  • Kill the xinit, XWin and other X server processes if they are running and close all your Cygwin windows. Open a new Cygwin window.
  • Set the DISPLAY environment variable:
$ export DISPLAY=:0.0
  • startx: If you run startx, you get X server opening a X window with XTerm running inside it. From here on, you can open X apps by running them from that XTerm. You can try by running xeyes inside that XTerm. Once you kill this X window, your X server is closed.

  • startxwin: If you run startxwin, you get X server running, but no windows are displayed. You can now open another Cygwin window and can invoke individual X apps there and they will be displayed in individual windows. You can try by running xeyes from the Cygwin shell. This is called the multiwindow mode.

Use Cygwin/X for remote apps

  • Follow all the steps shown above for local X apps. Make sure you have killed the earlier X server processes, set the DISPLAY environment variable locally and run startxwin. Make sure local X apps can display a window to the Cygwin/X server.
  • SSH to the remote computer running Linux using ssh on Cygwin or PuTTY. Make sure you have X11 Forwarding enabled for this SSH session. This is usually done by specifying -X (untrusted forwarding) or -Y (trusted forwarding) or setting the corresponding X11 Forwarding option in your SSH config file.
  • Check the DISPLAY on the remote computer. It should be set to some value like shown below:
$ echo $DISPLAY
localhost:11.0

If DISPLAY is empty, then either you did not set DISPLAY locally, did not cleanly restarted a X server locally or there was a problem with SSH X11 Forwarding. Please diagnose.

  • Run the xeyes program from the remote computer. It should display in a window on your local machine. Your Cygwin/X server is working! You can now run GUI/X apps from the remote computer and display them locally. Beware that they will be slow to display and refresh.

Tried with: Cygwin 2.10.0 and Windows 10

How to install Conda

Conda is a package and environment management system for Python. To install it on Ubuntu, I used these steps:

  • Download the bash shell script by clicking on the Download button here. For example, when I did this, I got a Anaconda3-5.1.0-Linux-x86_64.sh file.

  • Run the downloaded bash shell script through bash. For example, I did:

$ bash Anaconda3-5.1.0-Linux-x86_64.sh

  • The bash script asks you for a place to install Anaconda. Typically this is $HOME/anaconda3. But you can provide it any alternate location you wish. Conda will store the environments you create in an envs subdirectory inside this base directory.

  • The bash script asks if you want to add the bin subdirectory inside the base directory to your PATH environment variable. You can choose to do this on your own too. In either case, remember to make sure the Conda bin path is appended to PATH. If it is instead prepended to PATH, then Conda’s Python interpreter will be invoked as the default one at the shell. Typically, you do not want this behavior. You want Conda’s Python to be used only inside a Conda environment.

  • After the script is done, restart your shell, so that your PATH has the Anaconda bin subdirectory.

  • Run this command to check if everything is working:

$ conda info
  • I use the Fish shell, so I need to add this line to the config.fish to be able to use Conda’s environment activate and deactivate commands:
source (conda info --root)/etc/fish/conf.d/conda.fish

I was able to create environments and use Conda after this.

Tried with: Ubuntu 16.04

ImportError on IPython.paths

Problem

I installed a Python package using pip3. When I imported that newly installed package, it complained about IPython:

ImportError: No module named 'IPython.paths'

Solution

It turns out that only recent versions of the IPython package had IPython.paths. I had an older version of IPython installed using apt. The newly installed pip3 package was built around a more recent version of IPython. The problem went away once I removed the IPython packaged installed using apt and force installed its recent pip3 package:

$ sudo apt remove ipython
$ sudo pip3 install --upgrade ipython

Tried with: Ubuntu 16.04