This blog has moved

This blog has moved to: codeyarns.github.io/tech

After more than a decade of hosting on WordPress, I decided to move to my own light blog platform.
I will miss the ease of posting and the availability of WordPress hosting.

I will not be posting any new blog posts here. All the new posts will be happening at the above new location.

The posts here will remain until I figure how to redirect them or replace this domain with my new blog.

Lubuntu long boot on Inspiron 1525

Problem

I installed Lubuntu 18.04, the LXDE variant of Ubuntu, on an old Dell Inspiron 1525 laptop. It worked, but every bootup would take several long minutes.

Solution

Turns out that this is a known problem for this Dell laptop reported here. I followed these steps:

  • Opened Grub config file:
$ sudo vi /etc/default/grub
  • Change the Grub commandline option to be:
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash video=SVIDEO-1:d"
  • Update Grub and reboot:
$ sudo update-grub
$ sudo reboot

Boot time dropped dramatically to about a minute after this change.

WSL paths in Windows

Windows Subsystem for Linux (WSL) allows you to install Linux distributions such as Ubuntu under Windows. The root filesystem you see under WSL is located at a directory inside %LOCALAPPDATA% in Windows.

For example, the root directory of Ubuntu 18.04 can be located in Windows here: %LOCALAPPDATA%\Packages\CanonicalGroupLimited.Ubuntu18.04onWindows_79rhkp1fndgsc\LocalState\rootfs

The strange case of the notebook fan that would started spinning after a few hours

Problem

A problem started to happen last week with my Dell notebook running Windows 10. After I kept it open for a few hours, its fan would start spinning noisily and never stop. This started to happen without a fail every day. The only brute force solution that worked was to restart Windows. That would keep the fans silent for a few hours before they would start all over again.

Investigation

From the minimized Task Manager CPU graph I could see that all 8 logical CPU cores seem to have been engaged 100 percent by some process. Opening Task Manager to watch what process was causing this was a dead end because Task Manager window would freeze when restored. In the end, I had to try the same with Process Explorer and leave its window open to catch the rogue service: WmiPrvSE.exe.

Looking into this service turned out to be a dead end. It did not lead to any suitable culprit and it was not clear why this service was going all hammer and tongs on the cores.

I then looked at the problem from a different angle: this problem was new. What had changed on my Windows recently? I had installed the 1809 update for its WSL feature. And after that I had switched over from Cygwin to Ubuntu 18.04 running on WSL. And indeed the problem seemed to only occur when I had Ubuntu open for a few hours.

The next time this happened, I tried htop in Ubuntu. Just like Task Manager and Process Explorer, htop too hung. But I left it open nevertheless and after a while it showed that there were hundreds of wget and timeout 1 wget commands taking over all the cores! Who was running this automatically after a few hours in Ubuntu?

Using Google helped to reveal a new suspect: byobu. This is the terminal multiplexer that I use in Ubuntu. Checking its source code confirmed that it was using wget commands to get the host IP address (don’t ask me why!). I am guessing it does it periodically to keep its IP address updated. And this simple lookup was somehow acting like a virus outburst after a few hours. Why?

It turns out that WSL has a serious bug: if you open a large number of non-blocking TCP connections its ephemeral ports are exhausted. I guess the byobu’s regular requests, when performed over a span of hours, exhausted this. And after that the LxssManager service, which is the layer that enables ELF executables to be executed on Windows, would go crazy. The only solution at this point: restart LxssManager.

Solution

So the solution is: if possible do not use Byobu on Ubuntu WSL. If you get this problem, then restart LxssManager by opening a CMD window with Administrator privilege and running these commands:

sc stop LxssManager
sc start LxssManager

Note that stopping the LxssManager service kills Ubuntu.

How to add OCR to PDF with OCRMyPDF

PDF files created by scanning pages or by printing or exporting from other file formats may not have a text layer. Without a text layer, every page is just an image on which you will not be able to search or highlight. The OCRMyPDF tool can be used to add an OCR text layer to any PDF easily.

  • Installing is easy:
$ sudo apt install ocrmypdf
  • Usage is straightforward:
$ ocrmypdf in.pdf out.pdf

I noticed that adding an OCR text layer increased the PDF file size by 1.5x! The tool also mentions at the end that this file size increase is surprising.

How to convert DjVu to PDF using Foxit Reader

There are several ways to convert DjVu file to PDF. All these methods result in PDF file sizes that are quite large, in hundreds of MBs. I have found that the converting using Foxit Reader results in the smallest PDF file sizes.

To do this:

  • Install the free Foxit Reader from here. This also installs a PDF Printer which we will use for conversion.

  • Install any DjVu viewer. I use WinDjView from here.

  • Open the DjVu file in WinDjView. Choose to print it. In the printer list, choose Foxit Reader PDF Printer. This printer has settings that you can modify. For the print settings, choose a page size (Letter) and remember to choose Scale to fit media. This will use the Foxit PDF Printer to write a PDF file for you.

  • Note that the settings of the Foxit Reader PDF Printer did not seem to have much effect on the file size. Default is 600 DPI. I did not see much reduction in filesize by using 300 or 200 DPI.

  • Note that the resulting PDF does not have OCR. You would need to run it through a OCR tool to get text embedded in the PDF.

Tried with: Foxit Reader 7.2.8.1124, WinDjView 2.1 and Windows 10

How to sync MEGA on Raspbian

MEGA is a cloud storage provider that can be used to store files and sync them across multiple devices. MEGA has a fully featured commandline client and daemon for Linux distributions, including Raspbian.

  • Raspbian client can be downloaded here.

  • Install packages required by the MEGA client:

$ sudo apt install libc-ares2 libcrypto++6
  • Install the downloaded .deb file:
$ sudo dpkg -i megacmd-Raspbian_9.0_armhf.deb
  • To login into your MEGA service:
$ mega-login login your-login your-password
  • To list files and directories on your MEGA drive in the cloud:
$ mega-ls /
  • To sync a folder on MEGA to your local storage, use this command:
$ mega-exec sync /path/on/local/system /path/on/mega

This passes the command to the mega service running in the background and you get your shell back. Paths on your MEGA drive begin with / as the root directory.

USBMount not mounting problem

Problem

I had a portable harddisk connected using USB to my Raspberry Pi B+ Rev3 running Raspbian 9. The harddisk had two partitions: NTFS and ext4. I had usbmount installed and configured as described here. When Raspbian is booted up with the harddisk already plugged in, I could see using sudo fdisk -l that the ext4 partition was visible. So, usbmount should have automatically mounted this ext4 partition to /media/usb0, but it was not doing that.

I had verbose logging enabled for usbmount as described here. When I searched the system log I could see that usbmount was actually mounting the ext4 partition, but surprisingly it was not visible at /media/usb0:

$ cat /var/log/syslog | grep usbmount
980:May 18 11:42:28 my-pi usbmount[373]: /dev/sda2 contains filesystem type ext4
989:May 18 11:42:28 my-pi usbmount[373]: mountpoint /media/usb0 is available for /dev/sda2
991:May 18 11:42:28 my-pi usbmount[373]: executing command: mount -text4 -osync,noexec,nodev,noatime,nodiratime /dev/sda2 /media/usb0
1010:May 18 11:42:28 my-pi usbmount[373]: executing command: run-parts /etc/usbmount/mount.d
1013:May 18 11:42:28 my-pi usbmount[373]: usbmount execution finished

Solution

Turned out that systemd was the culprit. It was having a MountFlags=slave in its systemd-udevd.service file. So, the mount operation of usbmount was mounting into the namespace of systemd and not to the user.

To change this mount option, run this command:

$ sudo systemctl edit systemd-udevd

Add these lines to the file that is opened:

[Service]
MountFlags=shared

I found that this created the file /etc/systemd/system/systemd-udevd.service.d/override.conf with the above lines.

Restart systemd or your Pi:

$ sudo systemctl daemon-reload
$ sudo service systemd-udevd --full-restart

After the restart, I found that usbmount had automatically mounted my ext4 partition.

References:

How to run rsync on path with spaces

Problem

When either or both of the source and destination paths in a rsync command has spaces, it does not work as expected:

$ rsync joe@foobar:"path/to/some thing" .
$ rsync joe@foobar:path/to/some\ thing .

Enclosing the path in double quotes (like shown above) or using slashes to escape the space does not work.

Solution

The reason the above solutions do not work is that the local shell is acting on the path and stripping its quotes or slashes before passing it to rsync.

So, we double up using both strategies and it works:

$ rsync joe@foobar:"path/to/some\ thing" .