How to sync MEGA on Raspbian

MEGA is a cloud storage provider that can be used to store files and sync them across multiple devices. MEGA has a fully featured commandline client and daemon for Linux distributions, including Raspbian.

  • Raspbian client can be downloaded here.

  • Install packages required by the MEGA client:

$ sudo apt install libc-ares2 libcrypto++6
  • Install the downloaded .deb file:
$ sudo dpkg -i megacmd-Raspbian_9.0_armhf.deb
  • To login into your MEGA service:
$ mega-login login your-login your-password
  • To list files and directories on your MEGA drive in the cloud:
$ mega-ls /
  • To sync a folder on MEGA to your local storage, use this command:
$ mega-exec sync /path/on/local/system /path/on/mega

This passes the command to the mega service running in the background and you get your shell back. Paths on your MEGA drive begin with / as the root directory.

Advertisements

USBMount not mounting problem

Problem

I had a portable harddisk connected using USB to my Raspberry Pi B+ Rev3 running Raspbian 9. The harddisk had two partitions: NTFS and ext4. I had usbmount installed and configured as described here. When Raspbian is booted up with the harddisk already plugged in, I could see using sudo fdisk -l that the ext4 partition was visible. So, usbmount should have automatically mounted this ext4 partition to /media/usb0, but it was not doing that.

I had verbose logging enabled for usbmount as described here. When I searched the system log I could see that usbmount was actually mounting the ext4 partition, but surprisingly it was not visible at /media/usb0:

$ cat /var/log/syslog | grep usbmount
980:May 18 11:42:28 my-pi usbmount[373]: /dev/sda2 contains filesystem type ext4
989:May 18 11:42:28 my-pi usbmount[373]: mountpoint /media/usb0 is available for /dev/sda2
991:May 18 11:42:28 my-pi usbmount[373]: executing command: mount -text4 -osync,noexec,nodev,noatime,nodiratime /dev/sda2 /media/usb0
1010:May 18 11:42:28 my-pi usbmount[373]: executing command: run-parts /etc/usbmount/mount.d
1013:May 18 11:42:28 my-pi usbmount[373]: usbmount execution finished

Solution

Turned out that systemd was the culprit. It was having a MountFlags=slave in its systemd-udevd.service file. So, the mount operation of usbmount was mounting into the namespace of systemd and not to the user.

To change this mount option, run this command:

$ sudo systemctl edit systemd-udevd

Add these lines to the file that is opened:

[Service]
MountFlags=shared

I found that this created the file /etc/systemd/system/systemd-udevd.service.d/override.conf with the above lines.

Restart systemd or your Pi:

$ sudo systemctl daemon-reload
$ sudo service systemd-udevd --full-restart

After the restart, I found that usbmount had automatically mounted my ext4 partition.

References:

How to run rsync on path with spaces

Problem

When either or both of the source and destination paths in a rsync command has spaces, it does not work as expected:

$ rsync joe@foobar:"path/to/some thing" .
$ rsync joe@foobar:path/to/some\ thing .

Enclosing the path in double quotes (like shown above) or using slashes to escape the space does not work.

Solution

The reason the above solutions do not work is that the local shell is acting on the path and stripping its quotes or slashes before passing it to rsync.

So, we double up using both strategies and it works:

$ rsync joe@foobar:"path/to/some\ thing" .

How to install VeraCrypt on Raspbian

  • Install libraries required by VeraCrypt:
$ sudo apt install libfuse-dev libwxbase3.0-dev
  • Download the latest version of Veracrypt that has a Raspbian package from here. The latest version for Raspbian I could find was v1.21. It can be downloaded using this command:
$ wget -L -O veracrypt-1.21-raspbian-setup.tar.bz2 https://sourceforge.net/projects/veracrypt/files/VeraCrypt%201.21/veracrypt-1.21-raspbian-setup.tar.bz2/download
  • Uncompress the file:
$ tar xvf veracrypt-1.21-raspbian-setup.tar.bz2
  • Make the installer as executable and run it:
$ chmod +x veracrypt-1.21-setup-console-armv7
$ sudo ./veracrypt-1.21-setup-console-armv7
  • That is it! Try it out:
$ veracrypt --version
VeraCrypt 1.21
  • In case you need to uninstall it in the future, this is how to do it:
$ sudo /usr/bin/veracrypt-uninstall.sh

Tried with: Raspbian 9

CPU feature Caffe2 warning

Problem

Running a Caffe2 C++ application produces these messages:

$ ./foobar
E0514 20:37:31.503541 26925 init_intrinsics_check.cc:43] CPU feature avx is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
E0514 20:37:31.504768 26925 init_intrinsics_check.cc:43] CPU feature avx2 is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
E0514 20:37:31.504787 26925 init_intrinsics_check.cc:43] CPU feature fma is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.

Solution

Caffe2 is informing that the Intel CPU being used has AVX, AVX2 and FMA acceleration features. However, Caffe2 has not been compiled with support for these features. Compiling Caffe2 with support to use these features would speedup training and inference using Caffe2.

To enable use of these features when compiling Caffe2, enable the USE_NATIVE_ARCH option like this:

$ cmake -DUSE_NATIVE_ARCH ..

How to install Intel MKL

Intel Math Kernel Library (MKL) provides math routines optimized for Intel CPUs. It is available free for personal and community use.

  • Register and download MKL from here. I prefer to choose the full package.
  • For Linux, the downloaded file is a zipped tar file. For example: l_mkl_2019.3.199.tgz. Unzip its contents.
  • Run its installer script:
$ sudo ./install.sh

The installer takes you through a console install wizard. By default, it will install to /opt/intel. By default, it will install MKL libraries for C/C++ and Fortran. I typically request not to install the Fortran libraries which reduces the install size by half.

Tried with: Ubuntu 18.04

past.builtins ImportError

Problem

Calling a Caffe2 Python call gave this error:

$ python -c "from caffe2.python import core"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/caffe2/python/core.py", line 9, in <module>
    from past.builtins import basestring
ImportError: No module named past.builtins

Solution

This requires the future package. Installing that solved the problem:

$ pip install --user future

Tried with: Raspbian 9

serialized_options TypeError

Problem

I built and installed Caffe2. Running a simple Caffe2 Python call, gave this error:

$ python -c "from caffe2.python import core"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/caffe2/python/__init__.py", line 2, in <module>
    from caffe2.proto import caffe2_pb2
  File "/usr/local/lib/python2.7/dist-packages/caffe2/proto/__init__.py", line 11, in <module>
    from caffe2.proto import caffe2_pb2, metanet_pb2, torch_pb2
  File "/usr/local/lib/python2.7/dist-packages/caffe2/proto/caffe2_pb2.py", line 23, in <module>
    \x44\x65viceTypeProto\x12\r\n\tPROTO_CPU\x10\x00\x12\x0e\n\nPROTO_CUDA\x10\x01\x12\x10\n\x0cPROTO_MKLDNN\x10\x02\x12\x10\n\x0cPROTO_OPENGL\x10\x03\x12\x10\n\x0cPROTO_OPENCL\x10\x04\x12\x0f\n\x0bPROTO_IDEEP\x10\x05\x12\r\n\
tPROTO_HIP\x10\x06\x12\x0e\n\nPROTO_FPGA\x10\x07\x12\x0f\n\x0bPROTO_MSNPU\x10\x08\x12\r\n\tPROTO_XLA\x10\t\x12\'\n#PROTO_COMPILE_TIME_MAX_DEVICE_TYPES\x10\n\x12\x19\n\x13PROTO_ONLY_FOR_TEST\x10\xa5\xa3\x01')
TypeError: __new__() got an unexpected keyword argument 'serialized_options'

Solution

This is caused by an older version of Protobuf. Updating that solved the problem:

$ pip install -U protobuf

Tried with: Raspbian 9

Out of memory error on Raspbian

Problem

I was compiling some templated C++ on a Raspberry Pi when the compilation failed with this error:

cc1plus: out of memory allocating 66574076 bytes after a total of 98316160 bytes

Solution

The Raspberry Pi had 1GB of RAM. From the error it looked like the compiler needed more memory than that.

It turns out that Raspbian does not use a swap partition. Instead it uses a /var/swap file on the SD card. This is called dphys-swapfile. The default size of this swapfile is 100MB. As can be seen from the error above, this swap space was not enough.

Since I had more than 1GB of free space still available on my SD card, I decided to increase this swapfile. To do that, open the file /etc/dphys-swapfile and increase the number of MB set in the line CONF_SWAPSIZE=100.

Restart the swapfile using the command:

$ /etc/init.d/dphys-swapfile restart

Or alternately you could restart Raspbian.

Tried with: Raspbian 9 and Raspberry Pi B+ Rev3

VcXsrv X server for Windows

A X server is needed on Windows if you SSH to remote Linux computers and wish to start X or GUI applications from there. It is also necessary if you are working with a Linux distribution running inside Windows Subsystem for Linux (WSL). VcXsrv is a free open-source X server that can be used for all these purposes.

  • To install VcXsrv, download its installer from here and install it.
  • Launch XLaunch from the Windows start menu. This brings up a wizard to pick the X server configuration options. I choose Multiple Windows and go with the default options for the rest. Now the X server is running in the background.

  • Local: Go to your WSL shell (say Ubuntu) and set the DISPLAY environment variable:

$ export DISPLAY=localhost:0.0

Launch any X or GUI app and its window should now be displayed in its own individual Windows window.

  • Remote: Remember to SSH to the remote system with trusted X11 forwarding using option -Y. On the remote system, set the DISPLAY variable:
$ export DISPLAY=your-windows-ip:0.0

When you start XLaunch you may want to choose to Disable access control. Otherwise you may get errors like this:

$ xeyes
Authorization required, but no authorization protocol specified
Error: Can't open display: 10.0.0.99:0.0

Tried with: VcXsrv 1.20.1.4 and Ubuntu 18.04 WSL