How to set LMDB permissions in Caffe

Caffe can create, read and write images from an LMDB database. This is much faster than using the files from filesystem during training. However, I noticed that by default Caffe would not set write permissions for my group for the lmdb directory.

A quick investigation into src/caffe/util/db_lmdb.cpp showed that this was because new lmdb directory was being created with 0744 permissions. I changed this to 0774 in the code and recompiled Caffe.

Now comes the mystery: Caffe still created lmdb directories where my group did not have write permission!

Further investigation showed that the default umask in Linux is 0022. This umask does not allow the group write permissions to be set.

Ok, so then I set the umask at the shell to 0002 and tried to create the lmdb again. Found that lmdb directories still did not have group write permissions!

I was creating the lmdb databases from a script that ran the convert_imageset binary. Now the binary was run by using the exec command of the Bash shell. This replaces the currently running process with the convert_imageset binary. However, unlike a fork, the new process is not passed the umask of the parent process by the OS! This was the reason for the problem.

Once I knew this, the solution was easy. I modified the tools/convert_imageset.cpp code and added this code:

#include <sys/types.h>
#include <sys/stat.h>

// In the main function ...
umask(0002);
// ....

This solved the problem perfectly! 🙂

Advertisements

How to set umask for Docker container

Problem

Once you run a Docker container using docker run and get a shell inside it, you can set the file creation mode mask there with the umask command of the shell. This is usually 0022 and you can set it to whatever you want. All consecutive operations at the shell and child processes forked from the shell will have with umask.

What if you don’t want to manually type this umask command, but want it set automatically in the container?

Solution

There is no way to do this directly in the Dockerfile. You can have a RUN umask 0002 in the Dockerfile, but that does not have any effect for when you run the container.

One solution is to create a shell script that sets this umask:

$ cat set_umask.sh
#!/bin/bash
umask 0002
/bin/bash

To be able to run this script when the container is run, we first need to make this executable:

$ chmod +x set_umask.sh

Next we add commands to the Dockerfile to copy this into the image and make the script as the entry point:

COPY set_umask.sh set_umask.sh
ENTRYPOINT ["./set_umask.sh"]

Build the container and run it and see your umask already enabled at the shell! 🙂