CPU feature Caffe2 warning

Problem

Running a Caffe2 C++ application produces these messages:

$ ./foobar
E0514 20:37:31.503541 26925 init_intrinsics_check.cc:43] CPU feature avx is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
E0514 20:37:31.504768 26925 init_intrinsics_check.cc:43] CPU feature avx2 is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
E0514 20:37:31.504787 26925 init_intrinsics_check.cc:43] CPU feature fma is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.

Solution

Caffe2 is informing that the Intel CPU being used has AVX, AVX2 and FMA acceleration features. However, Caffe2 has not been compiled with support for these features. Compiling Caffe2 with support to use these features would speedup training and inference using Caffe2.

To enable use of these features when compiling Caffe2, enable the USE_NATIVE_ARCH option like this:

$ cmake -DUSE_NATIVE_ARCH ..
Advertisements

past.builtins ImportError

Problem

Calling a Caffe2 Python call gave this error:

$ python -c "from caffe2.python import core"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/caffe2/python/core.py", line 9, in <module>
    from past.builtins import basestring
ImportError: No module named past.builtins

Solution

This requires the future package. Installing that solved the problem:

$ pip install --user future

Tried with: Raspbian 9

serialized_options TypeError

Problem

I built and installed Caffe2. Running a simple Caffe2 Python call, gave this error:

$ python -c "from caffe2.python import core"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/caffe2/python/__init__.py", line 2, in <module>
    from caffe2.proto import caffe2_pb2
  File "/usr/local/lib/python2.7/dist-packages/caffe2/proto/__init__.py", line 11, in <module>
    from caffe2.proto import caffe2_pb2, metanet_pb2, torch_pb2
  File "/usr/local/lib/python2.7/dist-packages/caffe2/proto/caffe2_pb2.py", line 23, in <module>
    \x44\x65viceTypeProto\x12\r\n\tPROTO_CPU\x10\x00\x12\x0e\n\nPROTO_CUDA\x10\x01\x12\x10\n\x0cPROTO_MKLDNN\x10\x02\x12\x10\n\x0cPROTO_OPENGL\x10\x03\x12\x10\n\x0cPROTO_OPENCL\x10\x04\x12\x0f\n\x0bPROTO_IDEEP\x10\x05\x12\r\n\
tPROTO_HIP\x10\x06\x12\x0e\n\nPROTO_FPGA\x10\x07\x12\x0f\n\x0bPROTO_MSNPU\x10\x08\x12\r\n\tPROTO_XLA\x10\t\x12\'\n#PROTO_COMPILE_TIME_MAX_DEVICE_TYPES\x10\n\x12\x19\n\x13PROTO_ONLY_FOR_TEST\x10\xa5\xa3\x01')
TypeError: __new__() got an unexpected keyword argument 'serialized_options'

Solution

This is caused by an older version of Protobuf. Updating that solved the problem:

$ pip install -U protobuf

Tried with: Raspbian 9

dlopen: cannot load any more object with static TLS

Problem

I had a Python script that used Caffe2. It worked fine on one computer. On another computer with same setup, it would fail at the import caffe2.python line with this error:

WARNING:root:This caffe2 python run does not have GPU support. Will run in CPU only mode.
WARNING:root:Debug message: dlopen: cannot load any more object with static TLS
CRITICAL:root:Cannot load caffe2.python. Error: dlopen: cannot load any more object with static TLS

As I mentioned above, the GPU support warning is a red herring cause this Caffe2 Python was built with GPU support. The real error is the dlopen.

Solution

The only solution from Googling that gave a clue was this. As suggested there, I placed the import caffe2.python line at the top above all other imports. The error disappeared.

Tried with: Ubuntu 14.04

How to build Caffe2 from source

I followed these steps to build and use Caffe2 from source:

  • If you have a GPU, install CUDA and cuDNN as described here.

  • Install the package prerequisites:

$ sudo apt install build-essential cmake git libgoogle-glog-dev libprotobuf-dev protobuf-compiler python-dev python-pip libgflags2 libgtest-dev libiomp-dev libleveldb-dev liblmdb-dev libopencv-dev libopenmpi-dev libsnappy-dev openmpi-bin openmpi-doc python-pydot
  • Install the required Python packages using PIP:
$ sudo pip install numpy protobuf flask graphviz hypothesis jupyter matplotlib pydot python-nvd3 pyyaml requests scikit-image scipy setuptools tornado
  • Clone Caffe2 source code:
$ git clone git@github.com:caffe2/caffe2.git
  • Caffe2 is under rapid deployment, so I find that the master branch may sometimes not compile. It is better to check the available release tags and checkout the latest release:
$ git tag
$ git co -b v_0_7_0 v0.7.0
  • The install guide suggests running make to build. Note that this in turn creates a build directory, runs CMake from there and later runs make in a subshell. The child make running from inside a Makefile will not get the MAKEFLAGS of the parent make. So, you cannot make in parallel by using --jobs or -j settings. And believe me building Caffe2 without parallel make takes extremely long! So, I prefer doing the CMake and make myself:
$ mkdir build
$ cd build
$ cmake ..
$ make
  • After Caffe2 is built, you need to install the Caffe2 headers, libraries and Python files to a different location. If you do not configure anything, CMake will try to install Caffe2 to /usr/local, which requires superuser privileges. I prefer installing Caffe2 to a local directory, say /home/joe/caffe2_deploy. To be able to to this:
$ cmake -DCMAKE_INSTALL_PREFIX:PATH=/home/joe/caffe2_deploy ..
$ make install
  • Add the library path to LD_LIBRARY_PATH:
$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/joe/caffe2_deploy/lib
  • Add the install directory to PYTHONPATH to be able to use Caffe2 from Python:
$ export PYTHONPATH=$PYTHONPATH:/home/joe/caffe2_deploy
  • Use Caffe2 and enjoy:
$ python -m caffe2.python.operator_test.relu_op_test

Tried with: Ubuntu 14.04