How to visualize TensorFlow protobuf in Tensorboard

Tensorboard is a browser based visualization tool for the training of deep learning models using TensorFlow. It is typically invoked on the log directory that is output from the TensorFlow training process. It is not straightforward to use it to visualize a model stored as a single protobuf (.pb) file.

Here is how to do that:

  • Install TensorFlow and Tensorboard, if you do not have them already:
$ pip3 install -U --user tensorflow tensorboard
  • Convert the protobuf file to a file that Tensorboard can work with using an import script that ships with TensorFlow:
$ python3 ~/.local/lib/python3.6/site-packages/tensorflow/python/tools/import_pb_to_tensorboard.py --model_dir foobar_model.pb --log_dir foobar_log_dir

This script creates a log directory you requested for if it does not exist. It creates a file name of the form events.out.tfevents.1557253678.your-hostname that Tensorboard understands.
Note that it is better to pass in a different log directory for every different model.

Another thing to note is that the option is named --model_dir but it actually expects a protobuf file as input.

  • Now we can invoke Tensorboard with the log directory as input:
$ tensorboard --logdir=foobar_log_dir

The tensorboard executable file should be present in your ~/.local/bin directory. If this path is not in your PATH environment variable, consider adding it. Alternatively, you can invoke the executable with its absolute path too.

  • You can visualize and explore the structure of your model in Tensorboard by opening localhost:6006 in your browser.
Advertisements

How to build and install TensorFlow

TensorFlow (TF) can be built from source easily and installed as a Python wheel package. I used the following steps to build it using Python3 and with support for CUDA and TensorRT:

  • Install Python3 pre-requisites:
$ sudo apt install python3-dev python3-pip
  • Install necessary Python3 packages locally:
$ pip3 install -U --user six numpy wheel setuptools mock
$ pip3 install -U --user keras_applications==1.0.6 --no-deps
$ pip3 install -U --user keras_preprocessing==1.0.5 --no-deps

These packages are installed to your ~/.local/lib/python3.x/site-packages directory. TF documentation also installs the latest pip3 from PyPI. However, doing that causes the infamous “Cannot import name main” error, so I do not do that.

  • TF uses Bazel as its build tool. Install it as described here. I ended up placing its binary in my ~/bin. Since my ~/bin is in my PATH, the Bazel binary can be executed from any place.

  • I recommend creating a tensorflow_root directory. This is because the TF packaging tends to write out to a location outside the TF source directory. Also, TF needs to access other libraries. So this root directory makes it easy to create all TF related directories under one umbrella.

  • Clone the TF Git repository inside the root directory:

$ cd tensorflow_root
$ git clone git@github.com:tensorflow/tensorflow.git
$ cd tensorflow
  • Configure the build process using:
$ ./configure

Some of the questions it asks and my replies:

  • Please specify the location of python. /usr/bin/python3
  • Please input the desired Python library path to use. /usr/local/lib/python3.6/dist-packages
  • Enable: XLA JIT, CUDA and TensorRT. Be careful, TF might not work with latest versions of CUDA, cuDNN and TensorRT. I used CUDA 10.0 and cuDNN 7.3 and TensorRT 5.0.
  • Did not enable: OpenCL SYCL and ROCm.
    • It is time to build TF. The command to build is:
$ bazel build --config=opt //tensorflow/tools/pip_package:build_pip_package

The first build can take 2-4 hours to complete.

  • Now we are ready to package the build artifacts into a Python package. Specify where you want that package to be placed in the packaging command:
$ ./bazel-bin/tensorflow/tools/pip_package/build_pip_package /path_to_tensorflow_root/tensorflow_pkg

I found that it had generated a Python wheel file named: tensorflow-1.13.1-cp36-cp36m-linux_x86_64.whl

  • We are ready to install TF. Make sure you remove older versions of TF:
$ pip3 uninstall tensorflow tensorflow-estimator
  • Install TF from the Python wheel package:
$ pip3 install -U --user tensorflow-1.13.1-cp36-cp36m-linux_x86_64.whl
  • Check if TF is installed correctly:
$ python3 -c "import tensorflow"

Tried with: Tensorflow 1.13.1 and Ubuntu 18.04