The strange case of varying floats in Protobuf

Problem

I was using Google Protobuf in a Python program to read some text format Protobuf messages, merge them and write them out. Surprisingly, for the same set of input text format message files, I was getting different outputs on two computers! The values that were different were float values. The float values were generally correct, but varied slightly in precision between the two computers.

Solution

This strange observation took quite a long investigation. I initially assumed that maybe the Protobuf library (libprotobuf.so) or the Python Protobuf package were of different versions on these two computers. Surprisingly, they were exactly the same.

The mystery finally turned out to be the Protobuf implementation type. There are currently two possible types: cpp and python. By default, the cpp implementation is used. However, on one of the computers, the python implementation had been chosen by an engineer during the PIP package installation. The way to pick the engine is by setting an environment variable named PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION to either cpp or python. The engineer had set this environment variable in his shell when playing around with Protobuf and had later installed the PIP package.

Once I explicitly set the PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION environment variable manually in my Python code before importing protobuf, the float values were the same on both computers!

Now why should the engine affect the float value? Because Python’s float is actually double precision. On the other hand, when a 32-bit float moved between Python code and the C++ engine and back to Python code, it was sometimes changing precision. By using the same engine on all computers, we ensured that at least the float values did not vary between machines.

Tried with: Python Protobuf 3.3.0 and Ubuntu 14.04