How to use hgwatchman extension for Mercurial

hg status command can be very slow on a repository with thousands of files. This is because it has to explicitly check the status of every file. A great method to speed it up is the hgwatchman extension for Mercurial. It uses the Watchman tool which relies on the inotify mechanism of the Linux kernel to be notified directly of only the changes that occurred to the files in a directory. There are no packages of hgwatchman and Watchman for Ubuntu, so we need to build them from source. But it is pretty easy to do:

  • Install the Watchman tool as described here.

  • Download the hgwatchman source code:

$ hg clone https://bitbucket.org/facebook/hgwatchman
  • Build the code:
$ cd hgwatchman
$ make local

If you forget this step, everything seems to be fine until you start seeing this error:

$ hg status
abort: No module named bser!

bser.so is a module that is built by this step.

  • Add these lines to your hgrc:
[extensions]
hgwatchman = /path/to/hgwatchman/hgwatchman

[hgwatchman]
mode = on
  • That is it! You should see a nice improvement in the performance of hg status after using hgwatchman.

Tried with: Watchman 3.0.0, Mercurial. 2.8.3 and Ubuntu 14.04

How to install and use Watchman

Watchman is a server-client tool to watch files and directories for changes. It uses the inotify mechanism of the Linux kernel to achieve this.

There seems to be no package or PPA of Watchman for Ubuntu. But, installing it from source is easy:

$ git clone https://github.com/facebook/watchman.git
$ cd watchman
$ ./autogen.sh
$ ./configure
$ make
$ sudo make install

To ask Watchman to watch a directory for changes:

$ watchman watch /path/to/dir

By default, Watchman writes the changes it notices to its log file. The log file can be found in /tmp/.watchman.yourusername.log.

If you want to uninstall Watchman, go to the source directory and try:

$ sudo make uninstall
( cd '//usr/local/bin' && rm -f watchman )
( cd '//usr/local/share/doc/watchman-3.0.0' && rm -f README.markdown )

Tried with: Watchman 3.0.0 and Ubuntu 14.04

How to use inotify extension for Mercurial

hg status is the slowest command in Mercurial and can take several seconds on repositories with thousands of files. This is because it needs to check every single file and directory in your repository to see if it has changed. One method to speed up this command is to use the inotify extension for Mercurial.

Note: This extension is reported to be buggy and using the alternate hgwatchman extension is recommended.
Note: This extension only works in Mercurial 2.x. It has been explicitly disabled in Mercurial 3.x and will not work.

This extension ships with the mercurial-common package and so it will already be installed on your system if you have Mercurial. It relies on the inotify subsystem of the Linux kernel and any kernel later than 2.6.13 will have it. It uses the filesystem to watch for changes and reports these to applications. Using this Mercurial can check the status of gigantic repositories quickly.

To enable this extension, just add these lines to your hgrc:

[extensions]
inotify =

That is it! I noticed a speedup on a large repository with 500K files, but it was not as much as I wanted.

Tried with: Mercurial 2.8.2, Linux 3.13.0 and Ubuntu 14.04