How to work with Git submodules

Submodules in Git enable you add and manage external Git repos into an existing Git repo. This is useful for example, if the current Git repo depends on or requires the source code in other Git repo.

The presence of a .gitmodules file at the root of the repo indicates that this repo uses submodules. This file lists out the external Git repo URLs, the local path inside the repo where they will exist and their given name.

Working with submodules in Git can be a bit confusing and painful, especially when there is a hierarchy of submodules in the main module. However, organization of submodules in this manner are found in many Github repositories. So, you will need to know how to work with this setup. Many folks use Git clients like SmartGit that make it easy to work with submodules. However, sooner or later the complexity that Git exposes will bite back requiring one to revert back to the shell to operate.

Here are the Git practices and incantations I have found useful to work with submodules:

Clone

When you clone a Git repository that has submodules, the submodules are not cloned.

To clone the main module and all its submodules:

$ git clone --recursive git@github.com:jpe/some_repo.git

I prefer to use the normal clone, followed by initializing and updating all submodules:

$ git clone git@github.com:jpe/some_repo.git
$ git submodule update --init --recursive

Fetch

Each of the submodules are usually linked to remote repositories (like Github or NFS). To fetch changes for remotes of the main module and all remotes of the submodules:

$ git fetch --all --recurse-submodules

Update

After checking out a branch (or revision) on the main module, you need to update the submodules too:

$ git checkout foo_local_branch
$ git submodule update --recursive --force

Note that this is similar to the command used in clone above, except we do not use --init. The --force is optional, but I like to use since if some files or directories were removed or added between checkouts. If you do not use --force, then the update will not finish and quit with a warning like this:

warning: unable to rmdir foobar: Directory not empty

Status

To list all the submodules and their revisions:

$ git submodule status --recursive

Remotes

If you will be changing any of the submodules, then remember to fork those repositories (on Github) and add those remotes. Note that you will need to do this even if you have forked the main module and cloned from that forked repo.

For fruit and fruit/apple directories that are nested submodules under the main module:

$ cd fruit
$ git remote add joe_fruit_remote git@github.com:joe/fruit_repo.git

$ cd apple
$ git remote add joe_apple_remote git@github.com:joe/apple_repo.git

Commit

A revision or branch of the main module will be tied to particular revisions of its submodules. That is, the submodules appear as detached HEAD revisions. Also, it is the revision of a submodule that is committed to a parent module. All of this gets messy when you have a hierarchy of submodules.

So, when you checkout a local branch on the main module and you need to modify files in a hierarchy of submodules, the typical workflow is to:

• Create local branches from the detached HEAD in the submodules you will change.
• Commit in a bottom-up fashion: start from the leaf submodules that you changed and work your way up to the root module. All the modules on the path from changed submodule to root module need to be committed. This is because the parent module commit needs the revision of the child module commit.

Here is a sample workflow where the fruit and fruit/apple directories are nested submodules under the main module:

$ git checkout -b foo_local_branch origin/foo_remote_branch
$ git submodule update --recursive

$ cd fruit
$ git checkout -b fruit_local_branch
$ vim change_some_files_in_fruit_submodule.sh

$ cd apple
$ git checkout -b apple_local_branch
$ vim change_some_files_in_apple_submodule.sh

$ git add change_some_files_in_apple_submodule.sh
$ git commit -m "Changes in fruit/apple go first"

$ cd ..
$ git add change_some_files_in_fruit_submodule.sh
$ git commit -m "Changes in fruit go next"

$ cd ..
$ git add -u
$ git commit -m "Commit new fruit submodule revision to main module"

Push

Each of the submodules are usually linked to remote repositories (like Github or NFS). It is a good idea to push the local branches you created in submodules (see above) to remote branches in your fork on your remote. This makes it easy to give pull request later (see next section below).

Continuing from our example above:

$ cd fruit
$ cd apple

$ git push joe_apple_remote apple_remote_branch

$ cd ..
$ git push joe_fruit_remote fruit_remote_branch

$ cd ..
$ git push origin foo_remote_branch

Pull request

After your changes are done, you typically need to give a pull request (on Github) from your branch to a master branch which is typically owned by someone else. To do this:

• Push the local branches of all changed submodules to their remotes. (See Push section above).
• Give multiple pull requests, one from each submodule repository that has changed.