Working with submodules in Git can be a bit confusing and painful, especially when there is a hierarchy of submodules in the main module. However, organization of submodules in this manner are found in many Github repositories. So, you will need to know how to work with this setup. Many folks use Git clients like SmartGit that make it easy to work with submodules. However, sooner or later the complexity that Git exposes will bite back requiring one to revert back to the shell to operate.
Here are the Git practices and incantations I have found useful to work with submodules:
When you clone a Git repository that has submodules, the submodules are not cloned.
To clone the main module and all its submodules:
$ git clone --recursive email@example.com:jpe/some_repo.git
I prefer to use the normal clone, followed by initializing and updating all submodules:
$ git clone firstname.lastname@example.org:jpe/some_repo.git $ git submodule update --init --recursive
Each of the submodules are usually linked to remote repositories (like Github or NFS). To fetch changes for remotes of the main module and all remotes of the submodules:
$ git fetch --all --recurse-submodules
After checking out a branch (or revision) on the main module, you need to update the submodules too:
$ git checkout foo_local_branch $ git submodule update --recursive --force
Note that this is similar to the command used in clone above, except we do not use
--force is optional, but I like to use since if some files or directories were removed or added between checkouts. If you do not use
--force, then the update will not finish and quit with a warning like this:
warning: unable to rmdir foobar: Directory not empty
To list all the submodules and their revisions:
$ git submodule status --recursive
If you will be changing any of the submodules, then remember to fork those repositories (on Github) and add those remotes. Note that you will need to do this even if you have forked the main module and cloned from that forked repo.
fruit/apple directories that are nested submodules under the main module:
$ cd fruit $ git remote add joe_fruit_remote email@example.com:joe/fruit_repo.git $ cd apple $ git remote add joe_apple_remote firstname.lastname@example.org:joe/apple_repo.git
A revision or branch of the main module will be tied to particular revisions of its submodules. That is, the submodules appear as detached HEAD revisions. Also, it is the revision of a submodule that is committed to a parent module. All of this gets messy when you have a hierarchy of submodules.
So, when you checkout a local branch on the main module and you need to modify files in a hierarchy of submodules, the typical workflow is to:
- Create local branches from the detached HEAD in the submodules you will change
- Commit in a bottom-up fashion: start from the leaf submodules that you changed and work your way up to the root module. All the modules on the path from changed submodule to root module need to be committed. This is because the parent module commit needs the revision of the child module commit.
Here is a sample workflow where the
fruit/apple directories are nested submodules under the main module:
$ git checkout -b foo_local_branch origin/foo_remote_branch $ git submodule update --recursive $ cd fruit $ git checkout -b fruit_local_branch $ vim change_some_files_in_fruit_submodule.sh $ cd apple $ git checkout -b apple_local_branch $ vim change_some_files_in_apple_submodule.sh $ git add change_some_files_in_apple_submodule.sh $ git commit -m "Changes in fruit/apple go first" $ cd .. $ git add change_some_files_in_fruit_submodule.sh $ git commit -m "Changes in fruit go next" $ cd .. $ git add -u $ git commit -m "Commit new fruit submodule revision to main module"
Each of the submodules are usually linked to remote repositories (like Github or NFS). It is a good idea to push the local branches you created in submodules (see above) to remote branches in your fork on your remote. This makes it easy to give pull request later (see next section below).
Continuing from our example above:
$ cd fruit $ cd apple $ git push joe_apple_remote apple_remote_branch $ cd .. $ git push joe_fruit_remote fruit_remote_branch $ cd .. $ git push origin foo_remote_branch
After your changes are done, you typically need to give a pull request (on Github) from your branch to a master branch which is typically owned by someone else. To do this:
- Push the local branches of all changed submodules to their remotes. (See Push section above).
- Give multiple pull requests, one from each submodule repository that has changed.