How to resolve Git conflict with binary files

Updated post here: https://codeyarns.github.io/tech/2016-04-11-how-to-resolve-git-conflict-with-binary-files.html

GiTk

GiTk is a GUI tool written in Tcl/Tk for viewing the Git log as a directed acyclic graph (DAG) and has many features to browse and explore the DAG. This is a tool that ships along with Git, so it should be present wherever Git is there. It accepts many of the options that can be passed to git log.

  • By default, it shows only history of the current branch:
$ gitk
  • Colors: It is important to understand the various colors used in GiTk. Commit with yellow circle is the HEAD. When GiTk is opened, it always jumps to this commit. Commit with red circle is your uncommitted changes. All other commits are shown in blue circle. Local branche names are shown in green, while remote branch names are shown in combination of orange and green with the orange section being the remote name and the green section being the branch name.

  • Configuration file: Configuration settings of this tool can be changed or set in ~/.config/gitk/gitk

  • To view history of another branch:

$ gitk some_other_branch
  • To view the history of all branches:
$ gitk --all
  • To view the history of a certain file:
$ gitk /path/to/file
  • --date-order: Show commits in reverse chronological order of their dates. I use this when I want to get a sense of the progress of the team working on the repository. You might think this is very useful and may wonder why this is not turned on by default. Note that this makes the chain of development harder to visualize. The default view is --topo-order which tries to show as many commits of a branch together as possible. See the man page for an example scenario.

  • --merges: View only the pieces of the DAG with merges.

  • --no-merges: View the disconnected pieces of the DAG with the merge commits removed.

  • --first-parent: Show only the first parent in merges. This greatly simplifies the visual complexity of the DAG. This is especially useful when viewing the DAG of a repository that follows the GitFlow branching model. Note that the first parent of a merge is the branch to which the other branch was merged.

  • -n: Provide a number to this option to limit loading only that many commits. This is useful when viewing a repository with a large number of commits. For example: gitk -n 1000

  • To search and view all the commits with a certain string, type it in the Find box and press Enter. You will see that all the commits with that string are highlighted in bold. You can jump up and down between these commits using the Up and Down arrow buttons at the left.

  • To diff two commits: Click the first commit to highlight it. Next right-click on the second commit to get the context menu and in that menu choose Diff this and selected.

  • F2: To view all the branches and tags in a dialog. Clicking on one of them takes you to that tag or branch. This is useful to navigate a complex DAG.

  • Right-click on any line in the diff shown below and choose Show origin of this line to jump to the commit that added this line. You typically check this on line that are modified or removed in the current commit of course.

  • For complex DAGs, edges are broken into dangling arrowheads. Clicking on the arrow tip jumps you to the other end of the edge.

  • Click on any edge and all the edges that for that segment of the graph are shown in bold. The bottom pane changes to show the parents and children of that piece of the graph.

  • Mark: To bookmark a commit, right-click and choose Mark this commit. The commit will be shown in a box. Marking is useful to bookmark a commit and later return to it. For example, mark a commit, navigate to another part of the DAG and then right-click anywhere and choose Return to mark to go back to the marked commit. It is also useful for diff of two commits. For example, after marking a commit, right-click on another commit and you can choose to diff the two. Only one mark can be set. Setting a new mark, removes the previous one.

  • External Diff: No matter what you are doing in GitK, viewing a commit or comparing two commits, the lower-left box always shows the diffs and the lower-right box the files which are different. However, you may prefer to see diffs side-by-side or using a different external diff viewer, like Meld. This can be done easily. Just right-click any of the filepaths shown in the lower-right box and choose External Diff to view in your favorite diff viewer. The diff program it will use is the one you have set using the diff.tool configuration option in Git.

Tried with: Git 2.4.1 and Ubuntu 14.04

Git-GUI

Git-GUI is a rudimentary GUI tool for Git written in Tcl/Tk. It should be available on all platforms where Git is available. It is primarily meant for committing. However, its dinosaur GUI hides quite a few actions that can be very useful.

  • To install Git-GUI:
$ sudo apt install git-gui
  • To invoke Git-GUI:
$ git gui
  • By default, it shows the unstaged files in top-left and staged files in bottom-left. When any files in these sections are clicked, their diff is shown in the main window.

  • To stage a file in the Unstaged Changes section: click the file icon to the left of the file. I know this is totally non-intuitive, but that is how this GUI is designed!

  • To move all the unstaged files to staged, click the Stage Changed button at the bottom. The same can be achieved from the top menu: Commit -> Stage changed files to commit.

  • To unstage a staged file that is in the Unstaged Changes section: Click the checkbox icon to the left of the filename.

  • To commit, type commit message in the bottom box and press Commit button beside it.

  • To view graphical history log of any branch, go to Repository -> Visualize All Branch History. This uses gitk to show this info.

  • All other basic operations are accessible from the top menu. For example, create/checkout/delete of branches and so on.

  • To browse the directory tree and look inside files of any branch, use the Repository -> Browse branch files option. This is very useful. When you open a file, it is opened in a rudimentary File Viewer.

File Viewer

The information shown in the file viewer is incredibly informative!

  • You can hover mouse on any line and see its full history (who added it and so on)!

  • The file viewer also has rudimentary features to find text and other operations. Right-click anywhere to view these.

  • To view graphical log of just this file, right-click and choose Show History Context.

How to work with Git submodules

Working with submodules in Git can be a bit confusing and painful, especially when there is a hierarchy of submodules in the main module. However, organization of submodules in this manner are found in many Github repositories. So, you will need to know how to work with this setup. Many folks use Git clients like SmartGit that make it easy to work with submodules. However, sooner or later the complexity that Git exposes will bite back requiring one to revert back to the shell to operate.

Here are the Git practices and incantations I have found useful to work with submodules:

Clone

When you clone a Git repository that has submodules, the submodules are not cloned.

To clone the main module and all its submodules:

$ git clone --recursive git@github.com:jpe/some_repo.git

I prefer to use the normal clone, followed by initializing and updating all submodules:

$ git clone git@github.com:jpe/some_repo.git
$ git submodule update --init --recursive

Fetch

Each of the submodules are usually linked to remote repositories (like Github or NFS). To fetch changes for remotes of the main module and all remotes of the submodules:

$ git fetch --all --recurse-submodules

Update

After checking out a branch (or revision) on the main module, you need to update the submodules too:

$ git checkout foo_local_branch
$ git submodule update --recursive --force

Note that this is similar to the command used in clone above, except we do not use --init. The --force is optional, but I like to use since if some files or directories were removed or added between checkouts. If you do not use --force, then the update will not finish and quit with a warning like this:

warning: unable to rmdir foobar: Directory not empty

Status

To list all the submodules and their revisions:

$ git submodule status --recursive

Remotes

If you will be changing any of the submodules, then remember to fork those repositories (on Github) and add those remotes. Note that you will need to do this even if you have forked the main module and cloned from that forked repo.

For fruit and fruit/apple directories that are nested submodules under the main module:

$ cd fruit
$ git remote add joe_fruit_remote git@github.com:joe/fruit_repo.git

$ cd apple
$ git remote add joe_apple_remote git@github.com:joe/apple_repo.git

Commit

A revision or branch of the main module will be tied to particular revisions of its submodules. That is, the submodules appear as detached HEAD revisions. Also, it is the revision of a submodule that is committed to a parent module. All of this gets messy when you have a hierarchy of submodules.

So, when you checkout a local branch on the main module and you need to modify files in a hierarchy of submodules, the typical workflow is to:

  • Create local branches from the detached HEAD in the submodules you will change
  • Commit in a bottom-up fashion: start from the leaf submodules that you changed and work your way up to the root module. All the modules on the path from changed submodule to root module need to be committed. This is because the parent module commit needs the revision of the child module commit.

Here is a sample workflow where the fruit and fruit/apple directories are nested submodules under the main module:

$ git checkout -b foo_local_branch origin/foo_remote_branch
$ git submodule update --recursive

$ cd fruit
$ git checkout -b fruit_local_branch
$ vim change_some_files_in_fruit_submodule.sh


$ cd apple
$ git checkout -b apple_local_branch
$ vim change_some_files_in_apple_submodule.sh

$ git add change_some_files_in_apple_submodule.sh
$ git commit -m "Changes in fruit/apple go first"

$ cd ..
$ git add change_some_files_in_fruit_submodule.sh
$ git commit -m "Changes in fruit go next"

$ cd ..
$ git add -u
$ git commit -m "Commit new fruit submodule revision to main module"

Push

Each of the submodules are usually linked to remote repositories (like Github or NFS). It is a good idea to push the local branches you created in submodules (see above) to remote branches in your fork on your remote. This makes it easy to give pull request later (see next section below).

Continuing from our example above:

$ cd fruit
$ cd apple

$ git push joe_apple_remote apple_remote_branch

$ cd ..
$ git push joe_fruit_remote fruit_remote_branch

$ cd ..
$ git push origin foo_remote_branch

Pull request

After your changes are done, you typically need to give a pull request (on Github) from your branch to a master branch which is typically owned by someone else. To do this:

  • Push the local branches of all changed submodules to their remotes. (See Push section above).
  • Give multiple pull requests, one from each submodule repository that has changed.

Stow not installing file

Problem

I had a directory with a .gitignore file in it. When I run Stow with this directory as the input, symbolic links to all the files in the directory were created, except for the .gitignore file.

Solution

Stow ignores installing files and directories that are in its ignore list. The files and directories to be ignored can be specified as regular expressions in an ignore file. Stow looks for a ~/.stow-global-ignore file. If it does not find it, it will use an internal default ignore list.

The default ignore list from Stow source code is shown here:

It can be seen that .gitignore is one of the files that it will ignore, hence it is not installed. To override the internal ignore list, create a ~/.stow-global-ignore file filled with the regex of what files to ignore. If you make this an empty file, Stow will install all files without ignoring any of them.

Reference: Stow manual

Tried with: Stow 2.2.0 and Ubuntu 14.04

How to wrap git log

I display the log of Git in my own custom format. When the display text for a commit is longer than the width of the terminal, it is just truncated!

I could not find any way to make Git wrap the display of log message in the terminal. What I instead do is to pipe the git log through the classic Unix fold command:

$ git log | fold

By default, fold wraps to a width of 80 columns. You can specify the width of your terminal to it using the -w or --width option.

Tried with: Git 1.9.1 and Ubuntu 14.04

Slow speed on git clone

Sometimes when I have to clone a huge repository from Github, I find that the fetch speed can be extremely slow (KB/s).

There are two tricks I try for a quick cloning:

  • Kill the command and try again. Do this a few times, see if you can get a faster connection. This works most of the times.

  • Fetch only the latest revision first and then fetch the rest:

$ git clone --depth=1 git@github.com:joe/hello-world.git
$ cd hello-world
$ git fetch --unshallow

Tried with: Git 1.9.1 and Ubuntu 14.04

My Git Cheatsheet

Unlike Mercurial, Git is a far more complex and complicated beast. This post is a work-in-progress to gather the commands that I end up discovering.

$ git reset HEAD that_file
  • How to revert a staged or modified file back to its last commit:
$ git checkout -- file_path
  • How to revert all modified tracked files:
$ git reset --hard
  • How to revert the effect of a particular commit:
$ git revert sha_hash_of_that_commit
  • How to remove all untracked files:
$ git clean -fd
  • Create branch at current revision or branch:
$ git branch branch_name
  • Create branch at specific revision:
$ git branch branch_name hash_of_revision
  • Rename current branch:
$ git branch -m new_branch_name
  • Rename some branch:
$ git branch -m old_branch_name new_branch_name
  • How to delete a local branch:
$ git branch -d branch_name

Note that you cannot delete the current branch. Checkout to a different branch and then delete the branch you wanted to. This is because imagine chopping off the branch of a tree you are sitting on. You will fall down! 🙂

  • How to merge other branch to master branch:
$ git checkout master_branch
$ git merge other_branch
  • How to resolve merge conflicts using GUI tool:
$ git mergetool
  • How to see all info about remote, its branches and push-pull to it:
$ git remote show remote_name
  • How to rename a remote:
$ git remote rename old_remote_name new_remote_name
  • How to checkout a remote branch:
$ git checkout -b local_branch remote_name/remote_branch
  • How to push the current local branch to a remote branch:
$ git push remote_name remote_branch_name
  • How to push a branch on local to branch on remote:
$ git push remote_name local_branch_name:remote_branch_name
  • How to push the current local branch to a remote branch and make the current local branch track that remote branch:
$ git push -u remote_name remote_branch_name
  • How to push a local branch to a remote branch and make the local branch track that remote branch:
$ git push -u remote_name local_branch_name:remote_branch_name
  • How to revert non-staged non-committed changes:
$ git checkout that_file
  • How to delete a remote branch:
$ git push remote_name --delete remote_branch
  • How to rebase my_branch on top of main_branch:
$ git checkout my_branch
$ git rebase main_branch
  • How to checkout a remote branch to a new local branch (of same name) with tracking:
$ git checkout -t remote_name/remote_branch
  • How to make an existing local branch track a remote branch (or change remote tracking branch):
$ git branch local_branch -u remote_name/remote_branch
$ git branch -vva --contains 01234
  • If some branches have been removed at the remote and you want them removed from the local repository too:
$ git remote prune FOOBAR_REMOTE

Submodules

  • How to list all submodules:
$ git submodule status --recursive
  • How to fetch all submodules:
$ git fetch --all --recurse-submodules
  • How to update all submodules to revision required by main module revision:
$ git submodule update --recursive
  • How to initialize all submodules the first time and update them to revision required by main module revision:
$ git submodule update --init --recursive
  • How to view the commit of a submodule (named foobar) corresponding to a commit (say 999792b) of the main module:
$ git ls-tree 999792b foobar

Misc

  • How to view HEAD history of your local repository:
$ git reflog
  • Find which commit added or deleted a certain string:
$ git log -S foobar
  • Get the history of a function:
$ git log -L :funcname:filename
  • How to change the last commit message:
$ git commit --amend
  • Show who last modified lines of a file:
$ git blame -C -L 234,234 foobar.cpp
  • Check if a commit hash exists in the Git log:
$ git show 123456

How to ignore files and directories in SVN

It is easy to ignore files and directories in Git or Mercurial. Ignore files can be maintained at both the user and repository levels. Git or Mercurial automatically ignore the files and directories that match the patterns listed in these ignore files.

Being a version control system from an older era, Subversion does not have this simple method of ignore.

  • In SVN, ignoring a file or directory is a property that can be set at a specific directory in a repository.

For example, to ignore build subdirectory and *.tmp files:

$ svn propset svn:ignore build .
$ svn propset svn:ignore *.tmp .

Note that both these commands set ignore properties for the current directory .

  • To set an ignore property recursively to all subdirectories in a repository, go to the repository root directory and issue this:
$ svn propset svn:ignore -R *.tmp .
  • Thankfully, you can use an ignore file containing ignore patterns, one per line, as input:
$ svn propset svn:ignore -R -F ~/.svnignore .

Note the difference with Git or Mercurial. You have to manually set this ignore property on every directory. The ignore file we created is for our convenience. If you create a new subdirectory anywhere in this repository, the property needs to be applied again on it.

  • Remember to commit after setting the ignore property.

Tried with: Subversion 1.8.8 and Ubuntu 14.04

How to use Quick Diff in Eclipse

Added, modified and deleted lines marked by Quick Diff in Eclipse editor.
Added, modified and deleted lines marked by Quick Diff in Eclipse editor.

Eclipse has support for version control systems like Git, SVN and CVS. Support for Mercurial can be enabled by using the MercurialEclipse plugin. When viewing files in version control, it is common practice to quickly see which of the lines are uncommitted compared to the current head (or parent) revision. This information can be viewed in Eclipse by using its Quick Diff feature. This is easy to do:

  1. Go to Window -> Preferences -> General -> Editors -> Text Editors -> Quick Diff. This is the dialog from where it can be configured.

  2. Turn on Quick Diff, by enabling the option Enable quick diff.

  3. Choose the version control system for which Quick Diff is needed from the dropdown Use this reference source. Git, SVN and CVS should be present here. Mercurial appears if the MercurialEclipse plugin is installed.

  4. The overview ruler is the ruler to the right of the scrollbar in the editor. If you would like to see the uncommitted lines to be marked on it, enable the option Show differences in overview ruler.

  5. Choose Apply and OK. Close all editor windows and reopen the source file that you want in the editor.

You should be able to see the newly added, modified or deleted lines marked in colors in the left bar on which line numbers are usually displayed. The colors used are the ones set in the Quick Diff dialog (see above steps). You can also view these uncommitted locations marked along the overview ruler on the right.

Tried with: Eclipse Luna 4.4.1 and Ubuntu 14.04