log -G: ignore binary files

The -G<regex> option of log looks for the differences whose patch text
contains added/removed lines that match regex.

Currently -G looks also into patches of binary files (which
according to [1]) is binary as well.

This has a couple of issues:

- It makes the pickaxe search slow. In a proprietary repository of the
author with only ~5500 commits and a total .git size of ~300MB
searching takes ~13 seconds

$time git log -Gwave > /dev/null

real    0m13,241s
user    0m12,596s
sys     0m0,644s

whereas when we ignore binary files with this patch it takes ~4s

$time ~/devel/git/git log -Gwave > /dev/null

real    0m3,713s
user    0m3,608s
sys     0m0,105s

which is a speedup of more than fourfold.

- The internally used algorithm for generating patch text is based on
xdiff and its states in [1]

> The output format of the binary patch file is proprietary
> (and binary) and it is basically a collection of copy and insert
> commands [..]

which means that the current format could change once the internal
algorithm is changed as the format is not standardized. In addition
the git binary patch format used for preparing patches for git apply
is *different* from the xdiff format as can be seen by comparing

git log -p -a

commit 6e95bf4bafccf14650d02ab57f3affe669be10cf
Author: A U Thor <author@example.com>
Date:   Thu Apr 7 15:14:13 2005 -0700

modify binary file

diff --git a/data.bin b/data.bin
index f414c84..edfeb6f 100644
--- a/data.bin
+++ b/data.bin
@@ -1,2 +1,4 @@
a
a^@a
+a
+a^@a

with git log --binary

commit 6e95bf4bafccf14650d02ab57f3affe669be10cf
Author: A U Thor <author@example.com>
Date:   Thu Apr 7 15:14:13 2005 -0700

modify binary file

diff --git a/data.bin b/data.bin
index f414c84bd3aa25fa07836bb1fb73db784635e24b..edfeb6f501[..]
GIT binary patch
literal 12
QcmYe~N@Pgn0zx1O01)N^ZvX%Q

literal 6
NcmYe~N@Pgn0ssWg0XP5v

which seems unexpected.

To resolve these issues this patch makes -G<regex> ignore binary files
by default. Textconv filters are supported and also -a/--text for
getting the old and broken behaviour back.

The -S<block of text> option of log looks for differences that changes
the number of occurrences of the specified block of text (i.e.
addition/deletion) in a file. As we want to keep the current behaviour,
add a test to ensure it stays that way.

[1]: http://www.xmailserver.org/xdiff.html

Signed-off-by: Thomas Braun <thomas.braun@virtuell-zuhause.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

diff --git a/Documentation/diff-options.txt b/Documentation/diff-options.txt
index 0378cd574eb01069baba14221d511f580565c0eb..b94d332f71b0ff58fb12527f8ba13788b8aa6eeb 100644 (file)
--- a/Documentation/diff-options.txt
+++ b/Documentation/diff-options.txt
@@ -524,6 +524,8 @@ struct), and want to know the history of that block since it first
 came into being: use the feature iteratively to feed the interesting
 block in the preimage back into `-S`, and keep going until you get the
 very first version of the block.
++
+Binary files are searched as well.
 
 -G<regex>::
        Look for differences whose patch text contains added/removed
@@ -543,6 +545,9 @@ While `git log -G"regexec\(regexp"` will show this commit, `git log
 -S"regexec\(regexp" --pickaxe-regex` will not (because the number of
 occurrences of that string did not change).
 +
+Unless `--text` is supplied patches of binary files without a textconv
+filter will be ignored.
++
 See the 'pickaxe' entry in linkgit:gitdiffcore[7] for more
 information.
 

diff --git a/Documentation/gitdiffcore.txt b/Documentation/gitdiffcore.txt
index c0a60f315811c788f3431b5757c38ffcea487386..c970d9fe438a091dc19900a4de1973ac4bcc4434 100644 (file)
--- a/Documentation/gitdiffcore.txt
+++ b/Documentation/gitdiffcore.txt
@@ -242,7 +242,8 @@ textual diff has an added or a deleted line that matches the given
 regular expression.  This means that it will detect in-file (or what
 rename-detection considers the same file) moves, which is noise.  The
 implementation runs diff twice and greps, and this can be quite
-expensive.
+expensive.  To speed things up binary files without textconv filters
+will be ignored.
 
 When `-S` or `-G` are used without `--pickaxe-all`, only filepairs
 that match their respective criterion are kept in the output.  When

diff --git a/diffcore-pickaxe.c b/diffcore-pickaxe.c
index 69fc55ea1e34af69b7ed4d57e521cb4825f99fd2..4cea086f80e657e57ed5ccb2c5f7eeb5c6a640ab 100644 (file)
--- a/diffcore-pickaxe.c
+++ b/diffcore-pickaxe.c
@@ -154,6 +154,12 @@ static int pickaxe_match(struct diff_filepair *p, struct diff_options *o,
        if (textconv_one == textconv_two && diff_unmodified_pair(p))
                return 0;
 
+       if ((o->pickaxe_opts & DIFF_PICKAXE_KIND_G) &&
+           !o->flags.text &&
+           ((!textconv_one && diff_filespec_is_binary(o->repo, p->one)) ||
+            (!textconv_two && diff_filespec_is_binary(o->repo, p->two))))
+               return 0;
+
        mf1.size = fill_textconv(o->repo, textconv_one, p->one, &mf1.ptr);
        mf2.size = fill_textconv(o->repo, textconv_two, p->two, &mf2.ptr);
 

diff --git a/t/t4209-log-pickaxe.sh b/t/t4209-log-pickaxe.sh
index 844df760f7d1c2b1b25eb01fada039efd0354711..5d06f5f45eace6010c1a003e03a814b5ba03f324 100755 (executable)
--- a/t/t4209-log-pickaxe.sh
+++ b/t/t4209-log-pickaxe.sh
@@ -106,4 +106,39 @@ test_expect_success 'log -S --no-textconv (missing textconv tool)' '
        rm .gitattributes
 '
 
+test_expect_success 'setup log -[GS] binary & --text' '
+       git checkout --orphan GS-binary-and-text &&
+       git read-tree --empty &&
+       printf "a\na\0a\n" >data.bin &&
+       git add data.bin &&
+       git commit -m "create binary file" data.bin &&
+       printf "a\na\0a\n" >>data.bin &&
+       git commit -m "modify binary file" data.bin &&
+       git rm data.bin &&
+       git commit -m "delete binary file" data.bin &&
+       git log >full-log
+'
+
+test_expect_success 'log -G ignores binary files' '
+       git log -Ga >log &&
+       test_must_be_empty log
+'
+
+test_expect_success 'log -G looks into binary files with -a' '
+       git log -a -Ga >log &&
+       test_cmp log full-log
+'
+
+test_expect_success 'log -G looks into binary files with textconv filter' '
+       test_when_finished "rm .gitattributes" &&
+       echo "* diff=bin" >.gitattributes &&
+       git -c diff.bin.textconv=cat log -Ga >log &&
+       test_cmp log full-log
+'
+
+test_expect_success 'log -S looks into binary files' '
+       git log -Sa >log &&
+       test_cmp log full-log
+'
+
 test_done