This is the remainder of testcase fix by Mark Allen to make them
work on his Darwin box. I was using "xargs -r" (GNU) where it
was not needed, sed -ne '/^\(author\|committer\)/s|>.*|>|p'
where some sed does not know what to do with '\|', and also
"cmp - file" to compare standard input with a file, which his
cmp does not support.
Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
git-rev-list: add "end" commit and "--header" flag
The "end" commit is just faking it right now, it's sorting things
purely by date, so this is _not_ a reachability analysis. Some day.
The "--header" flag causes the commit message to be printed out,
with a NUL character separator after it for parseability. This
allows you to do things like use "grep -z" to grep for certain
authors etc.
commit: save the commit buffer off when parsing a commit
object.
A fair number of the users potentially want to look at the
commit objects more closely, and if you worry about memory
leaking in certain applications, you can always do a
- Mode-only changes were pruned incorrectly from the output.
- Added test to catch the above problem.
- Normalize rename/copy similarity score in the diff-raw output
to per-cent, no matter what scale we internally use.
Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
[PATCH] Fix type-change handling when assigning the status code to filepairs.
The interim single-liner '?' fix resulted delete entries that
should not have emitted coming out in the output as an
unintended side effect; I caught this with the "rename" test in
the test suite. This patch instead fixes the code that assigns
the status code to each filepair.
I verified this does not break the testcase in udev.git tree Kay
Sievers gave us, by running git-diff-tree on that tree which
showed 21 file to symlink changes.
Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
The earlier test was relying on the fact that dotfiles do not
appear in the output to prepare expected test results, which
inevitably got broken when we started handling dotfiles. Change
the test to be honest about what "--other" file it creates.
The problem was originally pointed out by Mark Allen.
Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
git-pull-script: pretty-print the merge head information
This also drops the common ".git" part from the end of the repo
name, and if a non-default head reference is given, makes a nicer
commit message about it.
Oh, I am an idiot. Repeating the same check against the first
element of pathspec array as many times as the pathspec array
has elements in it would not do us any good.
This patch allows you to specify more than one pathspec to
diff-tree family and have them actually used.
Signed-off-by: Junio C Hamano <junkio@cox.net>
;) Signed-off-by: Linus Torvalds <torvalds@osdl.org>
The second round similarity estimator simply used the size of
the xdelta itself to estimate the extent of damage. This patch
keeps that logic to detect big insertions to terminate the check
early, but otherwise looks at the generated delta in order to
estimate the extent of edit more accurately.
Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Earlier implementation had a major screw-up in the memory
management area. Rename/copy logic sometimes borrowed a pointer
to a structure without any provision for downstream to determine
which pointer is shared and which is not. This resulted in the
later clean-up code to sometimes double free such structure,
resulting in a segfault. This made -M and -C useless.
Another problem the earlier implementation had was that it
reordered the patches, and forced the logic to differentiate
renames and copies to depend on that particular order. This
problem was fixed by teaching rename/copy detection logic not to
do any reordering, and rename-copy differentiator not to depend
on the order of the patches. The diffs will leave rename/copy
detector in the same destination path order as the patch that
was fed into it. Some test vectors have been reordered to
accommodate this change.
It also adds a sanity check logic to the human-readable diff-raw
output to detect paths with embedded TAB and LF characters,
which cannot be expressed with that format. This idea came up
during a discussion with Chris Wedgwood.
Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
git-apply: more consistency checks on gitdiff filenames
There's some duplication of filenames when doing filename operations
(creates, deletes, renames and copies), and this makes us verify that
the pathnames match when they should.
[PATCH] Fix diff-pruning logic which was running prune too early.
For later stages to reorder patches, pruning logic and rename detection
logic should not decide which delete to discard (because another entry
said it will take over the file as a rename) until the very end.
Also fix some tests that were assuming the earlier "last one is rename
or keep everything else is copy" semantics of diff-raw format, which no
longer is true.
Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
This changes the diff-raw format again, following the mailing
list discussion. The new format explicitly expresses which one
is a rename and which one is a copy.
The documentation and tests are updated to match this change.
Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
In particular, give line numbers when detecting corrupt patches.
This makes the tool a lot more friendly (indeed, much more so
than regular "patch", I think).
[PATCH] NUL terminate diff-tree header lines under -z.
Thomas Glanzmann noticed that diff-tree -z HEAD piped to
diff-helper -z did not work. Since diff-helper -z expects NUL
terminated lines, we should generate such.
The output side of the diff-helper should always be using '\n'
termination; earlier it used the same line_termination used for
the input side, which was a mistake.
Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
The rename/copy detection logic in earlier round was only good
enough to show patch output and discussion on the mailing list
about the diff-raw format updates revealed many problems with
it. This patch fixes all the ones known to me, without making
things I want to do later impossible, mostly related to patch
reordering.
(1) Earlier rename/copy detector determined which one is rename
and which one is copy too early, which made it impossible
to later introduce diffcore transformers to reorder
patches. This patch fixes it by moving that logic to the
very end of the processing.
(2) Earlier output routine diff_flush() was pruning all the
"no-change" entries indiscriminatingly. This was done due
to my false assumption that one of the requirements in the
diff-raw output was not to show such an entry (which
resulted in my incorrect comment about "diff-helper never
being able to be equivalent to built-in diff driver"). My
special thanks go to Linus for correcting me about this.
When we produce diff-raw output, for the downstream to be
able to tell renames from copies, sometimes it _is_
necessary to output "no-change" entries, and this patch
adds diffcore_prune() function for doing it.
(3) Earlier diff_filepair structure was trying to be not too
specific about rename/copy operations, but the purpose of
the structure was to record one or two paths, which _was_
indeed about rename/copy. This patch discards xfrm_msg
field which was trying to be generic for this wrong reason,
and introduces a couple of fields (rename_score and
rename_rank) that are explicitly specific to rename/copy
logic. One thing to note is that the information in a
single diff_filepair structure _still_ does not distinguish
renames from copies, and it is deliberately so. This is to
allow patches to be reordered in later stages.
(4) This patch also adds some tests about diff-raw format
output and makes sure that necessary "no-change" entries
appear on the output.
Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
[PATCH] Be careful with symlinks when detecting renames and copies.
Earlier round was not treating symbolic links carefully enough,
and would have produced diff output that renamed/copied then
edited the contents of a symbolic link, which made no practical
sense. Change it to detect only pure renames.
Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Thomas Glanzmann points out that it doesn't work well with different
clients accessing the repository over NFS - they have different views
on what the "device" for the filesystem is.
Of course, other filesystems may not even have stable inode numbers.
But we don't care. At least for now.
Add <limits.h> to the include files handled by "cache.h", and remove
extraneous #include directives from various .c files. The rule is that
"cache.h" gets all the basic stuff, so that we'll have as few system
dependencies as possible.
This moves the path selection logic from individual programs to a new
diffcore transformer (diff-tree still needs to have its own for
performance reasons). Also the header printing code in diff-tree was
tweaked not to produce anything when pickaxe is in effect and there is
nothing interesting to report. An interesting example is the following
in the GIT archive itself:
$ git-whatchanged -p -C -S'or something in a real script'
Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
There was a screwy math bug in the estimator that confused what
-C1 meant and what -C9 meant, only in one of the early "cheap"
check, which resulted in quite confusing behaviour.
Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Update the diff-raw format as Linus and I discussed, except that
it does not use sequence of underscore '_' letters to express
nonexistence. All '0' mode is used for that purpose instead.
The new diff-raw format can express rename/copy, and the earlier
restriction that -M and -C _must_ be used with the patch format
output is no longer necessary. The patch makes -M and -C flags
independent of -p flag, so you need to say git-whatchanged -M -p
to get the diff/patch format.
Updated are both documentations and tests.
Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
The heuristics so far was to compare file size change and xdelta
size against the average of file size before and after the
change. This patch uses the smaller of pre- and post- change
file size instead.
It also makes a very small performance fix. I didn't measure
it; I do not expect it to make any practical difference, but
while scanning an already sorted list, breaking out in the
middle is the right thing.
Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Make the commit explanation buffer larger, and make sure that if
we truncate it, we put a "..." marker there to visually tell people
about the truncation (tested with a much smaller buffer to make
sure it looks sane).
Also make sure that the explanation is properly line-terminated,
and add an extra newline iff we have a diff.
[PATCH] Diff overhaul, adding the other half of copy detection.
This patch extends diff-cache and diff-files to report the
unmodified files to diff-core as well when -C (copy detection)
is in effect, so that the unmodified files can also be used as
the source candidates. The existing test t4003 has been
extended to cover this case.
Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
This steals the "pickaxe" feature from JIT and make it available
to the bare Plumbing layer. From the command line, the user
gives a string he is intersted in.
Using the diff-core infrastructure previously introduced, it
filters the differences to limit the output only to the diffs
between <src> and <dst> where the string appears only in one but
not in the other. For example:
$ ./git-rev-list HEAD | ./git-diff-tree -Sdiff-tree-helper --stdin -M
would show the diffs that touch the string "diff-tree-helper".
In real software-archaeologist application, you would typically
look for a few to several lines of code and see where that code
came from.
The "pickaxe" module runs after "rename/copy detection" module,
so it even crosses the file rename boundary, as the above
example demonstrates.
Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
[PATCH] Diff overhaul, adding half of copy detection.
This introduces the diff-core, the layer between the diff-tree
family and the external diff interface engine. The calls to the
interface diff-tree family uses (diff_change and diff_addremove)
have not changed and will not change. The purpose of the
diff-core layer is to provide an infrastructure to transform the
set of differences sent from the applications, before sending
them to the external diff interface.
The recently introduced rename detection code has been rewritten
to use the diff-core facility. When applications send in
separate creates and deletes, matching ones are transformed into
a single rename-and-edit diff, and sent out to the external diff
interface as such.
This patch also enhances the rename detection code further to be
able to detect copies. Currently this happens only as long as
copy sources appear as part of the modified files, but there
already is enough provision for callers to report unmodified
files to diff-core, so that they can be also used as copy source
candidates. Extending the callers this way will be done in a
separate patch.
Please see and marvel at how well this works by trying out the
newly added t/t4003-diff-rename-1.sh test script.
Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
This adds the ability to actually create delta objects using a new tool:
git-mkdelta. It uses an ordered list of potential objects to deltafy
against earlier objects in the list. A cap on the depth of delta
references can be provided as well, otherwise the default is to not have
any limit. A limit of 0 will also undeltafy any given object.
Also provided is the beginning of a script to deltafy an entire
repository.
Signed-off-by: Nicolas Pitre <nico@cam.org> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
This adds knowledge of delta objects to fsck-cache and various object
parsing code. A new switch to git-fsck-cache is provided to display the
maximum delta depth found in a repository.
Signed-off-by: Nicolas Pitre <nico@cam.org> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
This makes the core code aware of delta objects and undeltafy them as
needed. The convention is to use read_sha1_file() to have
undeltafication done automatically (most users do that already so this
is transparent).
If the delta object itself has to be accessed then it must be done
through map_sha1_file() and unpack_sha1_file().
In that context mktag.c has been switched to read_sha1_file() as there
is no reason to do the full map+unpack manually.
Signed-off-by: Nicolas Pitre <nico@cam.org> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Fix various things that sparse complains about:
- use NULL instead of 0
- make sure we declare everything properly, or mark it static
- use proper function declarations ("fn(void)" instead of "fn()")
[PATCH] Simplify "reverse-diff" logic in the diff core.
Instead of swapping the arguments just before output, this patch
makes the swapping happen on the input side of the diff core,
when "reverse-diff" is in effect. This greatly simplifies the
logic, but more importantly it is necessary for upcoming "copy
detection" work.
Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
The same check we added earlier to update-cache to catch ENOTDIR
turns out to be missing from diff-files. This causes a
difference not being reported when you have DF/DF (a file in a
subdirectory) in the cache and DF is a file on the filesystem.
Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
This one compares two pathnames that may be partial basenames, not
full paths. We need to get the path sorting right, since a directory
name will sort as if it had the final '/' at the end.
Add '-R' flag to diff-tree, and change the test subdirectory
shell files to be executable (something that Junio couldn't
get me to do through the pure patch with my current patch
handling infrastructure).
This cleans up the way calls are made into the diff core from diff-tree
family and diff-helper. Earlier, these programs had "if
(generating_patch)" sprinkled all over the place, but those ugliness are
gone and handled uniformly from the diff core, even when not generating
patch format.
This also allowed diff-cache and diff-files to acquire -R
(reverse) option to generate diff in reverse. Users of
diff-tree can swap two trees easily so I did not add -R there.
[ Linus' note: I'll add -R to "diff-tree" too, since a "commit
diff" doesn't have another tree to switch around: the other
tree is always the parent(s) of the commit ]
Also -M<digits-as-mantissa> suggestion made by Linus has been
implemented.
Documentation updates are also included.
Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
[PATCH] Implement git-checkout-cache -u to update stat information in the cache.
With -u flag, git-checkout-cache picks up the stat information
from newly created file and updates the cache. This removes the
need to run git-update-cache --refresh immediately after running
git-checkout-cache.
Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
This rips out the rename detection engine from diff-helper and moves it
to the diff core, and updates the internal calling convention used by
diff-tree family into the diff core. In order to give the same option
name to diff-tree family as well as to diff-helper, I've changed the
earlier diff-helper '-r' option to '-M' (stands for Move; sorry but the
natural abbreviation 'r' for 'rename' is already taken for 'recursive').
Although I did a fair amount of test with the git-diff-tree with
existing rename commits in the core GIT repository, this should still be
considered beta (preview) release. This patch depends on the diff-delta
infrastructure just committed.
This implements almost everything I wanted to see in this series of
patch, except a few minor cleanups in the calling convention into diff
core, but that will be a separate cleanup patch.
Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
I just remembered why I placed that bogus "sb->len ==0 implies
sb->eof" condition there. We need at least something like this
to catch the normal EOF (that is, line termination immediately
followed by EOF) case. "if (feof(fp))" fires when we have
already read the eof, not when we are about read it.
Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
diff-tree: don't match non-directories as partial pathnames
This normally doesn't matter, but if you have a filename that is
sometimes a directory and sometimes a regular file (or symlink),
we don't want the regular file case to trigger a "partial match".
diff-tree: fix up comparison of "interesting" sub-trees
We used to trigger the "interesting subdirectory" check for any
matching name that started with the same character series, regardless
of whether it had the matching slash or not.
We use "--" to mark end of command line switches, not "-". Also,
allow more flexibility in the passed-in sha1 names, in that a
single sha1 uses the "commit-diff" logic that compares against
its parent(s).