When upload-pack advertises refs, it dereferences any tags
it sees, and shows the resulting sha1 to the client. It does
this by calling deref_tag. That function must load and parse
each tag object to find the sha1 of the tagged object.
However, it also ends up parsing the tagged object itself,
which is not strictly necessary for upload-pack's use.
Each tag produces two object loads (assuming it is not a
recursive tag), when it could get away with only a single
one. Dropping the second load halves the effort we spend.
The downside is that we are no longer verifying the
resulting object by loading it. In particular:
1. We never cross-check the "type" field given in the tag
object with the type of the pointed-to object. If the
tag says it points to a tag but doesn't, then we will
keep peeling and realize the error. If the tag says it
points to a non-tag but actually points to a tag, we
will stop peeling and just advertise the pointed-to
tag.
2. If we are missing the pointed-to object, we will not
realize (because we never even look it up in the object
db).
However, both of these are errors in the object database,
and both will be detected if a client actually requests the
broken objects in question. So we are simply pushing the
verification away from the advertising stage, and down to
the actual fetching stage.
On my test repo with 120K refs, this drops the time to
advertise the refs from ~3.2s to ~2.0s.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
upload-pack: avoid parsing objects during ref advertisement
When we advertise a ref, the first thing we do is parse the
pointed-to object. This gives us two things:
1. a "struct object" we can use to store flags
2. the type of the object, so we know whether we need to
dereference it as a tag
Instead, we can just use lookup_unknown_object to get an
object struct, and then fill in just the type field using
sha1_object_info (which, in the case of packed files, can
find the information without actually inflating the object
data).
This can save time if you have a large number of refs, and
the client isn't actually going to request those refs (e.g.,
because most of them are already up-to-date).
The downside is that we are no longer verifying objects that
we advertise by fully parsing them (however, we do still
know we actually have them, because sha1_object_info must
find them to get the type). While we might fail to detect a
corrupt object here, if the client actually fetches the
object, we will parse (and verify) it then.
On a repository with 120K refs, the advertisement portion of
upload-pack goes from ~3.4s to 3.2s (the failure to speed up
more is largely due to the fact that most of these refs are
tags, which need dereferenced to find the tag destination
anyway).
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* jc/show-sig:
log --show-signature: reword the common two-head merge case
log-tree: show mergetag in log --show-signature output
log-tree.c: small refactor in show_signature()
commit --amend -S: strip existing gpgsig headers
verify_signed_buffer: fix stale comment
gpg-interface: allow use of a custom GPG binary
pretty: %G[?GS] placeholders
test "commit -S" and "log --show-signature"
log: --show-signature
commit: teach --gpg-sign option
Merge branch 'jn/maint-gitweb-utf8-fix' into maint
* jn/maint-gitweb-utf8-fix:
gitweb: Fix fallback mode of to_utf8 subroutine
gitweb: Output valid utf8 in git_blame_common('data')
gitweb: esc_html() site name for title in OPML
gitweb: Call to_utf8() on input string in chop_and_escape_str()
Documentation: rerere.enabled is the primary way to configure rerere
The wording seems to suggest that creating the directory is needed and the
setting of rerere.enabled is only for disabling the feature by setting it
to 'false'. But the configuration is meant to be the primary control and
setting it to 'true' will enable it; the rr-cache directory will be
created as necessary and the user does not have to create it.
Signed-off-by: Thomas Rast <trast@student.ethz.ch> Signed-off-by: Junio C Hamano <gitster@pobox.com>
generate array of SHA1s for alternate refs
for each unique SHA1 in array:
add_extra_ref(".have", sha1)
for each ref (including real refs and extra refs):
show_ref(refname, sha1)
But there is no need to stuff the alternate refs in extra_refs; we can
call show_ref() directly when iterating over the array, then handle
real refs separately. So change the code to:
generate array of SHA1s for alternate refs
for each unique SHA1 in array:
show_ref(".have", sha1)
for each ref (this now only includes real refs):
show_ref(refname, sha1)
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
receive-pack: move more work into write_head_info()
Move some more code from the calling site into write_head_info(), and
inline add_alternate_refs() there. (Some more simplification is
coming, and it is easier if all this code is in the same place.)
Move some helper functions to avoid the need for forward declarations.
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
xdiff: print post-image for common records instead of pre-image
Normally it doesn't matter if we show the pre-image or th post-image
for the common parts of a diff because they are the same. If
white-space changes are ignored they can differ, though. The
new text after applying the diff is more interesting in that case,
so show that instead of the old contents.
Note: GNU diff shows the pre-image.
Suggested-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx> Signed-off-by: Junio C Hamano <gitster@pobox.com>
parse_object: try internal cache before reading object db
When parse_object is called, we do the following:
1. read the object data into a buffer via read_sha1_file
2. call parse_object_buffer, which then:
a. calls the appropriate lookup_{commit,tree,blob,tag}
to either create a new "struct object", or to find
an existing one. We know the appropriate type from
the lookup in step 1.
b. calls the appropriate parse_{commit,tree,blob,tag}
to parse the buffer for the new (or existing) object
In step 2b, all of the called functions are no-ops for
object "X" if "X->object.parsed" is set. I.e., when we have
already parsed an object, we end up going to a lot of work
just to find out at a low level that there is nothing left
for us to do (and we throw away the data from read_sha1_file
unread).
We can optimize this by moving the check for "do we have an
in-memory object" from 2a before the expensive call to
read_sha1_file in step 1.
This might seem circular, since step 2a uses the type
information determined in step 1 to call the appropriate
lookup function. However, we can notice that all of the
lookup_* functions are backed by lookup_object. In other
words, all of the objects are kept in a master hash table,
and we don't actually need the type to do the "do we have
it" part of the lookup, only to do the "and create it if it
doesn't exist" part.
This can save time whenever we call parse_object on the same
sha1 twice in a single program. Some code paths already
perform this optimization manually, with either:
if you don't. This patch moves the optimization into
parse_object itself.
Most git operations won't notice any impact. Either they
don't parse a lot of duplicate sha1s, or the calling code
takes special care not to re-parse objects. I timed two
code paths that do benefit (there may be more, but these two
were immediately obvious and easy to time).
The first is fast-export, which calls parse_object on each
object it outputs, like this:
object = parse_object(sha1);
if (!object)
die(...);
if (object->flags & SHOWN)
return;
which means that just to realize we have already shown an
object, we will read the whole object from disk!
With this patch, my best-of-five time for "fast-export --all" on
git.git dropped from 26.3s to 21.3s.
The second case is upload-pack, which will call parse_object
for each advertised ref (because it needs to peel tags to
show "^{}" entries). This doesn't matter for most
repositories, because they don't have a lot of refs pointing
to the same objects. However, if you have a big alternates
repository with a shared object db for a number of child
repositories, then the alternates repository will have
duplicated refs representing each of its children.
For example, GitHub's alternates repository for git.git has
~120,000 refs, of which only ~3200 are unique. The time for
upload-pack to print its list of advertised refs dropped
from 3.4s to 0.76s.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
gitweb: Harden "grep" search against filenames with ':'
Run "git grep" in "grep" search with '-z' option, to be able to parse
response also for files with filename containing ':' character. The
':' character is otherwise (without '-z') used to separate filename
from line number and from matched line.
Note that this does not protect files with filename containing
embedded newline. This would be hard but doable for text files, and
harder or even currently impossible with binary files: git does not
quote filename in
"Binary file <foo> matches"
message, but new `--break` and/or `--header` options to git-grep could
help here.
Signed-off-by: Jakub Narebski <jnareb@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
log --show-signature: reword the common two-head merge case
While identifying the commit merged to our history as "parent #2" is
technically correct, we will never say "parent #1" (as that is the tip of
our history before the merge is made), and we rarely would say "parent #3"
(which would mean the merge is an octopus), especially when responding to
a request to pull a signed tag.
Treat the most common case to merge a single commit specially, and just
say "merged tag '<tagname>'" instead.
log-tree: show mergetag in log --show-signature output
A commit object that merges a signed tag records the "mergetag" extended
header. Check the validity of the GPG signature on it, and show it in a
way similar to how "gpgsig" extended header is shown.
The next patch needs to show the result of signature verification on a
mergetag extended header in a way similar to how embedded signature for
the commit object itself is shown. Separate out the logic to go through
the message lines and show them in the "error" color (highlighted) or the
"correct" color (dim).
Any existing commit signature was made against the contents of the old
commit, including its committer date that is about to change, and will
become invalid by amending it.
Merge branch 'jc/signed-commit' and 'jc/pull-signed-tag'
They both use the extended headers in commit objects, and the former has
necessary infrastructure to show them that is useful to view the result of
the latter.
Subsequently we assume that there is only one pack. Currently this is
true only by accident. Pass '-a -d' to repack in order to guarantee that
assumption to hold true.
The prune-packed command is now redundant since repack -d already calls
it.
Signed-off-by: Clemens Buchacher <drizzd@aon.at> Signed-off-by: Junio C Hamano <gitster@pobox.com>
The current code is removing the trailing "/", but computing the string
length on the previous value, i.e. with the trailing "/". Later in the
code, we do
my $path = substr($File::Find::name, $pfxlen + 1);
And the "$pfxlen + 1" is supposed to mean "the length of the prefix, plus
1 for the / separating the prefix and the path", but with an incorrect
$pfxlen, this basically eats the first character of the path, and yields
"404 - No projects found".
While we're there, also fix $pfxdepth to use $dir, although a change of 1
in the depth shouldn't really matter.
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
The FETCH_HEAD refname is supposed to refer to the ref that was fetched
and should be merged. However all fetched refs are written to
.git/FETCH_HEAD in an arbitrary order, and resolve_ref_unsafe simply
takes the first ref as the FETCH_HEAD, which is often the wrong one,
when other branches were also fetched.
The solution is to write the for-merge ref(s) to FETCH_HEAD first.
Then, unless --append is used, the FETCH_HEAD refname behaves as intended.
If the user uses --append, they presumably are doing so in order to
preserve the old FETCH_HEAD.
While we are at it, update an old example in the read-tree documentation
that implied that each entry in FETCH_HEAD only has the object name, which
is not true for quite a while.
[jc: adjusted tests]
Signed-off-by: Joey Hess <joey@kitenet.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Perforce client views can map individual files,
mapping one //depot file path to one //client file path.
These mappings contain no meta/masking characters.
This patch add support for these file maps to
the currently supported '...' view mappings.
[pw: one test now suceeds]
Signed-off-by: Gary Gibbons <ggibbons@perforce.com> Signed-off-by: Pete Wyckoff <pw@padd.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* pw/p4-docs-and-tests:
git-p4: document and test submit options
git-p4: test and document --use-client-spec
git-p4: test --keep-path
git-p4: test --max-changes
git-p4: document and test --import-local
git-p4: honor --changesfile option and test
git-p4: document and test clone --branch
git-p4: test cloning with two dirs, clarify doc
git-p4: clone does not use --git-dir
git-p4: introduce asciidoc documentation
rename git-p4 tests
* maint:
docs: describe behavior of relative submodule URLs
fix hang in git fetch if pointed at a 0 length bundle
Documentation: read-tree --prefix works with existing subtrees
Add MYMETA.json to perl/.gitignore
* maint-1.7.7:
docs: describe behavior of relative submodule URLs
Documentation: read-tree --prefix works with existing subtrees
Add MYMETA.json to perl/.gitignore
docs: describe behavior of relative submodule URLs
Since the relative submodule URLs have been introduced in f31a522a2d, they
do not conform to the rules for resolving relative URIs but rather to
those of relative directories.
Document that behavior.
Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
fix hang in git fetch if pointed at a 0 length bundle
git-repo if interupted at the exact wrong time will generate zero
length bundles- literal empty files. git-repo is wrong here, but
git fetch shouldn't effectively spin loop if pointed at a zero
length bundle.
Signed-off-by: Brian Harring <ferringb@chromium.org> Helped-by: Johannes Sixt Helped-by: Nguyen Thai Ngoc Duy Signed-off-by: Junio C Hamano <gitster@pobox.com>
Documentation: read-tree --prefix works with existing subtrees
Since 34110cd4 (Make 'unpack_trees()' have a separate source and
destination index) it is no longer true that a subdirectory with
the same prefix must not exist.
Signed-off-by: Clemens Buchacher <drizzd@aon.at> Signed-off-by: Junio C Hamano <gitster@pobox.com>
stash: Don't fail if work dir contains file named 'HEAD'
When performing a plain "git stash" (without --patch), git-diff would fail
with "fatal: ambiguous argument 'HEAD': both revision and filename". The
output was piped into git-update-index, masking the failed exit status.
The output is now sent to a temporary file (which is cleaned up by
existing code), and the exit status is checked. The "HEAD" arg to the
git-diff invocation has been disambiguated too, of course.
In patch mode, "git stash -p" would fail harmlessly, leaving the working
dir untouched. Interactive adding is fine, but the resulting tree was
diffed with an ambiguous 'HEAD' argument.
Use >foo (no space) when redirecting output.
In t3904, checks and operations on each file are in the order they'll
appear when interactively staging.
In t3905, fix a bug in "stash save --include-untracked -q is quiet": The
redirected stdout file was considered untracked, and so was removed from
the working directory. Use test path helper functions where appropriate.
Signed-off-by: Jonathon Mah <me@JonathonMah.com> Acked-by: Thomas Rast <trast@student.ethz.ch> Signed-off-by: Junio C Hamano <gitster@pobox.com>
ExtUtils::MakeMaker generates MYMETA.json in addition to MYMETA.yml
since version 6.57_07. As it suggests, it is just meta information about
the build and is cleaned up with 'make clean', so it should be ignored.
Signed-off-by: Jack Nagel <jacknagel@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* jc/checkout-m-twoway:
t/t2023-checkout-m.sh: fix use of test_must_fail
checkout_merged(): squelch false warning from some gcc
Test 'checkout -m -- path'
checkout -m: no need to insist on having all 3 stages
Merge branch 'jn/maint-sequencer-fixes' into maint
* jn/maint-sequencer-fixes:
revert: stop creating and removing sequencer-old directory
Revert "reset: Make reset remove the sequencer state"
revert: do not remove state until sequence is finished
revert: allow single-pick in the middle of cherry-pick sequence
revert: pass around rev-list args in already-parsed form
revert: allow cherry-pick --continue to commit before resuming
revert: give --continue handling its own function
* jk/maint-mv:
mv: be quiet about overwriting
mv: improve overwrite warning
mv: make non-directory destination error more clear
mv: honor --verbose flag
docs: mention "-k" for both forms of "git mv"
Merge branch 'jk/fetch-no-tail-match-refs' into maint
* jk/fetch-no-tail-match-refs:
connect.c: drop path_match function
fetch-pack: match refs exactly
t5500: give fully-qualified refs to fetch-pack
drop "match" parameter from get_remote_heads
Merge branch 'jk/refresh-porcelain-output' into maint
* jk/refresh-porcelain-output:
refresh_index: make porcelain output more specific
refresh_index: rename format variables
read-cache: let refresh_cache_ent pass up changed flags
The depot path is required, even with this option. Make sure
git-p4 fails and exits with non-zero.
Contents in the specified depot path will be rearranged according
to the client spec. Test this and add a note in the docs.
Leave an XXX suggesting that this is somewhat confusing behavior
that might be good to fix later.
Function stripRepoPath() looks at self.useClientSpec. Make sure
this is set both for command-line option --use-client-spec and
for configuration variable git-p4.useClientSpec. Test this.
Signed-off-by: Pete Wyckoff <pw@padd.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
When an explicit list of changes is given, it makes no sense to
use @all or @3,5 or any of the other p4 revision specifiers.
Make the code notice when this happens, instead of just ignoring
--changesfile. Test it.
Signed-off-by: Pete Wyckoff <pw@padd.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Clone with --branch will not checkout HEAD, unless the branch
happens to be called the default refs/remotes/p4/master. The
--branch option is most useful with sync; give an example of
that.
Signed-off-by: Pete Wyckoff <pw@padd.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add proper documentation for git-p4. Delete the old .txt
documentation from contrib/fast-import.
Cc: Frans Klaver <fransklaver@gmail.com> Cc: Luke Diamand <luke@diamand.org> Signed-off-by: Pete Wyckoff <pw@padd.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* ab/sun-studio-portability:
Appease Sun Studio by renaming "tmpfile"
Fix a bitwise negation assignment issue spotted by Sun Studio
Fix an enum assignment issue spotted by Sun Studio
* jn/maint-gitweb-utf8-fix:
gitweb: Fix fallback mode of to_utf8 subroutine
gitweb: Output valid utf8 in git_blame_common('data')
gitweb: esc_html() site name for title in OPML
gitweb: Call to_utf8() on input string in chop_and_escape_str()
* rs/diff-tree-combined-clean-up:
submodule: use diff_tree_combined_merge() instead of diff_tree_combined()
pass struct commit to diff_tree_combined_merge()
use struct sha1_array in diff_tree_combined()
* tr/grep-threading:
grep: disable threading in non-worktree case
grep: enable threading with -p and -W using lazy attribute lookup
grep: load funcname patterns for -W