Merge branch 'jc/maint-1.6.0-keep-pack' into maint-1.6.1
* jc/maint-1.6.0-keep-pack:
pack-objects: don't loosen objects available in alternate or kept packs
t7700: demonstrate repack flaw which may loosen objects unnecessarily
Remove --kept-pack-only option and associated infrastructure
pack-objects: only repack or loosen objects residing in "local" packs
git-repack.sh: don't use --kept-pack-only option to pack-objects
t7700-repack: add two new tests demonstrating repacking flaws
is_kept_pack(): final clean-up
Simplify is_kept_pack()
Consolidate ignore_packed logic more
has_sha1_kept_pack(): take "struct rev_info"
has_sha1_pack(): refactor "pretend these packs do not exist" interface
git-repack: resist stray environment variable
Merge branch 'bs/maint-1.6.0-tree-walk-prefix' into maint-1.6.1
* bs/maint-1.6.0-tree-walk-prefix:
match_tree_entry(): a pathspec only matches at directory boundaries
tree_entry_interesting: a pathspec only matches at directory boundary
The combine diff logic knew only about blobs (and their checked-out form
in the work tree, either regular files or symlinks), and barfed when fed
submodules. This "externalizes" gitlinks in the same way as the normal
patch generation codepath does (i.e. "Subproject commit Xxx\n") to fix the
issue.
* maint-1.6.0:
grep: fix segfault when "git grep '('" is given
Documentation: fix a grammatical error in api-builtin.txt
builtin-merge: fix a typo in an error message
When interpreting a config value, the config parser reads in 1+ space
character(s) and puts -one- space character in the buffer as soon as
the first non-space character is encountered (if not inside quotes).
Unfortunately the buffer size check lacks the extra space character
which gets inserted at the next non-space character, resulting in
a crash with a specially crafted config entry.
The unit test now uses Java to compile a platform independent
.NET framework to output the test string in C# :o)
Read: Thanks to Johannes Sixt for the correct printf call
which replaces the perl invocation.
Signed-off-by: Thomas Jarosch <thomas.jarosch@intra2net.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
On Wed, 8 Apr 2009, Björn Steinbrink wrote:
>
> The name of the processed object was duplicated for passing it to
> add_object(), but that already calls path_name, which allocates a new
> string anyway. So the memory allocated by the xstrdup calls just went
> nowhere, leaking memory.
Ack, ack.
There's another easy 5% or so for the built-in object walker: once we've
created the hash from the name, the name isn't interesting any more, and
so something trivial like this can help a bit.
Does it matter? Probably not on its own. But a few more memory saving
tricks and it might all make a difference.
The name of the processed object was duplicated for passing it to
add_object(), but that already calls path_name, which allocates a new
string anyway. So the memory allocated by the xstrdup calls just went
nowhere, leaking memory.
This reduces the RSS usage for a "rev-list --all --objects" by about 10% on
the gentoo repo (fully packed) as well as linux-2.6.git:
gentoo:
| old | new
----------------|-------------------------------
RSS | 1537284 | 1388408
VSZ | 1816852 | 1667952
time elapsed | 1:49.62 | 1:48.99
min. page faults| 417178 | 379919
linux-2.6.git:
| old | new
----------------|-------------------------------
RSS | 324452 | 292996
VSZ | 491792 | 460376
time elapsed | 0:14.53 | 0:14.28
min. page faults| 89360 | 81613
Signed-off-by: Björn Steinbrink <B.Steinbrink@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* maint-1.6.0:
Documentation: clarify .gitattributes search
git-checkout.txt: clarify that <branch> applies when no path is given.
git-checkout.txt: fix incorrect statement about HEAD and index
git-checkout.txt: fix incorrect statement about HEAD and index
The command "git checkout" checks out from the index by default, not
HEAD (the introducing comment were correct, but the detailled
explanation added below were not).
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Previously we ignored the result of calling add_interactive,
which meant that if an error occurred we simply committed
whatever happened to be in the index.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* maint-1.6.0:
Fix bash completion in path with spaces
bash completion: only show 'log --merge' if merging
git-tag(1): add hint about commit messages
Documentation: update graph api example.
Signed-off-by: Daniel Cheng (aka SDiZ) <j16sdiz+freenet@gmail.com> Trivially-acked-by: Shawn O. Pearce <spearce@spearce.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Embarrassingly, the common prefix calculation did not work properly, due
to a mistake in the assignment: instead of assigning the dirname of the
current file name, the dirname of the current common prefix needs to
be assigned to common prefix, when the current prefix does not match the
current file name.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Users were confused about the meaning and use of the --root option.
Notably, since 68c2ec7 (format-patch: show patch text for the root
commit, 2009-01-10), --root has nothing to do with showing the patch
text for the root commit any more.
Shorten and clarify the corresponding paragraph in the DESCRIPTION
section, document --root under OPTIONS, and add an explicit note that
root commits are formatted regardless.
Signed-off-by: Thomas Rast <trast@student.ethz.ch> Signed-off-by: Junio C Hamano <gitster@pobox.com>
diff --no-index: Do not generate patch output if other output is requested
Previously, 'git diff --no-index --stat a b' generated patch output in
addition to the --stat output (or whatever other output format was
requested). Now only the requested output is generated, and patch
output remains the default.
Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
avoid possible overflow in delta size filtering computation
On a 32-bit system, the maximum possible size for an object is less than
4GB, while 64-bit systems may cope with larger objects. Due to this
limitation, variables holding object sizes are using an unsigned long
type (32 bits on 32-bit systems, or 64 bits on 64-bit systems).
When large objects are encountered, and/or people play with large delta
depth values, it is possible for the maximum allowed delta size
computation to overflow, especially on a 32-bit system. When this
occurs, surviving result bits may represent a value much smaller than
what it is supposed to be, or even zero. This prevents some objects
from being deltified although they do get deltified when a smaller depth
limit is used. Fix this by always performing a 64-bit multiplication.
Signed-off-by: Nicolas Pitre <nico@cam.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
diff --cached: do not borrow from a work tree when a path is marked as assume-unchanged
When the index says that the file in the work tree that corresponds to the
blob object that is used for comparison is known to be unchanged, "diff"
reads from the file and applies convert_to_git(), instead of inflating the
object, to feed the internal diff engine with, because an earlier
benchnark found that it tends to be faster to use this optimization.
However, the index can lie when the path is marked as assume-unchanged.
Disable the optimization for such paths.
Merge branch 'js/maint-1.6.0-path-normalize' into maint-1.6.1
* js/maint-1.6.0-path-normalize:
Remove unused normalize_absolute_path()
Test and fix normalize_path_copy()
Fix GIT_CEILING_DIRECTORIES on Windows
Move sanitary_path_copy() to path.c and rename it to normalize_path_copy()
Make test-path-utils more robust against incorrect use
pack-objects: don't loosen objects available in alternate or kept packs
If pack-objects is called with the --unpack-unreachable option then it
will unpack (i.e. loosen) all unreferenced objects from local not-kept
packs, including those that also exist in packs residing in an alternate
object database or a locally kept pack. The only user of this option is
git-repack.
In this case, repack will follow the call to pack-objects with a call to
prune-packed, which will delete these newly loosened objects, making the
act of loosening a waste of time. The unnecessary loosening can be
avoided by checking whether an object exists in a non-local pack or a
locally kept pack before loosening it.
This fixes the 'local packed unreachable obs that exist in alternate ODB
are not loosened' test in t7700.
Signed-off-by: Brandon Casey <drafnel@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Propagate --exec-path setting to external commands via GIT_EXEC_PATH
Let PATH0=$PATH that was set before the invocation.
Let /foo be a build directory.
Let /pfx be the installation prefix.
Let pfxexecpath=/pfx/libexec/git-core.
The following is going on when 'git --exec-path=/foo gc' is invoked:
1. git sets PATH=/foo:$PATH0 using the path from --exec-path
2. gc execs 'git repack' (note: no dash).
3. Since there is a git in /foo (it's a build directory), /foo/git is
taken.
4. No explicit exec-path is set this time, hence, this secondary git sets
PATH=$pfxexecpath:/foo:$PATH
5. Since 'repack' is not a built-in, execv_dashed_external execs
'git-repack' (note: dash).
6. There is a $pfxexecpath/git-repack, and it is taken.
7. This git-repack runs 'git pack-objects' (note: no dash).
8. There is no git in $pfxexecpath, but there is one in /foo. Hence,
/foo/git is run.
9. pack-objects is a builtin, hence, in effect /foo/git-pack-objects
is run.
As you can see, the way in which we previously set the PATH allowed to
mix gits of different vintage. By setting GIT_EXEC_PATH when --exec-path
was given on the command line, we reduce the confusion.
Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
t7700: demonstrate repack flaw which may loosen objects unnecessarily
If an unreferenced object exists in both a local pack and in either a pack
residing in an alternate object database or a local kept pack, then the
pack-objects call made by repack will loosen that object only to have it
immediately pruned by repack's call to prune-packed.
Signed-off-by: Brandon Casey <drafnel@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Remove --kept-pack-only option and associated infrastructure
This option to pack-objects/rev-list was created to improve the -A and -a
options of repack. It was found to be lacking in that it did not provide
the ability to differentiate between local and non-local kept packs, and
found to be unnecessary since objects residing in local kept packs can be
filtered out by the --honor-pack-keep option.
Signed-off-by: Brandon Casey <casey@nrlssc.navy.mil> Signed-off-by: Junio C Hamano <gitster@pobox.com>
pack-objects: only repack or loosen objects residing in "local" packs
These two features were invented for use by repack when repack will delete
the local packs that have been made redundant. The packs accessible
through alternates are not deleted by repack, so the objects contained in
them are still accessible after the local packs are deleted. They do not
need to be repacked into the new pack or loosened. For the case of
loosening they would immediately be deleted by the subsequent prune-packed
that is called by repack anyway.
This fixes the test
'packed unreachable obs in alternate ODB are not loosened' in t7700.
Signed-off-by: Brandon Casey <casey@nrlssc.navy.mil> Signed-off-by: Junio C Hamano <gitster@pobox.com>
git-repack.sh: don't use --kept-pack-only option to pack-objects
The --kept-pack-only option to pack-objects treats all kept packs as equal.
This results in objects that reside in an alternate pack that has a .keep
file, not being packed into a newly created pack when the user specifies the
-a option to repack. Since the user may not have any control over the
alternate database, git should not refrain from repacking those objects
even though they are in a pack with a .keep file.
This fixes the 'packed obs in alternate ODB kept pack are repacked' test in
t7700.
Signed-off-by: Brandon Casey <casey@nrlssc.navy.mil> Signed-off-by: Junio C Hamano <gitster@pobox.com>
t7700-repack: add two new tests demonstrating repacking flaws
1) The new --kept-pack-only mechansim of rev-list/pack-objects has
replaced --unpacked=. This new mechansim does not operate solely on
"local" packs now. The result is that objects residing in an alternate
pack which has a .keep file will not be repacked with repack -a.
This flaw is only apparent when a commit object is the one residing in
an alternate kept pack.
2) The 'repack unpacked objects' and 'loosen unpacked objects' mechanisms
of pack-objects, i.e. --keep-unreachable and --unpack-unreachable,
now do not operate solely on local packs. The --keep-unreachable
option no longer has any callers, but --unpack-unreachable is used when
repack is called with '-A -d' and the local repo has existing packs.
In this case, objects residing in alternate, not-kept packs will be
loosened, and then immediately deleted by repack's call to
prune-packed.
The test must manually call pack-objects to avoid the call to
prune-packed that is made by repack when -d is used.
Signed-off-by: Brandon Casey <casey@nrlssc.navy.mil> Signed-off-by: Junio C Hamano <gitster@pobox.com>
blame: read custom grafts given by -S before calling setup_revisions()
setup_revisions() while getting the command line arguments parses the
given commits from the command line, which means their direct parents will
not be rewritten by the custom graft file.
Call read_ancestry() early to work around this issue.
* maint-1.6.0:
bisect: fix another instance of eval'ed string
bisect: fix quoting TRIED revs when "bad" commit is also "skip"ped
Support "\" in non-wildcard exclusion entries
Merge branch 'ks/maint-1.6.0-mailinfo-folded' into maint-1.6.0
* ks/maint-1.6.0-mailinfo-folded:
mailinfo: tests for RFC2047 examples
mailinfo: add explicit test for mails like '<a.u.thor@example.com> (A U Thor)'
mailinfo: 'From:' header should be unfold as well
mailinfo: correctly handle multiline 'Subject:' header
Merge branch 'js/maint-1.6.1-remote-remove-mirror' into maint-1.6.1
* js/maint-1.6.1-remote-remove-mirror:
builtin-remote: make rm operation safer in mirrored repository
builtin-remote: make rm() use properly named variable to hold return value
Merge branch 'ks/maint-1.6.0-mailinfo-folded' into maint-1.6.1
* ks/maint-1.6.0-mailinfo-folded:
mailinfo: tests for RFC2047 examples
mailinfo: add explicit test for mails like '<a.u.thor@example.com> (A U Thor)'
mailinfo: 'From:' header should be unfold as well
mailinfo: correctly handle multiline 'Subject:' header
Now is_kept_pack() is just a member lookup into a structure, we can write
it as such.
Also rewrite the sole caller of has_sha1_kept_pack() to switch on the
criteria the callee uses (namely, revs->kept_pack_only) between calling
has_sha1_kept_pack() and has_sha1_pack(), so that these two callees do not
have to take a pointer to struct rev_info as an argument.
This removes the header file dependency issue temporarily introduced by
the earlier commit, so we revert changes associated to that as well.
This removes --unpacked=<packfile> parameter from the revision parser, and
rewrites its use in git-repack to pass a single --kept-pack-only option
instead.
The new --kept-pack-only option means just that. When this option is
given, is_kept_pack() that used to say "not on the --unpacked=<packfile>
list" now says "the packfile has corresponding .keep file".
This refactors three loops that check if a given packfile is on the
ignore_packed list into a function is_kept_pack(). The function returns
false for a pack on the list, and true for a pack not on the list, because
this list is solely used by "git repack" to pass list of packfiles that do
not have corresponding .keep files, i.e. a packfile not on the list is
"kept".
Its "ignore_packed" parameter always comes from struct rev_info. This
patch makes the function take a pointer to the surrounding structure, so
that the refactoring in the next patch becomes easier to review.
There is an unfortunate header file dependency and the easiest workaround
is to temporarily move the function declaration from cache.h to
revision.h; this will be moved back to cache.h once the function loses
this "ignore_packed" parameter altogether in the later part of the
series.
has_sha1_pack(): refactor "pretend these packs do not exist" interface
Most of the callers of this function except only one pass NULL to its last
parameter, ignore_packed.
Introduce has_sha1_kept_pack() function that has the function signature
and the semantics of this function, and convert the sole caller that does
not pass NULL to call this new function.
All other callers and has_sha1_pack() lose the ignore_packed parameter.
The script used $args and $existing without initializing it to empty. It
would have been confused by an environment variable the end user had
before running it.
When there is nothing to be skipped, the output from
rev-list --bisect-vars was eval'ed without first being
strung together with &&; this is probably not a problem
as it is much less likely to be a bad input than the list
handcrafted by the filter_skip function, but it still is
a good discipline.
Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
bisect: fix quoting TRIED revs when "bad" commit is also "skip"ped
When the "bad" commit was also "skip"ped and when more than one
commit was skipped, the "filter_skipped" function would have
printed something like:
bisect_rev=<hash1>|<hash2>
(where <hash1> and <hash2> are hexadecimal sha1 hashes)
and this would have been evaled later as piping "bisect_rev=<hash1>"
into "<hash2>", which would have failed.
So this patch makes the "filter_skipped" function properly quote
what it outputs, so that it will print something like:
bisect_rev='<hash1>|<hash2>'
which will be properly evaled later. The caller was not stopping
properly because the scriptlet this function returned to be evaled
was not strung together with && and because of this, an error in
an earlier part of the output was simply ignored.
A test case is added to the test suite.
And while at it, we also initialize the VARS, FOUND and TRIED
variables, so that we protect ourselves from environment variables
the user may have with these names.
Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
The AIX mkstemp() modifies its template parameter to an empty string if
the call fails. The existing code had already recomputed the template,
but too late to be good.
See also 6ff6af62, which fixed this problem in a different spot.
Signed-off-by: Mike Ralphson <mike@abacus.co.uk> Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Make sure objects/pack exists before creating a new pack
In a repository created with git older than f49fb35 (git-init-db: create
"pack" subdirectory under objects, 2005-06-27), objects/pack/ directory is
not created upon initialization. It was Ok because subdirectories are
created as needed inside directories init-db creates, and back then,
packfiles were recent invention.
After the said commit, new codepaths started relying on the presense of
objects/pack/ directory in the repository. This was exacerbated with 8b4eb6b (Do not perform cross-directory renames when creating packs,
2008-09-22) that moved the location temporary pack files are created from
objects/ directory to objects/pack/ directory, because moving temporary to
the final location was done carefully with lazy leading directory creation.
Many packfile related operations in such an old repository can fail
mysteriously because of this.
This commit introduces two helper functions to make things work better.
- odb_mkstemp() is a specialized version of mkstemp() to refactor the
code and teach it to create leading directories as needed;
- odb_pack_keep() refactors the code to create a ".keep" file while
create leading directories as needed.
More friendly message when locking the index fails.
Just saying that index.lock exists doesn't tell the user _what_ to do
to fix the problem. We should give an indication that it's normally
safe to delete index.lock after making sure git isn't running here.
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>