Merge branch 'for-junio' of git://bogomips.org/git-svn
* 'for-junio' of git://bogomips.org/git-svn:
git-svn: use SVN::Ra::get_dir2 when possible
git-svn: add space after "W:" prefix in warning
git-svn: (cleanup) remove editor param passing
git-svn: prepare SVN::Ra config pieces once
Git.pm: add specified name to tempfile template
git-svn: disable _rev_list memoization
git-svn: save a little memory as fetch progresses
git-svn: remove unnecessary DESTROY override
git-svn: reload RA every log-window-size
git-svn.txt: advertise pushurl with dcommit
git-svn: remove mergeinfo rev caching
git-svn: cache only mergeinfo revisions
git-svn: reduce check_cherry_pick cache overhead
git-svn: only look at the root path for svn:mergeinfo
git-svn: only look at the new parts of svn:mergeinfo
* git://ozlabs.org/~paulus/gitk:
gitk: Remove boilerplate for configuration variables
gitk: Show detached HEAD if --all is specified
gitk: Do not depend on Cygwin's "kill" command on Windows
If HEAD is detached, 'gitk --all' does not show it. This is inconvenient
for frontend program, and for example git log does show the detached HEAD.
gitk uses git rev-parse to find a list of branches to show.
Apparently, the command does not include detached HEAD to output if
--all argument is specified. This has been discussed in [1] and stated
as expected behavior. So rev-parse's parameters should be tuned in gitk.
gitk: Do not depend on Cygwin's "kill" command on Windows
Windows does not necessarily mean Cygwin, it could also be MSYS. The
latter ships with a version of "kill" that does not understand "-f".
In msysgit this was addressed by shipping Cygwin's version of kill.
Properly fix this by using the stock Windows "taskkill" command instead,
which is available since Windows XP Professional.
Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com> Signed-off-by: Paul Mackerras <paulus@samba.org>
Memoizing these initialization functions saves some memory for
long fetches which require scanning many unwanted revisions
before any wanted revisions happen.
* dm/port2zos:
compat/bswap.h: detect endianness from XL C compiler macros
Makefile: reorder linker flags in the git executable rule
git-compat-util.h: support variadic macros with the XL C compiler
Tighten the logic to decide that an unreachable cruft is
sufficiently old by covering corner cases such as an ancient object
becoming reachable and then going unreachable again, in which case
its retention period should be prolonged.
* jk/prune-mtime: (28 commits)
drop add_object_array_with_mode
revision: remove definition of unused 'add_object' function
pack-objects: double-check options before discarding objects
repack: pack objects mentioned by the index
pack-objects: use argv_array
reachable: use revision machinery's --indexed-objects code
rev-list: add --indexed-objects option
rev-list: document --reflog option
t5516: test pushing a tag of an otherwise unreferenced blob
traverse_commit_list: support pending blobs/trees with paths
make add_object_array_with_context interface more sane
write_sha1_file: freshen existing objects
pack-objects: match prune logic for discarding objects
pack-objects: refactor unpack-unreachable expiration check
prune: keep objects reachable from recent objects
sha1_file: add for_each iterators for loose and packed objects
count-objects: use for_each_loose_file_in_objdir
count-objects: do not use xsize_t when counting object size
prune-packed: use for_each_loose_file_in_objdir
reachable: mark index blobs as SEEN
...
receive-pack: avoid minor leak in case start_async() fails
If the asynchronous start of copy_to_sideband() fails, then any
env_array entries added to struct child_process proc by
prepare_push_cert_sha1() are leaked. Call the latter function only
after start_async() succeeded so that the allocated entries are
cleaned up automatically by start_command() or finish_command().
Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Teach difftool to exit when a diff tool returns a non-zero exit
code when either --trust-exit-code is specified or
difftool.trustExitCode is true.
Forward exit codes from invoked diff tools to the caller when
--trust-exit-code is used.
Suggested-by: Adri Farr <14farresa@gmail.com> Helped-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: David Aguilar <davvid@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
grep: add color.grep.matchcontext and color.grep.matchselected
The config option color.grep.match can be used to specify the highlighting
color for matching strings. Add the options matchContext and matchSelected
to allow different colors to be specified for matching strings in the
context vs. in selected lines. This is similar to the ms and mc specifiers
in GNU grep's environment variable GREP_COLORS.
Tests are from Zoltan Klinger's earlier attempt to solve the same
issue in a different way.
Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
It took me a long time to notice the rider on the pack.threads
configuration option that it would multiple the memory consumption
by the number of CPUs in the machine. Clarify that the limit
applies per-thread.
Signed-off-by: Robert de Bath <rdebath@tvisiontech.co.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Asciidoctor provides an extension implementing a backend-independent
macro for dealing with manpage links just like the linkgit macro. As
this is more likely to be up-to-date with future changes in Asciidoctor,
prefer using it over reimplementing in Git.
Documentation: refactor common operations into variables
The Makefile performs several very similar tasks to convert AsciiDoc
files into either HTML or DocBook. Move these items into variables to
reduce the duplication.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
compat/bswap.h: detect endianness from XL C compiler macros
There is no /usr/include/endian.h equivalent on z/OS, but the
compiler will define macros to indicate endianness on host and
target hardware. This adds a test for these macros as a last
resort for determining byte order.
Signed-off-by: David Michael <fedora.dm0@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Makefile: reorder linker flags in the git executable rule
The XL C compiler can fail due to mixing library path and object
file arguments, for example when linking git while building with
"gmake LDFLAGS=-L$prefix/lib".
Move the ALL_LDFLAGS variable expansion in the git executable rule
to be consistent with all the other linking rules, namely to have
LDFLAGS such as -L$where before the object files *.o being linked
together.
Signed-off-by: David Michael <fedora.dm0@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
git-compat-util.h: support variadic macros with the XL C compiler
When the XL C compiler is run with an appropriate language level or
suboption, it defines a feature test macro to indicate support for
variadic macros by defining __C99_MACRO_WITH_VA_ARGS C preprocessor
macro.
This was tested on z/OS, but it should also work on AIX according
to IBM documentation.
Signed-off-by: David Michael <fedora.dm0@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
difftool--helper: exit when reading a prompt answer fails
An attempt to quit difftool by hitting Ctrl-D (EOF) at its prompt does
not quit it, but is treated as if 'yes' was answered to the prompt and
all following prompts, which is contrary to the user's intent. Fix the
error check.
Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
This memoization appears unneeded as the check_cherry_pick2 cache is
in front of it does enough.
With this change applied, importing from local svn+ssh and http copies
of the R repo[1] takes only 2:00 (2 hours) on my system and the git-svn
process never uses more than 60MB RSS on my x86-64 GNU/Linux system[2].
This 60M measurement is only for the git-svn Perl process itself and
does not include memory used by git subprocesses accessing large packs
(subprocess memory usage _is_ measured by my time(1) tool).
Before this change, an import took longer (2:20) on svn+ssh:// but
git-svn used around 240MB during the imports. Worse yet, git-svn
ballooned to over 400M when writing out the cache to the filesystem.
I also tried removing memoization for `has_no_changes', too, but a
local copy of the R repository(*) was not close to finishing within
10 hours on my system.
[1] http://svn.r-project.org/R
[2] file:// repos causes libsvn to use more memory internally
Signed-off-by: Eric Wong <normalperson@yhbt.net> Cc: Hin-Tak Leung <htl10@users.sourceforge.net>
Despite attempting to use local memory pools everywhere we can,
(including our call to SVN::Ra::do_update and all subsequent reporter
calls), there does not appear to be a way to force the Git::SVN::Fetcher
callbacks to use a pool other than the per-SVN::Ra pool.
Git::SVN::Fetcher ends up using the main RA pool which grows
monotonically in size for the lifetime of the RA object.
Thus the only way to free that memory appears to be to destroy and
recreate the RA connection for at every --log-window-size interval.
This reduces memory usage over the course of fetching 10K revisions
using a test repository created with the script at the end of this
commit message.
As reported by time(1) on my x86-64 system:
before: 54024k
after: 28680k
Unfortunately, there remains some yet-to-be-tracked-down slow memory
growth which would be evident as the `nr' parameter increases in
the repository generation script:
-----------------------------8<------------------------------
set -e
tmp=$(mktemp -d svntestrepo-XXXXXXXX)
svnadmin create "$tmp"
repo=file://"$(cd $tmp && pwd)"
svn co "$repo" "$tmp/wd"
cd "$tmp/wd"
if ! test -f a
then
> a
svn add a
svn commit -m 'A'
fi
nr=10000
while test $nr -gt 0
do
echo $nr > a
svn commit -q -m A
nr=$((nr - 1))
done
echo "repository created in $repo"
-----------------------------8<------------------------------
Advertise that the svn-remote.<name>.pushurl config key allows specifying
the commit URL for the entire SVN repository in the documentation of the
git svn dcommit command.
Signed-off-by: Sveinung Kvilhaugsvik <sveinung84@users.sourceforge.net> Signed-off-by: Eric Wong <normalperson@yhbt.net>
This should further reduce memory usage from the new mergeinfo
speedups without hurting performance too much, assuming
reasonable latency to the SVN server.
Cc: Hin-Tak Leung <htl10@users.sourceforge.net> Suggested-by: Jakob Stoklund Olesen <stoklund@2pi.dk> Signed-off-by: Eric Wong <normalperson@yhbt.net>
This should reduce excessive memory usage from the new mergeinfo
caches without hurting performance too much, assuming reasonable
latency to the SVN server.
Cc: Hin-Tak Leung <htl10@users.sourceforge.net> Suggested-by: Jakob Stoklund Olesen <stoklund@2pi.dk> Signed-off-by: Eric Wong <normalperson@yhbt.net>
We do not need to store entire lists of commits, only the
number of incomplete and the first commit for reference.
This reduces the amount of data we need to store in memory
and on disk stores.
git-svn: only look at the root path for svn:mergeinfo
Subversion can put mergeinfo on any sub-directory to track cherry-picks.
Since cherry-picks are not represented explicitly in git, git-svn should
just ignore it.
Signed-off-by: Jakob Stoklund Olesen <stoklund@2pi.dk> Signed-off-by: Eric Wong <normalperson@yhbt.net>
git-svn: only look at the new parts of svn:mergeinfo
In a Subversion repository where many feature branches are merged into a
trunk, the svn:mergeinfo property can grow very large. This severely
slows down git-svn's make_log_entry() because it is checking all
mergeinfo entries every time the property changes.
In most cases, the additions to svn:mergeinfo since the last commit are
pretty small, and there is nothing to gain by checking merges that were
already checked for the last commit in the branch.
Add a mergeinfo_changes() function which computes the set of interesting
changes to svn:mergeinfo since the last commit. Filter out merged
branches whose ranges haven't changed, and remove a common prefix of
ranges from other merged branches.
This speeds up "git svn fetch" by several orders of magnitude on a large
repository where thousands of feature branches have been merged.
Signed-off-by: Jakob Stoklund Olesen <stoklund@2pi.dk> Signed-off-by: Eric Wong <normalperson@yhbt.net>
Allow us build with NO_PTHREADS=NoThanks compilation option.
* eb/no-pthreads:
Handle atexit list internaly for unthreaded builds
pack-objects: set number of threads before checking and warning
index-pack: fix compilation with NO_PTHREADS
Allow a temporary directory specified to be used while running "git
mergetool" backend.
* da/mergetool-temporary-directory:
t7610-mergetool: add test cases for mergetool.writeToTemp
mergetool: add an option for writing to a temporary directory
Allow "git mergetool --help" to run outside a Git repository.
* da/mergetool-tool-help:
difftool: don't assume that default sh is sane
mergetool: don't require a work tree for --tool-help
git-sh-setup: move GIT_DIR initialization into a function
mergetool: use more conservative temporary filenames
test-lib-functions: adjust style to match CodingGuidelines
t7610-mergetool: prefer test_config over git config
Tweak the names of the three throw-away files "git mergetool" comes
up with to feed the merge tool backend, so that a file with a
single dot in its name in the original (e.g. "hello.c") will have
only one dot in these variants (e.g. "hello_BASE_4321.c").
* da/mergetool-temporary-filename:
mergetool: use more conservative temporary filenames
The clean-up of this test script was long overdue and is a very
welcome change.
* da/mergetool-tests:
test-lib-functions: adjust style to match CodingGuidelines
t7610-mergetool: use test_config to isolate tests
t7610-mergetool: add missing && and remove commented-out code
t7610-mergetool: use tabs instead of a mix of tabs and spaces
The API to update refs have been restructured to allow introducing
a true transactional updates later. We would even allow storing
refs in backends other than the traditional filesystem-based one.
* rs/ref-transaction: (25 commits)
ref_transaction_commit: bail out on failure to remove a ref
lockfile: remove unable_to_lock_error
refs.c: do not permit err == NULL
remote rm/prune: print a message when writing packed-refs fails
for-each-ref: skip and warn about broken ref names
refs.c: allow listing and deleting badly named refs
test: put tests for handling of bad ref names in one place
packed-ref cache: forbid dot-components in refnames
branch -d: simplify by using RESOLVE_REF_READING
branch -d: avoid repeated symref resolution
reflog test: test interaction with detached HEAD
refs.c: change resolve_ref_unsafe reading argument to be a flags field
refs.c: make write_ref_sha1 static
fetch.c: change s_update_ref to use a ref transaction
refs.c: ref_transaction_commit: distinguish name conflicts from other errors
refs.c: pass a list of names to skip to is_refname_available
refs.c: call lock_ref_sha1_basic directly from commit
refs.c: refuse to lock badly named refs in lock_ref_sha1_basic
rename_ref: don't ask read_ref_full where the ref came from
refs.c: pass the ref log message to _create/delete/update instead of _commit
...
Beyond Compare version 4 works the same way as version 3, so rename
the existing "bc3" adaptor to just "bc", while keeping "bc3" as a
backward compatible wrapper.
Noticed-by: Olivier Croquette <ocroquette@free.fr> Helped-by: David Aguilar <davvid@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
.mailmap: add Stefan Bellers corporate mail address
Note that despite the private address being first and primary,
Google owns the copyright on this patch as any other patch I'll be
sending signed off by the sbeller@google.com address.
Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
A new filter to programatically edit the tail end of the commit log
messages.
* cc/interpret-trailers:
Documentation: add documentation for 'git interpret-trailers'
trailer: add tests for commands in config file
trailer: execute command from 'trailer.<name>.command'
trailer: add tests for "git interpret-trailers"
trailer: add interpret-trailers command
trailer: put all the processing together and print
trailer: parse trailers from file or stdin
trailer: process command line trailer arguments
trailer: read and process config information
trailer: process trailers from input message and arguments
trailer: add data structures and basic functions
Test scripts were taught to notice "-x" option to show shell trace,
as if the tests were run under "sh -x".
* jk/test-shell-trace:
test-lib.sh: support -x option for shell-tracing
t5304: use helper to report failure of "test foo = bar"
t5304: use test_path_is_* instead of "test -f"
Revert "archive: honor tar.umask even for pax headers"
This reverts commit 10f343ea814f5c18a0913997904ee11cd9b7da24, whose
output is no longer bit-for-bit equivalent from the older versions
of Git, which the infrastructure to (pretend to) upload tarballs
kernel.org uses depends on.
core.filemode is set automatically when a repo is created.
But when a repo is exported via CIFS or cygwin is mixed with Git for Windows
or Eclipse core.filemode may better be set manually to false.
Update and improve the documentation
Helped-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
# ./t1304-default-acl.sh
ok 1 - checking for a working acl setup
ok 2 - Setup test repo
not ok 3 - Objects creation does not break ACLs with restrictive umask
#
# # SHA1 for empty blob
# check_perms_and_acl .git/objects/e6/9de29bb2d1d6434b8b29ae775ad8c2e48c5391
#
not ok 4 - git gc does not break ACLs with restrictive umask
#
# git gc &&
# check_perms_and_acl .git/objects/pack/*.pack
#
# failed 2 among 4 test(s)
1..4
on systems where USER isn't set. It's usually set by the login
process, but it isn't set when launching some Docker images. For
example:
$ docker run --rm debian env
HOME=/
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=b2dfdfe797ed
'id -u -n' has been in POSIX from Issue 2 through 2013 [1], so I don't
expect compatibility issues.
Handle atexit list internaly for unthreaded builds
Wrap atexit()s calls on unthreaded builds to handle callback list
internally.
This is needed because on unthreaded builds, asyncs inherits parent's
atexit() list, that gets run as soon as the async exit()s (and again at
the end of async's parent process). That led to remove temporary files
too early.
Also remove a by-atexit-callback guard against this kind of issue in
clone.c, as this patch makes it redundant.
Fixes test 5537 (temporary shallow file vanished before unpack-objects
could open it)
BTW remove an unused variable in shallow.c.
Helped-by: Duy Nguyen <pclouds@gmail.com> Helped-by: Andreas Schwab <schwab@linux-m68k.org> Helped-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Etienne Buira <etienne.buira@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
This is a thin compatibility wrapper around
add_pending_object_with_path. But the only caller is
add_object_array, which is itself just a thin compatibility
wrapper. There are no external callers, so we can just
remove this middle wrapper.
Noticed-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Convert users of struct child_process to using the managed env_array for
specifying environment variables instead of supplying an array on the
stack or bringing their own argv_array. This shortens and simplifies
the code and ensures automatically that the allocated memory is freed
after use.
Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
run-command: add env_array, an optional argv_array for env
Similar to args, add a struct argv_array member to struct child_process
that simplifies specifying the environment for children. It is freed
automatically by finish_command() or if start_command() encounters an
error.
Suggested-by: Jeff King <peff@peff.net> Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
pack-objects: turn off bitmaps when we split packs
If a pack.packSizeLimit is set, we may split the pack data
across multiple packfiles. This means we cannot generate
.bitmap files, as they require that all of the reachable
objects are in the same pack. We check that condition when
we are generating the list of objects to pack (and disable
bitmaps if we are not packing everything), but we forgot to
update it when we notice that we needed to split (which
doesn't happen until the actual write phase).
The resulting bitmaps are quite bogus (they mention entries
that do not exist in the pack!) and can cause a fetch or
push to send insufficient objects.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
pack-objects: double-check options before discarding objects
When we are given an expiration time like
--unpack-unreachable=2.weeks.ago, we avoid writing out old,
unreachable loose objects entirely, under the assumption
that running "prune" would simply delete them immediately
anyway. However, this is only valid if we computed the same
set of reachable objects as prune would.
In practice, this is the case, because only git-repack uses
the --unpack-unreachable option with an expiration, and it
always feeds as many objects into the pack as possible. But
we can double-check at runtime just to be sure.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
When we pack all objects, we use only the objects reachable
from references and reflogs. This misses any objects which
are reachable from the index, but not yet referenced.
By itself this isn't a big deal; the objects can remain
loose until they are actually used in a commit. However, it
does create a problem when we drop packed but unreachable
objects. We try to optimize out the writing of objects that
we will immediately prune, which means we must follow the
same rules as prune in determining what is reachable. And
prune uses the index for this purpose.
This is rather uncommon in practice, as objects in the index
would not usually have been packed in the first place. But
it could happen in a sequence like:
1. You make a commit on a branch that references blob X.
2. You repack, moving X into the pack.
3. You delete the branch (and its reflog), so that X is
unreferenced.
4. You "git add" blob X so that it is now referenced only
by the index.
5. You repack again with git-gc. The pack-objects we
invoke will see that X is neither referenced nor
recent and not bother loosening it.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
There is currently no easy way to ask the revision traversal
machinery to include objects reachable from the index (e.g.,
blobs and trees that have not yet been committed). This
patch adds an option to do so.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
t5516: test pushing a tag of an otherwise unreferenced blob
It's not unreasonable to have a tag that points to a blob
that is not part of the normal history. We do this in
git.git to distribute gpg keys. However, we never explicitly
checked in our test suite that this actually works (i.e.,
that pack-objects actually sends the blob because of the tag
mentioning it).
It does in fact work fine, but a recent patch under
discussion broke this, and the test suite didn't notice.
Let's make the test suite more complete.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>