The submodule code recently grew generic code to build a
dynamic argv array. Many other parts of the code can reuse
this, too, so let's make it generically available.
There are two enhancements not found in the original code:
1. We now handle the NULL-termination invariant properly,
even when no strings have been pushed (before, you
could have an empty, NULL argv). This was not a problem
for the submodule code, which always pushed at least
one argument, but was not sufficiently safe for
generic code.
2. There is a formatted variant of the "push" function.
This is a convenience function which was not needed by
the submodule code, but will make it easier to port
other users to the new code.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Commit 758e915 made sq_quote_next static, removing it from
quote.h. However, it forgot to update the related comment,
making it appear as a confusing description of sq_quote_to_argv.
Let's remove the crufty bits, and elaborate more on sq_quote_to_argv.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
This patch uses test_path_is_file and test_path_is_missing
instead of "test -f / ! test -f" checks. The former are more
verbose in case of failure and more precise (e.g., is_missing
will check that the entry is actually missing, not just not
a regular file).
As a bonus, this also fixes a few buggy tests that used
"test foo" instead of "test -f foo", and consequently always
reported success.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Allow git-svn to populate the svn:mergeinfo property automatically in
a narrow range of circumstances. Specifically, when dcommitting a
revision with multiple parents, all but (potentially) the first of
which have been committed to SVN in the same repository as the target
of the dcommit.
In this case, the merge info is the union of that given by each of the
parents, plus all changes introduced to the first parent by the other
parents.
In all other cases where a revision to be committed has multiple
parents, cause "git svn dcommit" to raise an error rather than
completing the commit and potentially losing history information in
the upstream SVN repository.
This behavior is disabled by default, and can be enabled by setting
the svn.pushmergeinfo config option.
[ew: minor style changes and manpage merge fix]
Acked-by: Eric Wong <normalperson@yhbt.net> Signed-off-by: Bryan Jacobs <bjacobs@woti.com>
Running a hook has to make complex set-up to establish web of
communication between child process and multiplexer, which is common
regardless of what kind of data is fed to the hook. Refactor the parts
that is specific to the data fed to the particular set of hooks from the
part that runs the hook, so that the code can be reused to drive hooks
that take different kind of data.
fetch: avoid quadratic loop checking for updated submodules
Recent versions of git can be slow to fetch repositories with a
large number of refs (or when they already have a large
number of refs). For example, GitHub makes pull-requests
available as refs, which can lead to a large number of
available refs. This slowness goes away when submodule
recursion is turned off:
[this takes ~10 seconds of CPU time to complete]
git fetch --recurse-submodules=no \
git://github.com/rails/rails.git "refs/*:refs/*"
[this still isn't done after 10 _minutes_ of pegging the CPU]
git fetch \
git://github.com/rails/rails.git "refs/*:refs/*"
You can produce a quicker and simpler test case like this:
doit() {
head=`git rev-parse HEAD`
for i in `seq 1 $1`; do
echo $head refs/heads/ref$i
done >.git/packed-refs
echo "==> $1"
rm -rf dest
git init -q --bare dest &&
(cd dest && time git.compile fetch -q .. refs/*:refs/*)
}
doit 100
doit 200
doit 400
doit 800
doit 1600
doit 3200
Which yields timings like:
# refs seconds of CPU
100 0.06
200 0.24
400 0.95
800 3.39
1600 13.66
3200 54.09
Notice that although the number of refs doubles in each
trial, the CPU time spent quadruples.
The problem is that the submodule recursion code works
something like:
- for each ref we fetch
- for each commit in git rev-list $new_sha1 --not --all
- add modified submodules to list
- fetch any newly referenced submodules
But that means if we fetch N refs, we start N revision
walks. Worse, because we use "--all", the number of refs we
must process that constitute "--all" keeps growing, too. And
you end up doing O(N^2) ref resolutions.
Instead, this patch structures the code like this:
- for each sha1 we already have
- add $old_sha1 to list $old
- for each ref we fetch
- add $new_sha1 to list $new
- for each commit in git rev-list $new --not $old
- add modified submodules to list
- fetch any newly referenced submodules
This yields timings like:
# refs seconds of CPU
100 0.00
200 0.04
400 0.04
800 0.10
1600 0.21
3200 0.39
Note that the amount of effort doubles as the number of refs
doubles. Similarly, the fetch of rails.git takes about as
much time as it does with --recurse-submodules=no.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
commit c9bfb953 (want_color: automatically fallback to color.ui,
2011-08-17) introduced a regression where format-patch produces colorized
patches when color.ui is set to "always".
In f3aafa4 (Disable color detection during format-patch, 2006-07-09),
git_format_config was taught to intercept diff.color to avoid passing it
down to git_log_config and later, git_diff_ui_config.
Teach git_format_config to intercept color.ui in the same way.
Helped-by: Jeff King <peff@peff.net> Signed-off-by: Pang Yan Han <pangyanhan@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* jn/remote-helpers-doc:
(short) documentation for the testgit remote helper
Documentation/git-remote-helpers: explain how import works with multiple refs
Documentation/remote-helpers: explain capabilities first
* jk/maint-config-param:
config: use strbuf_split_str instead of a temporary strbuf
strbuf: allow strbuf_split to work on non-strbufs
config: avoid segfault when parsing command-line config
config: die on error in command-line config
fix "git -c" parsing of values with equals signs
strbuf_split: add a max parameter
we would not update the fetch refspec and even if there is a ref
called "refs/remotes/origin/master", we should not rename it, since it
was not created by fetching from the remote.
Suggested-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Martin von Zweigbergk <martin.von.zweigbergk@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
When renaming a remote, we also try to update the fetch refspec
accordingly, but only if it has the default format. For others, such
as refs/heads/master:refs/heads/origin, we are conservative and leave
it untouched. Let's give the user a warning about refspecs that are
not updated, so he can manually update the config if necessary.
Suggested-by: Jeff King <peff@peff.net> Signed-off-by: Martin von Zweigbergk <martin.von.zweigbergk@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
remote: "rename o foo" should not rename ref "origin/bar"
When renaming a remote called 'o' using 'git remote rename o foo', git
should also rename any remote-tracking branches for the remote. This
does happen, but any remote-tracking branches starting with
'refs/remotes/o', such as 'refs/remotes/origin/bar', will also be
renamed (to 'refs/remotes/foorigin/bar' in this case).
Fix it by simply matching one more character, up to the slash
following the remote name.
Signed-off-by: Martin von Zweigbergk <martin.von.zweigbergk@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
remote: write correct fetch spec when renaming remote 'remote'
When renaming a remote whose name is contained in a configured fetch
refspec for that remote, we currently replace the first occurrence of
the remote name in the refspec. This is correct in most cases, but
breaks if the remote name occurs in the fetch refspec before the
expected place. For example, we currently change
Removing Cogito leaves just git and StGit, which is a rather
incomplete list of git diff tools available. Sidestep the problem
of deciding what tools to mention by not mentioning any.
Signed-off-by: Sverre Rabbelier <srabbelier@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Asking fwrite() to write one item of size bytes results in fwrite()
reporting "I wrote zero item", when size is zero. Instead, we could
ask it to write "size" items of 1 byte and expect it to report that
"I wrote size items" when it succeeds, with any value of size,
including zero.
In particular, sparse issues the following warnings:
compat/obstack.c:176:17: warning: Using plain integer as NULL pointer
compat/obstack.c:224:17: warning: Using plain integer as NULL pointer
compat/obstack.c:324:16: warning: Using plain integer as NULL pointer
compat/obstack.c:329:16: warning: Using plain integer as NULL pointer
compat/obstack.c:347:16: warning: Using plain integer as NULL pointer
compat/obstack.c:362:19: warning: Using plain integer as NULL pointer
compat/obstack.c:379:29: warning: Using plain integer as NULL pointer
compat/obstack.c:399:1: error: symbol 'print_and_abort' redeclared with \
different type (originally declared at compat/obstack.c:95) \
- different modifiers
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
check_everything_connected(): refactor to use an iterator
We will be using the same "rev-list --verify-objects" logic to add a
sanity check to the receiving end of "git push" in the same way, but the
list of commits that are checked come from a structure with a different
shape over there.
Update the function to take an iterator to make it easier to reuse it in
different contexts.
fetch: verify we have everything we need before updating our ref
The "git fetch" command works in two phases. The remote side tells us what
objects are at the tip of the refs we are fetching from, and transfers the
objects missing from our side. After storing the objects in our repository,
we update our remote tracking branches to point at the updated tips of the
refs.
A broken or malicious remote side could send a perfectly well-formed pack
data during the object transfer phase, but there is no guarantee that the
given data actually fill the gap between the objects we originally had and
the refs we are updating to.
Although this kind of breakage can be caught by running fsck after a
fetch, it is much cheaper to verify that everything that is reachable from
the tips of the refs we fetched are indeed fully connected to the tips of
our current set of refs before we update them.
fetch: skip on-demand checking when no submodules are configured
It makes no sense to do the - possibly very expensive - call to "rev-list
<new-ref-sha1> --not --all" in check_for_new_submodule_commits() when
there aren't any submodules configured.
Leave check_for_new_submodule_commits() early when no name <-> path
mappings for submodules are found in the configuration. To make that work
reading the configuration had to be moved further up in cmd_fetch(), as
doing that after the actual fetch of the superproject was too late.
Reported-by: Martin Fick <mfick@codeaurora.org> Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
"branch -v" without other options or parameters still works in the list
mode, but that is not because there is "-v" but because there is no
parameter nor option.
Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
for-each-ref: add split message parts to %(contents:*).
The %(body) placeholder returns the whole body of a tag or
commit, including the signature. However, callers may want
to get just the body without signature, or just the
signature.
Rather than change the meaning of %(body), which might break
some scripts, this patch introduces a new set of
placeholders which break down the %(contents) placeholder
into its constituent parts.
[jk: initial patch by mg, rebased on top of my refactoring
and with tests by me]
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
for-each-ref: handle multiline subjects like --pretty
Generally the format of a git tag or commit message is:
subject
body body body
body body body
However, we occasionally see multiline subjects like:
subject
with multiple
lines
body body body
body body body
The rest of git treats these multiline subjects as something
to be concatenated and shown as a single line (e.g., "git
log --pretty=format:%s" will do so since f53bd74). For
consistency, for-each-ref should do the same with its
"%(subject)".
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
for-each-ref: refactor subject and body placeholder parsing
The find_subpos function was a little hard to use, as well
as to read. It would sometimes write into the subject and
body pointers, and sometimes not. The body pointer sometimes
could be compared to subject, and sometimes not. When
actually duplicating the subject, the caller was forced to
figure out again how long the subject is (which is not too
big a deal when the subject is a single line, but hard to
extend).
The refactoring makes the function more straightforward, both
to read and to use. We will always put something into the
subject and body pointers, and we return explicit lengths
for them, too.
This lays the groundwork both for more complex subject
parsing (e.g., multiline), as well as splitting the body
into subparts (like the text versus the signature).
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Other test scripts may want to look at or verify signed
tags, and the setup is non-trivial. Let's factor this out
into lib-gpg.sh for other tests to use.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
sha1_file: normalize alt_odb path before comparing and storing
When it needs to compare and add an alt object path to the
alt_odb_list, we normalize this path first since comparing normalized
path is easy to get correct result.
Use strbuf to replace some string operations, since it is cleaner and
safer.
Helped-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Hui Wang <Hui.Wang@windriver.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
A few more parts of this document is stale that needs updating
to reflect the reality, but I do not regularly rebase topics that
are only in "pu" anymore, which may be noteworthy for a commit.
There is no need for a blank line between the detailed error message
and the later "fatal: HTTP request failed" notice. Keep the newline
written by error() itself and eliminate the extra one.
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Passing a prefix to a function that is supposed to find the prefix is
strange. And it's really only used if the pathspec is NULL. Make the
callers handle this case instead.
As we are always returning a fresh copy of a string (or NULL), change the
type of the returned value to non-const "char *".
Signed-off-by: Clemens Buchacher <drizzd@aon.at> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Adding a new command line option to receive-pack and feed it from
send-pack is not an acceptable way to add features, as there is no
guarantee that your updated send-pack will be talking to updated
receive-pack. New features need to be added via the capability mechanism
negotiated over the protocol.
This corresponds to receive.fsckobjects configuration variable added (a
lot) earlier in 20dc001 (receive-pack: allow using --strict mode for
unpacking objects, 2008-02-25).
* tr/maint-format-patch-empty-output:
Document negated forms of format-patch --to --cc --add-headers
t4014: "no-add-headers" is actually called "no-add-header"
t4014: invoke format-patch with --stdout where intended
t4014: check for empty files from git format-patch --stdout
* fk/use-kwset-pickaxe-grep-f:
obstack: Fix portability issues
Use kwset in grep
Use kwset in pickaxe
Adapt the kwset code to Git
Add string search routines from GNU grep
Add obstack.[ch] from EGLIBC 2.10
* en/merge-recursive-2: (57 commits)
merge-recursive: Don't re-sort a list whose order we depend upon
merge-recursive: Fix virtual merge base for rename/rename(1to2)/add-dest
t6036: criss-cross + rename/rename(1to2)/add-dest + simple modify
merge-recursive: Avoid unnecessary file rewrites
t6022: Additional tests checking for unnecessary updates of files
merge-recursive: Fix spurious 'refusing to lose untracked file...' messages
t6022: Add testcase for spurious "refusing to lose untracked" messages
t3030: fix accidental success in symlink rename
merge-recursive: Fix working copy handling for rename/rename/add/add
merge-recursive: add handling for rename/rename/add-dest/add-dest
merge-recursive: Have conflict_rename_delete reuse modify/delete code
merge-recursive: Make modify/delete handling code reusable
merge-recursive: Consider modifications in rename/rename(2to1) conflicts
merge-recursive: Create function for merging with branchname:file markers
merge-recursive: Record more data needed for merging with dual renames
merge-recursive: Defer rename/rename(2to1) handling until process_entry
merge-recursive: Small cleanups for conflict_rename_rename_1to2
merge-recursive: Fix rename/rename(1to2) resolution for virtual merge base
merge-recursive: Introduce a merge_file convenience function
merge-recursive: Fix modify/delete resolution in the recursive case
...
git-remote-mediawiki: allow push to set MediaWiki metadata
Push can not set the commit note "mediawiki_revision:" and update the
remote reference. This avoids having to "git pull --rebase" after each
push, and is probably more natural. Make it the default, but let it be
configurable with mediawiki.dumbPush or remote.<remotename>.dumbPush.
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add a remote helper to interact with mediawiki (fetch & push)
Implement a gate between git and mediawiki, allowing git users to push
and pull objects from mediawiki just as one would do with a classic git
repository thanks to remote-helpers.
The following packages need to be installed (available on common
repositories):
Use remote helpers in order to be as transparent as possible to the git
user.
Download Mediawiki revisions through the Mediawiki API and then
fast-import into git.
Mediawiki revision number and git commits are linked thanks to notes
bound to commits.
The import part is done on a refs/mediawiki/<remote> branch before
coming to refs/remote/origin/master (Huge thanks to Jonathan Nieder
for his help)
We use UTF-8 everywhere: use encoding 'utf8'; does most of the job, but
we also read the output of Git commands in UTF-8 with the small helper
run_git, and write to the console (STDERR) in UTF-8. This allows a
seamless use of non-ascii characters in page titles, but hasn't been
tested on non-UTF-8 systems. In particular, UTF-8 encoding for filenames
could raise problems if different file systems handle UTF-8 filenames
differently. A uri_escape of mediawiki filenames could be imaginable, and
is still to be discussed further.
Thanks to notes metadata, it is possible to compare remote and local last
mediawiki revision to warn non-fast forward pushes and "everything
up-to-date" case.
When allowed, push looks for each commit between remotes/origin/master
and HEAD, catches every blob related to these commit and push them in
chronological order. To do so, it uses git rev-list --children HEAD and
travels the tree from remotes/origin/master to HEAD through children. In
other words:
* Shortest path from remotes/origin/master to HEAD
* For each commit encountered, push blobs related to this commit
Signed-off-by: Jérémie Nikaes <jeremie.nikaes@ensimag.imag.fr> Signed-off-by: Arnaud Lacurie <arnaud.lacurie@ensimag.imag.fr> Signed-off-by: Claire Fousse <claire.fousse@ensimag.imag.fr> Signed-off-by: David Amouyal <david.amouyal@ensimag.imag.fr> Signed-off-by: Matthieu Moy <matthieu.moy@grenoble-inp.fr> Signed-off-by: Sylvain Boulmé <sylvain.boulme@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
(short) documentation for the testgit remote helper
While it's not a command meant to be used by actual users (hence, not
mentionned in git(1)), this command is a very precious help for
remote-helpers authors.
The best place for such technical doc is the source code, but users may
not find it without a link in a manpage.
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr> Acked-by: Sverre Rabbelier <srabbelier@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
fetch: verify we have everything we need before updating our ref
The "git fetch" command works in two phases. The remote side tells us what
objects are at the tip of the refs we are fetching from, and transfers the
objects missing from our side. After storing the objects in our repository,
we update our remote tracking branches to point at the updated tips of the
refs.
A broken or malicious remote side could send a perfectly well-formed pack
data during the object transfer phase, but there is no guarantee that the
given data actually fill the gap between the objects we originally had and
the refs we are updating to.
Although this kind of breakage can be caught by running fsck after a
fetch, it is much cheaper to verify that everything that is reachable from
the tips of the refs we fetched are indeed fully connected to the tips of
our current set of refs before we update them.
Often we want to verify everything reachable from a given set of commits
are present in our repository and connected without a gap to the tips of
our refs. We used to do this for this purpose:
Even though this is good enough for catching missing commits and trees,
we show the object name but do not verify their existence, let alone their
well-formedness, for the blob objects at the leaf level.
Add a new "--verify-object" option so that we can catch missing and broken
blobs as well.
list-objects: pass callback data to show_objects()
The traverse_commit_list() API takes two callback functions, one to show
commit objects, and the other to show other kinds of objects. Even though
the former has a callback data parameter, so that the callback does not
have to rely on global state, the latter does not.
Give the show_objects() callback the same callback data parameter.
git-svn: Teach dcommit --mergeinfo to handle multiple lines
"svn dcommit --mergeinfo" replaces the svn:mergeinfo property in an
upstream SVN repository with the given text. The svn:mergeinfo
property may contain commits originating on multiple branches,
separated by newlines.
Cause space characters in the mergeinfo to be replaced by newlines,
allowing a user to create history representing multiple branches being
merged into one.
Update the corresponding documentation and add a test for the new
functionality.
Signed-off-by: Bryan Jacobs <bjacobs@woti.com> Acked-by: Sam Vilain <sam@vilain.net> Acked-by: Eric Wong <normalperson@yhbt.net>
Adds a --preserve-empty-dirs flag to the clone operation that will detect
empty directories in the remote Subversion repository and create placeholder
files in the corresponding local Git directories. This allows "empty"
directories to exist in the history of a Git repository.
Also adds the --placeholder-file flag to control the name of any placeholder
files created. Default value is ".gitignore".
Signed-off-by: Ray Chen <rchen@cs.umd.edu> Acked-by: Eric Wong <normalperson@yhbt.net>
xdiff/xprepare: initialise xdlclassifier_t cf in xdl_prepare_env()
Ensure that the xdl_free_classifier() call on xdlclassifier_t cf is safe
even if xdl_init_classifier() isn't called. This may occur in the case
where diff is run with --histogram and a call to, say, xdl_prepare_ctx()
fails.
Signed-off-by: Tay Ray Chuan <rctay89@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
t6030: use $SHELL_PATH to invoke user's preferred shell instead of bare sh
Some platforms (IRIX, Solaris) provide an ancient /bin/sh which chokes on
modern shell syntax like $(). SHELL_PATH is provided to allow the user to
specify a working sh, let's use it here.
Signed-off-by: Brandon Casey <casey@nrlssc.navy.mil> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Symlink mergetools scriptlets into valgrind wrappers
Since bc7a96a (mergetool--lib: Refactor tools into separate files,
2011-08-18) the mergetools and difftools related tests fail under
--valgrind because the mergetools/* scriptlets are not in the exec
path.
For now, symlink the mergetools subdir into the t/valgrind/bin
directory as a whole, since it does not contain anything of interest
to the valgrind wrappers.
Signed-off-by: Thomas Rast <trast@student.ethz.ch> Acked-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
When running large git grep (ie: git grep regexp $(git rev-list --all)), glibc error sometimes occur:
*** glibc detected *** git: double free or corruption (!prev): 0x00000000010abdf0 ***
According to gdb the problem originate from release_delta_cash (sha1_file.c:1703)
free(ent->data);
>From my analysis it seems that git grep threads do acquire lock before calling read_sha1_file but not before calling
read_object_with_reference who ends up calling read_sha1_file too.
Adding the lock around read_object_with_reference seems to fix the issue for me.
I've ran git grep about a dozen time and seen no more error while
it usually happened half the time before.
Signed-off-by: Nicolas Morey-Chaisemartin <nicolas@morey-chaisemartin.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>