gitweb.git
push, fetch: error out for submodule entries not pointi... Stefan Beller Tue, 12 Sep 2017 17:30:27 +0000 (10:30 -0700)

push, fetch: error out for submodule entries not pointing to commits

The check_has_commit helper uses resolves a submodule entry to a
commit, when validating its existence. As a side effect this means
tolerates a submodule entry pointing to a tag, which is not a valid
submodule entry that git commands would know how to cope with.

Tighten the check to require an actual commit, not a tag pointing to a
commit.

Also improve the error handling when a submodule entry points to
non-commit (e.g., a blob) to error out instead of warning and
pretending the pointed to object doesn't exist.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

pack: make packed_git_mru global a value instead of... Jonathan Nieder Tue, 12 Sep 2017 17:28:39 +0000 (10:28 -0700)

pack: make packed_git_mru global a value instead of a pointer

The MRU cache that keeps track of recently used packs is represented
using two global variables:

struct mru packed_git_mru_storage;
struct mru *packed_git_mru = &packed_git_mru_storage;

Callers never assign to the packed_git_mru pointer, though, so we can
simplify by eliminating it and using &packed_git_mru_storage (renamed
to &packed_git_mru) directly. This variable is only used by the
packfile subsystem, making this a relatively uninvasive change (and
any new unadapted callers would trigger a compile error).

Noticed while moving these globals to the object_store struct.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Acked-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

help: change a message to be more preciseKaartic Sivaraam Tue, 12 Sep 2017 11:14:06 +0000 (16:44 +0530)

help: change a message to be more precise

When the user tries to use '--help' option on an aliased command
information about the alias is printed as sshown below,

$ git co --help
`git co' is aliased to `checkout'

This doesn't seem correct as the user has aliased only 'co' and not
'git co'. This might even be incorrect in cases in which the user has
used an alias like 'tgit'.

$ tgit co --help
`git co' is aliased to `checkout'

So, make the message more precise.

Signed-off-by: Kaartic Sivaraam <kaarticsivaraam91196@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

refs: strip out not allowed flags from ref_transaction_... Thomas Gummerer Tue, 12 Sep 2017 22:59:21 +0000 (23:59 +0100)

refs: strip out not allowed flags from ref_transaction_update

Callers are only allowed to pass certain flags into
ref_transaction_update, other flags are internal to it. To prevent
mistakes from the callers, strip the internal only flags out before
continuing.

This was noticed because of a compiler warning gcc 7.1.1 issued about
passing a NULL parameter as second parameter to memcpy (through
hashcpy):

In file included from refs.c:5:0:
refs.c: In function ‘ref_transaction_verify’:
cache.h:948:2: error: argument 2 null where non-null expected [-Werror=nonnull]
memcpy(sha_dst, sha_src, GIT_SHA1_RAWSZ);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from git-compat-util.h:165:0,
from cache.h:4,
from refs.c:5:
/usr/include/string.h:43:14: note: in a call to function ‘memcpy’ declared here
extern void *memcpy (void *__restrict __dest, const void *__restrict __src,
^~~~~~

The call to hascpy in ref_transaction_add_update is protected by the
passed in flags, but as we only add flags there, gcc notices
REF_HAVE_NEW or REF_HAVE_OLD flags could be passed in from the outside,
which would potentially result in passing in NULL as second parameter to
memcpy.

Fix both the compiler warning, and make the interface safer for its
users by stripping the internal flags out.

Suggested-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

doc/for-each-ref: explicitly specify option namesKevin Daudt Mon, 11 Sep 2017 19:33:38 +0000 (21:33 +0200)

doc/for-each-ref: explicitly specify option names

For count, sort and format, only the argument names were listed under
OPTIONS, not the option names.

Add the option names to make it clear the options exist

Signed-off-by: Kevin Daudt <me@ikke.info>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

doc/for-each-ref: consistently use '=' to between argum... Kevin Daudt Mon, 11 Sep 2017 19:33:37 +0000 (21:33 +0200)

doc/for-each-ref: consistently use '=' to between argument names and values

The synopsis and description inconsistently add a '=' between the
argument name and it's value. Make this consistent.

Signed-off-by: Kevin Daudt <me@ikke.info>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

cvsimport: shell-quote variable used in backticksJeff King Mon, 11 Sep 2017 14:24:26 +0000 (10:24 -0400)

cvsimport: shell-quote variable used in backticks

We run `git rev-parse` though the shell, and quote its
argument only with single-quotes. This prevents most
metacharacters from being a problem, but misses the obvious
case when $name itself has single-quotes in it. We can fix
this by applying the usual shell-quoting formula.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

archimport: use safe_pipe_capture for user inputJeff King Mon, 11 Sep 2017 14:24:11 +0000 (10:24 -0400)

archimport: use safe_pipe_capture for user input

Refnames can contain shell metacharacters which need to be
passed verbatim to sub-processes. Using safe_pipe_capture
skips the shell entirely.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

shell: drop git-cvsserver support by defaultJeff King Mon, 11 Sep 2017 15:27:51 +0000 (11:27 -0400)

shell: drop git-cvsserver support by default

The git-cvsserver script is old and largely unmaintained
these days. But git-shell allows untrusted users to run it
out of the box, significantly increasing its attack surface.

Let's drop it from git-shell's list of internal handlers so
that it cannot be run by default. This is not backwards
compatible. But given the age and development activity on
CVS-related parts of Git, this is likely to impact very few
users, while helping many more (i.e., anybody who runs
git-shell and had no intention of supporting CVS).

There's no configuration mechanism in git-shell for us to
add a boolean and flip it to "off". But there is a mechanism
for adding custom commands, and adding CVS support here is
fairly trivial. Let's document it to give guidance to
anybody who really is still running cvsserver.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

cvsserver: use safe_pipe_capture for `constant commands... Junio C Hamano Mon, 11 Sep 2017 05:45:54 +0000 (14:45 +0900)

cvsserver: use safe_pipe_capture for `constant commands` as well

This is not strictly necessary, but it is a good code hygiene.

Signed-off-by: Junio C Hamano <gitster@pobox.com>

cvsserver: use safe_pipe_capture instead of backticksjoernchen Mon, 11 Sep 2017 05:45:09 +0000 (14:45 +0900)

cvsserver: use safe_pipe_capture instead of backticks

This makes the script pass arguments that are derived from end-user
input in safer way when invoking subcommands.

Reported-by: joernchen <joernchen@phenoelit.de>
Signed-off-by: joernchen <joernchen@phenoelit.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

cvsserver: move safe_pipe_capture() to the main packageJunio C Hamano Mon, 11 Sep 2017 05:44:24 +0000 (14:44 +0900)

cvsserver: move safe_pipe_capture() to the main package

As a preparation for replacing `command` with a call to this
function from outside GITCVS::updater package, move it to the main
package.

Signed-off-by: Junio C Hamano <gitster@pobox.com>

subprocess: loudly die when subprocess asks for an... Junio C Hamano Mon, 11 Sep 2017 03:21:29 +0000 (12:21 +0900)

subprocess: loudly die when subprocess asks for an unsupported capability

The handshake_capabilities() function first advertises the set of
capabilities it supports, so that the other side can pick and choose
which ones to use and ask us to enable in its response. Then we
read the response that tells us what choice the other side made. If
we saw something that we never advertised, that indicates one of two
things. The other side, i.e. the "upgraded" filter, is not paying
attention of the capabilities advertisement, and asking something
its correct operation relies on, but we are not capable of giving
that unknown feature and operate without it, so after that point the
exchange of data is a garbage-in-garbage-out. Or the other side
wanted to ask for one of the capabilities we advertised, but the
code has typo and their wish to enable a capability that its correct
operation relies on is not understood on this end. The result is
the same garbage-in-garbage-out.

Instead of sweeping such a potential bug under the rug, die loudly
when we see a request for an unsupported capability in order to
force sloppily-written filter scripts to get corrected.

Signed-off-by: Junio C Hamano <gitster@pobox.com>

travis: dedent a few scripts that are indented overly... Junio C Hamano Mon, 11 Sep 2017 01:18:29 +0000 (10:18 +0900)

travis: dedent a few scripts that are indented overly deeply

Signed-off-by: Junio C Hamano <gitster@pobox.com>

travis-ci: skip a branch build if equal tag is presentLars Schneider Sun, 10 Sep 2017 14:44:29 +0000 (16:44 +0200)

travis-ci: skip a branch build if equal tag is present

If we push a branch and a tag pointing to the HEAD of this branch,
then Travis CI would run the build twice. This wastes resources and
slows the testing.

Add a function to detect this situation and skip the build the branch
if appropriate. Invoke this function on every build.

Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

travis-ci: move Travis CI code into dedicated scriptsLars Schneider Sun, 10 Sep 2017 14:44:28 +0000 (16:44 +0200)

travis-ci: move Travis CI code into dedicated scripts

Most of the Travis CI commands are in the '.travis.yml'. The yml format
does not support functions and therefore code duplication is necessary
to run commands across all builds.

To fix this, add a library for common CI functions. Move all Travis CI
code into dedicated scripts and make them call the library first.

Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Sync with maintJunio C Hamano Sun, 10 Sep 2017 08:15:43 +0000 (17:15 +0900)

Sync with maint

* maint:
RelNotes: further fixes for 2.14.2 from the master front

The seventh batch post 2.14Junio C Hamano Sun, 10 Sep 2017 08:15:09 +0000 (17:15 +0900)

The seventh batch post 2.14

Signed-off-by: Junio C Hamano <gitster@pobox.com>

Merge branch 'rs/apply-epoch'Junio C Hamano Sun, 10 Sep 2017 08:08:25 +0000 (17:08 +0900)

Merge branch 'rs/apply-epoch'

Code simplification.

* rs/apply-epoch:
apply: remove epoch date from regex
apply: check date of potential epoch timestamps first

Merge branch 'jk/drop-sha1-entry-pos'Junio C Hamano Sun, 10 Sep 2017 08:08:25 +0000 (17:08 +0900)

Merge branch 'jk/drop-sha1-entry-pos'

Code clean-up.

* jk/drop-sha1-entry-pos:
sha1-lookup: remove sha1_entry_pos() from header file

Merge branch 'nd/worktree-kill-parse-ref'Junio C Hamano Sun, 10 Sep 2017 08:08:23 +0000 (17:08 +0900)

Merge branch 'nd/worktree-kill-parse-ref'

"git branch -M a b" while on a branch that is completely unrelated
to either branch a or branch b misbehaved when multiple worktree
was in use. This has been fixed.

* nd/worktree-kill-parse-ref:
branch: fix branch renaming not updating HEADs correctly

Merge branch 'mm/send-email-cc-cruft'Junio C Hamano Sun, 10 Sep 2017 08:08:22 +0000 (17:08 +0900)

Merge branch 'mm/send-email-cc-cruft'

In addition to "cc: <a@dd.re.ss> # cruft", "cc: a@dd.re.ss # cruft"
was taught to "git send-email" as a valid way to tell it that it
needs to also send a carbon copy to <a@dd.re.ss> in the trailer
section.

* mm/send-email-cc-cruft:
send-email: don't use Mail::Address, even if available
send-email: fix garbage removal after address

Merge branch 'ls/convert-filter-progress'Junio C Hamano Sun, 10 Sep 2017 08:08:22 +0000 (17:08 +0900)

Merge branch 'ls/convert-filter-progress'

The codepath to call external process filter for smudge/clean
operation learned to show the progress meter.

* ls/convert-filter-progress:
convert: display progress for filtered objects that have been delayed

Merge branch 'ma/up-to-date'Junio C Hamano Sun, 10 Sep 2017 08:08:22 +0000 (17:08 +0900)

Merge branch 'ma/up-to-date'

Message and doc updates.

* ma/up-to-date:
treewide: correct several "up-to-date" to "up to date"
Documentation/user-manual: update outdated example output

Merge branch 'ma/ts-cleanups'Junio C Hamano Sun, 10 Sep 2017 08:08:22 +0000 (17:08 +0900)

Merge branch 'ma/ts-cleanups'

Assorted bugfixes and clean-ups.

* ma/ts-cleanups:
ThreadSanitizer: add suppressions
strbuf_setlen: don't write to strbuf_slopbuf
pack-objects: take lock before accessing `remaining`
convert: always initialize attr_action in convert_attrs

RelNotes: further fixes for 2.14.2 from the master... Junio C Hamano Sun, 10 Sep 2017 08:06:09 +0000 (17:06 +0900)

RelNotes: further fixes for 2.14.2 from the master front

Signed-off-by: Junio C Hamano <gitster@pobox.com>

Merge branch 'jt/doc-pack-objects-fix' into maintJunio C Hamano Sun, 10 Sep 2017 08:03:10 +0000 (17:03 +0900)

Merge branch 'jt/doc-pack-objects-fix' into maint

Doc updates.

* jt/doc-pack-objects-fix:
Doc: clarify that pack-objects makes packs, plural

Merge branch 'jn/vcs-svn-cleanup' into maintJunio C Hamano Sun, 10 Sep 2017 08:03:09 +0000 (17:03 +0900)

Merge branch 'jn/vcs-svn-cleanup' into maint

Code clean-up.

* jn/vcs-svn-cleanup:
vcs-svn: move remaining repo_tree functions to fast_export.h
vcs-svn: remove repo_delete wrapper function
vcs-svn: remove custom mode constants
vcs-svn: remove more unused prototypes and declarations

Merge branch 'bc/vcs-svn-cleanup' into maintJunio C Hamano Sun, 10 Sep 2017 08:03:08 +0000 (17:03 +0900)

Merge branch 'bc/vcs-svn-cleanup' into maint

Code clean-up.

* bc/vcs-svn-cleanup:
vcs-svn: rename repo functions to "svn_repo"
vcs-svn: remove unused prototypes

Merge branch 'jk/doc-the-this' into maintJunio C Hamano Sun, 10 Sep 2017 08:03:07 +0000 (17:03 +0900)

Merge branch 'jk/doc-the-this' into maint

Doc clean-up.

* jk/doc-the-this:
doc: fix typo in sendemail.identity

Merge branch 'rs/commit-h-single-parent-cleanup' into... Junio C Hamano Sun, 10 Sep 2017 08:03:07 +0000 (17:03 +0900)

Merge branch 'rs/commit-h-single-parent-cleanup' into maint

Code clean-up.

* rs/commit-h-single-parent-cleanup:
commit: remove unused inline function single_parent()

Merge branch 'mg/format-ref-doc-fix' into maintJunio C Hamano Sun, 10 Sep 2017 08:03:06 +0000 (17:03 +0900)

Merge branch 'mg/format-ref-doc-fix' into maint

Doc fix.

* mg/format-ref-doc-fix:
Documentation/git-for-each-ref: clarify peeling of tags for --format
Documentation: use proper wording for ref format strings

Merge branch 'sb/submodule-parallel-update' into maintJunio C Hamano Sun, 10 Sep 2017 08:03:06 +0000 (17:03 +0900)

Merge branch 'sb/submodule-parallel-update' into maint

Code clean-up.

* sb/submodule-parallel-update:
submodule.sh: remove unused variable

Merge branch 'hv/t5526-andand-chain-fix' into maintJunio C Hamano Sun, 10 Sep 2017 08:03:05 +0000 (17:03 +0900)

Merge branch 'hv/t5526-andand-chain-fix' into maint

Test fix.

* hv/t5526-andand-chain-fix:
t5526: fix some broken && chains

Merge branch 'sb/sha1-file-cleanup' into maintJunio C Hamano Sun, 10 Sep 2017 08:03:04 +0000 (17:03 +0900)

Merge branch 'sb/sha1-file-cleanup' into maint

Code clean-up.

* sb/sha1-file-cleanup:
sha1_file: make read_info_alternates static

Merge branch 'rs/t1002-do-not-use-sum' into maintJunio C Hamano Sun, 10 Sep 2017 08:03:04 +0000 (17:03 +0900)

Merge branch 'rs/t1002-do-not-use-sum' into maint

Test simplification.

* rs/t1002-do-not-use-sum:
t1002: stop using sum(1)

Merge branch 'ah/doc-empty-string-is-false' into maintJunio C Hamano Sun, 10 Sep 2017 08:03:03 +0000 (17:03 +0900)

Merge branch 'ah/doc-empty-string-is-false' into maint

Doc update.

* ah/doc-empty-string-is-false:
doc: clarify "config --bool" behaviour with empty string

Merge branch 'rs/merge-microcleanup' into maintJunio C Hamano Sun, 10 Sep 2017 08:03:02 +0000 (17:03 +0900)

Merge branch 'rs/merge-microcleanup' into maint

Code clean-up.

* rs/merge-microcleanup:
merge: use skip_prefix()

Merge branch 'rs/find-pack-entry-bisection' into maintJunio C Hamano Sun, 10 Sep 2017 08:03:02 +0000 (17:03 +0900)

Merge branch 'rs/find-pack-entry-bisection' into maint

Code clean-up.

* rs/find-pack-entry-bisection:
sha1_file: avoid comparison if no packed hash matches the first byte

Merge branch 'rs/apply-lose-prefix-length' into maintJunio C Hamano Sun, 10 Sep 2017 08:03:01 +0000 (17:03 +0900)

Merge branch 'rs/apply-lose-prefix-length' into maint

Code clean-up.

* rs/apply-lose-prefix-length:
apply: remove prefix_length member from apply_state

Merge branch 'rj/add-chmod-error-message' into maintJunio C Hamano Sun, 10 Sep 2017 08:03:00 +0000 (17:03 +0900)

Merge branch 'rj/add-chmod-error-message' into maint

Message fix.

* rj/add-chmod-error-message:
builtin/add: add detail to a 'cannot chmod' error message

Merge branch 'jk/hashcmp-memcmp' into maintJunio C Hamano Sun, 10 Sep 2017 08:02:59 +0000 (17:02 +0900)

Merge branch 'jk/hashcmp-memcmp' into maint

Code clean-up.

* jk/hashcmp-memcmp:
hashcmp: use memcmp instead of open-coded loop

Merge branch 'rs/t3700-clean-leftover' into maintJunio C Hamano Sun, 10 Sep 2017 08:02:58 +0000 (17:02 +0900)

Merge branch 'rs/t3700-clean-leftover' into maint

A test fix.

* rs/t3700-clean-leftover:
t3700: fix broken test under !POSIXPERM

Merge branch 'jc/perl-git-comment-typofix' into maintJunio C Hamano Sun, 10 Sep 2017 08:02:57 +0000 (17:02 +0900)

Merge branch 'jc/perl-git-comment-typofix' into maint

A comment fix.

* jc/perl-git-comment-typofix:
perl/Git.pm: typofix in a comment

Merge branch 'mf/no-dashed-subcommands' into maintJunio C Hamano Sun, 10 Sep 2017 08:02:56 +0000 (17:02 +0900)

Merge branch 'mf/no-dashed-subcommands' into maint

Code clean-up.

* mf/no-dashed-subcommands:
scripts: use "git foo" not "git-foo"

Merge branch 'ab/ref-filter-no-contains' into maintJunio C Hamano Sun, 10 Sep 2017 08:02:56 +0000 (17:02 +0900)

Merge branch 'ab/ref-filter-no-contains' into maint

A test fix.

* ab/ref-filter-no-contains:
tests: don't give unportable ">" to "test" built-in, use -gt

Merge branch 'rs/archive-excluded-directory' into maintJunio C Hamano Sun, 10 Sep 2017 08:02:55 +0000 (17:02 +0900)

Merge branch 'rs/archive-excluded-directory' into maint

"git archive" did not work well with pathspecs and the
export-ignore attribute.

We may want to resurrect the "we don't archive an empty directory"
bonus patch, but I do not mind merging the above early to 'next'
and leave it as a separate follow-up enhancement.
cf. <20170820090629.tumvqwzkromcykjf@sigill.intra.peff.net>

* rs/archive-excluded-directory:
archive: don't queue excluded directories
archive: factor out helper functions for handling attributes
t5001: add tests for export-ignore attributes and exclude pathspecs

Merge branch 'mg/killed-merge' into maintJunio C Hamano Sun, 10 Sep 2017 08:02:55 +0000 (17:02 +0900)

Merge branch 'mg/killed-merge' into maint

Killing "git merge --edit" before the editor returns control left
the repository in a state with MERGE_MSG but without MERGE_HEAD,
which incorrectly tells the subsequent "git commit" that there was
a squash merge in progress. This has been fixed.

* mg/killed-merge:
merge: save merge state earlier
merge: split write_merge_state in two
merge: clarify call chain
Documentation/git-merge: explain --continue

Merge branch 'tb/apply-with-crlf' into maintJunio C Hamano Sun, 10 Sep 2017 08:02:55 +0000 (17:02 +0900)

Merge branch 'tb/apply-with-crlf' into maint

"git apply" that is used as a better "patch -p1" failed to apply a
taken from a file with CRLF line endings to a file with CRLF line
endings. The root cause was because it misused convert_to_git()
that tried to do "safe-crlf" processing by looking at the index
entry at the same path, which is a nonsense---in that mode, "apply"
is not working on the data in (or derived from) the index at all.
This has been fixed.

* tb/apply-with-crlf:
apply: file commited with CRLF should roundtrip diff and apply
convert: add SAFE_CRLF_KEEP_CRLF

Merge branch 'cc/subprocess-handshake-missing-capabilit... Junio C Hamano Sun, 10 Sep 2017 08:02:55 +0000 (17:02 +0900)

Merge branch 'cc/subprocess-handshake-missing-capabilities' into maint

When handshake with a subprocess filter notices that the process
asked for an unknown capability, Git did not report what program
the offending subprocess was running. This has been corrected.

We may want a follow-up fix to tighten the error checking, though.

* cc/subprocess-handshake-missing-capabilities:
sub-process: print the cmd when a capability is unsupported

Merge branch 'as/grep-quiet-no-match-exit-code-fix... Junio C Hamano Sun, 10 Sep 2017 08:02:55 +0000 (17:02 +0900)

Merge branch 'as/grep-quiet-no-match-exit-code-fix' into maint

"git grep -L" and "git grep --quiet -L" reported different exit
codes; this has been corrected.

* as/grep-quiet-no-match-exit-code-fix:
git-grep: correct exit code with --quiet and -L

Merge branch 'kd/stash-with-bash-4.4' into maintJunio C Hamano Sun, 10 Sep 2017 08:02:54 +0000 (17:02 +0900)

Merge branch 'kd/stash-with-bash-4.4' into maint

bash 4.4 or newer gave a warning on NUL byte in command
substitution done in "git stash"; this has been squelched.

* kd/stash-with-bash-4.4:
stash: prevent warning about null bytes in input

Merge branch 'rs/win32-syslog-leakfix' into maintJunio C Hamano Sun, 10 Sep 2017 08:02:54 +0000 (17:02 +0900)

Merge branch 'rs/win32-syslog-leakfix' into maint

Memory leak in an error codepath has been plugged.

* rs/win32-syslog-leakfix:
win32: plug memory leak on realloc() failure in syslog()

Merge branch 'rs/unpack-entry-leakfix' into maintJunio C Hamano Sun, 10 Sep 2017 08:02:53 +0000 (17:02 +0900)

Merge branch 'rs/unpack-entry-leakfix' into maint

Memory leak in an error codepath has been plugged.

* rs/unpack-entry-leakfix:
sha1_file: release delta_stack on error in unpack_entry()

Merge branch 'rs/fsck-obj-leakfix' into maintJunio C Hamano Sun, 10 Sep 2017 08:02:52 +0000 (17:02 +0900)

Merge branch 'rs/fsck-obj-leakfix' into maint

Memory leak in an error codepath has been plugged.

* rs/fsck-obj-leakfix:
fsck: free buffers on error in fsck_obj()

Merge branch 'ur/svn-local-zone' into maintJunio C Hamano Sun, 10 Sep 2017 08:02:52 +0000 (17:02 +0900)

Merge branch 'ur/svn-local-zone' into maint

"git svn" used with "--localtime" option did not compute the tz
offset for the timestamp in question and instead always used the
current time, which has been corrected.

* ur/svn-local-zone:
git svn fetch: Create correct commit timestamp when using --localtime

Merge branch 'pw/am-signoff' into maintJunio C Hamano Sun, 10 Sep 2017 08:02:51 +0000 (17:02 +0900)

Merge branch 'pw/am-signoff' into maint

"git am -s" has been taught that some input may end with a trailer
block that is not Signed-off-by: and it should refrain from adding
an extra blank line before adding a new sign-off in such a case.

* pw/am-signoff:
am: fix signoff when other trailers are present

Merge branch 'rs/in-obsd-basename-dirname-take-const... Junio C Hamano Sun, 10 Sep 2017 08:02:51 +0000 (17:02 +0900)

Merge branch 'rs/in-obsd-basename-dirname-take-const' into maint

Portability fix.

* rs/in-obsd-basename-dirname-take-const:
test-path-utils: handle const parameter of basename and dirname

Merge branch 'rs/t4062-obsd' into maintJunio C Hamano Sun, 10 Sep 2017 08:02:51 +0000 (17:02 +0900)

Merge branch 'rs/t4062-obsd' into maint

Test portability fix.

* rs/t4062-obsd:
t4062: use less than 256 repetitions in regex

Merge branch 'rs/obsd-getcwd-workaround' into maintJunio C Hamano Sun, 10 Sep 2017 08:02:50 +0000 (17:02 +0900)

Merge branch 'rs/obsd-getcwd-workaround' into maint

Test portability fix for BSDs.

* rs/obsd-getcwd-workaround:
t0001: skip test with restrictive permissions if getpwd(3) respects them

Merge branch 'bw/clone-recursive-quiet' into maintJunio C Hamano Sun, 10 Sep 2017 08:02:49 +0000 (17:02 +0900)

Merge branch 'bw/clone-recursive-quiet' into maint

"git clone --recurse-submodules --quiet" did not pass the quiet
option down to submodules.

* bw/clone-recursive-quiet:
clone: teach recursive clones to respect -q

Merge branch 'pw/sequence-rerere-autoupdate' into maintJunio C Hamano Sun, 10 Sep 2017 08:02:49 +0000 (17:02 +0900)

Merge branch 'pw/sequence-rerere-autoupdate' into maint

Commands like "git rebase" accepted the --rerere-autoupdate option
from the command line, but did not always use it. This has been
fixed.

* pw/sequence-rerere-autoupdate:
cherry-pick/revert: reject --rerere-autoupdate when continuing
cherry-pick/revert: remember --rerere-autoupdate
t3504: use test_commit
rebase -i: honor --rerere-autoupdate
rebase: honor --rerere-autoupdate
am: remember --rerere-autoupdate setting

Merge branch 'bw/push-options-recursively-to-submodules... Junio C Hamano Sun, 10 Sep 2017 08:02:49 +0000 (17:02 +0900)

Merge branch 'bw/push-options-recursively-to-submodules' into maint

"git push --recurse-submodules $there HEAD:$target" was not
propagated down to the submodules, but now it is.

* bw/push-options-recursively-to-submodules:
submodule--helper: teach push-check to handle HEAD

Merge branch 'ma/pager-per-subcommand-action' into... Junio C Hamano Sun, 10 Sep 2017 08:02:48 +0000 (17:02 +0900)

Merge branch 'ma/pager-per-subcommand-action' into maint

The "tag.pager" configuration variable was useless for those who
actually create tag objects, as it interfered with the use of an
editor. A new mechanism has been introduced for commands to enable
pager depending on what operation is being carried out to fix this,
and then "git tag -l" is made to run pager by default.

If this works out OK, I think there are low-hanging fruits in
other commands like "git branch" that outputs long list in one mode
while taking input in another.

* ma/pager-per-subcommand-action:
git.c: ignore pager.* when launching builtin as dashed external
tag: change default of `pager.tag` to "on"
tag: respect `pager.tag` in list-mode only
t7006: add tests for how git tag paginates
git.c: provide setup_auto_pager()
git.c: let builtins opt for handling `pager.foo` themselves
builtin.h: take over documentation from api-builtin.txt

Merge branch 'jk/rev-list-empty-input' into maintJunio C Hamano Sun, 10 Sep 2017 08:02:48 +0000 (17:02 +0900)

Merge branch 'jk/rev-list-empty-input' into maint

"git log --tag=no-such-tag" showed log starting from HEAD, which
has been fixed---it now shows nothing.

* jk/rev-list-empty-input:
revision: do not fallback to default when rev_input_given is set
rev-list: don't show usage when we see empty ref patterns
revision: add rev_input_given flag
t6018: flesh out empty input/output rev-list tests

Merge branch 'st/lib-gpg-kill-stray-agent' into maintJunio C Hamano Sun, 10 Sep 2017 08:02:48 +0000 (17:02 +0900)

Merge branch 'st/lib-gpg-kill-stray-agent' into maint

Some versions of GnuPG fails to kill gpg-agent it auto-spawned
and such a left-over agent can interfere with a test. Work it
around by attempting to kill one before starting a new test.

* st/lib-gpg-kill-stray-agent:
t: lib-gpg: flush gpg agent on startup

wt-status: release strbuf after use in wt_longstatus_pr... Rene Scharfe Wed, 30 Aug 2017 18:20:18 +0000 (20:20 +0200)

wt-status: release strbuf after use in wt_longstatus_print_tracking()

If format_tracking_info() returns 0, then it didn't touch its strbuf
parameter, so it's OK to exit early in that case. Clean up sb in the
other case.

Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

refs/files-backend: add `refname`, not "HEAD", to listMartin Ågren Sat, 9 Sep 2017 06:57:18 +0000 (08:57 +0200)

refs/files-backend: add `refname`, not "HEAD", to list

An earlier patch rewrote `split_symref_update()` to add a copy of a
string to a string list instead of adding the original string. That was
so that the original string could be freed in a later patch, but it is
also conceptually cleaner, since now all calls to `string_list_insert()`
and `string_list_append()` add `update->refname`. --- Except a literal
"HEAD" is added in `split_head_update()`.

Restructure `split_head_update()` in the same way as the earlier patch
did for `split_symref_update()`. This does not correct any practical
problem, but makes things conceptually cleaner. The downside is a call
to `string_list_has_string()`, which should be relatively cheap.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

refs/files-backend: correct return value in lock_ref_fo... Martin Ågren Sat, 9 Sep 2017 06:57:17 +0000 (08:57 +0200)

refs/files-backend: correct return value in lock_ref_for_update

In one code path we return a literal -1 and not a symbolic constant. The
value -1 would be interpreted as TRANSACTION_NAME_CONFLICT, which is
wrong. Use TRANSACTION_GENERIC_ERROR instead (that is the only other
return value we have to choose from).

Noticed-by: Michael Haggerty <mhagger@alum.mit.edu>
Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

refs/files-backend: fix memory leak in lock_ref_for_updateMartin Ågren Sat, 9 Sep 2017 06:57:16 +0000 (08:57 +0200)

refs/files-backend: fix memory leak in lock_ref_for_update

After the previous patch, none of the functions we call hold on to
`referent.buf`, so we can safely release the string buffer before
returning.

Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

refs/files-backend: add longer-scoped copy of string... Martin Ågren Sat, 9 Sep 2017 06:57:15 +0000 (08:57 +0200)

refs/files-backend: add longer-scoped copy of string to list

split_symref_update() receives a string-pointer `referent` and adds it
to the list of `affected_refnames`. The list simply holds on to the
pointers it is given, it does not copy the strings and it does not ever
free them. The `referent` string in split_symref_update() belongs to a
string buffer in the caller. After we return, the string will be leaked.

In the next patch, we want to properly release the string buffer in the
caller, but we can't safely do so until we've made sure that
`affected_refnames` will not be holding on to a pointer to the string.
We could configure the list to handle its own resources, but it would
mean some alloc/free-churning. The list is already handling other
strings (through other code paths) which we do not need to worry about,
and we'd be memory-churning those strings too, completely unnecessary.

Observe that split_symref_update() creates a `new_update`-object through
ref_transaction_add_update(), after which `new_update->refname` is a
copy of `referent`. The difference is, this copy will be freed, and it
will be freed *after* `affected_refnames` has been cleared.

Rearrange the handling of `referent`, so that we don't add it directly
to `affected_refnames`. Instead, first just check whether `referent`
exists in the string list, and later add `new_update->refname`.

Helped-by: Michael Haggerty <mhagger@alum.mit.edu>
Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

commit-tree: do not complete line in -F inputRoss Kabus Thu, 7 Sep 2017 14:41:11 +0000 (10:41 -0400)

commit-tree: do not complete line in -F input

"git commit-tree -F <file>", unlike "cat <file> | git
commit-tree" (i.e. feeding the same contents from the standard
input), added a missing final newline when the input ended in an
incomplete line.

Correct this inconsistency by leaving the incomplete line as-is,
as erring on the side of not touching the input is preferrable
and expected for a plumbing command like "commit-tree".

Signed-off-by: Ross Kabus <rkabus@aerotech.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

files_transaction_finish(): delete reflogs before refer... Michael Haggerty Fri, 8 Sep 2017 13:51:53 +0000 (15:51 +0200)

files_transaction_finish(): delete reflogs before references

If the deletion steps unexpectedly fail, it is less bad to leave a
reference without its reflog than it is to leave a reflog without its
reference, since the latter is an invalid repository state.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

packed-backend: rip out some now-unused codeMichael Haggerty Fri, 8 Sep 2017 13:51:52 +0000 (15:51 +0200)

packed-backend: rip out some now-unused code

Now the outside world interacts with the packed ref store only via the
generic refs API plus a few lock-related functions. This allows us to
delete some functions that are no longer used, thereby completing the
encapsulation of the packed ref store.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

files_ref_store: use a transaction to update packed... Michael Haggerty Fri, 8 Sep 2017 13:51:51 +0000 (15:51 +0200)

files_ref_store: use a transaction to update packed refs

When processing a `files_ref_store` transaction, it is sometimes
necessary to delete some references from the "packed-refs" file. Do
that using a reference transaction conducted against the
`packed_ref_store`.

This change further decouples `files_ref_store` from
`packed_ref_store`. It also fixes multiple problems, including the two
revealed by test cases added in the previous commit.

First, the old code didn't obtain the `packed-refs` lock until
`files_transaction_finish()`. This means that a failure to acquire the
`packed-refs` lock (e.g., due to contention with another process)
wasn't detected until it was too late (problems like this are supposed
to be detected in the "prepare" phase). The new code acquires the
`packed-refs` lock in `files_transaction_prepare()`, the same stage of
the processing when the loose reference locks are being acquired,
removing another reason why the "prepare" phase might succeed and the
"finish" phase might nevertheless fail.

Second, the old code deleted the loose version of a reference before
deleting any packed version of the same reference. This left a moment
when another process might think that the packed version of the
reference is current, which is incorrect. (Even worse, the packed
version of the reference can be arbitrarily old, and might even point
at an object that has since been garbage-collected.)

Third, if a reference deletion fails to acquire the `packed-refs` lock
altogether, then the old code might leave the repository in the
incorrect state (possibly corrupt) described in the previous
paragraph.

Now we activate the new "packed-refs" file (sans any references that
are being deleted) *before* deleting the corresponding loose
references. But we hold the "packed-refs" lock until after the loose
references have been finalized, thus preventing a simultaneous
"pack-refs" process from packing the loose version of the reference in
the time gap, which would otherwise defeat our attempt to delete it.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t1404: demonstrate two problems with reference transactionsMichael Haggerty Fri, 8 Sep 2017 13:51:50 +0000 (15:51 +0200)

t1404: demonstrate two problems with reference transactions

Currently, a loose reference is deleted even before locking the
`packed-refs` file, let alone deleting any packed version of the
reference. This leads to two problems, demonstrated by two new tests:

* While a reference is being deleted, other processes might see the
old, packed value of the reference for a moment before the packed
version is deleted. Normally this would be hard to observe, but we
can prolong the window by locking the `packed-refs` file externally
before running `update-ref`, then unlocking it before `update-ref`'s
attempt to acquire the lock times out.

* If the `packed-refs` file is locked so long that `update-ref` fails
to lock it, then the reference can be left permanently in the
incorrect state described in the previous point.

In a moment, both problems will be fixed.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

files_initial_transaction_commit(): use a transaction... Michael Haggerty Fri, 8 Sep 2017 13:51:49 +0000 (15:51 +0200)

files_initial_transaction_commit(): use a transaction for packed refs

Use a `packed_ref_store` transaction in the implementation of
`files_initial_transaction_commit()` rather than using internal
features of the packed ref store. This further decouples
`files_ref_store` from `packed_ref_store`.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

prune_refs(): also free the linked listMichael Haggerty Fri, 8 Sep 2017 13:51:48 +0000 (15:51 +0200)

prune_refs(): also free the linked list

At least since v1.7, the elements of the `refs_to_prune` linked list
have been leaked. Fix the leak by teaching `prune_refs()` to free the
list elements as it processes them.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

files_pack_refs(): use a reference transaction to write... Michael Haggerty Fri, 8 Sep 2017 13:51:47 +0000 (15:51 +0200)

files_pack_refs(): use a reference transaction to write packed refs

Now that the packed reference store supports transactions, we can use
a transaction to write the packed versions of references that we want
to pack. This decreases the coupling between `files_ref_store` and
`packed_ref_store`.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

packed_delete_refs(): implement methodMichael Haggerty Fri, 8 Sep 2017 13:51:46 +0000 (15:51 +0200)

packed_delete_refs(): implement method

Implement `packed_delete_refs()` using a reference transaction. This
means that `files_delete_refs()` can use `refs_delete_refs()` instead
of `repack_without_refs()` to delete any packed references, decreasing
the coupling between the classes.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

packed_ref_store: implement reference transactionsMichael Haggerty Fri, 8 Sep 2017 13:51:45 +0000 (15:51 +0200)

packed_ref_store: implement reference transactions

Implement the methods needed to support reference transactions for
the packed-refs backend. The new methods are not yet used.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

struct ref_transaction: add a place for backends to... Michael Haggerty Fri, 8 Sep 2017 13:51:44 +0000 (15:51 +0200)

struct ref_transaction: add a place for backends to store data

`packed_ref_store` is going to want to store some transaction-wide
data, so make a place for it.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

packed-backend: don't adjust the reference count on... Michael Haggerty Fri, 8 Sep 2017 13:51:43 +0000 (15:51 +0200)

packed-backend: don't adjust the reference count on lock/unlock

The old code incremented the packed ref cache reference count when
acquiring the packed-refs lock, and decremented the count when
releasing the lock. This is unnecessary because:

* Another process cannot change the packed-refs file because it is
locked.

* When we ourselves change the packed-refs file, we do so by first
modifying the packed ref-cache, and then writing the data from the
ref-cache to disk. So the packed ref-cache remains fresh because any
changes that we plan to make to the file are made in the cache first
anyway.

So there is no reason for the cache to become stale.

Moreover, the extra reference count causes a problem if we
intentionally clear the packed refs cache, as we sometimes need to do
if we change the cache in anticipation of writing a change to disk,
but then the write to disk fails. In that case, `packed_refs_unlock()`
would have no easy way to find the cache whose reference count it
needs to decrement.

This whole issue will soon become moot due to upcoming changes that
avoid changing the in-memory cache as part of updating the packed-refs
on disk, but this change makes that transition easier.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

load_subtree(): check that `prefix_len` is in the expec... Michael Haggerty Fri, 8 Sep 2017 16:10:10 +0000 (18:10 +0200)

load_subtree(): check that `prefix_len` is in the expected range

This value, which is stashed in the last byte of an object_id hash,
gets handed around a lot. So add a sanity check before using it in
`load_subtree()`.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

shortlog: skip format/parse roundtrip for internal... Jeff King Fri, 8 Sep 2017 09:21:27 +0000 (05:21 -0400)

shortlog: skip format/parse roundtrip for internal traversal

The original git-shortlog command parsed the output of
git-log, and the logic went something like this:

1. Read stdin looking for "author" lines.

2. Parse the identity into its name/email bits.

3. Apply mailmap to the name/email.

4. Reformat the identity into a single buffer that is our
"key" for grouping entries (either a name by default,
or "name <email>" if --email was given).

The first part happens in read_from_stdin(), and the other
three steps are part of insert_one_record().

When we do an internal traversal, we just swap out the stdin
read in step 1 for reading the commit objects ourselves.
Prior to 2db6b83d18 (shortlog: replace hand-parsing of
author with pretty-printer, 2016-01-18), that made sense; we
still had to parse the ident in the commit message.

But after that commit, we use pretty.c's "%an <%ae>" to get
the author ident (for simplicity). Which means that the
pretty printer is doing a parse/format under the hood, and
then we parse the result, apply the mailmap, and format the
result again.

Instead, we can just ask pretty.c to do all of those steps
for us (including the mailmap via "%aN <%aE>", and not
formatting the address when --email is missing).

And then we can push steps 2-4 into read_from_stdin(). This
speeds up "git shortlog -ns" on linux.git by about 3%, and
eliminates a leak in insert_one_record() of the namemailbuf
strbuf.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

add UNLEAK annotation for reducing leak false positivesJeff King Fri, 8 Sep 2017 06:38:41 +0000 (02:38 -0400)

add UNLEAK annotation for reducing leak false positives

It's a common pattern in git commands to allocate some
memory that should last for the lifetime of the program and
then not bother to free it, relying on the OS to throw it
away.

This keeps the code simple, and it's fast (we don't waste
time traversing structures or calling free at the end of the
program). But it also triggers warnings from memory-leak
checkers like valgrind or LSAN. They know that the memory
was still allocated at program exit, but they don't know
_when_ the leaked memory stopped being useful. If it was
early in the program, then it's probably a real and
important leak. But if it was used right up until program
exit, it's not an interesting leak and we'd like to suppress
it so that we can see the real leaks.

This patch introduces an UNLEAK() macro that lets us do so.
To understand its design, let's first look at some of the
alternatives.

Unfortunately the suppression systems offered by
leak-checking tools don't quite do what we want. A
leak-checker basically knows two things:

1. Which blocks were allocated via malloc, and the
callstack during the allocation.

2. Which blocks were left un-freed at the end of the
program (and which are unreachable, but more on that
later).

Their suppressions work by mentioning the function or
callstack of a particular allocation, and marking it as OK
to leak. So imagine you have code like this:

int cmd_foo(...)
{
/* this allocates some memory */
char *p = some_function();
printf("%s", p);
return 0;
}

You can say "ignore allocations from some_function(),
they're not leaks". But that's not right. That function may
be called elsewhere, too, and we would potentially want to
know about those leaks.

So you can say "ignore the callstack when main calls
some_function". That works, but your annotations are
brittle. In this case it's only two functions, but you can
imagine that the actual allocation is much deeper. If any of
the intermediate code changes, you have to update the
suppression.

What we _really_ want to say is that "the value assigned to
p at the end of the function is not a real leak". But
leak-checkers can't understand that; they don't know about
"p" in the first place.

However, we can do something a little bit tricky if we make
some assumptions about how leak-checkers work. They
generally don't just report all un-freed blocks. That would
report even globals which are still accessible when the
leak-check is run. Instead they take some set of memory
(like BSS) as a root and mark it as "reachable". Then they
scan the reachable blocks for anything that looks like a
pointer to a malloc'd block, and consider that block
reachable. And then they scan those blocks, and so on,
transitively marking anything reachable from a global as
"not leaked" (or at least leaked in a different category).

So we can mark the value of "p" as reachable by putting it
into a variable with program lifetime. One way to do that is
to just mark "p" as static. But that actually affects the
run-time behavior if the function is called twice (you
aren't likely to call main() twice, but some of our cmd_*()
functions are called from other commands).

Instead, we can trick the leak-checker by putting the value
into _any_ reachable bytes. This patch keeps a global
linked-list of bytes copied from "unleaked" variables. That
list is reachable even at program exit, which confers
recursive reachability on whatever values we unleak.

In other words, you can do:

int cmd_foo(...)
{
char *p = some_function();
printf("%s", p);
UNLEAK(p);
return 0;
}

to annotate "p" and suppress the leak report.

But wait, couldn't we just say "free(p)"? In this toy
example, yes. But UNLEAK()'s byte-copying strategy has
several advantages over actually freeing the memory:

1. It's recursive across structures. In many cases our "p"
is not just a pointer, but a complex struct whose
fields may have been allocated by a sub-function. And
in some cases (e.g., dir_struct) we don't even have a
function which knows how to free all of the struct
members.

By marking the struct itself as reachable, that confers
reachability on any pointers it contains (including those
found in embedded structs, or reachable by walking
heap blocks recursively.

2. It works on cases where we're not sure if the value is
allocated or not. For example:

char *p = argc > 1 ? argv[1] : some_function();

It's safe to use UNLEAK(p) here, because it's not
freeing any memory. In the case that we're pointing to
argv here, the reachability checker will just ignore
our bytes.

3. Likewise, it works even if the variable has _already_
been freed. We're just copying the pointer bytes. If
the block has been freed, the leak-checker will skip
over those bytes as uninteresting.

4. Because it's not actually freeing memory, you can
UNLEAK() before we are finished accessing the variable.
This is helpful in cases like this:

char *p = some_function();
return another_function(p);

Writing this with free() requires:

int ret;
char *p = some_function();
ret = another_function(p);
free(p);
return ret;

But with unleak we can just write:

char *p = some_function();
UNLEAK(p);
return another_function(p);

This patch adds the UNLEAK() macro and enables it
automatically when Git is compiled with SANITIZE=leak. In
normal builds it's a noop, so we pay no runtime cost.

It also adds some UNLEAK() annotations to show off how the
feature works. On top of other recent leak fixes, these are
enough to get t0000 and t0001 to pass when compiled with
LSAN.

Note the case in commit.c which actually converts a
strbuf_release() into an UNLEAK. This code was already
non-leaky, but the free didn't do anything useful, since
we're exiting. Converting it to an annotation means that
non-leak-checking builds pay no runtime cost. The cost is
minimal enough that it's probably not worth going on a
crusade to convert these kinds of frees to UNLEAKS. I did it
here for consistency with the "sb" leak (though it would
have been equally correct to go the other way, and turn them
both into strbuf_release() calls).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t6120: test describe and name-rev with deep reposMichael J Gruber Thu, 7 Sep 2017 14:02:23 +0000 (16:02 +0200)

t6120: test describe and name-rev with deep repos

Depending on the implementation of walks, limitted stack size may lead
to problems (for recursion).

Test name-rev and describe with deep repos and limitted stack size and
mark the former with known failure.

We add these tests (which add gazillions of commits) last so as to keep
the runtime of other subtests the same.

Signed-off-by: Michael J Gruber <git@grubix.eu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t6120: clean up state after breaking repoMichael J Gruber Thu, 7 Sep 2017 14:02:22 +0000 (16:02 +0200)

t6120: clean up state after breaking repo

t6120 breaks the repo state intentionally in the last tests.

Clean up the breakage afterwards (and before adding more tests).

Signed-off-by: Michael J Gruber <git@grubix.eu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t6120: test name-rev --all and --stdinMichael J Gruber Thu, 7 Sep 2017 14:02:21 +0000 (16:02 +0200)

t6120: test name-rev --all and --stdin

name-rev is used in a few tests, but tested only in t6120 along with
describe so far.

Add tests for name-rev with --all and --stdin.

Signed-off-by: Michael J Gruber <git@grubix.eu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t7004: move limited stack prereq to test-libMichael J Gruber Thu, 7 Sep 2017 14:02:20 +0000 (16:02 +0200)

t7004: move limited stack prereq to test-lib

The lazy prerequisite ULIMIT_STACK_SIZE is used only in t7004 so far.

Move it to test-lib.sh so that it can be used in other tests (which it will
be in a follow-up commit).

Signed-off-by: Michael J Gruber <git@grubix.eu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

merge-recursive: change current file dir string_lists... Kevin Willford Thu, 7 Sep 2017 16:25:56 +0000 (10:25 -0600)

merge-recursive: change current file dir string_lists to hashmap

The code was using two string_lists, one for the directories and
one for the files. The code never checks the lists independently
so we should be able to only use one list. The string_list also
is a O(log n) for lookup and insertion. Switching this to use a
hashmap will give O(1) which will save some time when there are
millions of paths that will be checked.

Signed-off-by: Kevin Willford <kewillf@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

builtin/merge: honor commit-msg hook for mergesStefan Beller Thu, 7 Sep 2017 22:04:29 +0000 (15:04 -0700)

builtin/merge: honor commit-msg hook for merges

Similar to 65969d43d1 (merge: honor prepare-commit-msg hook, 2011-02-14)
merge should also honor the commit-msg hook: When a merge is stopped due
to conflicts or --no-commit, the subsequent commit calls the commit-msg
hook. However, it is not called after a clean merge. Fix this
inconsistency by invoking the hook after clean merges as well.

This change is motivated by Gerrit's commit-msg hook to install a ChangeId
trailer into the commit message. Without such a ChangeId, Gerrit refuses
to accept any commit by default, such that the inconsistency of (not)
running the commit-msg hook between commit and merge leads to confusion
and might block people from getting their work done.

As the githooks man page is very vocal about the possibility of skipping
the commit-msg hook via the --no-verify option, implement the option
in merge, too.

'git merge --continue' is currently implemented as calling cmd_commit
with no further arguments. This works for most other merge related options,
such as demonstrated via the --allow-unrelated-histories flag in the
test. The --no-verify option however is not remembered across invocations
of git-merge. Originally the author assumed an alternative in which the
'git merge --continue' command accepts the --no-verify flag, but that
opens up the discussion which flags are allows to the continued merge
command and which must be given in the first invocation.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

read-cache: fix index corruption with index v4Thomas Gummerer Thu, 7 Sep 2017 19:24:12 +0000 (20:24 +0100)

read-cache: fix index corruption with index v4

ce012deb98 ("read-cache: avoid allocating every ondisk entry when
writing", 2017-08-21) changed the way cache entries are written to the
index file. While previously it wrote the name to an struct that was
allocated using xcalloc(), it now uses ce_write() directly. Previously
ce_namelen - common bytes were written to the cache entry, which would
automatically make it nul terminated, as it was allocated using calloc.

Now we are writing ce_namelen - common + 1 bytes directly from the
ce->name to the index. If CE_STRIP_NAME however gets set in the split
index case ce->ce_namelen is set to 0 without changing the actual
ce->name buffer. When index-v4, this results in the first character of
ce->name being written out instead of just a terminating nul charcter.

As index-v4 requires the terminating nul character as terminator of
the name when reading it back, this results in a corrupted index.

Fix that by only writing ce_namelen - common bytes directly from
ce->name to the index, and adding the nul terminator in an extra call to
ce_write.

This bug was turned up by setting TEST_GIT_INDEX_VERSION = 4 in
config.mak and running the test suite (t1700 specifically broke).

Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

pull: honor submodule.recurse config optionNicolas Morey-Chaisemartin Wed, 6 Sep 2017 06:48:09 +0000 (08:48 +0200)

pull: honor submodule.recurse config option

"git pull" supports a --recurse-submodules option but does not parse the
submodule.recurse configuration item to set the default for that option.
Meanwhile "git fetch" does support submodule.recurse, producing
confusing behavior: when submodule.recurse is enabled, "git pull"
recursively fetches submodules but does not update them after fetch.

Handle submodule.recurse in "git pull" to fix this.

Reported-by: Magnus Homann <magnus@homann.se>
Signed-off-by: Nicolas Morey-Chaisemartin <nicolas@morey-chaisemartin.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

pull: fix cli and config option parsing orderNicolas Morey-Chaisemartin Wed, 6 Sep 2017 06:48:06 +0000 (08:48 +0200)

pull: fix cli and config option parsing order

pull parses first the cli options and then the config option.
The expected behavior is the other way around, so that config
options can not override the cli ones.

This patch changes the parsing order so config options are
parsed first.

Signed-off-by: Nicolas Morey-Chaisemartin <nicolas@morey-chaisemartin.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

config: remove git_config_maybe_boolMartin Ågren Tue, 5 Sep 2017 18:39:59 +0000 (20:39 +0200)

config: remove git_config_maybe_bool

The function was deprecated in commit 89576613 ("treewide: deprecate
git_config_maybe_bool, use git_parse_maybe_bool", 2017-08-07) and has no
users.

Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

hashmap: add API to disable item counting when threadedJeff Hostetler Wed, 6 Sep 2017 15:43:48 +0000 (15:43 +0000)

hashmap: add API to disable item counting when threaded

This is to address concerns raised by ThreadSanitizer on the mailing list
about threaded unprotected R/W access to map.size with my previous "disallow
rehash" change (0607e10009ee4e37cb49b4cec8d28a9dda1656a4).

See:
https://public-inbox.org/git/adb37b70139fd1e2bac18bfd22c8b96683ae18eb.1502780344.git.martin.agren@gmail.com/

Add API to hashmap to disable item counting and thus automatic rehashing.
Also include API to later re-enable them.

When item counting is disabled, the map.size field is invalid. So to
prevent accidents, the field has been renamed and an accessor function
hashmap_get_size() has been added. All direct references to this
field have been been updated. And the name of the field changed
to map.private_size to communicate this.

Here is the relevant output from ThreadSanitizer showing the problem:

WARNING: ThreadSanitizer: data race (pid=10554)
Read of size 4 at 0x00000082d488 by thread T2 (mutexes: write M16):
#0 hashmap_add hashmap.c:209
#1 hash_dir_entry_with_parent_and_prefix name-hash.c:302
#2 handle_range_dir name-hash.c:347
#3 handle_range_1 name-hash.c:415
#4 lazy_dir_thread_proc name-hash.c:471
#5 <null> <null>

Previous write of size 4 at 0x00000082d488 by thread T1 (mutexes: write M31):
#0 hashmap_add hashmap.c:209
#1 hash_dir_entry_with_parent_and_prefix name-hash.c:302
#2 handle_range_dir name-hash.c:347
#3 handle_range_1 name-hash.c:415
#4 handle_range_dir name-hash.c:380
#5 handle_range_1 name-hash.c:415
#6 lazy_dir_thread_proc name-hash.c:471
#7 <null> <null>

Martin gives instructions for running TSan on test t3008 in this post:
https://public-inbox.org/git/CAN0heSoJDL9pWELD6ciLTmWf-a=oyxe4EXXOmCKvsG5MSuzxsA@mail.gmail.com/

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

git_extract_argv0_path: do nothing without RUNTIME_PREFIXJeff King Wed, 6 Sep 2017 12:32:10 +0000 (08:32 -0400)

git_extract_argv0_path: do nothing without RUNTIME_PREFIX

When the RUNTIME_PREFIX compile-time knob isn't set, we
never look at the argv0_path we extract. We can push its
declaration inside the #ifdef to make it more clear that the
extract code is effectively a noop.

This also un-confuses leak-checking of the argv0_path
variable when RUNTIME_PREFIX isn't set. The compiler is free
to drop this static variable that we set but never look at
(and "gcc -O2" does so). But the compiler still must call
strbuf_detach(), since it doesn't know whether that function
has side effects; it just throws away the result rather than
putting it into the global.

Leak-checkers which work by scanning the data segment for
pointers to heap blocks would normally consider the block
as reachable at program end. But if the compiler removes the
variable entirely, there's nothing to find.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

system_path: move RUNTIME_PREFIX to a sub-functionJeff King Wed, 6 Sep 2017 12:30:28 +0000 (08:30 -0400)

system_path: move RUNTIME_PREFIX to a sub-function

The system_path() function has an #ifdef in the middle of
it. Let's move the conditional logic into a sub-function.
This isolates it more, which will make it easier to change
and add to.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Add t/helper/test-write-cache to .gitignoreJonathan Tan Mon, 28 Aug 2017 20:06:18 +0000 (13:06 -0700)

Add t/helper/test-write-cache to .gitignore

This new binary was introduced in commit 3921a0b ("perf: add test for
writing the index", 2017-08-21), but a .gitignore entry was not added
for it. Add that entry.

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>