gitweb.git
commit-graph: prepare for commit-graph chainsDerrick Stolee Tue, 18 Jun 2019 18:14:24 +0000 (11:14 -0700)

commit-graph: prepare for commit-graph chains

To prepare for a chain of commit-graph files, augment the
commit_graph struct to point to a base commit_graph. As we load
commits from the graph, we may actually want to read from a base
file according to the graph position.

The "graph position" of a commit is given by concatenating the
lexicographic commit orders from each of the commit-graph files in
the chain. This means that we must distinguish two values:

* lexicographic index : the position within the lexicographic
order in a single commit-graph file.

* graph position: the position within the concatenated order
of multiple commit-graph files

Given the lexicographic index of a commit in a graph, we can
compute the graph position by adding the number of commits in
the lower-level graphs. To find the lexicographic index of
a commit, we subtract the number of commits in lower-level graphs.

While here, change insert_parent_or_die() to take a uint32_t
position, as that is the type used by its only caller and that
makes more sense with the limits in the commit-graph format.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

commit-graph: document commit-graph chainsDerrick Stolee Tue, 18 Jun 2019 18:14:23 +0000 (11:14 -0700)

commit-graph: document commit-graph chains

Add a basic description of commit-graph chains. More details about the
feature will be added as we add functionality. This introduction gives a
high-level overview to the goals of the feature and the basic layout of
commit-graph chains.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

packfile: rename close_all_packs to close_object_storeDerrick Stolee Fri, 17 May 2019 18:41:49 +0000 (11:41 -0700)

packfile: rename close_all_packs to close_object_store

The close_all_packs() method is now responsible for more than just pack-files.
It also closes the commit-graph and the multi-pack-index. Rename the function
to be more descriptive of its larger role. The name also fits because the
input parameter is a raw_object_store.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

packfile: close commit-graph in close_all_packsDerrick Stolee Fri, 17 May 2019 18:41:48 +0000 (11:41 -0700)

packfile: close commit-graph in close_all_packs

The close_all_packs() method is used to close all read handles to
pack-files and the multi-pack-index before running 'git gc --auto'.
This is particularly important on the Windows platform, where read
handles block any writes to those files. Replacing one of these
files with a rename() will fail in this situation.

The commit-graph also performs a rename, so is susceptable to this
problem. We are careful to close the commit-graph before writing,
but that doesn't work when a 'git fetch' (or similar) process runs
'git gc --auto' which may write a commit-graph.

Here, close the commit-graph as part of close_all_packs().

Reported-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

commit-graph: use raw_object_store when closingDerrick Stolee Fri, 17 May 2019 18:41:47 +0000 (11:41 -0700)

commit-graph: use raw_object_store when closing

The close_commit_graph() method took a repository struct, but then
only uses the raw_object_store within. Change the function prototype
to make the method more flexible.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

commit-graph: extract write_commit_graph_file()Derrick Stolee Wed, 12 Jun 2019 13:29:45 +0000 (06:29 -0700)

commit-graph: extract write_commit_graph_file()

The write_commit_graph() method is too complex, so we are
extracting helper functions one by one.

Extract write_commit_graph_file() that takes all of the information
in the context struct and writes the data to a commit-graph file.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

commit-graph: extract copy_oids_to_commits()Derrick Stolee Wed, 12 Jun 2019 13:29:44 +0000 (06:29 -0700)

commit-graph: extract copy_oids_to_commits()

The write_commit_graph() method is too complex, so we are
extracting helper functions one by one.

Extract copy_oids_to_commits(), which fills the commits list
with the distinct commits from the oids list. During this loop,
it also counts the number of "extra" edges from octopus merges.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

commit-graph: extract count_distinct_commits()Derrick Stolee Wed, 12 Jun 2019 13:29:43 +0000 (06:29 -0700)

commit-graph: extract count_distinct_commits()

The write_commit_graph() method is too complex, so we are
extracting helper functions one by one.

Extract count_distinct_commits(), which sorts the oids list, then
iterates through to find duplicates.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

commit-graph: extract fill_oids_from_all_packs()Derrick Stolee Wed, 12 Jun 2019 13:29:42 +0000 (06:29 -0700)

commit-graph: extract fill_oids_from_all_packs()

The write_commit_graph() method is too complex, so we are
extracting helper functions one by one.

Extract fill_oids_from_all_packs() that reads all pack-files
for commits and fills the oid list in the context.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

commit-graph: extract fill_oids_from_commit_hex()Derrick Stolee Wed, 12 Jun 2019 13:29:42 +0000 (06:29 -0700)

commit-graph: extract fill_oids_from_commit_hex()

The write_commit_graph() method is too complex, so we are
extracting helper functions one by one.

Extract fill_oids_from_commit_hex() that reads the given commit
id list and fille the oid list in the context.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

commit-graph: extract fill_oids_from_packs()Derrick Stolee Wed, 12 Jun 2019 13:29:41 +0000 (06:29 -0700)

commit-graph: extract fill_oids_from_packs()

The write_commit_graph() method is too complex, so we are
extracting helper functions one by one.

This extracts fill_oids_from_packs() that reads the given
pack-file list and fills the oid list in the context.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

commit-graph: create write_commit_graph_contextDerrick Stolee Wed, 12 Jun 2019 13:29:40 +0000 (06:29 -0700)

commit-graph: create write_commit_graph_context

The write_commit_graph() method is too large and complex. To simplify
it, we should extract several helper functions. However, we will risk
repeating a lot of declarations related to progress incidators and
object id or commit lists.

Create a new write_commit_graph_context struct that contains the
core data structures used in this process. Replace the other local
variables with the values inside the context object. Following this
change, we will start to lift code segments wholesale out of the
write_commit_graph() method and into helper functions.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

commit-graph: remove Future Work sectionDerrick Stolee Wed, 12 Jun 2019 13:29:39 +0000 (06:29 -0700)

commit-graph: remove Future Work section

The commit-graph feature began with a long list of planned
benefits, most of which are now complete. The future work
section has only a few items left.

As for making more algorithms aware of generation numbers,
some are only waiting for generation number v2 to ensure the
performance matches the existing behavior using commit date.

It is unlikely that we will ever send a commit-graph file
as part of the protocol, since we would need to verify the
data, and that is expensive. If we want to start trusting
remote content, then that item can be investigated again.

While there is more work to be done on the feature, having
a section of the docs devoted to a TODO list is wasteful and
hard to keep up-to-date.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

commit-graph: collapse parameters into flagsDerrick Stolee Wed, 12 Jun 2019 13:29:38 +0000 (06:29 -0700)

commit-graph: collapse parameters into flags

The write_commit_graph() and write_commit_graph_reachable() methods
currently take two boolean parameters: 'append' and 'report_progress'.
As we update these methods, adding more parameters this way becomes
cluttered and hard to maintain.

Collapse these parameters into a 'flags' parameter, and adjust the
callers to provide flags as necessary.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

commit-graph: return with errors during writeDerrick Stolee Wed, 12 Jun 2019 13:29:37 +0000 (06:29 -0700)

commit-graph: return with errors during write

The write_commit_graph() method uses die() to report failure and
exit when confronted with an unexpected condition. This use of
die() in a library function is incorrect and is now replaced by
error() statements and an int return type. Return zero on success
and a negative value on failure.

Now that we use 'goto cleanup' to jump to the terminal condition
on an error, we have new paths that could lead to uninitialized
values. New initializers are added to correct for this.

The builtins 'commit-graph', 'gc', and 'commit' call these methods,
so update them to check the return value. Test that 'git commit-graph
write' returns a proper error code when hitting a failure condition
in write_commit_graph().

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

commit-graph: fix the_repository referenceDerrick Stolee Thu, 9 May 2019 14:22:31 +0000 (07:22 -0700)

commit-graph: fix the_repository reference

The parse_commit_buffer() method takes a repository pointer, so it
should not refer to the_repository anymore.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

commit-graph: improve & i18n error messagesÆvar Arnfjörð Bjarmason Mon, 25 Mar 2019 12:08:34 +0000 (13:08 +0100)

commit-graph: improve & i18n error messages

Change the error emitted when a commit-graph file is corrupt so that
we actually mention the commit-graph, e.g. change errors like:

error: improper chunk offset 0000000000385e0c

To:

error: commit-graph improper chunk offset 0000000000385e0c

As discussed in the commits leading up to this one the commit-graph
machinery is now used by common commands like "status". If the graph
was corrupt we'd often emit some error that gave no indication what
was wrong. Now some of them are still cryptic, but they'll at least
mention "commit-graph" to give the user a hint as to where to look.

While I'm at it mark some of the strings that hadn't been marked for
translation. It's clear from the commit history and the code that this
was merely forgotten at the time, and wasn't intentional.p5

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

commit-graph write: don't die if the existing graph... Ævar Arnfjörð Bjarmason Mon, 25 Mar 2019 12:08:33 +0000 (13:08 +0100)

commit-graph write: don't die if the existing graph is corrupt

When the commit-graph is written we end up calling
parse_commit(). This will in turn invoke code that'll consult the
existing commit-graph about the commit, if the graph is corrupted we
die.

We thus get into a state where a failing "commit-graph verify" can't
be followed-up with a "commit-graph write" if core.commitGraph=true is
set, the graph either needs to be manually removed to proceed, or
core.commitGraph needs to be set to "false".

Change the "commit-graph write" codepath to use a new
parse_commit_no_graph() helper instead of parse_commit() to avoid
this. The latter will call repo_parse_commit_internal() with
use_commit_graph=1 as seen in 177722b344 ("commit: integrate commit
graph with commit parsing", 2018-04-10).

Not using the old graph at all slows down the writing of the new graph
by some small amount, but is a sensible way to prevent an error in the
existing commit-graph from spreading.

Just fixing the current issue would be likely to result in code that's
inadvertently broken in the future. New code might use the
commit-graph at a distance. To detect such cases introduce a
"GIT_TEST_COMMIT_GRAPH_DIE_ON_LOAD" setting used when we do our
corruption tests, and test that a "write/verify" combo works after
every one of our current test cases where we now detect commit-graph
corruption.

Some of the code changes here might be strictly unnecessary, e.g. I
was unable to find cases where the parse_commit() called from
write_graph_chunk_data() didn't exit early due to
"item->object.parsed" being true in
repo_parse_commit_internal() (before the use_commit_graph=1 has any
effect). But let's also convert those cases for good measure, we do
not have exhaustive tests for all possible types of commit-graph
corruption.

This might need to be re-visited if we learn to write the commit-graph
incrementally, but probably not. Hopefully we'll just start by finding
out what commits we have in total, then read the old graph(s) to see
what they cover, and finally write a new graph file with everything
that's missing. In that case the new graph writing code just needs to
continue to use e.g. a parse_commit() that doesn't consult the
existing commit-graphs.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

commit-graph verify: detect inability to read the graphÆvar Arnfjörð Bjarmason Mon, 25 Mar 2019 12:08:32 +0000 (13:08 +0100)

commit-graph verify: detect inability to read the graph

Change "commit-graph verify" to error on open() failures other than
ENOENT. As noted in the third paragraph of 283e68c72f ("commit-graph:
add 'verify' subcommand", 2018-06-27) and the test it added it's
intentional that "commit-graph verify" doesn't error out when the file
doesn't exist.

But let's not be overly promiscuous in what we accept. If we can't
read the file for other reasons, e.g. permission errors, bad file
descriptor etc. we'd like to report an error to the user.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

commit-graph: don't pass filename to load_commit_graph_... Ævar Arnfjörð Bjarmason Mon, 25 Mar 2019 12:08:31 +0000 (13:08 +0100)

commit-graph: don't pass filename to load_commit_graph_one_fd_st()

An earlier change implemented load_commit_graph_one_fd_st() in a way
that was bug-compatible with earlier code in terms of the "graph file
%s is too small" error message printing out the path to the
commit-graph (".git/objects/info/commit-graph").

But change that, because:

* A function that takes an already-open file descriptor also needing
the filename isn't very intuitive.

* The vast majority of errors we might emit when loading the graph
come from parse_commit_graph(), which doesn't report the
filename. Let's not do that either in this case for consistency.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

commit-graph: don't early exit(1) on e.g. "git status"Ævar Arnfjörð Bjarmason Mon, 25 Mar 2019 12:08:30 +0000 (13:08 +0100)

commit-graph: don't early exit(1) on e.g. "git status"

Make the commit-graph loading code work as a library that returns an
error code instead of calling exit(1) when the commit-graph is
corrupt. This means that e.g. "status" will now report commit-graph
corruption as an "error: [...]" at the top of its output, but then
proceed to work normally.

This required splitting up the load_commit_graph_one() function so
that the code that deals with open()-ing and stat()-ing the graph can
now be called independently as open_commit_graph().

This is needed because "commit-graph verify" where the graph doesn't
exist isn't an error. See the third paragraph in
283e68c72f ("commit-graph: add 'verify' subcommand",
2018-06-27). There's a bug in that logic where we conflate the
intended ENOENT with other errno values (e.g. EACCES), but this change
doesn't address that. That'll be addressed in a follow-up change.

I'm then splitting most of the logic out of load_commit_graph_one()
into load_commit_graph_one_fd_st(), which allows for providing an
existing file descriptor and stat information to the loading
code. This isn't strictly needed, but it would be redundant and
confusing to open() and stat() the file twice for some of the
codepaths, this allows for calling open_commit_graph() followed by
load_commit_graph_one_fd_st(). The "graph_file" still needs to be
passed to that function for the the "graph file %s is too small" error
message.

This leaves load_commit_graph_one() unused by everything except the
internal prepare_commit_graph_one() function, so let's mark it as
"static". If someone needs it in the future we can remove the "static"
attribute. I could also rewrite its sole remaining
user ("prepare_commit_graph_one()") to use
load_commit_graph_one_fd_st() instead, but let's leave it at this.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Ramsay Jones <ramsay@ramsayjones.plus.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

commit-graph: fix segfault on e.g. "git status"Ævar Arnfjörð Bjarmason Mon, 25 Mar 2019 12:08:29 +0000 (13:08 +0100)

commit-graph: fix segfault on e.g. "git status"

When core.commitGraph=true is set, various common commands now consult
the commit graph. Because the commit-graph code is very trusting of
its input data, it's possibly to construct a graph that'll cause an
immediate segfault on e.g. "status" (and e.g. "log", "blame", ...). In
some other cases where git immediately exits with a cryptic error
about the graph being broken.

The root cause of this is that while the "commit-graph verify"
sub-command exhaustively verifies the graph, other users of the graph
simply trust the graph, and will e.g. deference data found at certain
offsets as pointers, causing segfaults.

This change does the bare minimum to ensure that we don't segfault in
the common fill_commit_in_graph() codepath called by
e.g. setup_revisions(), to do this instrument the "commit-graph
verify" tests to always check if "status" would subsequently
segfault. This fixes the following tests which would previously
segfault:

not ok 50 - detect low chunk count
not ok 51 - detect missing OID fanout chunk
not ok 52 - detect missing OID lookup chunk
not ok 53 - detect missing commit data chunk

Those happened because with the commit-graph enabled setup_revisions()
would eventually call fill_commit_in_graph(), where e.g.
g->chunk_commit_data is used early as an offset (and will be
0x0). With this change we get far enough to detect that the graph is
broken, and show an error instead. E.g.:

$ git status; echo $?
error: commit-graph is missing the Commit Data chunk
1

That also sucks, we should *warn* and not hard-fail "status" just
because the commit-graph is corrupt, but fixing is left to a follow-up
change.

A side-effect of changing the reporting from graph_report() to error()
is that we now have an "error: " prefix for these even for
"commit-graph verify". Pseudo-diff before/after:

$ git commit-graph verify
-commit-graph is missing the Commit Data chunk
+error: commit-graph is missing the Commit Data chunk

Changing that is OK. Various errors it emits now early on are prefixed
with "error: ", moving these over and changing the output doesn't
break anything.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

commit-graph tests: test a graph that's too smallÆvar Arnfjörð Bjarmason Thu, 21 Feb 2019 22:37:47 +0000 (23:37 +0100)

commit-graph tests: test a graph that's too small

Use the recently split-up components of the corrupt_graph_and_verify()
function to assert that we error on graphs that are too small. The
error was added in 2a2e32bdc5 ("commit-graph: implement git
commit-graph read", 2018-04-10), but there was no test for it.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

commit-graph tests: split up corrupt_graph_and_verify()Ævar Arnfjörð Bjarmason Thu, 21 Feb 2019 22:37:46 +0000 (23:37 +0100)

commit-graph tests: split up corrupt_graph_and_verify()

Split up the corrupt_graph_and_verify() function added in
d9b9f8a6fd ("commit-graph: verify catches corrupt signature",
2018-06-27) into its logical components of setting up the test itself,
doing the corruption in a particular way with "dd", and then finally
testing that stderr is what we expect.

This allows for re-using everything except the now slimmer
corrupt_graph_and_verify() to corrupt the graph in a way that doesn't
involve inserting a given byte sequence at a given position,
e.g. truncating it entirely to a custom value.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Git 2.21-rc2 v2.21.0-rc2Junio C Hamano Tue, 19 Feb 2019 21:20:23 +0000 (13:20 -0800)

Git 2.21-rc2

Signed-off-by: Junio C Hamano <gitster@pobox.com>

Merge branch 'js/test-tool-gen-nuls'Junio C Hamano Tue, 19 Feb 2019 21:18:08 +0000 (13:18 -0800)

Merge branch 'js/test-tool-gen-nuls'

* js/test-tool-gen-nuls:
tests: teach the test-tool to generate NUL bytes and use it

Merge branch 'mk/t5562-no-input-to-too-large-an-input... Junio C Hamano Tue, 19 Feb 2019 21:18:08 +0000 (13:18 -0800)

Merge branch 'mk/t5562-no-input-to-too-large-an-input-test'

* mk/t5562-no-input-to-too-large-an-input-test:
t5562: do not depend on /dev/zero
Revert "t5562: replace /dev/zero with a pipe from generate_zero_bytes"

Merge branch 'mk/t5562-do-not-reuse-output-files'Junio C Hamano Tue, 19 Feb 2019 21:18:08 +0000 (13:18 -0800)

Merge branch 'mk/t5562-do-not-reuse-output-files'

* mk/t5562-do-not-reuse-output-files:
t5562: do not reuse output files

t5562: do not reuse output filesMax Kirillov Sat, 24 Nov 2018 09:37:19 +0000 (11:37 +0200)

t5562: do not reuse output files

Some expected failures of git-http-backend leaves running its children
(receive-pack or upload-pack) which still hold opened descriptors
to act.err and with some probability they live long enough to write
there their failure messages after next test has already truncated
the files. This causes occasional failures of the test script.

Avoid the issue by using separated output and error file for each test,
apprending the test number to their name.

Reported-by: Carlo Arenas <carenas@gmail.com>
Helped-by: Carlo Arenas <carenas@gmail.com>
Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Max Kirillov <max@max630.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

tests: teach the test-tool to generate NUL bytes and... Johannes Schindelin Thu, 14 Feb 2019 21:33:12 +0000 (13:33 -0800)

tests: teach the test-tool to generate NUL bytes and use it

In cc95bc2025 (t5562: replace /dev/zero with a pipe from
generate_zero_bytes, 2019-02-09), we replaced usage of /dev/zero (which
is not available on NonStop, apparently) by a Perl script snippet to
generate NUL bytes.

Sadly, it does not seem to work on NonStop, as t5562 reportedly hangs.

Worse, this also hangs in the Ubuntu 16.04 agents of the CI builds on
Azure Pipelines: for some reason, the Perl script snippet that is run
via `generate_zero_bytes` in t5562's 'CONTENT_LENGTH overflow ssite_t'
test case tries to write out an infinite amount of NUL bytes unless a
broken pipe is encountered, that snippet never encounters the broken
pipe, and keeps going until the build times out.

Oddly enough, this does not reproduce on the Windows and macOS agents,
nor in a local Ubuntu 18.04.

This developer tried for a day to figure out the exact circumstances
under which this hang happens, to no avail, the details remain a
mystery.

In the end, though, what counts is that this here change incidentally
fixes that hang (maybe also on NonStop?). Even more positively, it gets
rid of yet another unnecessary Perl invocation.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t5562: do not depend on /dev/zeroMax Kirillov Fri, 15 Feb 2019 16:42:37 +0000 (18:42 +0200)

t5562: do not depend on /dev/zero

It was reported [1] that NonStop platform does not have /dev/zero.

The test uses /dev/zero as a dummy input. Passing case (http-backed
failed because of too big input size) should not be reading anything
from it. If http-backend would erroneously try to read any data
returning EOF probably would be even safer than providing some
meaningless data.

Replace /dev/zero with /dev/null to avoid issues with platforms which do
not have /dev/zero.

[1] https://public-inbox.org/git/20190209185930.5256-4-randall.s.becker@rogers.com/

Reported-by: Randall S. Becker <rsbecker@nexbridge.com>
Signed-off-by: Max Kirillov <max@max630.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Revert "t5562: replace /dev/zero with a pipe from gener... Junio C Hamano Tue, 19 Feb 2019 18:18:15 +0000 (10:18 -0800)

Revert "t5562: replace /dev/zero with a pipe from generate_zero_bytes"

Revert cc95bc20 ("t5562: replace /dev/zero with a pipe from
generate_zero_bytes", 2019-02-09), as not feeding anything to the
command is a better way to test it.

mingw: safe-guard a bit more against getenv() problemsJohannes Schindelin Fri, 15 Feb 2019 15:17:45 +0000 (07:17 -0800)

mingw: safe-guard a bit more against getenv() problems

Running up to v2.21.0, we fixed two bugs that were made prominent by the
Windows-specific change to retain copies of only the 30 latest getenv()
calls' returned strings, invalidating any copies of previous getenv()
calls' return values.

While this really shines a light onto bugs of the form where we hold
onto getenv()'s return values without copying them, it is also a real
problem for users.

And even if Jeff King's patches merged via 773e408881 (Merge branch
'jk/save-getenv-result', 2019-01-29) provide further work on that front,
we are far from done. Just one example: on Windows, we unset environment
variables when spawning new processes, which potentially invalidates
strings that were previously obtained via getenv(), and therefore we
have to duplicate environment values that are somehow involved in
spawning new processes (e.g. GIT_MAN_VIEWER in show_man_page()).

We do not have a chance to investigate, let address, all of those issues
in time for v2.21.0, so let's at least help Windows users by increasing
the number of getenv() calls' return values that are kept valid. The
number 64 was determined by looking at the average number of getenv()
calls per process in the entire test suite run on Windows (which is
around 40) and then adding a bit for good measure. And it is a power of
two (which would have hit yesterday's theme perfectly).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Merge branch 'ea/rebase-compat-doc-fix'Junio C Hamano Thu, 14 Feb 2019 22:28:22 +0000 (14:28 -0800)

Merge branch 'ea/rebase-compat-doc-fix'

* ea/rebase-compat-doc-fix:
docs/git-rebase: remove redundant entry in incompatible options list

Merge branch 'jc/no-grepping-for-strerror-in-tests'Junio C Hamano Thu, 14 Feb 2019 22:28:21 +0000 (14:28 -0800)

Merge branch 'jc/no-grepping-for-strerror-in-tests'

* jc/no-grepping-for-strerror-in-tests:
t1404: do not rely on the exact phrasing of strerror()

Merge branch 'jt/fetch-v2-sideband'Junio C Hamano Thu, 14 Feb 2019 22:28:20 +0000 (14:28 -0800)

Merge branch 'jt/fetch-v2-sideband'

"git fetch" and "git upload-pack" learned to send all exchange over
the sideband channel while talking the v2 protocol.

* jt/fetch-v2-sideband:
t/lib-httpd: pass GIT_TEST_SIDEBAND_ALL through Apache

Merge branch 'en/rebase-merge-on-sequencer'Junio C Hamano Thu, 14 Feb 2019 22:28:20 +0000 (14:28 -0800)

Merge branch 'en/rebase-merge-on-sequencer'

"git rebase --merge" as been reimplemented by reusing the internal
machinery used for "git rebase -i".

* en/rebase-merge-on-sequencer:
git-rebase.txt: update to reflect merge now implemented on sequencer

git-rebase.txt: update to reflect merge now implemented... Elijah Newren Thu, 14 Feb 2019 20:25:41 +0000 (12:25 -0800)

git-rebase.txt: update to reflect merge now implemented on sequencer

Since commit 8fe9c3f21dff (Merge branch 'en/rebase-merge-on-sequencer',
2019-02-06), --merge now uses the interactive backend (and matches its
behavior) so there is no separate merge backend anymore. Fix an
oversight in the docs that should have been updated with the previous
change.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t/lib-httpd: pass GIT_TEST_SIDEBAND_ALL through ApacheTodd Zullinger Thu, 14 Feb 2019 06:35:13 +0000 (01:35 -0500)

t/lib-httpd: pass GIT_TEST_SIDEBAND_ALL through Apache

07c3c2aa16 ("tests: define GIT_TEST_SIDEBAND_ALL", 2019-01-16) added
GIT_TEST_SIDEBAND_ALL to the apache.conf PassEnv list. Avoid warnings
from Apache when the variable is unset, as we do for GIT_VALGRIND* and
GIT_TRACE, from f628825481 ("t/lib-httpd: handle running under
--valgrind", 2012-07-24) and 89c57ab3f0 ("t: pass GIT_TRACE through
Apache", 2015-03-13), respectively.

Signed-off-by: Todd Zullinger <tmz@pobox.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t1404: do not rely on the exact phrasing of strerror()Junio C Hamano Thu, 14 Feb 2019 20:16:20 +0000 (12:16 -0800)

t1404: do not rely on the exact phrasing of strerror()

Not even in C locale, it is wrong to expect that the exact phrasing
"File exists" is used to show EEXIST.

Reported-by: Randall S. Becker <rsbecker@nexbridge.com>
Helped-by: Duy Nguyen <pclouds@gmail.com>
Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

docs/git-rebase: remove redundant entry in incompatible... Emilio Cobos Álvarez Wed, 13 Feb 2019 23:44:33 +0000 (00:44 +0100)

docs/git-rebase: remove redundant entry in incompatible options list

The --autosquash option is implied by the earlier --[no-]autosquash
entry in the list.

Signed-off-by: Emilio Cobos Álvarez <emilio@crisal.io>
Reviewed-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Git 2.21-rc1 v2.21.0-rc1Junio C Hamano Thu, 14 Feb 2019 02:18:11 +0000 (18:18 -0800)

Git 2.21-rc1

Signed-off-by: Junio C Hamano <gitster@pobox.com>

Merge branch 'ab/rebase-test-fix'Junio C Hamano Thu, 14 Feb 2019 02:18:43 +0000 (18:18 -0800)

Merge branch 'ab/rebase-test-fix'

* ab/rebase-test-fix:
rebase: fix regression in rebase.useBuiltin=false test mode

Merge branch 'rb/no-dev-zero-in-test'Junio C Hamano Thu, 14 Feb 2019 02:18:43 +0000 (18:18 -0800)

Merge branch 'rb/no-dev-zero-in-test'

* rb/no-dev-zero-in-test:
t5562: replace /dev/zero with a pipe from generate_zero_bytes
t5318: replace use of /dev/zero with generate_zero_bytes
test-lib-functions.sh: add generate_zero_bytes function

Merge branch 'rs/bash-is-in-coreutils-on-nonstop'Junio C Hamano Thu, 14 Feb 2019 02:18:43 +0000 (18:18 -0800)

Merge branch 'rs/bash-is-in-coreutils-on-nonstop'

* rs/bash-is-in-coreutils-on-nonstop:
config.mak.uname: move location of bash on NonStop to CoreUtils

Merge branch 'js/mingw-host-cpu'Junio C Hamano Thu, 14 Feb 2019 02:18:43 +0000 (18:18 -0800)

Merge branch 'js/mingw-host-cpu'

Windows update.

* js/mingw-host-cpu:
mingw: use a more canonical method to fix the CPU reporting

Merge branch 'sg/stress-test'Junio C Hamano Thu, 14 Feb 2019 02:18:42 +0000 (18:18 -0800)

Merge branch 'sg/stress-test'

Test improvement.

* sg/stress-test:
test-lib: fix non-portable pattern bracket expressions
test-lib: make '--stress' more bisect-friendly

Merge branch 'kd/t0028-octal-del-is-377-not-777'Junio C Hamano Thu, 14 Feb 2019 02:18:42 +0000 (18:18 -0800)

Merge branch 'kd/t0028-octal-del-is-377-not-777'

Test fix.

* kd/t0028-octal-del-is-377-not-777:
t0028: fix wrong octal values for BOM in setup

Merge branch 'bc/utf16-portability-fix'Junio C Hamano Thu, 14 Feb 2019 02:18:41 +0000 (18:18 -0800)

Merge branch 'bc/utf16-portability-fix'

The code and tests assume that the system supplied iconv() would
always use BOM in its output when asked to encode to UTF-16 (or
UTF-32), but apparently some implementations output big-endian
without BOM. A compile-time knob has been added to help such
systems (e.g. NonStop) to add BOM to the output to increase
portability.

* bc/utf16-portability-fix:
utf8: handle systems that don't write BOM for UTF-16

Merge branch 'nd/fileno-may-be-macro'Junio C Hamano Thu, 14 Feb 2019 02:18:41 +0000 (18:18 -0800)

Merge branch 'nd/fileno-may-be-macro'

* nd/fileno-may-be-macro:
git-compat-util: work around fileno(fp) that is a macro

Merge branch 'nd/get-oid-with-context-returns-an-enum'Junio C Hamano Thu, 14 Feb 2019 02:18:41 +0000 (18:18 -0800)

Merge branch 'nd/get-oid-with-context-returns-an-enum'

* nd/get-oid-with-context-returns-an-enum:
get_oid_with_context(): match prototype and implementation

Merge branch 'rj/sequencer-sign-off-header-static'Junio C Hamano Thu, 14 Feb 2019 02:18:41 +0000 (18:18 -0800)

Merge branch 'rj/sequencer-sign-off-header-static'

Code clean-up.

* rj/sequencer-sign-off-header-static:
sequencer: make sign_off_header a file local symbol

rebase: fix regression in rebase.useBuiltin=false test... Ævar Arnfjörð Bjarmason Wed, 13 Feb 2019 21:49:08 +0000 (22:49 +0100)

rebase: fix regression in rebase.useBuiltin=false test mode

Fix a recently introduced regression in c762aada1a ("rebase -x: sanity
check command", 2019-01-29) triggered when running the tests with
GIT_TEST_REBASE_USE_BUILTIN=false. See 62c23938fa ("tests: add a
special setup where rebase.useBuiltin is off", 2018-11-14) for how
that test mode works.

As discussed on-list[1] it's not worth it to implement the sanity
check in the legacy rebase code, we plan to remove it after the 2.21
release. So let's do the bare minimum to make the tests pass under the
GIT_TEST_REBASE_USE_BUILTIN=false special setup.

1. https://public-inbox.org/git/xmqqva1nbeno.fsf@gitster-ct.c.googlers.com/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

mingw: use a more canonical method to fix the CPU reportingJohannes Schindelin Wed, 13 Feb 2019 10:19:49 +0000 (02:19 -0800)

mingw: use a more canonical method to fix the CPU reporting

In `git version --build-options`, we report also the CPU, but in Git for
Windows we actually cross-compile the 32-bit version in a 64-bit Git for
Windows, so we cannot rely on the auto-detected value.

In 3815f64b0dd9 (mingw: fix CPU reporting in `git version
--build-options`, 2019-02-07), we fixed this by a Windows-only
workaround, making use of magic pre-processor constants, which works in
GCC, but most likely not all C compilers.

As pointed out by Eric Sunshine, there is a better way, anyway: to set
the Makefile variable HOST_CPU explicitly for cross-compiled Git. So
let's do that!

This reverts commit 3815f64b0dd983bdbf9242a0547706d5d81cb3e6 partially.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

config.mak.uname: move location of bash on NonStop... Randall S. Becker Sat, 9 Feb 2019 17:26:11 +0000 (12:26 -0500)

config.mak.uname: move location of bash on NonStop to CoreUtils

The default bash is now officially in /usr/coreutils/bin instead
of in /usr/local/bin. This version of bash is more stable and
recommended for all use as of the J06.22 and L18.02 operating
system revision levels. This new version provides more stability
of test results.

Signed-off-by: Randall S. Becker <rsbecker@nexbridge.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t5562: replace /dev/zero with a pipe from generate_zero... Randall S. Becker Sat, 9 Feb 2019 18:59:30 +0000 (13:59 -0500)

t5562: replace /dev/zero with a pipe from generate_zero_bytes

To help platforms that lack /dev/zero (e.g. NonStop), replace use
of /dev/zero to feed "git http-backend" with a pipe of output from
the generate_zero_bytes helper.

Signed-off-by: Randall S. Becker <rsbecker@nexbridge.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t5318: replace use of /dev/zero with generate_zero_bytesRandall S. Becker Sat, 9 Feb 2019 18:59:29 +0000 (13:59 -0500)

t5318: replace use of /dev/zero with generate_zero_bytes

There are platforms (e.g. NonStop) that lack /dev/zero; use the
generate_zero_bytes helper we just introduced to append stream
of NULs at the end of the file.

The original, even though it uses "dd seek=... count=..." to make it
look like it is overwriting the middle part of an existing file, has
truncated the file before this step with another use of "dd", which
may make it tricky to see why this rewrite is a correct one.

Signed-off-by: Randall S. Becker <rsbecker@nexbridge.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

get_oid_with_context(): match prototype and implementationDuy Nguyen Tue, 12 Feb 2019 12:43:23 +0000 (19:43 +0700)

get_oid_with_context(): match prototype and implementation

The get_oid_with_context() function is declared to return an enum in
cache.h, but defined to return an int in sha1-name.c. The compiler
notices this on AIX and rejects the build, since d1dd94b308 (Do not
print 'dangling' for cat-file in case of ambiguity - 2019-01-17) was
merged.

Return the correct type from the implementation to fix this.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

git-compat-util: work around fileno(fp) that is a macroDuy Nguyen Tue, 12 Feb 2019 14:14:41 +0000 (21:14 +0700)

git-compat-util: work around fileno(fp) that is a macro

On various BSD's, fileno(fp) is implemented as a macro that directly
accesses the fields in the FILE * object, which breaks a function that
accepts a "void *fp" parameter and calls fileno(fp) and expect it to
work.

Work it around by adding a compile-time knob FILENO_IS_A_MACRO that
inserts a real helper function in the middle of the callchain.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

test-lib-functions.sh: add generate_zero_bytes functionRandall S. Becker Sat, 9 Feb 2019 18:59:28 +0000 (13:59 -0500)

test-lib-functions.sh: add generate_zero_bytes function

t5318 and t5562 used /dev/zero, which is not portable. This function
provides both a fixed block of NUL bytes and an infinite stream of NULs.

Signed-off-by: Randall S. Becker <rsbecker@nexbridge.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

RelNotes/2.21: misc typo/English fixupsJeff King Tue, 12 Feb 2019 06:33:04 +0000 (01:33 -0500)

RelNotes/2.21: misc typo/English fixups

These are just some small fixes I noticed doing a complete read-through
(there are a few cases I left that are incomplete or abbreviated
sentences, but I think those are OK in this sort of bullet-list style).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

RelNotes/2.21: tweak "--date=auto" mentionJeff King Tue, 12 Feb 2019 06:32:48 +0000 (01:32 -0500)

RelNotes/2.21: tweak "--date=auto" mention

In the feature that was eventually committed, "--date=auto" doesn't do
anything. It was generalized to "--date=auto:<format>".

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Merge branch 'nd/imap-send-typofix'Junio C Hamano Tue, 12 Feb 2019 17:00:25 +0000 (09:00 -0800)

Merge branch 'nd/imap-send-typofix'

* nd/imap-send-typofix:
imap-send.c: add a missing space in error message

imap-send.c: add a missing space in error messageNguyễn Thái Ngọc Duy Mon, 11 Feb 2019 09:40:11 +0000 (16:40 +0700)

imap-send.c: add a missing space in error message

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

utf8: handle systems that don't write BOM for UTF-16brian m. carlson Tue, 12 Feb 2019 00:52:06 +0000 (00:52 +0000)

utf8: handle systems that don't write BOM for UTF-16

When serializing UTF-16 (and UTF-32), there are three possible ways to
write the stream. One can write the data with a BOM in either big-endian
or little-endian format, or one can write the data without a BOM in
big-endian format.

Most systems' iconv implementations choose to write it with a BOM in
some endianness, since this is the most foolproof, and it is resistant
to misinterpretation on Windows, where UTF-16 and the little-endian
serialization are very common. For compatibility with Windows and to
avoid accidental misuse there, Git always wants to write UTF-16 with a
BOM, and will refuse to read UTF-16 without it.

However, musl's iconv implementation writes UTF-16 without a BOM,
relying on the user to interpret it as big-endian. This causes t0028 and
the related functionality to fail, since Git won't read the file without
a BOM.

Add a Makefile and #define knob, ICONV_OMITS_BOM, that can be set if the
iconv implementation has this behavior. When set, Git will write a BOM
manually for UTF-16 and UTF-32 and then force the data to be written in
UTF-16BE or UTF-32BE. We choose big-endian behavior here because the
tests use the raw "UTF-16" encoding, which will be big-endian when the
implementation requires this knob to be set.

Update the tests to detect this case and write test data with an added
BOM if necessary. Always write the BOM in the tests in big-endian
format, since all iconv implementations that omit a BOM must use
big-endian serialization according to the Unicode standard.

Preserve the existing behavior for systems which do not have this knob
enabled, since they may use optimized implementations, including
defaulting to the native endianness, which may improve performance.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t0028: fix wrong octal values for BOM in setupKevin Daudt Mon, 11 Feb 2019 21:38:18 +0000 (22:38 +0100)

t0028: fix wrong octal values for BOM in setup

The setup code uses octal values with printf to generate a BOM for
UTF-16/32 BE/LE. It specifically uses '\777' to emit a 0xff byte. This
relies on the fact that most shells truncate the value above 0o377.

Ash however interprets '\777' as '\77' + a literal '7', resulting in an
invalid BOM.

Fix this by using the proper value of 0xff: '\377'.

Signed-off-by: Kevin Daudt <me@ikke.info>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

test-lib: fix non-portable pattern bracket expressionsSZEDER Gábor Mon, 11 Feb 2019 19:58:03 +0000 (20:58 +0100)

test-lib: fix non-portable pattern bracket expressions

Use a '!' character to start a non-matching pattern bracket
expression, as specified by POSIX in Shell Command Language section
2.13.1 Patterns Matching a Single Character [1].

I used '^' instead in three places in the previous three commits, to
verify that the arguments of the '--stress=' and '--stress-limit='
options and the values of various '*_PORT' environment variables are
valid numbers. With certain shells, at least with dash (upstream and
in Ubuntu 14.04) and mksh, this led to various undesired behaviors:

# error message in case of a valid number
$ ~/src/dash/src/dash ./t3903-stash.sh --stress=8
error: --stress=<N> requires the number of jobs to run

# not the expected error message
$ ~/src/dash/src/dash ./t3903-stash.sh --stress=foo
./t3903-stash.sh: 238: test: Illegal number: foo

# no error message at all?!
$ mksh ./t3903-stash.sh --stress=foo
$ echo $?
0

Some other shells, e.g. Bash (even in posix mode), ksh, dash in Ubuntu
16.04 or later, are apparently happy to accept '^' just as well.

[1] http://pubs.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html#tag_02_13

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

sequencer: make sign_off_header a file local symbolRamsay Jones Mon, 11 Feb 2019 17:16:58 +0000 (17:16 +0000)

sequencer: make sign_off_header a file local symbol

Commit d0aaa46fd3 ("commit: move empty message checks to libgit",
2017-11-10) removes the last use of 'sign_off_header' outside of
the "sequencer.c" source file. Remove the extern declaration from
the header file and mark the definition of the symbol with the
static keyword.

Signed-off-by: Ramsay Jones <ramsay@ramsayjones.plus.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

config.mak.uname: add FREAD_READS_DIRECTORIES for NonSt... Randall S. Becker Sun, 10 Feb 2019 00:20:16 +0000 (19:20 -0500)

config.mak.uname: add FREAD_READS_DIRECTORIES for NonStop platform

The NonStop platform needs this configuration item specified as
UnfortunatelyYes so that config directory files are correctly processed.

Signed-off-by: Randall S. Becker <rsbecker@nexbridge.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Fix typos in translatable strings for v2.21.0Jean-Noël Avila Mon, 11 Feb 2019 06:44:53 +0000 (07:44 +0100)

Fix typos in translatable strings for v2.21.0

Signed-off-by: Jean-Noël Avila <jn.avila@free.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Seventh batch for 2.21Junio C Hamano Sat, 9 Feb 2019 04:45:48 +0000 (20:45 -0800)

Seventh batch for 2.21

Signed-off-by: Junio C Hamano <gitster@pobox.com>

Merge branch 'js/mingw-host-cpu'Junio C Hamano Sat, 9 Feb 2019 04:44:53 +0000 (20:44 -0800)

Merge branch 'js/mingw-host-cpu'

Windows update.

* js/mingw-host-cpu:
mingw: fix CPU reporting in `git version --build-options`

Merge branch 'js/fuzz-commit-graph-update'Junio C Hamano Sat, 9 Feb 2019 04:44:53 +0000 (20:44 -0800)

Merge branch 'js/fuzz-commit-graph-update'

Update to the fuzzer.

* js/fuzz-commit-graph-update:
object: fix leak of shallow_stat
fuzz-commit-graph: initialize repo object

Merge branch 'kl/pretty-doc-markup-fix'Junio C Hamano Sat, 9 Feb 2019 04:44:52 +0000 (20:44 -0800)

Merge branch 'kl/pretty-doc-markup-fix'

Doc update.

* kl/pretty-doc-markup-fix:
doc: prevent overflowing <code> tag in rendered HTML

Merge branch 'sg/ci-parallel-build'Junio C Hamano Sat, 9 Feb 2019 04:44:52 +0000 (20:44 -0800)

Merge branch 'sg/ci-parallel-build'

Build update.

* sg/ci-parallel-build:
ci: clear and mark MAKEFLAGS exported just once
ci: make sure we build Git parallel

Merge branch 'ld/git-p4-remove-flakey-test'Junio C Hamano Sat, 9 Feb 2019 04:44:52 +0000 (20:44 -0800)

Merge branch 'ld/git-p4-remove-flakey-test'

A flakey "p4" test has been removed.

* ld/git-p4-remove-flakey-test:
git-p4: remove ticket expiry test

Merge branch 'js/rebase-i-redo-exec-fix'Junio C Hamano Sat, 9 Feb 2019 04:44:52 +0000 (20:44 -0800)

Merge branch 'js/rebase-i-redo-exec-fix'

For "rebase -i --reschedule-failed-exec", we do not want the "-y"
shortcut after all.

* js/rebase-i-redo-exec-fix:
Revert "rebase: introduce a shortcut for --reschedule-failed-exec"

Merge branch 'nd/checkout-noisy-unmerge'Junio C Hamano Sat, 9 Feb 2019 04:44:51 +0000 (20:44 -0800)

Merge branch 'nd/checkout-noisy-unmerge'

"git checkout [<tree-ish>] <pathspec>" started reporting the number
of paths that have got updated recently, but the same messages were
given when "git checkout -m <pathspec>" to unresolve conflicts that
have just been resolved. The message now reports these unresolved
paths separately from the paths that are checked out from the index.

* nd/checkout-noisy-unmerge:
checkout: count and print -m paths separately
checkout: update count-checkouts messages

Merge branch 'js/smart-http-detect-remote-error'Junio C Hamano Sat, 9 Feb 2019 04:44:51 +0000 (20:44 -0800)

Merge branch 'js/smart-http-detect-remote-error'

Some errors from the other side coming over smart HTTP transport
were not noticed, which has been corrected.

* js/smart-http-detect-remote-error:
t5551: test server-side ERR packet
remote-curl: tighten "version 2" check for smart-http
remote-curl: refactor smart-http discovery

Merge branch 'ds/coverage-prove'Junio C Hamano Sat, 9 Feb 2019 04:44:51 +0000 (20:44 -0800)

Merge branch 'ds/coverage-prove'

A new target "coverage-prove" to run the coverage test under
"prove" has been added.

* ds/coverage-prove:
Makefile: add coverage-prove target

Merge branch 'tz/gpg-test-fix'Junio C Hamano Sat, 9 Feb 2019 04:44:50 +0000 (20:44 -0800)

Merge branch 'tz/gpg-test-fix'

Test fix.

* tz/gpg-test-fix:
t/lib-gpg: drop redundant killing of gpg-agent
t/lib-gpg: quote path to ${GNUPGHOME}/trustlist.txt

Merge branch 'os/rebase-runs-post-checkout-hook'Junio C Hamano Sat, 9 Feb 2019 04:44:50 +0000 (20:44 -0800)

Merge branch 'os/rebase-runs-post-checkout-hook'

Test fix.

* os/rebase-runs-post-checkout-hook:
t5403: correct bash ambiguous redirect error in subtest 8 by quoting $GIT_DIR

test-lib: make '--stress' more bisect-friendlySZEDER Gábor Fri, 8 Feb 2019 11:50:45 +0000 (12:50 +0100)

test-lib: make '--stress' more bisect-friendly

Let's suppose that a test somehow becomes flaky between 'master' and
'pu', and tends to fail within the first 50 repetitions when run with
'--stress'. In such a case we could use 'git bisect' to find the
culprit: if the test script fails with '--stress', then the commit is
definitely bad, but if it survives, say, 300 repetitions, then we could
consider it good with reasonable confidence.

Unfortunately, all this could only be done manually, because
'--stress' would run the test script repeatedly for all eternity on a
good commit, and it would exit with success even when it found a
failure on a bad commit.

So let's make '--stress' usable with 'git bisect run':

- Make it exit with failure if a failure is found.

- Add the '--stress-limit=<N>' option to repeat the test script
at most N times in each of the parallel jobs, and exit with
success when the limit is reached.

And then we could simply run something like:

$ git bisect start origin/pu master
$ git bisect run sh -c 'make && cd t &&
./t1234-foo.sh --stress --stress-limit=300'

Sure, as a brand new feature it won't be any useful right now, but in
a release or three most cooking topics will already contain this, so
we could automatically bisect at least newly introduced flakiness.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t5403: correct bash ambiguous redirect error in subtest... Randall S. Becker Fri, 8 Feb 2019 11:32:33 +0000 (06:32 -0500)

t5403: correct bash ambiguous redirect error in subtest 8 by quoting $GIT_DIR

The embedded blanks in the full path of the test git repository cased bash
to generate an ambugious redirect error.

Signed-off-by: Randall S. Becker <rsbecker@nexbridge.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

.mailmap: map Clemens Buchacher's mail addressesJohannes Schindelin Fri, 8 Feb 2019 10:01:20 +0000 (02:01 -0800)

.mailmap: map Clemens Buchacher's mail addresses

We have three email addresses for Clemens in our commit history, two of
them bouncing. Let's map the latter to the only one that still works.

Pointed out by Gábor Szeder.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t/lib-gpg: drop redundant killing of gpg-agentTodd Zullinger Fri, 8 Feb 2019 03:17:46 +0000 (22:17 -0500)

t/lib-gpg: drop redundant killing of gpg-agent

In 53fc999306 ("gpg-interface t: extend the existing GPG tests with
GPGSM", 2018-07-20), the gpgconf call which kills gpg-agent was copied
from the existing gpg setup code.

The reason for killing gpg-agent is given in 29ff1f8f74 ("t: lib-gpg:
flush gpg agent on startup", 2017-07-20):

When running gpg-relevant tests, a gpg-daemon is spawned for each
GNUPGHOME used. This daemon may stay running after the test and cache
file descriptors for the trash directories, even after the trash
directory is removed. This leads to ENOENT errors when attempting to
create files if tests are run multiple times.

Add a cleanup script to force flushing the gpg-agent for that GNUPGHOME
(if any) before setting up the GPG relevant-environment.

Killing gpg-agent once per test is sufficient.

Signed-off-by: Todd Zullinger <tmz@pobox.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t/lib-gpg: quote path to ${GNUPGHOME}/trustlist.txtTodd Zullinger Fri, 8 Feb 2019 03:17:45 +0000 (22:17 -0500)

t/lib-gpg: quote path to ${GNUPGHOME}/trustlist.txt

When gpgsm is installed, lib-gpg.sh attempts to update trustlist.txt to
relax the checking of some root certificate requirements. The path to
"${GNUPGHOME}" contains spaces which cause an "ambiguous redirect"
warning when bash is used to run the tests:

$ bash t7030-verify-tag.sh
/git/t/lib-gpg.sh: line 66: ${GNUPGHOME}/trustlist.txt: ambiguous redirect
ok 1 - create signed tags
ok 2 # skip create signed tags x509 (missing GPGSM)
...

No warning is issued when using bash called as /bin/sh, dash, or mksh.

Quote the path to ensure the redirect works as intended and sets the
GPGSM prereq. While we're here, drop the space after ">>".

Signed-off-by: Todd Zullinger <tmz@pobox.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

mingw: fix CPU reporting in `git version --build-options`Johannes Schindelin Thu, 7 Feb 2019 10:46:06 +0000 (02:46 -0800)

mingw: fix CPU reporting in `git version --build-options`

We cannot rely on `uname -m` in Git for Windows' SDK to tell us what
architecture we are compiling for, as we can compile both 32-bit and
64-bit `git.exe` from a 64-bit SDK, but the `uname -m` in that SDK will
always report `x86_64`.

So let's go back to our original design. And make it explicitly
Windows-specific.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

object: fix leak of shallow_statJosh Steadmon Thu, 7 Feb 2019 20:05:54 +0000 (12:05 -0800)

object: fix leak of shallow_stat

In eee4502baaf ("shallow: migrate shallow information into the object
parser", 2018-05-17), we added a stat_validity pointer into the
parsed_object_pool struct, but did not add code to free this in
parsed_object_pool_clear(). This leak was found by fuzz-commit-graph.

Clear the struct and then free it in parsed_object_pool_clear() to
prevent the leak.

Signed-off-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

fuzz-commit-graph: initialize repo objectJosh Steadmon Thu, 7 Feb 2019 20:05:53 +0000 (12:05 -0800)

fuzz-commit-graph: initialize repo object

Various #DEFINE "constants" in commit-graph.c now depend on
the_hash_algo->rawsz, but this object must be initialized before it can
be used.

Signed-off-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

doc: prevent overflowing <code> tag in rendered HTMLKatrin Leinweber Thu, 7 Feb 2019 19:16:37 +0000 (20:16 +0100)

doc: prevent overflowing <code> tag in rendered HTML

Add an apparently missing back-tick to fix a multi-line <code> section
on https://git-scm.com/docs/git-log which seems to have been caused by
commit 18fb7ffc ("pretty: respect color settings [...]", 2017-07-13).

Signed-off-by: Katrin Leinweber <katrin.leinweber@uni-konstanz.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

ci: clear and mark MAKEFLAGS exported just onceJunio C Hamano Thu, 7 Feb 2019 19:36:28 +0000 (11:36 -0800)

ci: clear and mark MAKEFLAGS exported just once

Clearing it once upfront, and turning all the assignment into
appending, would future-proof the code even more, to prevent
mistakes the previous one fixed from happening again.

Also, mark the variable exported just once at the beginning. There
is no point in marking it exported repeatedly.

Signed-off-by: Junio C Hamano <gitster@pobox.com>

ci: make sure we build Git parallelSZEDER Gábor Thu, 7 Feb 2019 18:37:36 +0000 (19:37 +0100)

ci: make sure we build Git parallel

Commit 2c8921db2b (travis-ci: build with the right compiler,
2019-01-17) started to use MAKEFLAGS to specify which compiler to use
to build Git. A bit later, and in a different topic branch commit
eaa62291ff (ci: inherit --jobs via MAKEFLAGS in run-build-and-tests,
2019-01-27) started to use MAKEFLAGS as well. Unfortunately, there is
a semantic conflict between these two commits: both of them set
MAKEFLAGS, and since the line adding CC from 2c8921db2b comes later in
'ci/lib.sh', it overwrites the number of parallel jobs added in
eaa62291ff.

Consequently, since both commits have been merged all our CI jobs have
been building Git, building its documentation, and applying semantic
patches sequentially, making all build jobs a bit slower. Running
the test suite is unaffected, because the number of test jobs comes
from GIT_PROVE_OPTS.

Append to MAKEFLAGS when setting the compiler to use, to ensure that
the number of parallel jobs to use is preserved.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Git 2.21-rc0 v2.21.0-rc0Junio C Hamano Thu, 7 Feb 2019 05:30:09 +0000 (21:30 -0800)

Git 2.21-rc0

Signed-off-by: Junio C Hamano <gitster@pobox.com>

Merge branch 'jk/add-ignore-errors-bit-assignment-fix'Junio C Hamano Thu, 7 Feb 2019 06:05:30 +0000 (22:05 -0800)

Merge branch 'jk/add-ignore-errors-bit-assignment-fix'

A hotfix to an incomplete fix made earlier.

* jk/add-ignore-errors-bit-assignment-fix:
add_to_index(): convert forgotten HASH_RENORMALIZE check

Merge branch 'rj/sparse-flags'Junio C Hamano Thu, 7 Feb 2019 06:05:30 +0000 (22:05 -0800)

Merge branch 'rj/sparse-flags'

Use of the sparse tool got easier to customize from the command
line to help developers.

* rj/sparse-flags:
Makefile: improve SPARSE_FLAGS customisation
config.mak.uname: remove obsolete SPARSE_FLAGS setting

Merge branch 'bc/fetch-pack-clear-alternate-shallow'Junio C Hamano Thu, 7 Feb 2019 06:05:30 +0000 (22:05 -0800)

Merge branch 'bc/fetch-pack-clear-alternate-shallow'

"git fetch" over protocol v2 that needs to make a second connection
to backfill tags did not clear a variable that holds shallow
repository information correctly, leading to an access of freed
piece of memory.

* bc/fetch-pack-clear-alternate-shallow:
fetch-pack: clear alternate shallow in one more place
fetch-pack: clear alternate shallow when complete

Merge branch 'ma/doc-diff-usage-fix'Junio C Hamano Thu, 7 Feb 2019 06:05:29 +0000 (22:05 -0800)

Merge branch 'ma/doc-diff-usage-fix'

Running "Documentation/doc-diff x" from anywhere other than the
top-level of the working tree did not show the usage string
correctly, which has been fixed.

* ma/doc-diff-usage-fix:
doc-diff: don't `cd_to_toplevel`

Merge branch 'ab/diff-tree-doc-fix'Junio C Hamano Thu, 7 Feb 2019 06:05:29 +0000 (22:05 -0800)

Merge branch 'ab/diff-tree-doc-fix'

Doc fix.

* ab/diff-tree-doc-fix:
diff-tree doc: correct & remove wrong documentation

Merge branch 'ss/describe-dirty-in-the-right-directory'Junio C Hamano Thu, 7 Feb 2019 06:05:29 +0000 (22:05 -0800)

Merge branch 'ss/describe-dirty-in-the-right-directory'

"git --work-tree=$there --git-dir=$here describe --dirty" did not
work correctly as it did not pay attention to the location of the
worktree specified by the user by mistake, which has been
corrected.

* ss/describe-dirty-in-the-right-directory:
t6120: test for describe with a bare repository
describe: setup working tree for --dirty