perf-lib.sh: forbid the use of GIT_TEST_INSTALLED

As noted in preceding commits setting GIT_TEST_INSTALLED has never
been supported or documented, and as noted in an earlier t/perf/README
change to the extent that it's been documented nobody's notices that
the example hasn't worked since 3c8f12c96c ("test-lib: reorder and
include GIT-BUILD-OPTIONS a lot earlier", 2012-06-24).

We could directly support GIT_TEST_INSTALLED for invocations without
the "run" script, such as:

GIT_TEST_INSTALLED=../../ ./p0000-perf-lib-sanity.sh
GIT_TEST_INSTALLED=/home/avar/g/git ./p0000-perf-lib-sanity.sh

But while not having this "error" will "work", it won't write the the
resulting "test-results/*" files to the right place, and thus a
subsequent call to aggregate.perl won't work as expected.

Let's just tell the user that they need to use the "run" script,
which'll correctly deal with this and set the right
PERF_RESULTS_PREFIX.

If someone's in desperate need of bypassing "run" for whatever reason
they can trivially do so by setting "PERF_SET_GIT_TEST_INSTALLED", but
not we won't have people who expect GIT_TEST_INSTALLED to just work
wondering why their aggregation doesn't work, even though they're
running the right "git".

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>

perf tests: add "bindir" prefix to git tree test results

Change the output file names in test-results/ to be
"test-results/bindir_<munged dir>" rather than just
"test-results/<munged dir>".

This is for consistency with the "build_" directories we have for
built revisions, i.e. "test-results/build_<SHA-1>".

There's no user-visible functional changes here, it just makes it
easier to see at a glance what "test-results" files are of what "type"
as they're all explicitly grouped together now, and to grep this code
to find both the run_dirs_helper() implementation and its
corresponding aggregate.perl code.

Note that we already guarantee that the rest of the
PERF_RESULTS_PREFIX is an absolute path, and since it'll start with
e.g. "/" which we munge to "_" we'll up with a readable string like
"bindir_home_avar[...]".

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>

perf-lib.sh: remove GIT_TEST_INSTALLED from perf-lib.sh

Follow-up my preceding change which fixed the immediate "./run
<revisions>" regression in 0baf78e7bc ("perf-lib.sh: rely on
test-lib.sh for --tee handling", 2019-03-15) and entirely get rid of
GIT_TEST_INSTALLED from perf-lib.sh (and aggregate.perl).

As noted in that change the dance we're doing with GIT_TEST_INSTALLED
perf-lib.sh isn't necessary, but there I was doing the most minimal
set of changes to quickly fix a regression.

But it's much simpler to never deal with the "GIT_TEST_INSTALLED" we
were setting in perf-lib.sh at all. Instead the run_dirs_helper() sets
the previously inferred $PERF_RESULTS_PREFIX directly.

Setting this at the callsite that's already best positioned to
exhaustively know about all the different cases we need to handle
where PERF_RESULTS_PREFIX isn't what we want already (the empty
string) makes the most sense. In one-off cases like:

./run ./p0000-perf-lib-sanity.sh
./p0000-perf-lib-sanity.sh

We'll just do the right thing because PERF_RESULTS_PREFIX will be
empty, and test-lib.sh takes care of finding where our git is.

Any refactoring of this code needs to change both the shell code and
the Perl code in aggregate.perl, because when running e.g.:

./run ../../ -- <test>

The "../../" path to a relative bindir needs to be munged to a
filename containing the results, and critically aggregate.perl does
not get passed the path to those aggregations, just "../..".

Let's fix cases where aggregate.perl would print e.g. ".." in its
report output for this, and "git" for "/home/avar/g/git", i.e. it
would always pick the last element. Now'll always print the full path
instead.

This also makes the code sturdier, e.g. you can feed "../.." to
"./run" and then an absolute path to the aggregate.perl script, as
long as the absolute path and "../.." resolved to the same directory
printing the aggregation will work.

Also simplify the "[_*]" on the RHS of "tr -c", we're trimming
everything to "_", so we don't need that.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>

perf-lib.sh: make "./run <revisions>" use the correct gits

Fix a really bad regression in 0baf78e7bc ("perf-lib.sh: rely on
test-lib.sh for --tee handling", 2019-03-15). Since that change all
runs of different <revisions> of git have used the git found in the
user's $PATH, e.g. /usr/bin/git instead of the <revision> we just
built and wanted to performance test.

The problem starts with GIT_TEST_INSTALLED not working like our
non-perf tests with the "run" script. I.e. you can't run performance
tests against a given installed git. Instead we expect to use it
ourselves to point GIT_TEST_INSTALLED to the <revision> we just built.

However, we had been relying on '$(cd "$GIT_TEST_INSTALLED" && pwd)'
to resolve that relative $GIT_TEST_INSTALLED to an absolute
path *before* test-lib.sh was loaded, in cases where it was
e.g. "build/<rev>/bin-wrappers" and we wanted "<abs_path>build/...".

This change post-dates another proposed solution by a few days[1], I
didn't notice that version when I initially wrote this. I'm doing the
most minimal thing to solve the regression here, a follow-up change
will move this result prefix selection logic entirely into the "run"
script.

This makes e.g. these cases all work:

./run . $PWD/../../ origin/master origin/next HEAD -- <tests>

As well as just a plain one-off:

./run <tests>

And, since we're passing down the new GIT_PERF_DIR_MYDIR_REL we make
sure the bug relating to aggregate.perl not finding our files as
described in 0baf78e7bc doesn't happen again.

What *doesn't* work is setting GIT_TEST_INSTALLED to a relative path,
this will subtly fail in test-lib.sh. This has always been the case
even before 0baf78e7bc, and as documented in t/README the
GIT_TEST_INSTALLED variable should be set to an absolute path (needs
to be set "to the bindir", which is always absolute), and the "perf"
framework expects to munge it itself.

Perhaps that should be dealt with in the future to allow manually
setting GIT_TEST_INSTALLED, but as a preceding commit showed the user
can just use the "run" script, which'll also pick the right output
directory for the test results as expected by aggregate.perl.

1. https://public-inbox.org/git/20190502222409.GA15631@sigill.intra.peff.net/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>

perf aggregate: remove GIT_TEST_INSTALLED from --codespeed

Remove the setting of the "environment" from the --codespeed output. I
don't think this is useful, and it helps with a later refactoring
where we GIT_TEST_INSTALLED stop munging/reading GIT_TEST_INSTALLED in
the perf tests in so many places.

This was added in 05eb1c37ed ("perf/aggregate: implement codespeed
JSON output", 2018-01-05), but since the "run" scripts uses
"GIT_TEST_INSTALLED" internally this was only ever useful for one-off
runs of a single revision as all the "environment" values would be
ones for whatever directory the "run" script ran last.

Let's instead fall back on the "uname -r" case, which is the sort of
thing the environment should be set to, not something that duplicates
other parts of the codpseed output. For setting the "environment" to
something custom the perf.repoName variable can be used. See
19cf57a92e ("perf/run: read GIT_PERF_REPO_NAME from perf.repoName",
2018-01-05).

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>

perf README: correct docs for 3c8f12c96c regression

Since 3c8f12c96c ("test-lib: reorder and include GIT-BUILD-OPTIONS a
lot earlier", 2012-06-24) the suggested advice of overriding
GIT_BUILD_DIR has not worked. We've printed a hard error like this
given e.g. GIT_BUILD_DIR=/home/avar/g/git:

/bin-wrappers/git is not executable; using GIT_EXEC_PATH
error: You haven't built things yet, have you?

Let's just suggest that the user run other gits via the "run"
script. That'll do the right thing for setting the path to the other
git, and running the "aggregate.perl" scripts afterwards will work.

As an aside, if setting GIT_BUILD_DIR had still worked, then the
MODERN_GIT feature/fix added in 1a0962dee5 ("t/perf: fix regression in
testing older versions of git", 2016-06-22) would have broke.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>

revisions.txt: remove ambibuity between <rev>:<path> and :<path>

The revision ':README' is mentioned as an example for '<rev>:<path>'
but the explanation forwards to the ':<n>:<path>' syntax. At the same
time ':<n>:<path>' did not mark the '<n>:' as optional.

Signed-off-by: Andreas Heiduk <asheiduk@gmail.com>
Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

revisions.txt: mention <rev>~ form

In revisions.txt, the '<rev>^' form is mentioned but the '<rev>~' form
is missing. Although both forms are essentially equivalent (they each
get the first parent of the specified revision), we should mention the
latter for completeness. Make this change.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

revisions.txt: mark optional rev arguments with []

In revisions.txt, an optional rev argument was not distinguised.
Instead, a user had to continue and read the description in order to
learn that the argument was optional.

Since the [] notation for an optional argument is common-knowledge in
the Git documentation, mark optional arguments with [] so that it's more
obvious for the reader.

Helped-by: Andreas Heiduk <asheiduk@gmail.com>
Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

revisions.txt: change "rev" to "<rev>"

In revisions.txt, there were some instances of a rev argument being
written as "rev". However, since they didn't mean the string literal,
write "<rev>", instead.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

notes: correct documentation of format_display_notes()

'flags' parameter was replaced by 'raw' in commit:
76141e2e62 ("format_note(): simplify API", 2012-10-17)

Signed-off-by: Chris Mayo <aklhfex@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

mingw: remove obsolete IPv6-related code

To support IPv6, Git provided fall back functions for Windows versions
that did not support IPv6. However, as Git dropped support for Windows
XP and prior, those functions are not needed anymore.

Remove those fallbacks by reverting fe3b2b7b827c (Enable support for
IPv6 on MinGW, 2009-11-24) and using the functions directly (without
'ipv6_' prefix).

Signed-off-by: Tanushree Tumane <tanushreetumane@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t5580: verify that alternates can be UNC paths

On Windows, UNC paths are a very convenient way to share data, and
alternates are all about sharing data.

We fixed a bug where alternates specifying UNC paths were not handled
properly, and it is high time that we add a regression test to ensure
that this bug is not reintroduced.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

cvsexportcommit: force crlf translation

When using cvsnt + msys + git, it seems like the output of cvs status
had \r\n in it, and caused the command to fail.

This fixes that.

Signed-off-by: Dustin Spicuzza <dustin@virtualroadside.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

ci: install 'libsvn-perl' instead of 'git-svn'

Since e7e9f5e7a1 (travis-ci: enable Git SVN tests t91xx on Linux,
2016-05-19) some of our Travis CI build jobs install the 'git-svn'
package, because it was a convenient way to install its dependencies,
which are necessary to run our 'git-svn' tests (we don't actually need
the 'git-svn' package itself). However, from those dependencies,
namely the 'libsvn-perl', 'libyaml-perl', and 'libterm-readkey-perl'
packages, only 'libsvn-perl' is necessary to run those tests, the
others arent, not even to fulfill some prereqs.

So update 'ci/install-dependencies.sh' to install only 'libsvn-perl'
instead of 'git-svn' and its additional dependencies.

Note that this change has more important implications than merely not
installing three unnecessary packages, as it keeps our builds working
with Travis CI's Xenial images. In our '.travis.yml' we never
explicitly specified which Linux image we want to use to run our Linux
build jobs, and so far they have been run on the default Ubuntu 14.04
Trusty image. However, 14.04 just reached its EOL, and Travis CI has
already began the transition to use 16.04 Xenial as the default Linux
build environment [1]. Alas, our Linux Clang and GCC build jobs can't
simply 'apt-get install git-svn' in the current Xenial images [2],
like they did in the Trusty images, and, consequently, fail.
Installing only 'libsvn-perl' avoids this issue, while the 'git svn'
tests are still run as they should.

[1] https://blog.travis-ci.com/2019-04-15-xenial-default-build-environment

[2] 'apt-get install git-svn' in the Xenial image fails with:

The following packages have unmet dependencies:
git-svn : Depends: git (< 1:2.7.4-.)
E: Unable to correct problems, you have held broken packages.

The reason is that both the Trusty and Xenial images contain the
'git' package installed from 'ppa:git-core/ppa', so it's
considerably newer than the 'git' package in the corresponding
standard Ubuntu package repositories. The difference is that the
Trusty image still contains these third-party apt repositories, so
the 'git-svn' package was installed from the same PPA, and its
version matched the version of the already installed 'git'
package. In the Xenial image, however, these third-party
apt-repositories are removed (to reduce the risk of unrelated
interference and faster 'apt-get update') [3], and the version of
the 'git-svn' package coming from the standard Ubuntu package
repositories doesn't match the much more recent version of the
'git' package installed from the PPA, resulting in this dependecy
error.

Adding back the 'ppa:git-core/ppa' package repository would solve
this dependency issue as well, but since the troublesome package
happens to be unnecessary, not installing it in the first place is
better.

[3] https://docs.travis-ci.com/user/reference/xenial/#third-party-apt-repositories-removed

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t4253-am-keep-cr-dos: avoid using pipes

The exit code of the upstream in a pipe is ignored thus we should avoid
using it. By writing out the output of the git command to a file, we can
test the exit codes of both the commands.

Signed-off-by: Boxuan Li <liboxuan@connect.hku.hk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

commit-graph: fix memory leak

Free the commit graph when verify_commit_graph_lite() reports an error.
Credit to OSS-Fuzz for finding this leak.

Signed-off-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

midx: add packs to packed_git linked list

The multi-pack-index allows searching for objects across multiple
packs using one object list. The original design gains many of
these performance benefits by keeping the packs in the
multi-pack-index out of the packed_git list.

Unfortunately, this has one major drawback. If the multi-pack-index
covers thousands of packs, and a command loads many of those packs,
then we can hit the limit for open file descriptors. The
close_one_pack() method is used to limit this resource, but it
only looks at the packed_git list, and uses an LRU cache to prevent
thrashing.

Instead of complicating this close_one_pack() logic to include
direct references to the multi-pack-index, simply add the packs
opened by the multi-pack-index to the packed_git list. This
immediately solves the file-descriptor limit problem, but requires
some extra steps to avoid performance issues or other problems:

1. Create a multi_pack_index bit in the packed_git struct that is
one if and only if the pack was loaded from a multi-pack-index.

2. Skip packs with the multi_pack_index bit when doing object
lookups and abbreviations. These algorithms already check the
multi-pack-index before the packed_git struct. This has a very
small performance hit, as we need to walk more packed_git
structs. This is acceptable, since these operations run binary
search on the other packs, so this walk-and-ignore logic is
very fast by comparison.

3. When closing a multi-pack-index file, do not close its packs,
as those packs will be closed using close_all_packs(). In some
cases, such as 'git repack', we run 'close_midx()' without also
closing the packs, so we need to un-set the multi_pack_index bit
in those packs. This is necessary, and caught by running
t6501-freshen-objects.sh with GIT_TEST_MULTI_PACK_INDEX=1.

To manually test this change, I inserted trace2 logging into
close_pack_fd() and set pack_max_fds to 10, then ran 'git rev-list
--all --objects' on a copy of the Git repo with 300+ pack-files and
a multi-pack-index. The logs verified the packs are closed as
we read them beyond the file descriptor limit.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

midx: pass a repository pointer

Much of the multi-pack-index code focuses on the multi_pack_index
struct, and so we only pass a pointer to the current one. However,
we will insert a dependency on the packed_git linked list in a
future change, so we will need a repository reference. Inserting
these parameters is a significant enough change to split out.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

branch: make create_branch accept a merge base rev

When we ran something like

$ git checkout -b test master...

it would fail with the message

fatal: Not a valid object name: 'master...'.

This was caused by the call to `create_branch` where `start_name` is
expected to be a valid rev. However, git-checkout allows the branch to
be a valid _merge base_ rev (i.e. with a "...") so it was possible for
an invalid rev to be passed in.

Make `create_branch` accept a merge base rev so that this case does not
error out.

As a side-effect, teach git-branch how to handle merge base revs as
well.

Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t2018: cleanup in current test

Before, in t2018, if do_checkout failed to create `branch2`, the next
test-case would run `git branch -D branch2` but then fail because it was
expecting `branch2` to exist, even though it doesn't. As a result, an
early failure could cause a cascading failure of tests.

Make test-case responsible for cleaning up their own branches so that
future tests can start with a sane environment.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

parse-options: don't emit "ambiguous option" for aliases

Change the option parsing machinery so that e.g. "clone --recurs ..."
doesn't error out because "clone" understands both "--recursive" and
"--recurse-submodules" to mean the same thing.

Initially "clone" just understood --recursive until the
--recurses-submodules alias was added in ccdd3da652 ("clone: Add the
--recurse-submodules option as alias for --recursive",
2010-11-04). Since bb62e0a99f ("clone: teach --recurse-submodules to
optionally take a pathspec", 2017-03-17) the longer form has been
promoted to the default.

But due to the way the options parsing machinery works this resulted
in the rather absurd situation of:

$ git clone --recurs [...]
error: ambiguous option: recurs (could be --recursive or --recurse-submodules)

Add OPT_ALIAS() to express this link between two or more options and use
it in git-clone. Multiple aliases of an option could be written as

OPT_ALIAS(0, "alias1", "original-name"),
OPT_ALIAS(0, "alias2", "original-name"),
...

The current implementation is not exactly optimal in this case. But we
can optimize it when it becomes a problem. So far we don't even have two
aliases of any option.

A big chunk of code is actually from Junio C Hamano.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

coccicheck: optionally batch spatch invocations

In our "make coccicheck" rule, we currently feed each source file to its
own individual invocation of spatch. This has a few downsides:

- it repeats any overhead spatch has for starting up and reading the
patch file

- any included header files may get processed from multiple
invocations. This is slow (we see the same header files multiple
times) and may produce a resulting patch with repeated hunks (which
cannot be applied without further cleanup)

Ideally we'd just invoke a single instance of spatch per rule-file and
feed it all source files. But spatch can be rather memory hungry when
run in this way. I measured the peak RSS going from ~90MB for a single
file to ~1900MB for all files. Multiplied by multiple rule files being
processed at the same time (for "make -j"), this can make things slower
or even cause them to fail (e.g., this is reported to happen on our
Travis builds).

Instead, let's provide a tunable knob. We'll leave the default at "1",
but it can be cranked up to "999" for maximum CPU/memory tradeoff, or
people can find points in between that serve their particular machines.

Here are a few numbers running a single rule via:

SIZES='1 4 16 999'
RULE=contrib/coccinelle/object_id.cocci
for i in $SIZES; do
make clean
/usr/bin/time -o $i.out --format='%e | %U | %S | %M' \
make $RULE.patch SPATCH_BATCH_SIZE=$i
done
for i in $SIZES; do
printf '%4d | %s\n' $i "$(cat $i.out)"
done

which yields:

1 | 97.73 | 93.38 | 4.33 | 100128
4 | 52.80 | 51.14 | 1.69 | 135204
16 | 35.82 | 35.09 | 0.76 | 284124
999 | 23.30 | 23.13 | 0.20 | 1903852

The implementation is done with xargs, which should be widely available;
it's in POSIX, we rely on it already in the test suite. And "coccicheck"
is really a developer-only tool anyway, so it's not a big deal if
obscure systems can't run it.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

trace2: fixup access problem on /etc/gitconfig in read_very_early_config

Teach do_git_config_sequence() to optionally gently check for access to
the system config. Use this option in read_very_early_config() when
initializing trace2.

In [1] SZEDER Gábor reported that my changes in [2] introduced a
regression when the user does not have permission to read the system
config.

This commit addresses that problem by optionally ignoring that error.

[1] https://public-inbox.org/git/285beb2b2d740ce20fdd8af1becf371ab39703db.1554995916.git.gitgitgadget@gmail.com/T/#m342e839289aec515523a98b5e34d7f42d3f1fd79
[2] https://public-inbox.org/git/285beb2b2d740ce20fdd8af1becf371ab39703db.1554995916.git.gitgitgadget@gmail.com/T/#m11b59c9228c698442f750ee8f9b10c629399ae48

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

*.[ch]: manually align parameter lists

In previous patches, extern was mechanically removed from function
declarations without care to formatting, causing parameter lists to be
misaligned. Manually format changed sections such that the parameter
lists should be realigned.

Viewing this patch with 'git diff -w' should produce no output.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

*.[ch]: remove extern from function declarations using sed

There has been a push to remove extern from function declarations.
Finish the job by removing all instances of "extern" for function
declarations in headers using sed.

This was done by running the following on my system with sed 4.2.2:

$ git ls-files \*.{c,h} |
grep -v ^compat/ |
xargs sed -i'' -e 's/^\(\s*\)extern \([^(]*([^*]\)/\1\2/'

Files under `compat/` are intentionally excluded as some are directly
copied from external sources and we should avoid churning them as much
as possible.

Then, leftover instances of extern were found by running

$ git grep -w -C3 extern \*.{c,h}

and manually checking the output. No other instances were found.

Note that the regex used specifically excludes function variables which
_should_ be left as extern.

Not the most elegant way to do it but it gets the job done.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

*.[ch]: remove extern from function declarations using spatch

There has been a push to remove extern from function declarations.
Remove some instances of "extern" for function declarations which are
caught by Coccinelle. Note that Coccinelle has some difficulty with
processing functions with `__attribute__` or varargs so some `extern`
declarations are left behind to be dealt with in a future patch.

This was the Coccinelle patch used:

@@
type T;
identifier f;
@@
- extern
T f(...);

and it was run with:

$ git ls-files \*.{c,h} |
grep -v ^compat/ |
xargs spatch --sp-file contrib/coccinelle/noextern.cocci --in-place

Files under `compat/` are intentionally excluded as some are directly
copied from external sources and we should avoid churning them as much
as possible.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t/perf: add perf script for partial clones

We don't cover the partial clone feature at all in t/perf. Let's at
least run a few basic tests so that we'll notice any regressions.

We'll do a no-blob clone, and split it into two parts: the actual object
transfer, and the subsequent checkout (which will of course require
another transfer to get the blobs). That will help us more clearly
assess the performance of each.

There are obviously a lot more possibilities besides just a no-blob
partial clone, but this should serve as a canary that alerts us to any
generic slow-downs (and we can add more tests later for cases that
aren't exercised here).

There are a few non-ideal things here that make this not an entirely
accurate test, but are probably OK for our purposes:

1. We have to do some extra prep/cleanup work inside the timing tests,
since they impact the on-disk state and the perf harness may run
each one multiple times.

In practice this is probably OK, since these bits should be much
less expensive than the operations we are measuring.

2. The clone time is likely to be dominated by the server's object
enumeration. In the real world, a repo large enough to drive people
to partial clones is likely to have reachability bitmaps enabled.

And in the opposite direction, our object transfer is happening at
the speed of a local pipe, whereas in the real world it would
bottle-neck on the network.

So any percentage speedups should be taken with a grain of salt.
But hopefully any regressions will produce enough of an effect to
be noticeable.

This script also demonstrates the recent improvement from dfa33a298d
(clone: do faster object check for partial clones, 2019-04-19):

Test dfa33a298d^ dfa33a298d
-------------------------------------------------------------------------
5600.2: clone without blobs 18.41(22.72+1.09) 6.83(11.65+0.50) -62.9%
5600.3: checkout of result 1.82(3.24+0.26) 1.84(3.24+0.26) +1.1%

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

trace2: fix incorrect function pointer check

Fix trace2_data_json_fl() to check for the presence of pfn_data_json_fl
in its targets, rather than pfn_data_fl, which is not actually called.

Signed-off-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Honor core.precomposeUnicode in more places

On Mac's HFS where git sets core.precomposeUnicode to true automatically
by git init/clone, when a user creates a simple unicode refname (in NFC
format) such as españa:

$ git branch españa

different commands would display the branch name differently. For
example, git branch, git log --decorate, and git fast-export all used

65 73 70 61 c3 b1 61 (or "espa\xc3\xb1a")

(NFC form) while show-ref would use

65 73 70 61 6e cc 83 61 (or "espan\xcc\x83a")

(NFD form). A stress test for git filter-repo was tripped up by this
inconsistency, though digging in I found that the problems could
compound; for example, if the user ran

$ git pack-refs --all

and then tried to check out the branch, they would be met with:

$ git checkout españa
error: pathspec 'españa' did not match any file(s) known to git

$ git checkout españa --
fatal: invalid reference: españa

$ git branch
españa
* master

Note that the user could run the `git branch` command first and copy and
paste the `españa` portion of the output and still see the same two
errors. Also, if the user added --no-prune to the pack-refs command,
then they would see three branches: master, españa, and españa (those
last two are NFC vs. NFD forms, even if they render the same).

Further, if the user had the `españa` branch checked out before
running `git pack-refs --all`, the user would be greeted with (note
that I'm trimming trailing output with an ellipsis):

$ git rev-parse HEAD
fatal: ambiguous argument 'HEAD': unknown revision or path...

$ git status
On branch españa

No commits yet...

Or worse, if the user didn't check this stuff first, running `git
commit` will create a new commit with all changes of all of history
being squashed into it.

In addition to pack-refs, one could also get into this state with
upload-pack or anything that calls either pack-refs or upload-pack (e.g.
gc or clone).

Add code in a few places (pack-refs, show-ref, upload-pack) to check and
honor the setting of core.precomposeUnicode to avoid these bugs.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

git-compat-util: work around for access(X_OK) under root

On AIX, access(X_OK) may succeed when run as root even if the
execution isn't possible. This behavior is allowed by POSIX
which says:

... for a process with appropriate privileges, an implementation
may indicate success for X_OK even if execute permission is not
granted to any user.

It can lead hook programs to have their execution refused:

git commit -m content
fatal: cannot exec '.git/hooks/pre-commit': Permission denied

Add NEED_ACCESS_ROOT_HANDLER in order to use an access helper function.
It checks with stat if any executable flags is set when the current user
is root.

Signed-off-by: Clément Chigot <clement.chigot@atos.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Makefile: use fileno macro work around on AIX

Declare FILENO_IS_A_MACRO on AIX

On AIX, fileno(fp) is a macro and need to use the work around already made for BSD's.

Signed-off-by: Clément Chigot <clement.chigot@atos.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

The seventh batch

Signed-off-by: Junio C Hamano <gitster@pobox.com>

Merge branch 'js/macos-gettext-build'

Build with gettext breaks on recent macOS w/ Homebrew when
/usr/local/bin is not on PATH, which has been corrected.

* js/macos-gettext-build:
macOS: make sure that gettext is found

Merge branch 'bs/sendemail-tighten-anything-by'

The recently added feature to add addresses that are on
anything-by: trailers in 'git send-email' was found to be way too
eager and considered nonsense strings as if they can be legitimate
beginning of *-by: trailer. This has been tightened.

* bs/sendemail-tighten-anything-by:
send-email: don't cc *-by lines with '-' prefix

Merge branch 'bc/send-email-qp-cr'

"git send-email" has been taught to use quoted-printable when the
payload contains carriage-return. The use of the mechanism is in
line with the design originally added the codepath that chooses QP
when the payload has overly long lines.

* bc/send-email-qp-cr:
send-email: default to quoted-printable when CR is present

Merge branch 'nd/submodule-foreach-quiet'

"git submodule foreach <command> --quiet" did not pass the option
down correctly, which has been corrected.

* nd/submodule-foreach-quiet:
submodule foreach: fix "<command> --quiet" not being respected

Merge branch 'js/iso8895-test-on-apfs'

Test fix on APFS that is incapable of store paths in Latin-1.

* js/iso8895-test-on-apfs:
t9822: skip tests if file names cannot be ISO-8859-1 encoded

Merge branch 'jc/gettext-test-fix'

The GETTEXT_POISON test option has been quite broken ever since it
was made runtime-tunable, which has been fixed.

* jc/gettext-test-fix:
gettext tests: export the restored GIT_TEST_GETTEXT_POISON

Merge branch 'jk/fetch-reachability-error-fix'

Code clean-up and a fix for "git fetch" by an explicit object name
(as opposed to fetching refs by name).

* jk/fetch-reachability-error-fix:
fetch: do not consider peeled tags as advertised tips
remote.c: make singular free_ref() public
fetch: use free_refs()
pkt-line: prepare buffer before handling ERR packets
upload-pack: send ERR packet for non-tip objects
t5530: check protocol response for "not our ref"
t5516: drop ok=sigpipe from unreachable-want tests

Merge branch 'jk/xmalloc'

The code is updated to check the result of memory allocation before
it is used in more places, by using xmalloc and/or xcalloc calls.

* jk/xmalloc:
progress: use xmalloc/xcalloc
xdiff: use xmalloc/xrealloc
xdiff: use git-compat-util
test-prio-queue: use xmalloc

Merge branch 'km/t3000-retitle'

A test update.

* km/t3000-retitle:
t3000 (ls-files -o): widen description to reflect current tests

Merge branch 'js/untracked-cache-allocfix'

An underallocation in the code to read the untracked cache
extension has been corrected.

* js/untracked-cache-allocfix:
untracked cache: fix off-by-one

Merge branch 'js/t3301-unbreak-notes-test'

Test fix.

* js/t3301-unbreak-notes-test:
t3301: fix false negative

Merge branch 'tz/doc-apostrophe-no-longer-needed'

Doc formatting fix.

* tz/doc-apostrophe-no-longer-needed:
Documentation/git-show-branch: avoid literal {apostrophe}

Merge branch 'sg/blame-in-bare-start-at-head'

"git blame -- path" in a non-bare repository starts blaming from
the working tree, and the same command in a bare repository errors
out because there is no working tree by definition. The command
has been taught to instead start blaming from the commit at HEAD,
which is more useful.

* sg/blame-in-bare-start-at-head:
blame: default to HEAD in a bare repo when no start commit is given

Merge branch 'tg/ls-files-debug-format-fix'

Debugging code fix.

* tg/ls-files-debug-format-fix:
ls-files: use correct format string

Merge branch 'po/describe-not-necessarily-7'

Docfix.

* po/describe-not-necessarily-7:
describe doc: remove '7-char' abbreviation reference

Merge branch 'po/rerere-doc-fmt'

Docfix.

* po/rerere-doc-fmt:
rerere doc: quote `rerere.enabled`

Merge branch 'sg/overlong-progress-fix'

Updating the display with progress message has been cleaned up to
deal better with overlong messages.

* sg/overlong-progress-fix:
progress: break too long progress bar lines
progress: clear previous progress update dynamically
progress: assemble percentage and counters in a strbuf before printing
progress: make display_progress() return void

Merge branch 'jt/batch-fetch-blobs-in-diff'

While running "git diff" in a lazy clone, we can upfront know which
missing blobs we will need, instead of waiting for the on-demand
machinery to discover them one by one. Aim to achieve better
performance by batching the request for these promised blobs.

* jt/batch-fetch-blobs-in-diff:
diff: batch fetching of missing blobs
sha1-file: support OBJECT_INFO_FOR_PREFETCH

Merge branch 'ab/gc-docs'

Update docs around "gc".

* ab/gc-docs:
gc docs: remove incorrect reference to gc.auto=0
gc docs: clarify that "gc" doesn't throw away referenced objects
gc docs: note "gc --aggressive" in "fast-import"
gc docs: downplay the usefulness of --aggressive
gc docs: note how --aggressive impacts --window & --depth
gc docs: fix formatting for "gc.writeCommitGraph"
gc docs: re-flow the "gc.*" section in "config"
gc docs: include the "gc.*" section from "config" in "gc"
gc docs: clean grammar for "gc.bigPackThreshold"
gc docs: stop noting "repack" flags
gc docs: modernize the advice for manually running "gc"

Merge branch 'dl/submodule-set-branch'

"git submodule" learns "set-branch" subcommand that allows the
submodule.*.branch settings to be modified.

* dl/submodule-set-branch:
submodule: teach set-branch subcommand
submodule--helper: teach config subcommand --unset
git-submodule.txt: "--branch <branch>" option defaults to 'master'

Merge branch 'jk/revision-rewritten-parents-in-prio-queue'

Performance fix for "rev-list --parents -- pathspec".

* jk/revision-rewritten-parents-in-prio-queue:
revision: use a prio_queue to hold rewritten parents

Merge branch 'dk/blame-keep-origin-blob'

Performance fix around "git blame", especially in a linear history
(which is the norm we should optimize for).

* dk/blame-keep-origin-blob:
blame.c: don't drop origin blobs as eagerly

Merge branch 'bc/hash-transition-16'

Conversion from unsigned char[20] to struct object_id continues.

* bc/hash-transition-16: (35 commits)
gitweb: make hash size independent
Git.pm: make hash size independent
read-cache: read data in a hash-independent way
dir: make untracked cache extension hash size independent
builtin/difftool: use parse_oid_hex
refspec: make hash size independent
archive: convert struct archiver_args to object_id
builtin/get-tar-commit-id: make hash size independent
get-tar-commit-id: parse comment record
hash: add a function to lookup hash algorithm by length
remote-curl: make hash size independent
http: replace sha1_to_hex
http: compute hash of downloaded objects using the_hash_algo
http: replace hard-coded constant with the_hash_algo
http-walker: replace sha1_to_hex
http-push: remove remaining uses of sha1_to_hex
http-backend: allow 64-character hex names
http-push: convert to use the_hash_algo
builtin/pull: make hash-size independent
builtin/am: make hash size independent
...

Merge branch 'en/fast-import-parsing-fix'

"git fast-import" update.

* en/fast-import-parsing-fix:
fast-import: fix erroneous handling of get-mark with empty orphan commits
fast-import: only allow cat-blob requests where it makes sense
fast-import: check most prominent commands first
git-fast-import.txt: fix wording about where ls command can appear
t9300: demonstrate bug with get-mark and empty orphan commits

Merge branch 'jt/fetch-no-update-shallow-in-proto-v2'

Fix for protocol v2 support in "git fetch-pack" of shallow clones.

* jt/fetch-no-update-shallow-in-proto-v2:
fetch-pack: respect --no-update-shallow in v2
fetch-pack: call prepare_shallow_info only if v0

Merge branch 'jt/fetch-pack-wanted-refs-optim'

Performance fix around "git fetch" that grabs many refs.

* jt/fetch-pack-wanted-refs-optim:
fetch-pack: binary search when storing wanted-refs

Merge branch 'ab/test-lib-pass-trace2-env'

Allow tracing of Git executable while running the testsuite.

* ab/test-lib-pass-trace2-env:
test-lib: whitelist GIT_TR2_* in the environment

Merge branch 'sg/index-pack-progress'

A progress indicator has been added to the "index-pack" step, which
often makes users wait for completion during "git clone".

* sg/index-pack-progress:
index-pack: show progress while checking objects

Merge branch 'ab/commit-graph-fixes'

Code cleanup with more careful error checking before using data
read from the commit-graph file.

* ab/commit-graph-fixes:
commit-graph: improve & i18n error messages
commit-graph write: don't die if the existing graph is corrupt
commit-graph verify: detect inability to read the graph
commit-graph: don't pass filename to load_commit_graph_one_fd_st()
commit-graph: don't early exit(1) on e.g. "git status"
commit-graph: fix segfault on e.g. "git status"
commit-graph tests: test a graph that's too small
commit-graph tests: split up corrupt_graph_and_verify()

Merge branch 'ab/gc-reflog'

Fix various glitches in "git gc" around reflog handling.

* ab/gc-reflog:
gc: handle & check gc.reflogExpire config
reflog tests: assert lack of early exit with expiry="never"
reflog tests: test for the "points nowhere" warning
reflog tests: make use of "test_config" idiom
gc: refactor a "call me once" pattern
gc: convert to using the_hash_algo
gc: remove redundant check for gc_auto_threshold

Merge branch 'nd/checkout-m'

"git checkout -m <other>" was about carrying the differences
between HEAD and the working-tree files forward while checking out
another branch, and ignored the differences between HEAD and the
index. The command has been taught to abort when the index and the
HEAD are different.

* nd/checkout-m:
checkout: prevent losing staged changes with --merge
read-tree: add --quiet
unpack-trees: rename "gently" flag to "quiet"
unpack-trees: keep gently check inside add_rejected_path

Merge branch 'js/difftool-no-index'

"git difftool" can now run outside a repository.

* js/difftool-no-index:
difftool: allow running outside Git worktrees with --no-index
parse-options: make OPT_ARGUMENT() more useful
difftool: remove obsolete (and misleading) comment

Merge branch 'pw/cherry-pick-continue'

"git cherry-pick --options A..B", after giving control back to the
user to ask help resolving a conflicted step, did not honor the
options it originally received, which has been corrected.

* pw/cherry-pick-continue:
cherry-pick --continue: remember options
cherry-pick: demonstrate option amnesia
sequencer: break some long lines

Merge branch 'nd/commit-a-with-paths-msg-update'

The message given when "git commit -a <paths>" errors out has been
updated.

* nd/commit-a-with-paths-msg-update:
commit: improve error message in "-a <paths>" case

Merge branch 'jk/server-info-rabbit-hole'

Code clean-up around a much-less-important-than-it-used-to-be
update_server_info() funtion.

* jk/server-info-rabbit-hole:
update_info_refs(): drop unused force parameter
server-info: drop objdirlen pointer arithmetic
server-info: drop nr_alloc struct member
server-info: use strbuf to read old info/packs file
server-info: simplify cleanup in parse_pack_def()
server-info: fix blind pointer arithmetic
http: simplify parsing of remote objects/info/packs
packfile: fix pack basename computation
midx: check both pack and index names for containment
t5319: drop useless --buffer from cat-file
t5319: fix bogus cat-file argument
pack-revindex: open index if necessary
packfile.h: drop extern from function declarations

Merge branch 'nd/diff-parseopt-4'

Fourth batch to teach the diff machinery to use the parse-options
API.

* nd/diff-parseopt-4:
am: avoid diff_opt_parse()
diff --no-index: use parse_options() instead of diff_opt_parse()
range-diff: use parse_options() instead of diff_opt_parse()
diff.c: allow --no-color-moved-ws
diff-parseopt: convert --color-moved-ws
diff-parseopt: convert --[no-]color-moved
diff-parseopt: convert --inter-hunk-context
diff-parseopt: convert --no-prefix
diff-parseopt: convert --line-prefix
diff-parseopt: convert --[src|dst]-prefix
diff-parseopt: convert --[no-]abbrev
diff-parseopt: convert --diff-filter
diff-parseopt: convert --find-object
diff-parseopt: convert -O
diff-parseopt: convert --pickaxe-all|--pickaxe-regex
diff-parseopt: convert -S|-G
diff-parseopt: convert -l
diff-parseopt: convert -z
diff-parseopt: convert --ita-[in]visible-in-index
diff-parseopt: convert --ws-error-highlight

Merge branch 'jk/unused-params-even-more'

Code cleanup.

* jk/unused-params-even-more:
parse_opt_ref_sorting: always use with NONEG flag
pretty: drop unused strbuf from parse_padding_placeholder()
pretty: drop unused "type" parameter in needs_rfc2047_encoding()
parse-options: drop unused ctx parameter from show_gitcomp()
fetch_pack(): drop unused parameters
report_path_error(): drop unused prefix parameter
unpack-trees: drop unused error_type parameters
unpack-trees: drop name_entry from traverse_by_cache_tree()
test-date: drop unused "now" parameter from parse_dates()
update-index: drop unused prefix_length parameter from do_reupdate()
log: drop unused "len" from show_tagger()
log: drop unused rev_info from early output
revision: drop some unused "revs" parameters

Merge branch 'sg/test-atexit'

Test framework update to more robustly clean up leftover files and
processes after tests are done.

* sg/test-atexit:
t9811-git-p4-label-import: fix pipeline negation
git p4 test: disable '-x' tracing in the p4d watchdog loop
git p4 test: simplify timeout handling
git p4 test: clean up the p4d cleanup functions
git p4 test: use 'test_atexit' to kill p4d and the watchdog process
t0301-credential-cache: use 'test_atexit' to stop the credentials helper
tests: use 'test_atexit' to stop httpd
git-daemon: use 'test_atexit` to stop 'git-daemon'
test-lib: introduce 'test_atexit'
t/lib-git-daemon: make sure to kill the 'git-daemon' process
test-lib: fix interrupt handling with 'dash' and '--verbose-log -x'

Merge branch 'ag/sequencer-reduce-rewriting-todo'

The scripted version of "git rebase -i" wrote and rewrote the todo
list many times during a single step of its operation, and the
recent C-rewrite made a faithful conversion of the logic to C. The
implementation has been updated to carry necessary information
around in-core to avoid rewriting the same file over and over
unnecessarily.

* ag/sequencer-reduce-rewriting-todo:
rebase--interactive: move transform_todo_file()
sequencer: use edit_todo_list() in complete_action()
rebase-interactive: rewrite edit_todo_list() to handle the initial edit
rebase-interactive: append_todo_help() changes
rebase-interactive: use todo_list_write_to_file() in edit_todo_list()
sequencer: refactor skip_unnecessary_picks() to work on a todo_list
rebase--interactive: move rearrange_squash_in_todo_file()
rebase--interactive: move sequencer_add_exec_commands()
sequencer: change complete_action() to use the refactored functions
sequencer: make sequencer_make_script() write its script to a strbuf
sequencer: refactor rearrange_squash() to work on a todo_list
sequencer: refactor sequencer_add_exec_commands() to work on a todo_list
sequencer: refactor check_todo_list() to work on a todo_list
sequencer: introduce todo_list_write_to_file()
sequencer: refactor transform_todos() to work on a todo_list
sequencer: remove the 'arg' field from todo_item
sequencer: make the todo_list structure public
sequencer: changes in parse_insn_buffer()

Merge branch 'bp/post-index-change-hook'

A new hook "post-index-change" is called when the on-disk index
file changes, which can help e.g. a virtualized working tree
implementation.

* bp/post-index-change-hook:
read-cache: add post-index-change hook

t7610: add mergetool --gui tests

In 063f2bdbf7 (mergetool: accept -g/--[no-]gui as arguments,
2018-10-24), mergetool was taught the --gui option but no tests were
added to ensure that it was working properly. Add a test to ensure that
it works.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t7610: unsuppress output

The output for commands used to be suppressed by redirecting both stdout
and stderr to /dev/null. However, this should not happen since the
output is useful for debugging and, without the "-v" flag, test scripts
don't output anyway.

Unsuppress the output by removing the redirections to /dev/null.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t/perf: depend on perl JSON only when using --codespeed

Commit 05eb1c37ed (perf/aggregate: implement codespeed JSON output,
2018-01-05) added a dependency on the perl JSON module to show output
from aggregate.perl, but we only need it when the user asks for
--codespeed output. While the module is pretty common, it's not part of
the base system, and this dependency can get in the way of producing the
default human-readable output.

Let's bump the "use" down to a "require" in the code path that needs it,
which will be interpreted at run-time instead of compile-time. People
not using "--codespeed" won't even load the module, and anybody using it
should see the same results (including the same perl error if they don't
have it).

Note that this skips the importing step, so we'll have to fully qualify
our function call. We could accomplish the same thing in other ways.
E.g., calling JSON->import() ourselves, or wrapping "use JSON" in an
eval. Since there's only one such call, this seems like the
least-magical way of doing it.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

doc/ls-files: put nested list for "-t" option into block

The description for the "-t" option contains a sub-list of all of the
possible file status outputs. But because of the newline separating that
list from the description paragraph, asciidoc treats the sub-list
entries as a continuation of the overall options list, rather than as
children of the "-t" description.

We could fix it by adding a "+" before the sub-list to connect it to the
rest of the "-t" text. But using a pair of "--" to delimit the block is
perhaps more readable, and may have better compatibility with
asciidoctor, as in 39a869b2f2 (Documentation/rev-list-options: wrap
--date=<format> block with "--", 2019-03-30).

The extra blank line comes from 5bc0e247c4 (Document ls-files -t as
semi-obsolete., 2010-07-28), but the problem actually seems older than
that. Before then, we did:

-t:: some text...
H:: cached
M:: unmerged
etc...

but asciidoc also treats that as one big list. So this problem seems to
have been around forever.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

p5302: create the repo in each index-pack test

The p5302 script runs "index-pack --stdin" in each timing test. It does
two things to try to get good timings:

1. we do the repo creation in a separate (non-timed) setup test, so
that our timing is purely the index-pack run

2. we use a separate repo for each test; this is important because the
presence of existing objects in the repo influences the result
(because we'll end up doing collision checks against them)

But this forgets one thing: we generally run each timed test multiple
times to reduce the impact of noise. Which means that repeats of each
test after the first will be subject to the collision slowdown from
point 2, and we'll generally just end up taking the first time anyway.

Instead, let's create the repo in the test (effectively undoing point
1). That does add a constant amount of extra work to each iteration, but
it's quite small compared to the actual effects we're interested in
measuring.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Makefile: dedup list of files obtained from ls-files

Since 33533975 ("Makefile: ask "ls-files" to list source files if
available", 2011-10-18), we optionally asked "ls-files" to list the
source files that ought to exist, as a faster approximation for
"find" on working tree files.

This works reasonably well, except that it ends up listing the same
path multiple times if the index is unmerged. Because the original
use of this construct was to name files to run etags over, and the
etags command happily takes the same filename multiple times without
causing any harm, there was no problem (other than perhaps spending
slightly more cycles, but who cares how fast the TAGS file gets
updated).

We however recently added a similar call to "ls-files" to list *.h
files, instead of using "find", in 92b88eba ("Makefile: use `git
ls-files` to list header files, if possible", 2019-03-04). In this
new use of "ls-files", the resulting list $(LIB_H) is used for,
among other things, generating the header files to run hdr-check
target, and the duplicate unfortunately becomes a true problem. It
causes $(MAKE) to notice that there are multiple %.hco targets and
complain.

Let the resulting list consumed by $(sort), which deduplicates,
to fix this.

Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

The sixth batch

Signed-off-by: Junio C Hamano <gitster@pobox.com>

Merge branch 'da/smerge'

"git mergetool" learned to offer Sublime Merge (smerge) as one of
its backends.

* da/smerge:
contrib/completion: add smerge to the mergetool completion candidates
mergetools: add support for smerge (Sublime Merge)

Merge branch 'dl/flex-str-cocci'

Code clean-up.

* dl/flex-str-cocci:
cocci: FLEX_ALLOC_MEM to FLEX_ALLOC_STR
midx.c: convert FLEX_ALLOC_MEM to FLEX_ALLOC_STR

Merge branch 'js/spell-out-options-in-tests'

The tests have been updated not to rely on the abbreviated option
names the parse-options API offers, to protect us from an
abbreviated form of an option that used to be unique within the
command getting non-unique when a new option that share the same
prefix is added.

* js/spell-out-options-in-tests:
tests: disallow the use of abbreviated options (by default)
tests (pack-objects): use the full, unabbreviated `--revs` option
tests (status): spell out the `--find-renames` option in full
tests (push): do not abbreviate the `--follow-tags` option
t5531: avoid using an abbreviated option
t7810: do not abbreviate `--no-exclude-standard` nor `--invert-match`
tests (rebase): spell out the `--force-rebase` option
tests (rebase): spell out the `--keep-empty` option

Merge branch 'nd/read-tree-reset-doc'

The documentation for "git read-tree --reset -u" has been updated.

* nd/read-tree-reset-doc:
read-tree.txt: clarify --reset and worktree changes

Merge branch 'nd/interpret-trailers-docfix'

Doc update.

* nd/interpret-trailers-docfix:
interpret-trailers.txt: start the desc line with a capital letter

Merge branch 'sg/asciidoctor-in-ci'

Update our support to format documentation in the CI environment,
either with AsciiDoc ro Asciidoctor.

* sg/asciidoctor-in-ci:
ci: fix AsciiDoc/Asciidoctor stderr check in the documentation build job
ci: stick with Asciidoctor v1.5.8 for now
ci: install Asciidoctor in 'ci/install-dependencies.sh'
Documentation/technical/protocol-v2.txt: fix formatting
Documentation/technical/api-config.txt: fix formatting
Documentation/git-diff-tree.txt: fix formatting

Merge branch 'tz/asciidoctor-fixes'

Doc updates.

* tz/asciidoctor-fixes:
Documentation/git-status: fix titles in porcelain v2 section
Documentation/rev-list-options: wrap --date=<format> block with "--"

Merge branch 'js/check-docs-exe'

Dev support update.

* js/check-docs-exe:
check-docs: fix for setups where executables have an extension
check-docs: do not expect guide pages to correspond to commands
check-docs: really look at the documented commands again
docs: do not document the `git remote-testgit` command
docs: move gitremote-helpers into section 7

Merge branch 'nd/include-if-wildmatch'

A buglet in configuration parser has been fixed.

* nd/include-if-wildmatch:
config: correct '**' matching in includeIf patterns

Merge branch 'cb/doco-mono'

Clean-up markup in the documentation suite.

* cb/doco-mono:
doc: format pathnames and URLs as monospace.
doc/CodingGuidelines: URLs and paths as monospace

Merge branch 'tg/stash-in-c-show-default-to-p-fix'

A regression fix.

* tg/stash-in-c-show-default-to-p-fix:
stash: setup default diff output format if necessary

Merge branch 'js/stash-in-c-pathspec-fix'

Further fixes to "git stash" reimplemented in C.

* js/stash-in-c-pathspec-fix:
stash: pass pathspec as pointer
built-in stash: handle :(glob) pathspecs again
legacy stash: fix "rudimentary backport of -q"

Merge branch 'tb/stash-in-c-unused-param-fix'

Code clean-up.

* tb/stash-in-c-unused-param-fix:
stash: drop unused parameter

Merge branch 'ps/stash-in-c'

"git stash" rewritten in C.

* ps/stash-in-c: (28 commits)
tests: add a special setup where stash.useBuiltin is off
stash: optionally use the scripted version again
stash: add back the original, scripted `git stash`
stash: convert `stash--helper.c` into `stash.c`
stash: replace all `write-tree` child processes with API calls
stash: optimize `get_untracked_files()` and `check_changes()`
stash: convert save to builtin
stash: make push -q quiet
stash: convert push to builtin
stash: convert create to builtin
stash: convert store to builtin
stash: convert show to builtin
stash: convert list to builtin
stash: convert pop to builtin
stash: convert branch to builtin
stash: convert drop and clear to builtin
stash: convert apply to builtin
stash: mention options in `show` synopsis
stash: add tests for `git stash show` config
stash: rename test cases to be more descriptive
...

clone: do faster object check for partial clones

For partial clones, doing a full connectivity check is wasteful; we skip
promisor objects (which, for a partial clone, is all known objects), and
enumerating them all to exclude them from the connectivity check can
take a significant amount of time on large repos.

At most, we want to make sure that we get the objects referred to by any
wanted refs. For partial clones, just check that these objects were
transferred.

Signed-off-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

worktree: update is_bare heuristics

When "git branch -D <name>" is run, Git usually first checks if that
branch is currently checked out. But this check is not performed if the
Git directory of that repository is not at "<repo>/.git", which is the
case if that repository is a submodule that has its Git directory stored
as "super/.git/modules/<repo>", for example. This results in the branch
being deleted even though it is checked out.

This is because get_main_worktree() in worktree.c sets is_bare on a
worktree only using the heuristic that a repo is bare if the worktree's
path does not end in "/.git", and not bare otherwise. This is_bare code
was introduced in 92718b7438 ("worktree: add details to the worktree
struct", 2015-10-08), following a pre-core.bare heuristic. This patch
does 2 things:

- Teach get_main_worktree() to use is_bare_repository() instead,
introduced in 7d1864ce67 ("Introduce is_bare_repository() and
core.bare configuration variable", 2007-01-07) and updated in
e90fdc39b6 ("Clean up work-tree handling", 2007-08-01). This solves
the "git branch -D <name>" problem described above. However...

- If a repository has core.bare=1 but the "git" command is being run
from one of its secondary worktrees, is_bare_repository() returns
false (which is fine, since there is a worktree available). However,
treating the main worktree as non-bare when it is bare causes issues:
for example, failure to delete a branch from a secondary worktree
that is referred to by a main worktree's HEAD, even if that main
worktree is bare.

In order to avoid that, also check core.bare when setting is_bare. If
core.bare=1, trust it, and otherwise, use is_bare_repository().

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

rebase -i: run without forking rebase--interactive

When the builtin rebase starts an interactive rebase it parses the
options and then repackages them and forks
`rebase--interactive`. Separate the option parsing in
cmd_rebase__interactive() from the business logic to allow interactive
rebases can be run without forking `rebase__interactive` by calling
run_rebase_interactive() directly.

Starting interactive rebases without forking makes it easy to debug
the sequencer without worrying about attaching to child
processes. Ævar has also reported that some of the rebase perf tests
are 30% faster [1].

This patch also makes it easy to remove cmd_rebase__interactive() in
the future when git-legacy-rebase.sh and
git-rebase--preserve-merges.sh are retired.

[1] https://public-inbox.org/git/87y359cfjj.fsf@evledraar.gmail.com/

Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

rebase: use a common action enum

cmd_rebase() and cmd_rebase__interactive() used different enums to hold
the current action. Change to using a common enum so the values are the
same when we change `rebase -i` to avoid forking `rebase--interactive`.

Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

rebase -i: use struct rebase_options in do_interactive_rebase()

All the parameters that are passed to do_interactive_rebase() apart from
`flags` are already in `struct rebase_options` so there is no need to
pass them separately.

Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

rebase -i: use struct rebase_options to parse args

In order to run `rebase -i` without forking `rebase--interactive` it
will be convenient to use the same structure when parsing the options in
cmd_rebase() and cmd_rebase__interactive().

Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>