Merge branch 'js/test-git-installed'

Test fix for Windows.

* js/test-git-installed:
tests: explicitly use `test-tool.exe` on Windows

Merge branch 'js/abspath-part-inside-repo'

On a case-insensitive filesystem, we failed to compare the part of
the path that is above the worktree directory in an absolute
pathname, which has been corrected.

* js/abspath-part-inside-repo:
abspath_part_inside_repo: respect core.ignoreCase

Merge branch 'jt/namespaced-ls-refs-fix'

Fix namespace support in protocol v2.

* jt/namespaced-ls-refs-fix:
ls-refs: filter refs using namespace-stripped name

Merge branch 'ab/commit-graph-write-progress'

The codepath to show progress meter while writing out commit-graph
file has been improved.

* ab/commit-graph-write-progress:
commit-graph write: emit a percentage for all progress
commit-graph write: add itermediate progress
commit-graph write: remove empty line for readability
commit-graph write: add more descriptive progress output
commit-graph write: show progress for object search
commit-graph write: more descriptive "writing out" output
commit-graph write: add "Writing out" progress output
commit-graph: don't call write_graph_chunk_extra_edges() unnecessarily
commit-graph: rename "large edges" to "extra edges"

Merge branch 'ab/commit-graph-write-optim'

The codepath to write out commit-graph has been optimized by
following the usual pattern of visiting objects in in-pack order.

* ab/commit-graph-write-optim:
commit-graph write: use pack order when finding commits

Merge branch 'js/t6042-timing-fix'

Test update.

* js/t6042-timing-fix:
t6042: work around speed optimization on Windows

Merge branch 'jk/add-ignore-errors-bit-assignment-fix'

"git add --ignore-errors" did not work as advertised and instead
worked as an unintended synonym for "git add --renormalize", which
has been fixed.

* jk/add-ignore-errors-bit-assignment-fix:
add: use separate ADD_CACHE_RENORMALIZE flag

Merge branch 'js/mingw-unc-path-w-backslashes'

In Git for Windows, "git clone \\server\share\path" etc. that uses
UNC paths from command line had bad interaction with its shell
emulation.

* js/mingw-unc-path-w-backslashes:
mingw: special-case arguments to `sh`
mingw (t5580): document bug when cloning from backslashed UNC paths

Merge branch 'cc/test-ref-store-typofix'

An obvious typo in an assertion error message has been fixed.

* cc/test-ref-store-typofix:
helper/test-ref-store: fix "new-sha1" vs "old-sha1" typo

Merge branch 'jt/fetch-v2-sideband'

"git fetch" and "git upload-pack" learned to send all exchange over
the sideband channel while talking the v2 protocol.

* jt/fetch-v2-sideband:
tests: define GIT_TEST_SIDEBAND_ALL
{fetch,upload}-pack: sideband v2 fetch response
sideband: reverse its dependency on pkt-line
pkt-line: introduce struct packet_writer
pack-protocol.txt: accept error packets in any context
Use packet_reader instead of packet_read_line

Merge branch 'sg/obstack-cast-function-type-fix'

The compat/obstack code had casts that -Wcast-function-type
compilation option found questionable.

* sg/obstack-cast-function-type-fix:
compat/obstack: fix -Wcast-function-type warnings

Merge branch 'js/commit-graph-chunk-table-fix'

The codepath to read from the commit-graph file attempted to read
past the end of it when the file's table-of-contents was corrupt.

* js/commit-graph-chunk-table-fix:
Makefile: correct example fuzz build
commit-graph: fix buffer read-overflow
commit-graph, fuzz: add fuzzer for commit-graph

Merge branch 'ld/git-p4-shelve-update-fix'

"git p4" failed to update a shelved change when there were moved
files, which has been corrected.

* ld/git-p4-shelve-update-fix:
git-p4: handle update of moved/copied files when updating a shelve
git-p4: add failing test for shelved CL update involving move/copy

Merge branch 'jt/get-reference-with-commit-graph'

Micro-optimize the code that prepares commit objects to be walked
by "git rev-list" when the commit-graph is available.

* jt/get-reference-with-commit-graph:
revision: use commit graph in get_reference()

Merge branch 'js/filter-options-should-use-plain-int'

Update the protocol message specification to allow only the limited
use of scaled quantities. This is ensure potential compatibility
issues will not go out of hand.

* js/filter-options-should-use-plain-int:
filter-options: expand scaled numbers
tree:<depth>: skip some trees even when collecting omits
list-objects-filter: teach tree:# how to handle >0

Merge branch 'sb/more-repo-in-api'

The in-core repository instances are passed through more codepaths.

* sb/more-repo-in-api: (23 commits)
t/helper/test-repository: celebrate independence from the_repository
path.h: make REPO_GIT_PATH_FUNC repository agnostic
commit: prepare free_commit_buffer and release_commit_memory for any repo
commit-graph: convert remaining functions to handle any repo
submodule: don't add submodule as odb for push
submodule: use submodule repos for object lookup
pretty: prepare format_commit_message to handle arbitrary repositories
commit: prepare logmsg_reencode to handle arbitrary repositories
commit: prepare repo_unuse_commit_buffer to handle any repo
commit: prepare get_commit_buffer to handle any repo
commit-reach: prepare in_merge_bases[_many] to handle any repo
commit-reach: prepare get_merge_bases to handle any repo
commit-reach.c: allow get_merge_bases_many_0 to handle any repo
commit-reach.c: allow remove_redundant to handle any repo
commit-reach.c: allow merge_bases_many to handle any repo
commit-reach.c: allow paint_down_to_common to handle any repo
commit: allow parse_commit* to handle any repo
object: parse_object to honor its repository argument
object-store: prepare has_{sha1, object}_file to handle any repo
object-store: prepare read_object_file to deal with any repo
...

Makefile: improve SPARSE_FLAGS customisation

In order to enable greater user customisation of the SPARSE_FLAGS
variable, we introduce a new SP_EXTRA_FLAGS variable to use for
target specific settings. Without using the new variable, setting
the SPARSE_FLAGS on the 'make' command-line would also override the
value set by the target-specific rules in the Makefile (effectively
making them useless). Also, this enables the SP_EXTRA_FLAGS to be
used in the future for any other internal customisations, such as
for some platform specific values.

In addition, we initialise the SPARSE_FLAGS to the default (empty)
value using a conditional assignment (?=). This allows SPARSE_FLAGS
to be set from the environment as well as from the command-line.

Signed-off-by: Ramsay Jones <ramsay@ramsayjones.plus.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

config.mak.uname: remove obsolete SPARSE_FLAGS setting

An upcoming commit will change the semantics of the SPARSE_FLAGS
variable from an internal to a user only customisation variable.
The MinGW configuration section contains an obsolete setting for
this variable which was used (some years ago) to cater to an error
in the Win32 system header files. Since 'sparse' does not currently
support the MinGW platform, nobody on that platform can be relying
on this setting today. Remove this use of the SPARSE_FLAGS variable.

Signed-off-by: Ramsay Jones <ramsay@ramsayjones.plus.com>
Acked-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

git-submodule.sh: shorten submodule SHA-1s using rev-parse

Until now, `git submodule summary` was always emitting 7-character
SHA-1s that have a higher chance of being ambiguous for larger
repositories. Use `git rev-parse --short` instead, which will
determine suitable short SHA-1 lengths.

When a submodule hasn't been initialized with "submodule init" or
not cloned, `git rev-parse` would not work in it yet; as a fallback,
use the original method of cutting at 7 hexdigits.

Signed-off-by: Sven van Haastregt <svenvh@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

fetch-pack: clear alternate shallow when complete

When we write an alternate shallow file in update_shallow, we write it
into the lock file. The string stored in alternate_shallow_file is
copied from the lock file path, but it is freed the moment that the lock
file is closed, since we call strbuf_release to free that path.

This used to work, since we did not invoke git index-pack more than
once, but now that we do, we reuse the freed memory. Ensure we reset the
value to NULL to avoid using freed memory. git index-pack will read the
repository's shallow file, which will have been updated with the correct
information.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

doc-diff: don't `cd_to_toplevel`

`usage` tries to call $0, which might very well be "./doc-diff", so if
we `cd_to_toplevel` before calling `usage`, we'll end with an error to
the effect of "./doc-diff: not found" rather than a friendly `doc-diff
-h` output. This regressed in ad51743007 ("doc-diff: add --clean mode to
remove temporary working gunk", 2018-08-31) where we moved the call to
`cd_to_toplevel` to much earlier.

A general fix might be to teach git-sh-setup to save away the absolute
path for $0 and then use that, instead. I'm not aware of any portable
way of doing that, see, e.g., d2addc3b96 ("t7800: readlink may not be
available", 2016-05-31).

An early version of this patch moved `cd_to_toplevel` back to where it
was before ad51743007 and taught the "--clean" code to cd on its own.
But let's try instead to get rid of the cd-ing entirely. We don't really
need it and we can work with absolute paths instead. There's just one
use of $PWD that we need to adjust by simply dropping it.

Suggested-by: Jeff King <peff@peff.net>
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

diff-tree doc: correct & remove wrong documentation

The documentation saying that diff-tree didn't support anything except
literal prefixes hasn't been true since
d38f28093e ("tree_entry_interesting(): support wildcard matching",
2010-12-15), but this documentation was not updated at the time.

Since this command uses pathspecs like most other commands, there's no
need to show examples of how the various "cmd <revs> <paths>"
invocations work.

Furthermore, the "git diff-tree --abbrev 5319e4" example shown here
never worked. We'd ended up with that through a combination of
62b42d3487 ("docs: fix some antique example output", 2011-05-26) and
ac4e086929 ("Adjust core-git documentation to more recent Linus GIT.",
2005-05-05), but "git diff-tree <tree>" was always invalid.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t6120: test for describe with a bare repository

This ensures that nothing breaks the basic functionality of describe for
bare repositories. Please note that --broken and --dirty need a working
tree.

Signed-off-by: Sebastian Staudt <koraktor@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

describe: setup working tree for --dirty

We don't use NEED_WORK_TREE when running the git-describe builtin,
since you should be able to describe a commit even in a bare repository.
However, the --dirty flag does need a working tree. Since we don't call
setup_work_tree(), it uses whatever directory we happen to be in. That's
unlikely to match our index, meaning we'd say "dirty" even when the real
working tree is clean.

We can fix that by calling setup_work_tree() once we know that the user
has asked for --dirty.

The --broken option also needs a working tree. But because its
implementation calls git-diff-index we don‘t have to setup the working
tree in the git-describe process.

Signed-off-by: Sebastian Staudt <koraktor@gmail.com>
Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

travis-ci: make the OSX build jobs' 'brew update' more quiet

Before installing the necessary dependencies, our OSX build jobs run
'brew update --quiet'. This is problematic for two reasons:

- This '--quiet' flag apparently broke overnight, resulting in
errored builds:

+brew update --quiet
==> Downloading https://homebrew.bintray.com/bottles-portable-ruby/portable-ruby-2.3.7.mavericks.bottle.tar.gz
######################################################################## 100.0%
==> Pouring portable-ruby-2.3.7.mavericks.bottle.tar.gz
Usage: brew update_report [--preinstall]
The Ruby implementation of brew update. Never called manually.
--preinstall Run in 'auto-update' mode (faster, less
output).
-f, --force Override warnings and enable potentially
unsafe operations.
-d, --debug Display any debugging information.
-v, --verbose Make some output more verbose.
-h, --help Show this message.
Error: invalid option: --quiet
The command "ci/install-dependencies.sh" failed and exited with 1 during .

I belive that this breakage will be noticed and fixed soon-ish, so
we could probably just wait a bit for this issue to solve itself,
but:

- 'brew update --quiet' wasn't really quiet in the first place, as
it listed over about 2000 lines worth of available packages that
we absolutely don't care about, see e.g. one of the latest
'master' builds:

https://travis-ci.org/git/git/jobs/486134962#L113

So drop this '--quiet' option and redirect 'brew update's standard
output to /dev/null to make it really quiet, thereby making the OSX
builds work again despite the above mentioned breakage.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

git-commit.txt: better description what it does

The description of git-commit jumps right into the commit content, which
is important, but it fails to mention how the commit is "added" to the
repository. Update the first paragraph saying a bit more about branch
update to fill this gap.

While at there, add a couple linkgit references when the command is
first mentioned.

Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

various: tighten constness of some local variables

Signed-off-by: Shahzad Lone <shahzadlone@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

help: align the longest command in the command listing

"longest" is used to determine how many extra spaces we need to print
to keep the command description aligned. For the longest command, we
should print no extra space instead of one, or we'll get unaligned
output like this (notice the "checkout" line):

grow, mark and tweak your common history
branch List, create, or delete branches
checkout Switch branches or restore working tree files
commit Record changes to the repository
diff Show changes between commits, commit and ...
merge Join two or more development histories together
rebase Reapply commits on top of another base tip
tag Create, list, delete or verify a tag ...

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t1512: test ambiguous cat-file --batch and --batch-output

Test the new "ambiguous" result from cat-file --batch and
--batch-check. This is in t1512 instead of t1006 since
we need a repo with ambiguous object_id names.

Signed-off-by: Eric Wong <e@80x24.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Support working-tree-encoding "UTF-16LE-BOM"

Users who want UTF-16 files in the working tree set the .gitattributes
like this:
test.txt working-tree-encoding=UTF-16

The unicode standard itself defines 3 allowed ways how to encode UTF-16.
The following 3 versions convert all back to 'g' 'i' 't' in UTF-8:

a) UTF-16, without BOM, big endian:
$ printf "\000g\000i\000t" | iconv -f UTF-16 -t UTF-8 | od -c
0000000 g i t

b) UTF-16, with BOM, little endian:
$ printf "\377\376g\000i\000t\000" | iconv -f UTF-16 -t UTF-8 | od -c
0000000 g i t

c) UTF-16, with BOM, big endian:
$ printf "\376\377\000g\000i\000t" | iconv -f UTF-16 -t UTF-8 | od -c
0000000 g i t

Git uses libiconv to convert from UTF-8 in the index into ITF-16 in the
working tree.
After a checkout, the resulting file has a BOM and is encoded in "UTF-16",
in the version (c) above.
This is what iconv generates, more details follow below.

iconv (and libiconv) can generate UTF-16, UTF-16LE or UTF-16BE:

d) UTF-16
$ printf 'git' | iconv -f UTF-8 -t UTF-16 | od -c
0000000 376 377 \0 g \0 i \0 t

e) UTF-16LE
$ printf 'git' | iconv -f UTF-8 -t UTF-16LE | od -c
0000000 g \0 i \0 t \0

f) UTF-16BE
$ printf 'git' | iconv -f UTF-8 -t UTF-16BE | od -c
0000000 \0 g \0 i \0 t

There is no way to generate version (b) from above in a Git working tree,
but that is what some applications need.
(All fully unicode aware applications should be able to read all 3 variants,
but in practise we are not there yet).

When producing UTF-16 as an output, iconv generates the big endian version
with a BOM. (big endian is probably chosen for historical reasons).

iconv can produce UTF-16 files with little endianess by using "UTF-16LE"
as encoding, and that file does not have a BOM.

Not all users (especially under Windows) are happy with this.
Some tools are not fully unicode aware and can only handle version (b).

Today there is no way to produce version (b) with iconv (or libiconv).
Looking into the history of iconv, it seems as if version (c) will
be used in all future iconv versions (for compatibility reasons).

Solve this dilemma and introduce a Git-specific "UTF-16LE-BOM".
libiconv can not handle the encoding, so Git pick it up, handles the BOM
and uses libiconv to convert the rest of the stream.
(UTF-16BE-BOM is added for consistency)

Rported-by: Adrián Gimeno Balaguer <adrigibal@gmail.com>
Signed-off-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

init docs: correct a punctuation typo

Signed-off-by: Kyle Meyer <kyle@kyleam.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

rebase -x: sanity check command

If the user gives an empty argument to --exec then git creates a todo
list that it cannot parse. The rebase starts to run before erroring out
with

error: missing arguments for exec
error: invalid line 2: exec
You can fix this with 'git rebase --edit-todo' and then run 'git rebase --continue'.
Or you can abort the rebase with 'git rebase --abort'.

Instead check for empty commands before starting the rebase.

Also check that the command does not contain any newlines as the
todo-list format is unable to cope with multiline commands. Note that
this changes the behavior, before this change one could do

git rebase --exec='echo one
exec echo two'

and it would insert two exec lines in the todo list, now it will error
out.

Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Fourth batch after 2.20

Signed-off-by: Junio C Hamano <gitster@pobox.com>

Merge branch 'it/log-format-source'

Custom userformat "log --format" learned %S atom that stands for
the tip the traversal reached the commit from, i.e. --source.

* it/log-format-source:
log: add %S option (like --source) to log --format

Merge branch 'js/add-e-clear-patch-before-stating'

"git add -e" got confused when the change it wants to let the user
edit is smaller than the previous change that was left over in a
temporary file.

* js/add-e-clear-patch-before-stating:
add --edit: truncate the patch file

Merge branch 'bc/tree-walk-oid'

The code to walk tree objects has been taught that we may be
working with object names that are not computed with SHA-1.

* bc/tree-walk-oid:
cache: make oidcpy always copy GIT_MAX_RAWSZ bytes
tree-walk: store object_id in a separate member
match-trees: use hashcpy to splice trees
match-trees: compute buffer offset correctly when splicing
tree-walk: copy object ID before use

Merge branch 'jt/upload-pack-deepen-relative-proto-v2'

"git fetch --deepen=<more>" has been corrected to work over v2
protocol.

* jt/upload-pack-deepen-relative-proto-v2:
upload-pack: teach deepen-relative in protocol v2
fetch-pack: do not take shallow lock unnecessarily

Merge branch 'jk/remote-insteadof-cleanup'

Code clean-up.

* jk/remote-insteadof-cleanup:
remote: check config validity before creating rewrite struct

Merge branch 'ms/http-no-more-failonerror'

Debugging help for http transport.

* ms/http-no-more-failonerror:
test: test GIT_CURL_VERBOSE=1 shows an error
remote-curl: unset CURLOPT_FAILONERROR
remote-curl: define struct for CURLOPT_WRITEFUNCTION
http: enable keep_error for HTTP requests
http: support file handles for HTTP_KEEP_ERROR

Merge branch 'os/rebase-runs-post-checkout-hook'

"git rebase" internally runs "checkout" to switch between branches,
and the command used to call the post-checkout hook, but the
reimplementation stopped doing so, which is getting fixed.

* os/rebase-runs-post-checkout-hook:
rebase: run post-checkout hook on checkout
t5403: simplify by using a single repository

Merge branch 'bc/sha-256'

Add sha-256 hash and plug it through the code to allow building Git
with the "NewHash".

* bc/sha-256:
hash: add an SHA-256 implementation using OpenSSL
sha256: add an SHA-256 implementation using libgcrypt
Add a base implementation of SHA-256 support
commit-graph: convert to using the_hash_algo
t/helper: add a test helper to compute hash speed
sha1-file: add a constant for hash block size
t: make the sha1 test-tool helper generic
t: add basic tests for our SHA-1 implementation
cache: make hashcmp and hasheq work with larger hashes
hex: introduce functions to print arbitrary hashes
sha1-file: provide functions to look up hash algorithms
sha1-file: rename algorithm to "sha1"

Merge branch 'sb/submodule-recursive-fetch-gets-the-tip'

"git fetch --recurse-submodules" may not fetch the necessary commit
that is bound to the superproject, which is getting corrected.

* sb/submodule-recursive-fetch-gets-the-tip:
fetch: ensure submodule objects fetched
submodule.c: fetch in submodules git directory instead of in worktree
submodule: migrate get_next_submodule to use repository structs
repository: repo_submodule_init to take a submodule struct
submodule: store OIDs in changed_submodule_names
submodule.c: tighten scope of changed_submodule_names struct
submodule.c: sort changed_submodule_names before searching it
submodule.c: fix indentation
sha1-array: provide oid_array_filter

Merge branch 'jt/fetch-pack-v2'

"git fetch-pack" now can talk the version 2 protocol.

* jt/fetch-pack-v2:
fetch-pack: support protocol version 2

Merge branch 'jk/proto-v2-hidden-refs-fix'

The v2 upload-pack protocol implementation failed to honor
hidden-ref configuration, which has been corrected.
An earlier attempt reverted out of 'next'.

* jk/proto-v2-hidden-refs-fix:
upload-pack: support hidden refs with protocol v2

Merge branch 'jk/save-getenv-result'

There were many places the code relied on the string returned from
getenv() to be non-volatile, which is not true, that have been
corrected.

* jk/save-getenv-result:
builtin_diff(): read $GIT_DIFF_OPTS closer to use
merge-recursive: copy $GITHEAD strings
init: make a copy of $GIT_DIR string
config: make a copy of $GIT_CONFIG string
commit: copy saved getenv() result
get_super_prefix(): copy getenv() result

Merge branch 'pw/diff-color-moved-ws-fix'

"git diff --color-moved-ws" updates.

* pw/diff-color-moved-ws-fix:
diff --color-moved-ws: handle blank lines
diff --color-moved-ws: modify allow-indentation-change
diff --color-moved-ws: optimize allow-indentation-change
diff --color-moved=zebra: be stricter with color alternation
diff --color-moved-ws: fix false positives
diff --color-moved-ws: demonstrate false positives
diff: allow --no-color-moved-ws
Use "whitespace" consistently
diff: document --no-color-moved

Merge branch 'ja/doc-build-l10n'

Prepare Documentation/Makefile so that manpage localization can
reuse it by overriding and tweaking the list of build products.

* ja/doc-build-l10n:
Documentation/Makefile add optional targets for l10n

Merge branch 'js/rebase-i-redo-exec'

"git rebase -i" learned to re-execute a command given with 'exec'
to run after it failed the last time.

* js/rebase-i-redo-exec:
rebase: introduce a shortcut for --reschedule-failed-exec
rebase: add a config option to default to --reschedule-failed-exec
rebase: introduce --reschedule-failed-exec

Merge branch 'cc/fetch-error-message-fix'

Error message fix.

* cc/fetch-error-message-fix:
fetch: fix extensions.partialclone name in error message

Merge branch 'cc/partial-clone-doc-typofix'

Doc fix.

* cc/partial-clone-doc-typofix:
partial-clone: add missing 'is' in doc

Merge branch 'kg/external-diff-save-env'

The code to drive GIT_EXTERNAL_DIFF command relied on the string
returned from getenv() to be non-volatile, which is not true, that
has been corrected.

* kg/external-diff-save-env:
diff: ensure correct lifetime of external_diff_cmd

Makefile: add coverage-prove target

Sometimes there are test failures in the 'pu' branch. This
is somewhat expected for a branch that takes the very latest
topics under development, and those sometimes have semantic
conflicts that only show up during test runs. This also can
happen when running the test suite with different GIT_TEST_*
environment variables that interact in unexpected ways

This causes a problem for the test coverage reports, as
the typical 'make coverage-test coverage-report' run halts
at the first failed test. If that test is early in the
suite, then many valuable tests are not exercising the code
and the coverage report becomes noisy with false positives.

Add a new 'coverage-prove' target to the Makefile,
modeled after the 'coverage-test' target. This compiles
the source using the coverage flags, then runs the test
suite using the 'prove' tool. Since the coverage
machinery is not thread-safe, enforce that the tests
are run in sequence by appending '-j1' to GIT_PROVE_OPTS.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Add `human` date format tests.

When using `human` several fields are suppressed depending on the time
difference between the reference date and the local computer date. In
cases where the difference is less than a year, the year field is
supppressed. If the time is less than a day; the month and year is
suppressed.

Use TEST_DATE_NOW environment variable when using the test-tool to
hold the expected output strings constant.

Signed-off-by: Stephen P. Smith <ischis2@cox.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Add `human` format to test-tool

Add the human format support to the test tool so that
GIT_TEST_DATE_NOW can be used to specify the current time.

The get_time() helper function was created and and checks the
GIT_TEST_DATE_NOW environment variable. If GIT_TEST_DATE_NOW is set,
then that date is used instead of the date returned by by
gettimeofday().

All calls to gettimeofday() were replaced by calls to get_time().

Renamed occurances of TEST_DATE_NOW to GIT_TEST_DATE_NOW since the
variable is now used in the get binary and not just in the test-tool.

Signed-off-by: Stephen P. Smith <ischis2@cox.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

ci: parallelize testing on Windows

The fact that Git's test suite is implemented in Unix shell script that
is as portable as we can muster, combined with the fact that Unix shell
scripting is foreign to Windows (and therefore has to be emulated),
results in pretty abysmal speed of the test suite on that platform, for
pretty much no other reason than that language choice.

For comparison: while the Linux build & test is typically done within
about 8 minutes, the Windows build & test typically lasts about 80
minutes in Azure Pipelines.

To help with that, let's use the Azure Pipeline feature where you can
parallelize jobs, make jobs depend on each other, and pass artifacts
between them.

The tests are distributed using the following heuristic: listing all
test scripts ordered by size in descending order (as a cheap way to
estimate the overall run time), every Nth script is run (where N is the
total number of parallel jobs), starting at the index corresponding to
the parallel job. This slicing is performed by a new function that is
added to the `test-tool`.

To optimize the overall runtime of the entire Pipeline, we need to move
the Windows jobs to the beginning (otherwise there would be a very
decent chance for the Pipeline to be run only the Windows build, while
all the parallel Windows test jobs wait for this single one).

We use Azure Pipelines Artifacts for both the minimal Git for Windows
SDK as well as the built executables, as deduplication and caching close
to the agents makes that really fast. For comparison: while downloading
and unpacking the minimal Git for Windows SDK via PowerShell takes only
one minute (down from anywhere between 2.5 to 7 when using a shallow
clone), uploading it as Pipeline Artifact takes less than 30s and
downloading and unpacking less than 20s (sometimes even as little as
only twelve seconds).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

ci: speed up Windows phase

As Unix shell scripting comes at a hefty price on Windows, we have to
see where we can save some time to run the test suite.

Let's skip the chain linting and the bin-wrappers/ redirection on
Windows; this seems to shave of anywhere between 10-30% from the overall
runtime.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

tests: optionally skip bin-wrappers/

This speeds up the tests by a bit on Windows, where running Unix shell
scripts (and spawning processes) is not exactly a cheap operation.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t0061: workaround issues with --with-dashes and RUNTIME_PREFIX

When building Git with RUNTIME_PREFIX and starting a test helper from
t/helper/, it fails to detect a system prefix. The reason is that the
RUNTIME_PREFIX feature wants to use the location of the Git executable
to determine where the support files can be found, e.g. system-wide Git
config or the translations. This does not make any sense for the test
helpers, though, as they are distinctly not in a directory structure
resembling the final installation location of Git.

That is the reason why the test helpers rely on environment variables to
indicate the location of the needed support files, e.g.
GIT_TEXTDOMAINDIR. If this information is missing, the output will
contain warnings like this one:

RUNTIME_PREFIX requested, but prefix computation failed. [...]

In t0061, we did not expect that to happen, and it actually does not
happen in the regular case, because bin-wrappers/test-tool specifically
sets GIT_TEXTDOMAINDIR (and as a consequence, nothing in test-tool needs
to know anything about any runtime prefix).

However, with --with-dashes, bin-wrappers/test-tool is no longer called,
but t/helper/test-tool is called directly instead.

So let's just ignore the RUNTIME_PREFIX warning.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

tests: add t/helper/ to the PATH with --with-dashes

We really need to be able to find the test helpers... Really. This
change was forgotten when we moved the test helpers into t/helper/

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

mingw: try to work around issues with the test cleanup

It seems that every once in a while in the Git for Windows SDK, there
are some transient file locking issues preventing the test clean up to
delete the trash directory. Let's be gentle and try again five seconds
later, and only error out if it still fails the second time.

This change helps Windows, and does not hurt any other platform
(normally, it is highly unlikely that said deletion fails, and if it
does, normally it will fail again even 5 seconds later).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

tests: include detailed trace logs with --write-junit-xml upon failure

The JUnit XML format lends itself to be presented in a powerful UI,
where you can drill down to the information you are interested in very
quickly.

For test failures, this usually means that you want to see the detailed
trace of the failing tests.

With Travis CI, we passed the `--verbose-log` option to get those
traces. However, that seems excessive, as we do not need/use the logs in
almost all of those cases: only when a test fails do we have a way to
include the trace.

So let's do something different when using Azure DevOps: let's run all
the tests with `--quiet` first, and only if a failure is encountered,
try to trace the commands as they are executed.

Of course, we cannot turn on `--verbose-log` after the fact. So let's
just re-run the test with all the same options, adding `--verbose-log`.
And then munging the output file into the JUnit XML on the fly.

Note: there is an off chance that re-running the test in verbose mode
"fixes" the failures (and this does happen from time to time!). That is
a possibility we should be able to live with. Ideally, we would label
this as "Passed upon rerun", and Azure Pipelines even know about that
outcome, but it is not available when using the JUnit XML format for
now:
https://github.com/Microsoft/azure-pipelines-agent/blob/master/src/Agent.Worker/TestResults/JunitResultReader.cs

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

tests: avoid calling Perl just to determine file sizes

It is a bit ridiculous to spin up a full-blown Perl instance (especially
on Windows, where that means spinning up a full POSIX emulation layer,
AKA the MSYS2 runtime) just to tell how large a given file is.

So let's just use the test-tool to do that job instead.

This command will also be used over the next commits, to allow for
cutting out individual test cases' verbose log from the file generated
via --verbose-log.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

README: add a build badge (status of the Azure Pipelines build)

Just like so many other OSS projects, we now also have a build badge.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

mingw: be more generous when wrapping up the setitimer() emulation

Every once in a while, the Azure Pipeline fails with some semi-random

error: timer thread did not terminate timely

This error message means that the thread that is used to emulate the
setitimer() function did not terminate within 1,000 milliseconds.

The most likely explanation (and therefore the one we should assume to
be true, according to Occam's Razor) is that the timeout of one second
is simply not enough because we try to run so many tasks in parallel.

So let's give it ten seconds instead of only one. That should be enough.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

ci: use git-sdk-64-minimal build artifact

Instead of a shallow fetch followed by a sparse checkout, we are
better off by using a separate, dedicated Pipeline that bundles
the SDK as a build artifact, and then consuming that build artifact
here.

In fact, since this artifact will be used a lot, we spent substantial
time on figuring out a minimal subset of the Git for Windows SDK, just
enough to build and test Git. The result is a size reduction from around
1GB (compressed) to around 55MB (compressed). This also comes with the
change where we now call `usr\bin\bash.exe` directly, as `git-cmd.exe`
is not included in the minimal SDK.

That reduces the time to initialize Git for Windows' SDK from anywhere
between 2m30s-7m to a little over 1m.

Note: in theory, we could also use the DownloadBuildArtifacts@0 task
here. However, restricted permissions that are in effect when building
from forks would let this fail for PR builds, defeating the whole
purpose of the Azure Pipelines support for git.git.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

ci: add a Windows job to the Azure Pipelines definition

Previously, we did not have robust support for Windows in our CI
definition, simply because Travis cannot accommodate our needs (even
after Travis added experimental Windows support very recently, it takes
longer than Travis' 50 minute timeout to build Git and run the test
suite on Windows). Instead, we used a hack that started a dedicated
Azure Pipeline from Travis and waited for the output, often timing out
(which is quite fragile, as we found out).

With this commit, we finally have first-class support for Windows in our
CI definition (in the Azure Pipelines one, that is).

Due to our reliance on Unix shell scripting in the test suite, combined
with the challenges on executing such scripts on Windows, the Windows
job currently takes a whopping ~1h20m to complete. Which is *far* longer
than the next-longest job takes (linux-gcc, ~35m).

Now, Azure Pipelines's free tier for open source projects (such as Git)
offers up to 10 concurrent jobs for free, meaning that the overall run
time will be dominated by the slowest job(s).

Therefore, it makes sense to start the Windows job first, to minimize
the time the entire build takes from start to end (which is now pretty
safely the run time of the Windows job).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Add a build definition for Azure DevOps

This commit adds an azure-pipelines.yml file which is Azure DevOps'
equivalent to Travis CI's .travis.yml.

The main idea is to replicate the Travis configuration as faithfully as
possible, to make it easy to compare the Azure Pipeline builds to the
Travis ones (spoiler: some parts, especially the macOS jobs, are way
faster in Azure Pileines). Meaning: the number and the order of the jobs
added in this commit faithfully replicates what we have in .travis.yml.

Note: Our .travis.yml configuration has a Windows part that is *not*
replicated in the Azure Pipelines definition. The reason is easy to see:
As Travis cannot support our Windws needs (even with the preliminary
Windows support that was recently added to Travis after waiting for
*years* for that feature, our test suite would simply hit Travis'
timeout every single time).

To make things a bit easier to understand, we refrain from using the
`matrix` feature here because (while it is powerful) it can be a bit
confusing to users who are not familiar with CI setups. Therefore, we
use a separate phase even for similar configurations (such as GCC vs
Clang on Linux, GCC vs Clang on macOS).

Also, we make use of the shiny new feature we just introduced where the
test suite can output JUnit-style .xml files. This information is made
available in a nice UI that allows the viewer to filter by phase and/or
test number, and to see trends such as: number of (failing) tests, time
spent running the test suite, etc. (While this seemingly contradicts the
intention to replicate the Travis configuration as faithfully as
possible, it is just too nice to show off that capability here already.)

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

ci/lib.sh: add support for Azure Pipelines

This patch introduces a conditional arm that defines some environment
variables and a function that displays the URL given the job id (to
identify previous runs for known-good trees).

Because Azure Pipeline's macOS agents already have git-lfs and gettext
installed, we can leave `BREW_INSTALL_PACKAGES` empty (unlike in
Travis' case).

Note: this patch does not introduce an Azure Pipelines definition yet;
That is left for the next patch.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

tests: optionally write results as JUnit-style .xml

This will come in handy when publishing the results of Git's test suite
during an automated Azure DevOps run.

Note: we need to make extra sure that invalid UTF-8 encoding is turned
into valid UTF-8 (using the Replacement Character, \uFFFD) because
t9902's trace contains such invalid byte sequences, and the task in the
Azure Pipeline that uploads the test results would refuse to do anything
if it was asked to parse an .xml file with invalid UTF-8 in it.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

docs/config: clarify "text property" in core.eol

The word "property" is vague here. Let's spell out that we mean the path
must be marked with the text attribute.

While we're here, let's make the paragraph a little easier to read by
de-emphasizing the "when core.autocrlf is false" bit. Putting it in the
first sentence obscures the main content, and many readers won't care
about autocrlf (i.e., anyone who is just following the gitattributes(7)
advice, which mainly discusses "text" and "core.eol").

Helped-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

doc/gitattributes: clarify "autocrlf overrides eol"

We only override core.eol with core.autocrlf when the latter is set to
something besides "false". Let's make this more clear, and point the
reader to the git-config definitions, which discuss this in more detail.

Noticed-by: Sergey Lukashev <lukashev.s@ya.ru>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

test-lint: only use only sed [-n] [-e command] [-f command_file]

From `man sed` (on a Mac OS X box):
The -E, -a and -i options are non-standard FreeBSD extensions and may not be available
on other operating systems.

From `man sed` on a Linux box:
REGULAR EXPRESSIONS
POSIX.2 BREs should be supported, but they aren't completely because of
performance problems. The \n sequence in a regular expression matches the newline
character, and similarly for \a, \t, and other sequences.
The -E option switches to using extended regular expressions instead; the -E option
has been supported for years by GNU sed, and is now included in POSIX.

Well, there are still a lot of systems out there, which don't support it.
Beside that, IEEE Std 1003.1TM-2017, see
http://pubs.opengroup.org/onlinepubs/9699919799/
does not mention -E either.

To be on the safe side, don't allow -E (or -r, which is GNU).
Change check-non-portable-shell.pl to only accept the portable options:
sed [-n] [-e command] [-f command_file]

Reported-by: SZEDER Gábor <szeder.dev@gmail.com>
Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

pack-objects: merge read_lock and lock in packing_data struct

Rename the packing_data lock to obd_lock and upgrade it to a recursive
mutex to make it suitable for current read_lock usages. Additionally
remove the superfluous #ifndef NO_PTHREADS guard around mutex
initialization in prepare_packing_data as the mutex functions
themselves are already protected.

Signed-off-by: Patrick Hogg <phogg@novamoon.net>
Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

pack-objects: move read mutex to packing_data struct

ac77d0c37 ("pack-objects: shrink size field in struct object_entry",
2018-04-14) added an extra usage of read_lock/read_unlock in the newly
introduced oe_get_size_slow for thread safety in parallel calls to
try_delta(). Unfortunately oe_get_size_slow is also used in serial
code, some of which is called before the first invocation of
ll_find_deltas. As such the read mutex is not guaranteed to be
initialized.

Resolve this by moving the read mutex to packing_data and initializing
it in prepare_packing_data which is initialized in cmd_pack_objects.

Signed-off-by: Patrick Hogg <phogg@novamoon.net>
Reviewed-by: Duy Nguyen <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

git-instaweb: add Python builtin http.server support

With this patch it is possible to launch git-instaweb by using
Python http.server CGI handler via `-d python` option.

git-instaweb generates a small wrapper around the http.server
(in GIT_DIR/gitweb/) that address a limitation of the CGI handler
where CGI scripts have to be in a cgi-bin subdirectory and
directory index can't be easily changed. To keep the implementation
small, gitweb is running on url `/cgi-bin/gitweb.cgi` and an automatic
redirection is done when opening `/`.

The generated wrapper is compatible with both Python 2 and 3.

Python is by default installed on most modern Linux distributions
which enables running `git instaweb -d python` without needing
anything else.

Signed-off-by: Arti Zirk <arti.zirk@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

test-date: add a subcommand to measure times in shell scripts

In the next commit, we want to teach Git's test suite to optionally
output test results in JUnit-style .xml files. These files contain
information about the time spent. So we need a way to measure time.

While we could use `date +%s` for that, this will give us only seconds,
i.e. very coarse-grained timings.

GNU `date` supports `date +%s.%N` (i.e. nanosecond-precision output),
but there is no equivalent in BSD `date` (read: on macOS, we would not
be able to obtain precise timings).

So let's introduce `test-tool date getnanos`, with an optional start
time, that outputs preciser values. Note that this might not actually
give us nanosecond precision on some platforms, but it will give us as
precise information as possible, without the portability issues of shell
commands.

Granted, it is a bit pointless to try measuring times accurately in
shell scripts, certainly to nanosecond precision. But it is better than
second-granularity.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

ci: use a junction on Windows instead of a symlink

Symbolic links are still not quite as easy to use on Windows as on Linux
(for example, on versions older than Windows 10, only administrators can
create symlinks, and on Windows 10 you still need to be in developer
mode for regular users to have permission), but NTFS junctions can give
us a way out.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

ci: inherit --jobs via MAKEFLAGS in run-build-and-tests

Let's not decide in the generic ci/ part how many jobs to run in
parallel; different CI configurations would favor a different number of
parallel jobs, and it is easy enough to hand that information down via
the `MAKEFLAGS` variable.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

ci/lib.sh: encapsulate Travis-specific things

The upcoming patches will allow building git.git via Azure Pipelines
(i.e. Azure DevOps' Continuous Integration), where variable names and
URLs look a bit different than in Travis CI.

Also, the configurations of the available agents are different. For
example, Travis' and Azure Pipelines' macOS agents are set up
differently, so that on Travis, we have to install the git-lfs and
gettext Homebrew packages, and on Azure Pipelines we do not need to.
Likewise, Azure Pipelines' Ubuntu agents already have asciidoctor
installed.

Finally, on Azure Pipelines the natural way is not to base64-encode tar
files of the trash directories of failed tests, but to publish build
artifacts instead. Therefore, that code to log those base64-encoded tar
files is guarded to be Travis-specific.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

ci: rename the library of common functions

The name is hard-coded to reflect that we use Travis CI for continuous
testing.

In the next commits, we will extend this to be able use Azure DevOps,
too.

So let's adjust the name to make it more generic.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

travis: fix skipping tagged releases

When building a PR, TRAVIS_BRANCH refers to the *target branch*.
Therefore, if a PR targets `master`, and `master` happened to be tagged,
we skipped the build by mistake.

Fix this by using TRAVIS_PULL_REQUEST_BRANCH (i.e. the *source branch*)
when available, falling back to TRAVIS_BRANCH (i.e. for CI builds, also
known as "push builds").

Let's give it a new variable name, too: CI_BRANCH (as it is different
from TRAVIS_BRANCH). This also prepares for the upcoming patches which
will make our ci/* code a bit more independent from Travis and open it
to other CI systems (in particular to Azure Pipelines).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

implicit interactive rebase: don't run sequence editor

If GIT_SEQUENCE_EDITOR is set then rebase runs it when executing
implicit interactive rebases which are supposed to appear
non-interactive to the user. Fix this by setting GIT_SEQUENCE_EDITOR=:
rather than GIT_EDITOR=:. A couple of tests relied on the old behavior
so they are updated to work with the new regime.

Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

object_as_type: initialize commit-graph-related fields of 'struct commit'

When the commit graph and generation numbers were introduced in
commits 177722b344 (commit: integrate commit graph with commit
parsing, 2018-04-10) and 83073cc994 (commit: add generation number to
struct commit, 2018-04-25), they tried to make sure that the
corresponding 'graph_pos' and 'generation' fields of 'struct commit'
are initialized conservatively, as if the commit were not included in
the commit-graph file.

Alas, initializing those fields only in alloc_commit_node() missed the
case when an object that happens to be a commit is first looked up via
lookup_unknown_object(), and is then later converted to a 'struct
commit' via the object_as_type() helper function (either calling it
directly, or as part of a subsequent lookup_commit() call).
Consequently, both of those fields incorrectly remain set to zero,
which means e.g. that the commit is present in and is the first entry
of the commit-graph file. This will result in wrong timestamp, parent
and root tree hashes, if such a 'struct commit' instance is later
filled from the commit-graph.

Extract the initialization of 'struct commit's fields from
alloc_commit_node() into a helper function, and call it from
object_as_type() as well, to make sure that it properly initializes
the two commit-graph-related fields, too. With this helper function
it is hopefully less likely that any new fields added to 'struct
commit' in the future would remain uninitialized.

With this change alloc_commit_index() won't have any remaining callers
outside of 'alloc.c', so mark it as static.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

strbuf.cocci: suggest strbuf_addbuf() to add one strbuf to an other

The best way to add one strbuf to an other is via:

strbuf_addbuf(&sb, &sb2);

This is a bit more idiomatic and efficient than:

strbuf_addstr(&sb, sb2.buf);

because the size of the second strbuf is known and thus it can spare a
strlen() call, and much more so than:

strbuf_addf(&sb, "%s", sb2.buf);

because it can spare the whole vsnprintf() formatting magic.

Add new semantic patches to 'contrib/coccinelle/strbuf.cocci' to catch
these undesired patterns and to suggest strbuf_addbuf() instead.

Luckily, our codebase is already clean from any such undesired
patterns (but one of the in-flight topics just tried to sneak in such
a strbuf_addf() call).

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

fetch: prefer suffix substitution in compact fetch.output

I have a remote named "jch" and it has a branch with the same name. And
fetch.output is set to "compact". Fetching this remote looks like this

From https://github.com/gitster/git
+ eb7fd39f6b...835363af2f jch -> */jch (forced update)
6f11fd5edb..59b12ae96a nd/config-move-to -> jch/*
* [new branch] nd/diff-parseopt -> jch/*
* [new branch] nd/the-index-final -> jch/*

Notice that the local side of branch jch starts with "*" instead of
ending with it like the rest. It's not exactly wrong. It just looks
weird.

This patch changes the find-and-replace code a bit to try finding prefix
first before falling back to strstr() which finds a substring from left
to right. Now we have something less OCD

From https://github.com/gitster/git
+ eb7fd39f6b...835363af2f jch -> jch/* (forced update)
6f11fd5edb..59b12ae96a nd/config-move-to -> jch/*
* [new branch] nd/diff-parseopt -> jch/*
* [new branch] nd/the-index-final -> jch/*

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

convert: drop path parameter from actual conversion functions

The caller is responsible for looking up the attributes,
after which point we no longer care about the path at which
the content is found.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

convert: drop len parameter from conversion checks

We've already extracted the useful information into our text_stat
struct, so the length is no longer needed.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

config: drop unused parameter from maybe_remove_section()

We don't need the contents buffer to drop a section; the parse
information in the config_store_data parameter is enough for our logic.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

show_date_relative(): drop unused "tz" parameter

The timestamp we receive is in epoch time, so there's no need for a
timezone parameter to interpret it. The matching show_date() uses "tz"
to show dates in author local time, but relative dates show only the
absolute time difference. The author's location is irrelevant, barring
relativistic effects from using Git close to the speed of light.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

column: drop unused "opts" parameter in item_length()

There are no column options which impact the length computation. In
theory there might be, but this is a file-local function, so it will be
trivial to re-add the parameter should it ever be useful.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

create_bundle(): drop unused "header" parameter

There's no need to pass a header struct to create_bundle(); it writes
the header information directly to a descriptor (and does not report
back details to the caller).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

apply: drop unused "def" parameter from find_name_gnu()

We use the "def" parameter in find_name_common() for some heuristics,
but they are not necessary with the less-ambiguous GNU format. Let's
drop this unused parameter.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

match-trees: drop unused path parameter from score functions

The scores do not take the particular path into account at
all. It's possible they could, but these are all static file-local
functions. It won't be a big deal to re-add the parameter if they ever
need it.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

combine-diff: treat --dirstat like --stat

Currently "--cc --dirstat" will show nothing for a merge. Like
--shortstat and --summary in the previous two patches, it probably makes
sense to treat it like we do --stat, and show a stat against the
first-parent.

This case is less obviously correct than for --shortstat and --summary,
as those are basically variants of --stat themselves. It's possible we
could develop a multi-parent combined dirstat format, in which case we
might regret defining this first-parent behavior. But the same could be
said for --stat, and in the 12+ years of it showing first-parent stats,
nobody has complained.

So showing the first-parent dirstat is at least _useful_, and if we
later develop a clever multi-parent stat format, we'd probably have to
deal with --stat anyway.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

combine-diff: treat --summary like --stat

Currently "--cc --summary" on a merge shows nothing. Since we show "--cc
--stat" as a stat against the first parent, and because --summary is
typically used in combination with --stat, it makes sense to treat them
both the same way.

Note that we have to tweak t4013's setup a bit to test this case, as the
existing merges do not have any --summary results against their first
parent. But since the merge at the tip of 'master' does add and remove
files with respect to the second parent, we can just make a reversed
doppelganger merge where the parents are swapped.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

combine-diff: treat --shortstat like --stat

The --stat of a combined diff is defined as the first-parent stat,
going all the way back to 965f803c32 (combine-diff: show diffstat with
the first parent., 2006-04-17).

Naturally, we gave --numstat the same treatment in 74e2abe5b7 (diff
--numstat, 2006-10-12).

But --shortstat, which is really just the final line of --stat, does
nothing, which produces confusing results:

$ git show --oneline --stat eab7584e37
eab7584e37 Merge branch 'en/show-ref-doc-fix'

Documentation/git-show-ref.txt | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

$ git show --oneline --shortstat eab7584e37
eab7584e37 Merge branch 'en/show-ref-doc-fix'

[nothing! We'd expect to see the "1 file changed..." line]

This patch teaches combine-diff to treats the two formats identically.

Reported-by: David Turner <novalis@novalis.org>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

combine-diff: factor out stat-format mask

There are several conditionals in the combine diff code that check if
we're doing --stat or --numstat output. Since these must all remain in
sync, let's pull them out into a separate bit-mask.

Arguably this could go into diff.h along with the other DIFF_FORMAT
macros, but it's not clear that the definition of "which formats are
stat" is universal (e.g., does --dirstat count? --summary?). So let's
keep this local to combine-diff.c, where the meaning is more clearly
"stat formats that combine-diff supports".

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

diff: clear emitted_symbols flag after use

There's an odd bug when "log --color-moved" is used with the combination
of "--cc --stat -p": the stat for merge commits is erroneously shown
with the diff of the _next_ commit.

The included test demonstrates the issue. Our history looks something
like this:

A-B-M--D
\ /
C

When we run "git log --cc --stat -p --color-moved" starting at D, we get
this sequence of events:

1. The diff for D is using -p, so diff_flush() calls into
diff_flush_patch_all_file_pairs(). There we see that o->color_moved
is in effect, so we point o->emitted_symbols to a static local
struct, causing diff_flush_patch() to queue the symbols instead of
actually writing them out.

We then do our move detection, emit the symbols, and clear the
struct. But we leave o->emitted_symbols pointing to our struct.

2. Next we compute the diff for M. This is a merge, so we use the
combined diff code. In find_paths_generic(), we compute the
pairwise diff between each commit and its parent. Normally this is
done with DIFF_FORMAT_NO_OUTPUT, since we're just looking for
intersecting paths. But since "--stat --cc" shows the first-parent
stat, and since we're computing that diff anyway, we enable
DIFF_FORMAT_DIFFSTAT for the first parent. This outputs the stat
information immediately, saving us from running a separate
first-parent diff later.

But where does that output go? Normally it goes directly to stdout,
but because o->emitted_symbols is set, we queue it. As a result, we
don't actually print the diffstat for the merge commit (yet), which
is wrong.

3. Next we compute the diff for C. We're actually showing a patch
again, so we end up in diff_flush_patch_all_file_pairs(), but this
time we have the queued stat from step 2 waiting in our struct.

We add new elements to it for C's diff, and then flush the whole
thing. And we see the diffstat from M as part of C's diff, which is
wrong.

So triggering the bug really does require the combination of all of
those options.

To fix it, we can simply restore o->emitted_symbols to NULL after
flushing it, so that it does not affect anything outside of
diff_flush_patch_all_file_pairs(). This intuitively makes sense, since
nobody outside of that function is going to bother flushing it, so we
would not want them to write to it either.

In fact, we could take this a step further and turn the local "esm"
struct into a non-static variable that goes away after the function
ends. However, since it contains a dynamically sized array, we benefit
from amortizing the cost of allocations over many calls. So we'll leave
it as static to retain that benefit.

But let's push the zero-ing of esm.nr into the conditional for "if
(o->emitted_symbols)" to make it clear that we do not expect esm to hold
any values if we did not just try to use it. With the code as it is
written now, if we did encounter such a case (which I think would be a
bug), we'd silently leak those values without even bothering to display
them. With this change, we'd at least eventually show them, and somebody
would notice.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t4006: resurrect commented-out tests

This set of tests was added by 4434e6ba6c (tests: check --[short]stat
output after chmod, 2012-05-01), and is primarily about the handling of
binary versus text files.

Later, 74faaa16f0 (Fix "git diff --stat" for interesting - but empty -
file changes, 2012-10-17) changed the stat output so that the empty text
file is mentioned rather than omitted. That commit just comments out
these tests. There's no discussion in the commit message, but the
original email[1] says:

NOTE! This does break two of our tests, so we clearly did this on
purpose, or at least tested for it. I just uncommented the subtests
that this makes irrelevant, and changed the output of another one.

I don't think they're irrelevant, though. We should be testing this
"mode change only" case and making sure that it has the post-74faaa16f0
behavior. So this commit brings back those tests, with the current
expected output.

[1] https://public-inbox.org/git/CA+55aFz88GPJcfMSqiyY+u0Cdm48bEyrsTGxHVJbGsYsDg=Q5w@mail.gmail.com/

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

cache.h: flip NO_THE_INDEX_COMPATIBILITY_MACROS switch

By default, index compat macros are off from now on, because they
could hide the_index dependency.

Only those in builtin can use it.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>