t/test-lib: add an SHA1 prerequisite

There are some basic tests in our codebase that test that we get fixed
SHA-1 values. These are valuable because they make sure that our SHA-1
implementation is free of bugs, but obviously these tests will fail with
a different hash.

There are also tests which intentionally produce objects that have
collisions when truncated to a certain length to test our handling of
these cases. These tests, too, will fail with a different hash.

Add an SHA1 prerequisite to annotate both of these types of tests and
disable them when we're using a different hash. In the future, we will
create versions of these tests which handle both SHA-1 and NewHash.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

git-credential-netrc: accept gpg option

git-credential-netrc was hardcoded to decrypt with 'gpg' regardless of
the gpg.program option. This is a problem on distributions like Debian
that call modern GnuPG something else, like 'gpg2'.
Set the command according to these settings in descending precedence
1. the git-credential-netrc command -g|--gpg option
2. the git gpg.program configuration option
3. the default: 'gpg'

For conformance with Documentation/CodingGuidelines
- use Git.pm for repository and global option queries
- document -g|--gpg command option in command usage
- test repository & command options
- write documentation placeholders according to main standards

Signed-off-by: Luis Marsano <luis.marsano@gmail.com>
Acked-by: Ted Zlatanov <tzz@lifelogs.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

git-credential-netrc: adapt to test framework for git

git-credential-netrc tests did not run in a test repository.
Reuse the main test framework to stage a temporary repository.
To imitate Perl tests under t/
- switch to Test::More module
- use File::Basename & File::Spec::Functions

Signed-off-by: Luis Marsano <luis.marsano@gmail.com>
Acked-by: Ted Zlatanov <tzz@lifelogs.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t5512: run git fetch inside test

Do the preparatory fetch inside the test of ls-remote --symref to avoid
cluttering the test output and to be able to catch unexpected fetch
failures.

Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

mark_parents_uninteresting(): avoid most allocation

Commit 941ba8db57 (Eliminate recursion in setting/clearing
marks in commit list, 2012-01-14) used a clever double-loop
to avoid allocations for single-parent chains of history.
However, it did so only when following parents of parents
(which was an uncommon case), and _always_ incurred at least
one allocation to populate the list of pending parents in
the first place.

We can turn this into zero-allocation in the common case by
iterating directly over the initial parent list, and then
following up on any pending items we might have discovered.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

mark_parents_uninteresting(): replace list with stack

The mark_parents_uninteresting() function uses two loops:
an outer one to process our queue of pending parents, and an
inner one to process first-parent chains. This is a clever
optimization from 941ba8db57 (Eliminate recursion in
setting/clearing marks in commit list, 2012-01-14) to limit
the number of linked-list allocations when following
single-parent chains.

Unfortunately, this makes the result a little hard to read.
Let's replace the list with a stack. Then we don't have to
worry about doing this double-loop optimization, as we'll
just reuse the top element of the stack as we pop/push.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

mark_parents_uninteresting(): drop missing object check

We allow UNINTERESTING objects in a traversal to be
unavailable. As part of this, mark_parents_uninteresting()
checks whether we have a particular uninteresting parent; if
not, we will mark it as "parsed" so that later code skips
it.

This code is redundant and even a little bit harmful, so
let's drop it.

It's redundant because when our parse_object() call in
add_parents_to_list() fails, we already quietly skip
UNINTERESTING parents. This redundancy is a historical
artifact. The mark_parents_uninteresting() protection is
from 454fbbcde3 (git-rev-list: allow missing objects when
the parent is marked UNINTERESTING, 2005-07-10). Much later,
aeeae1b771 (revision traversal: allow UNINTERESTING objects
to be missing, 2009-01-27) covered more cases by making the
actual parse more gentle.

As an aside, even if this weren't redundant, it would be
insufficient. The gentle parsing handles both missing and
corrupted objects, whereas the has_object_file() check
we're getting rid of covers only missing ones.

And the code we're dropping is harmful for two reasons:

1. We spend extra time on the object lookup, even though
we don't actually need the information at this point
(and will just repeat that lookup later when we parse
for the common case that we _do_ have the object).

2. It "lies" about the commit by setting the parsed flag,
even though we didn't load any useful data into the
struct. This shouldn't matter for the UNINTERESTING
case, but we may later clear our flags and do another
traversal in the same process. While pretty unlikely,
it's possible that we could then look at the same
commit without the UNINTERESTING flag, in which case
we'd produce the wrong result (we'd think it's a commit
with no parents, when in fact we should probably die
due to the missing object).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

mark_tree_contents_uninteresting(): drop missing object check

It's generally acceptable for UNINTERESTING objects in a
traversal to be unavailable (e.g., see aeeae1b771). When
marking trees UNINTERESTING, we access the object database
twice: once to check if the object is missing (and return
quietly if it is), and then again to actually parse it.

We can instead just try to parse; if that fails, we can then
return quietly. That halves the effort we spend on locating
the object.

Note that this isn't _exactly_ the same as the original
behavior, as the parse failure could be due to other
problems than a missing object: it could be corrupted, in
which case the original code would have died. But the new
behavior is arguably better, as it covers the object being
unavailable for any reason. We'll also still issue a warning
to stderr in such a case.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

commit.h: rearrange 'index' to shrink struct commit

On linux 64-bit architecture, pahole finds that there's a 4 bytes
padding after 'index'. Moving it to the end reduces this struct's size
from 72 to 64 bytes (because of another 4 bytes padding after
graph_pos). On linux 32-bit, the struct size remains 52 bytes like
before.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

add status config and command line options for rename detection

After performing a merge that has conflicts git status will, by default,
attempt to detect renames which causes many objects to be examined. In a
virtualized repo, those objects do not exist locally so the rename logic
triggers them to be fetched from the server. This results in the status call
taking hours to complete on very large repos vs seconds with this patch.

Add a new config status.renames setting to enable turning off rename
detection during status and commit. This setting will default to the value
of diff.renames.

Add a new config status.renamelimit setting to to enable bounding the time
spent finding out inexact renames during status and commit. This setting
will default to the value of diff.renamelimit.

Add --no-renames command line option to status that enables overriding the
config setting from the command line. Add --find-renames[=<n>] command line
option to status that enables detecting renames and optionally setting the
similarity index.

Reviewed-by: Elijah Newren <newren@gmail.com>
Original-Patch-by: Alejandro Pauly <alpauly@microsoft.com>
Signed-off-by: Ben Peart <Ben.Peart@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

column: fix off-by-one default width

By default we want to fill the whole screen if possible, but we do not
want to use up _all_ terminal columns because the last character is
going hit the border, push the cursor over and wrap. Keep it at
default value zero, which will make print_columns() set the width at
term_columns() - 1.

This affects the test in t7004 because effective column width before
was 40 but now 39 so we need to compensate it by one or the output at
39 columns has a different layout.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

pager: set COLUMNS to term_columns()

After we invoke the pager, our stdout goes to a pipe, not the
terminal, meaning we can no longer use an ioctl to get the
terminal width. For that reason, ad6c3739a3 (pager: find out
the terminal width before spawning the pager, 2012-02-12)
started caching the terminal width.

But that cache is only an in-process variable. Any programs
we spawn will also not be able to run that ioctl, but won't
have access to our cache. They'll end up falling back to our
80-column default.

You can see the problem with:

git tag --column=row

Since git-tag spawns a pager these days, its spawned
git-column helper will see neither the terminal on stdout
nor a useful COLUMNS value (assuming you do not export it
from your shell already). And you'll end up with 80-column
output in the pager, regardless of your terminal size.

We can fix this by setting COLUMNS right before spawning the
pager. That fixes this case, as well as any more complicated
ones (e.g., a paged program spawns another script which then
generates columnized output).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

refs: handle zero oid for pseudorefs

According to the documentation, it is possible to "specify 40 '0' or an
empty string as <oldvalue> to make sure that the ref you are creating
does not exist." But in the code for pseudorefs, we do not implement
this, as demonstrated by the failing tests added in the previous commit.
If we fail to read the old ref, we immediately die. But a failure to
read would actually be a good thing if we have been given the zero oid.

With the zero oid, allow -- and even require -- the ref-reading to fail.
This implements the "make sure that the ref ... does not exist" part of
the documentation and fixes both failing tests from the previous commit.

Since we have a `strbuf err` for collecting errors, let's use it and
signal an error to the caller instead of dying hard.

Reported-by: Rafael Ascensão <rafa.almas@gmail.com>
Helped-by: Rafael Ascensão <rafa.almas@gmail.com>
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t1400: add tests around adding/deleting pseudorefs

I have not been able to find any tests around adding pseudorefs using
`git update-ref`. Add some as outlined in this table (original design by
Michael Haggerty; modified and extended by me):

Pre-update value | ref-update old OID | Expected result
-------------------|----------------------|----------------
missing | value | reject
missing | none given | accept
set | none given | accept
set | correct value | accept
set | wrong value | reject
missing | zero | accept *
set | zero | reject *

The tests marked with a * currently fail, despite git-update-ref(1)
claiming that it is possible to "specify 40 '0' or an empty string as
<oldvalue> to make sure that the ref you are creating does not exist."
These failing tests will be fixed in the next commit.

It is only natural to test deletion as well. Test deletion without an
old OID, with a correct one and with an incorrect one.

Suggested-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

refs.c: refer to "object ID", not "sha1", in error messages

We have two error messages that complain about the "sha1". Because we
are about to touch one of these sites and add some tests, let's first
modernize the messages to say "object ID" instead.

While at it, make the second one use `error()` instead of `warning()`.
After printing the message, we do not continue, but actually drop the
lock and return -1 without deleting the pseudoref.

Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

pack-format.txt: more details on pack file format

The current document mentions OBJ_* constants without their actual
values. A git developer would know these are from cache.h but that's
not very friendly to a person who wants to read this file to implement
a pack file parser.

Similarly, the deltified representation is not documented at all (the
"document" is basically patch-delta.c). Translate that C code to
English with a bit more about what ofs-delta and ref-delta mean.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

get_short_oid: sort ambiguous objects by type, then SHA-1

Change the output emitted when an ambiguous object is encountered so
that we show tags first, then commits, followed by trees, and finally
blobs. Within each type we show objects in hashcmp() order. Before
this change the objects were only ordered by hashcmp().

The reason for doing this is that the output looks better as a result,
e.g. the v2.17.0 tag before this change on "git show e8f2" would
display:

hint: The candidates are:
hint: e8f2093055 tree
hint: e8f21caf94 commit 2013-06-24 - bash prompt: print unique detached HEAD abbreviated object name
hint: e8f21d02f7 blob
hint: e8f21d577c blob
hint: e8f25a3a50 tree
hint: e8f26250fa commit 2017-02-03 - Merge pull request #996 from jeffhostetler/jeffhostetler/register_rename_src
hint: e8f2650052 tag v2.17.0
hint: e8f2867228 blob
hint: e8f28d537c tree
hint: e8f2a35526 blob
hint: e8f2bc0c06 commit 2015-05-10 - Documentation: note behavior for multiple remote.url entries
hint: e8f2cf6ec0 tree

Now we'll instead show:

hint: e8f2650052 tag v2.17.0
hint: e8f21caf94 commit 2013-06-24 - bash prompt: print unique detached HEAD abbreviated object name
hint: e8f26250fa commit 2017-02-03 - Merge pull request #996 from jeffhostetler/jeffhostetler/register_rename_src
hint: e8f2bc0c06 commit 2015-05-10 - Documentation: note behavior for multiple remote.url entries
hint: e8f2093055 tree
hint: e8f25a3a50 tree
hint: e8f28d537c tree
hint: e8f2cf6ec0 tree
hint: e8f21d02f7 blob
hint: e8f21d577c blob
hint: e8f2867228 blob
hint: e8f2a35526 blob

Since we show the commit data in the output that's nicely aligned once
we sort by object type. The decision to show tags before commits is
pretty arbitrary. I don't want to order by object_type since there
tags come last after blobs, which doesn't make sense if we want to
show the most important things first.

I could display them after commits, but it's much less likely that
we'll display a tag, so if there is one it makes sense to show it
prominently at the top.

A note on the implementation: Derrick rightly pointed out[1] that
we're bending over backwards here in get_short_oid() to first
de-duplicate the list, and then emit it, but could simply do it in one
step.

The reason for that is that oid_array_for_each_unique() doesn't
actually require that the array be sorted by oid_array_sort(), it just
needs to be sorted in some order that guarantees that all objects with
the same ID are adjacent to one another, which (barring a hash
collision, which'll be someone else's problem) the sort_ambiguous()
function does.

I agree that would be simpler for this code, and had forgotten why I
initially wrote it like this[2]. But on further reflection I think
it's better to do more work here just so we're not underhandedly using
the oid-array API where we lie about the list being sorted. That would
break any subsequent use of oid_array_lookup() in subtle ways.

I could get around that by hacking the API itself to support this
use-case and documenting it, which I did as a WIP patch in [3], but I
think it's too much code smell just for this one call site. It's
simpler for the API to just introduce a oid_array_for_each() function
to eagerly spew out the list without sorting or de-duplication, and
then do the de-duplication and sorting in two passes.

1. https://public-inbox.org/git/20180501130318.58251-1-dstolee@microsoft.com/
2. https://public-inbox.org/git/876047ze9v.fsf@evledraar.gmail.com/
3. https://public-inbox.org/git/874ljrzctc.fsf@evledraar.gmail.com/

Helped-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

apply: clarify "-p" documentation

We're not really removing slashes, but slash-separated path
components. Let's make that more clear.

Reported-by: kelly elton <its.the.doc@gmail.com>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t5516-fetch-push: fix broken &&-chain

b2dc968e60 (t5516: refactor oddball tests, 2008-11-07) accidentaly
broke the &&-chain in the test 'push does not update local refs on
failure', but since it was in a subshell, chain-lint couldn't notice
it.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t5516-fetch-push: fix 'push with dry-run' test

In a while-at-it cleanup replacing a 'cd dir && <...> && cd ..' with a
subshell, commit 28391a80a9 (receive-pack: allow deletion of corrupt
refs, 2007-11-29) also moved the assignment of the $old_commit
variable to that subshell. This variable, however, is used outside of
that subshell as a parameter of check_push_result(), to check that a
ref still points to the commit where it is supposed to. With the
variable remaining unset outside the subshell check_push_result()
doesn't perform that check at all.

Use 'git -C <dir> cmd...', so we don't need to change directory, and
thus don't need the subshell either when setting $old_commit.

Furthermore, change check_push_result() to require at least three
parameters (the repo, the oid, and at least one ref), so it will catch
similar issues earlier should they ever arise.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

sha1-name.c: move around the collect_ambiguous() function

A subsequent change will make use of this static function in the
get_short_oid() function, which is defined above where the
collect_ambiguous() function is now. Without this we'd then have a
compilation error due to a forward declaration.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t5310-pack-bitmaps: make JGit tests work with GIT_TEST_SPLIT_INDEX

The two JGit tests 'we can read jgit bitmaps' and 'jgit can read our
bitmaps' in 't5310-pack-bitmaps.sh' fail when run with
GIT_TEST_SPLIT_INDEX=YesPlease. Both tests create a clone of the test
repository to check bitmap interoperability with JGit. With split
indexes enabled the index in the clone repositories contains the
'link' extension, which JGit doesn't support and, consequently, an
exception aborts it:

<...>
org.eclipse.jgit.api.errors.JGitInternalException: DIRC extension 'link' not supported by this version.
at org.eclipse.jgit.dircache.DirCache.readFrom(DirCache.java:562)
<...>

Since testing bitmaps doesn't need a worktree in the first place,
let's just create bare clones for the two JGit tests, so the cloned
won't have an index, and these two tests can be executed even with
split index enabled.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

git-p4: change "commitish" typo to "committish"

This was the only occurrence of "commitish" in the tree, but as the
log will reveal we've had others in the past. Fixes up code added in
00ad6e3182 ("git-p4: work with a detached head", 2015-11-21).

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Acked-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

sha1-array.h: align function arguments

The arguments weren't lined up with the opening parenthesis, after
910650d2 ("Rename sha1_array to oid_array", 2017-03-31) renamed the
function.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

sha1-name.c: remove stray newline

This stray newline was accidentally introduced in
d2b7d9c7ed ("sha1_name: convert disambiguate_hint_fn to take
object_id", 2017-03-26).

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

doc: fix config API documentation about config_with_options

In commit dc8441fdb ("config: don't implicitly use gitdir or commondir",
2017-06-14) the function git_config_with_options was renamed to
config_with_options to better reflect the fact that it does not access
the git global config or the repo config by default.

However Documentation/technical/api-config.txt still refers to the
previous name, fix that.

While at it also update the documentation about the extra parameters,
because they too changed since the initial definition.

Signed-off-by: Antonio Ospite <ao2@ao2.it>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

replace-object.c: remove the_repository from prepare_replace_object

This was missed in 5982da9d2ce (replace-object: allow
prepare_replace_object to handle arbitrary repositories, 2018-04-11)

Technically the code works correctly as the replace_map is the same
size in different repositories, however it is hard to read. So convert
the code to the familiar pattern of dereferencing the pointer that we
assign in the sizeof itself.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

object.c: free replace map in raw_object_store_clear

The replace map for objects was missed to free in the object store in
the conversion of 174774cd519 (Merge branch 'sb/object-store-replace',
2018-05-08)

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

repository: fix free problem with repo_clear(the_repository)

the_repository is special. One of the special things about it is that
it does not allocate a new index_state object like submodules but
points to the global the_index variable instead. As a global variable,
the_index cannot be free()'d.

Add an exception for this in repo_clear(). In the future perhaps we
would be able to allocate the_repository's index on heap too. Then we
can revert this.

the_repository->index remains pointed to a clean the_index even after
repo_clear() so that it could still be used next time (e.g. in a crazy
use case where a dev switches repo in the same process).

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

fast-export: avoid NULL pointer arithmetic

Clang 6 reports the following warning, which is turned into an error in a
DEVELOPER build:

builtin/fast-export.c:162:28: error: performing pointer arithmetic on a null pointer has undefined behavior [-Werror,-Wnull-pointer-arithmetic]
return ((uint32_t *)NULL) + mark;
~~~~~~~~~~~~~~~~~~ ^
1 error generated.

The compiler is correct, and the error message speaks for itself. There
is no need for any undefined operation -- just cast mark to void * or
uint32_t after an intermediate cast to uintptr_t. That encodes the
integer value into a pointer and later decodes it as intended.

While at it remove an outdated comment -- intptr_t has been used since
ffe659f94d (parse-options: make some arguments optional, add callbacks),
committed in October 2007.

Signed-off-by: Rene Scharfe <l.s.r@web.de>
Acked-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

BUG_exit_code: fix sparse "symbol not declared" warning

Signed-off-by: Ramsay Jones <ramsay@ramsayjones.plus.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

lock_file: move static locks into functions

Placing `struct lock_file`s on the stack used to be a bad idea, because
the temp- and lockfile-machinery would keep a pointer into the struct.
But after 076aa2cbd (tempfile: auto-allocate tempfiles on heap,
2017-09-05), we can safely have lockfiles on the stack. (This applies
even if a user returns early, leaving a locked lock behind.)

Each of these `struct lock_file`s is used from within a single function.
Move them into the respective functions to make the scope clearer and
drop the staticness.

For good measure, I have inspected these sites and come to believe that
they always release the lock, with the possible exception of bailing out
using `die()` or `exit()` or by returning from a `cmd_foo()`.

As pointed out by Jeff King, it would be bad if someone held on to a
`struct lock_file *` for some reason. After some grepping, I agree with
his findings: no-one appears to be doing that.

After this commit, the remaining occurrences of "static struct
lock_file" are locks that are used from within different functions. That
is, they need to remain static. (Short of more intrusive changes like
passing around pointers to non-static locks.)

Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

lock_file: make function-local locks non-static

Placing `struct lock_file`s on the stack used to be a bad idea, because
the temp- and lockfile-machinery would keep a pointer into the struct.
But after 076aa2cbd (tempfile: auto-allocate tempfiles on heap,
2017-09-05), we can safely have lockfiles on the stack. (This applies
even if a user returns early, leaving a locked lock behind.)

These `struct lock_file`s are local to their respective functions and we
can drop their staticness.

For good measure, I have inspected these sites and come to believe that
they always release the lock, with the possible exception of bailing out
using `die()` or `exit()` or by returning from a `cmd_foo()`.

As pointed out by Jeff King, it would be bad if someone held on to a
`struct lock_file *` for some reason. After some grepping, I agree with
his findings: no-one appears to be doing that.

Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

refs.c: do not die if locking fails in `delete_pseudoref()`

After taking the lock we check whether we got it and die otherwise. But
since we take the lock using `LOCK_DIE_ON_ERROR`, we would already have
died.

Considering the choice between dropping the dead code and dropping the
flag, let's go for option number three: Drop the flag, write an error
instead of dying, then return -1. This function already returns -1 for
another error, so the caller (or rather, its callers) should be able to
handle this. There is some inconsistency around how we handle errors in
this function and elsewhere in this file, but let's take this small step
towards gentle error-reporting now and leave the rest for another time.

While at it, make the lock non-static and reduce its scope. (Placing
`struct lock_file`s on the stack used to be a bad idea, because the
temp- and lockfile-machinery would keep a pointer into the struct. But
after 076aa2cbd (tempfile: auto-allocate tempfiles on heap, 2017-09-05),
we can safely have lockfiles on the stack.)

Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

refs.c: do not die if locking fails in `write_pseudoref()`

If we could not take the lock, we add an error to the `strbuf err` and
return. However, this code is dead. The reason is that we take the lock
using `LOCK_DIE_ON_ERROR`. Drop the flag to allow our more gentle
error-handling to actually kick in.

We could instead just drop the dead code and die here. But everything is
prepared for gently propagating the error, so let's do that instead.

There is similar dead code in `delete_pseudoref()`, but let's save that
for the next patch.

While at it, make the lock non-static. (Placing `struct lock_file`s on
the stack used to be a bad idea, because the temp- and
lockfile-machinery would keep a pointer into the struct. But after
076aa2cbd (tempfile: auto-allocate tempfiles on heap, 2017-09-05), we
can safely have lockfiles on the stack.)

Reviewed-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t/helper/test-write-cache: clean up lock-handling

Die in case writing the index fails, so that the caller can notice
(instead of, say, being impressed by how performant the writing is).

While at it, note that after opening a lock with `LOCK_DIE_ON_ERROR`, we
do not need to worry about whether we succeeded. Also, we can move the
`struct lock_file` into the function and drop the staticness. (Placing
`struct lock_file`s on the stack used to be a bad idea, because the
temp- and lockfile-machinery would keep a pointer into the struct. But
after 076aa2cbd (tempfile: auto-allocate tempfiles on heap, 2017-09-05),
we can safely have lockfiles on the stack.)

Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t6050-replace: don't disable stdin for the whole test script

The test script 't6050-replace.sh' starts off with redirecting the
whole test script's stdin from /dev/null. This redirection has been
there since the test script was introduced in a3e8267225
(replace_object: add a test case, 2009-01-23), but the commit message
doesn't explain why it was deemed necessary. AFAICT, it has never
been necessary, and t6050 runs just fine and succeeds even without it,
not only the current version but past versions as well.

Besides being unnecessary, this redirection is also harmful, as it
prevents the test helper functions 'test_pause' and 'debug' from
working properly in t6050, because we can't enter any commands to the
shell and the debugger, respectively.

So let's remove that redirection.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

merge: pass aggressive when rename detection is turned off

Set aggressive flag in git_merge_trees() when rename detection is turned off.
This allows read_tree() to auto resolve more cases that would have otherwise
been handled by the rename detection.

Reviewed-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Ben Peart <benpeart@microsoft.com>
Reviewed-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

merge: add merge.renames config setting

Add the ability to control rename detection for merge via a config setting.
This setting behaves the same and defaults to the value of diff.renames but only
applies to merge.

Reviewed-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Helped-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Ben Peart <benpeart@microsoft.com>
Reviewed-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

merge: update documentation for {merge,diff}.renameLimit

Update the documentation to better indicate that the renameLimit setting is
ignored if rename detection is turned off via command line options or config
settings.

Signed-off-by: Ben Peart <benpeart@microsoft.com>
Reviewed-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

merge-recursive: fix check for skipability of working tree updates

The can-working-tree-updates-be-skipped check has had a long and blemished
history. The update can be skipped iff:
a) The merge is clean
b) The merge matches what was in HEAD (content, mode, pathname)
c) The target path is usable (i.e. not involved in D/F conflict)

Traditionally, we split b into parts:
b1) The merged result matches the content and mode found in HEAD
b2) The merged target path existed in HEAD

Steps a & b1 are easy to check; we have always gotten those right. While
it is easy to overlook step c, this was fixed seven years ago with commit
4ab9a157d069 ("merge_content(): Check whether D/F conflicts are still
present", 2010-09-20). merge-recursive didn't have a readily available
way to directly check step b2, so various approximations were used:

* In commit b2c8c0a76274 ("merge-recursive: When we detect we can skip
an update, actually skip it", 2011-02-28), it was noted that although
the code claimed it was skipping the update, it did not actually skip
the update. The code was made to skip it, but used lstat(path, ...)
as an approximation to path-was-tracked-in-index-before-merge.

* In commit 5b448b853030 ("merge-recursive: When we detect we can skip
an update, actually skip it", 2011-08-11), the problem with using
lstat was noted. It was changed to the approximation
path2 && strcmp(path, path2)
which is also wrong. !path2 || strcmp(path, path2) would have been
better, but would have fallen short with directory renames.

* In c5b761fb2711 ("merge-recursive: ensure we write updates for
directory-renamed file", 2018-02-14), the problem with the previous
approximation was noted and changed to
was_tracked(path)
That looks close to what we were trying to answer, but was_tracked()
as implemented at the time should have been named is_tracked(); it
returned something different than what we were looking for.

* To make matters more complex, fixing was_tracked() isn't sufficient
because the splitting of b into b1 and b2 is wrong. Consider the
following merge with a rename/add conflict:
side A: modify foo, add unrelated bar
side B: rename foo->bar (but don't modify the mode or contents)
In this case, the three-way merge of original foo, A's foo, and B's
bar will result in a desired pathname of bar with the same
mode/contents that A had for foo. Thus, A had the right mode and
contents for the file, and it had the right pathname present (namely,
bar), but the bar that was present was unrelated to the contents, so
the working tree update was not skippable.

Fix this by introducing a new function:
was_tracked_and_matches(o, path, &mfi.oid, mfi.mode)
and use it to directly check for condition b.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

merge-recursive: make "Auto-merging" comment show for other merges

Previously, merge_content() would print "Auto-merging" whenever the final
content and mode aren't already available from HEAD. There are a few
problems with this:

1) There are other code paths doing merges that should probably have the
same message printed, in particular rename/rename(2to1) which cannot
call into the normal rename logic.

2) If both sides of the merge have modifications, then a content merge
is needed. It may turn out that the end result matches one of the
sides (because the other only had a subset of the same changes), but
the merge was still needed. Currently, the message will not print in
that case, though it seems like it should.

Move the printing of this message to merge_file_1() in order to address
both issues.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

merge-recursive: fix remainder of was_dirty() to use original index

was_dirty() uses was_tracked(), which has been updated to use the original
index rather than the current one. However, was_dirty() also had a
separate call to cache_file_exists(), causing it to still implicitly use
the current index. Update that to instead use index_file_exists().

Also, was_dirty() had a hack where it would mark any file as non-dirty if
we simply didn't know its modification time. This was due to using the
current index rather than the original index, because D/F conflicts and
such would cause unpack_trees() to not copy the modification times from
the original index to the current one. Now that we are using the original
index, we can dispense with this hack.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

merge-recursive: fix was_tracked() to quit lying with some renamed paths

In commit aacb82de3ff8 ("merge-recursive: Split was_tracked() out of
would_lose_untracked()", 2011-08-11), was_tracked() was split out of
would_lose_untracked() with the intent to provide a function that could
answer whether a path was tracked in the index before the merge. Sadly,
it instead returned whether the path was in the working tree due to having
been tracked in the index before the merge OR having been written there by
unpack_trees(). The distinction is important when renames are involved,
e.g. for a merge where:

HEAD: modifies path b
other: renames b->c

In this case, c was not tracked in the index before the merge, but would
have been added to the index at stage 0 and written to the working tree by
unpack_trees(). would_lose_untracked() is more interested in the
in-working-copy-for-either-reason behavior, while all other uses of
was_tracked() want just was-it-tracked-in-index-before-merge behavior.

Unsplit would_lose_untracked() and write a new was_tracked() function
which answers whether a path was tracked in the index before the merge
started.

This will also affect was_dirty(), helping it to return better results
since it can base answers off the original index rather than an index that
possibly only copied over some of the stat information. However,
was_dirty() will need an additional change that will be made in a
subsequent patch.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t6046: testcases checking whether updates can be skipped in a merge

Add several tests checking whether updates can be skipped in a merge.
Also add several similar testcases for where updates cannot be skipped in
a merge to make sure that we skip if and only if we should.

In particular:

* Testcase 1a (particularly 1a-check-L) would have pointed out the
problem Linus has been dealing with for year with his merges[1].

* Testcase 2a (particularly 2a-check-L) would have pointed out the
problem with my directory-rename-series before it broke master[2].

* Testcases 3[ab] (particularly 3a-check-L) provide a simpler testcase
than 12b of t6043 making that one easier to understand.

* There are several complementary testcases to make sure we're not just
fixing those particular issues while regressing in the opposite
direction.

* There are also a pair of tests for the special case when a merge
results in a skippable update AND the user has dirty modifications to
the path.

[1] https://public-inbox.org/git/CA+55aFzLZ3UkG5svqZwSnhNk75=fXJRkvU1m_RHBG54NOoaZPA@mail.gmail.com/
[2] https://public-inbox.org/git/xmqqmuya43cs.fsf@gitster-ct.c.googlers.com/

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

merge-recursive: avoid triggering add_cacheinfo error with dirty mod

If a cherry-pick or merge with a rename results in a skippable update
(due to the merged content matching what HEAD already had), but the
working directory is dirty, avoid trying to refresh the index as that
will fail.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

merge-recursive: move more is_dirty handling to merge_content

conflict_rename_normal() was doing some handling for dirty files that
more naturally belonged in merge_content. Move it, and rename a
parameter for clarity while at it.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

merge-recursive: improve add_cacheinfo error handling

Four closely related changes all with the purpose of fixing error handling
in this function:
- fix reported function name in add_cacheinfo error messages
- differentiate between the two error messages
- abort early when we hit the error (stop ignoring return code)
- mark a test which was hitting this error as failing until we get the
right fix

In more detail...

In commit 0424138d5715 ("Fix bogus error message from merge-recursive
error path", 2007-04-01), it was noted that the name of the function which
the error message claimed it was reported from did not match the actual
function name. This was changed to something closer to the real function
name, but it still didn't match the actual function name. Fix the
reported name to match.

Second, the two errors in this function had identical messages, preventing
us from knowing which error had been triggered. Add a couple words to the
second error message to differentiate the two.

Next, make sure callers do not ignore the return code so that it will stop
processing further entries (processing further entries could result in
more output which could cause the error to scroll off the screen, or at
least be missed by the user) and make it clear the error is the cause of
the early abort. These errors should never be triggered in production; if
either one is, it represents a bug in the calling path somewhere and is
likely to have resulted in mis-merged content. The combination of
ignoring of the return code and continuing to print other standard
messages after hitting the error resulted in the following bug report from
Junio: "...the command pretends that everything went well and merged
cleanly in that path...[Behaving] in a buggy and unexplainable way is bad
enough, doing so silently is unexcusable." Fix this.

Finally, there was one test in the testsuite that did hit this error path,
but was passing anyway. This would have been easy to miss since it had a
test_must_fail and thus could have failed for the wrong reason, but in a
separate testing step I added an intentional NULL-dereference to the
codepath where these error messages are printed in order to flush out such
cases. I could modify that test to explicitly check for this error and
fail the test if it is hit, but since this test operates in a bit of a
gray area and needed other changes, I went for a different fix. The gray
area this test operates in is the following: If the merge of a certain
file results in the same version of the file that existed in HEAD, but
there are dirty modifications to the file, is that an error with a
"Refusing to overwrite existing file" expected, or a case where the merge
should succeed since we shouldn't have to touch the dirty file anyway?
Recent discussion on the list leaned towards saying it should be a
success. Therefore, change the expected behavior of this test to match.
As a side effect, this makes the failed-due-to-hitting-add_cacheinfo-error
very clear, and we can mark the test as test_expect_failure. A subsequent
commit will implement the necessary changes to get this test to pass
again.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

merge-recursive: avoid spurious rename/rename conflict from dir renames

If a file on one side of history was renamed, and merely modified on the
other side, then applying a directory rename to the modified side gives us
a rename/rename(1to2) conflict. We should only apply directory renames to
pairs representing either adds or renames.

Making this change means that a directory rename testcase that was
previously reported as a rename/delete conflict will now be reported as a
modify/delete conflict.

Reviewed-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

directory rename detection: new testcases showcasing a pair of bugs

Add a testcase showing spurious rename/rename(1to2) conflicts occurring
due to directory rename detection.

Also add a pair of testcases dealing with moving directory hierarchies
around that were suggested by Stefan Beller as "food for thought" during
his review of an earlier patch series, but which actually uncovered a
bug. Round things out with a test that is a cross between the two
testcases that showed existing bugs in order to make sure we aren't
merely addressing problems in isolation but in general.

Reviewed-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

merge-recursive: fix remaining directory rename + dirty overwrite cases

Reviewed-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

merge-recursive: fix overwriting dirty files involved in renames

This fixes an issue that existed before my directory rename detection
patches that affects both normal renames and renames implied by
directory rename detection. Additional codepaths that only affect
overwriting of dirty files that are involved in directory rename
detection will be added in a subsequent commit.

Reviewed-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

merge-recursive: avoid clobbering untracked files with directory renames

Reviewed-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

merge-recursive: apply necessary modifications for directory renames

This commit hooks together all the directory rename logic by making the
necessary changes to the rename struct, it's dst_entry, and the
diff_filepair under consideration.

Reviewed-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

merge-recursive: when comparing files, don't include trees

get_renames() would look up stage data that already existed (populated
in get_unmerged(), taken from whatever unpack_trees() created), and if
it didn't exist, would call insert_stage_data() to create the necessary
entry for the given file. The insert_stage_data() fallback becomes
much more important for directory rename detection, because that creates
a mechanism to have a file in the resulting merge that didn't exist on
either side of history. However, insert_stage_data(), due to calling
get_tree_entry() loaded up trees as readily as files. We aren't
interested in comparing trees to files; the D/F conflict handling is
done elsewhere. This code is just concerned with what entries existed
for a given path on the different sides of the merge, so create a
get_tree_entry_if_blob() helper function and use it.

Reviewed-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

merge-recursive: check for file level conflicts then get new name

Before trying to apply directory renames to paths within the given
directories, we want to make sure that there aren't conflicts at the
file level either. If there aren't any, then get the new name from
any directory renames.

Reviewed-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

merge-recursive: add computation of collisions due to dir rename & merging

directory renaming and merging can cause one or more files to be moved to
where an existing file is, or to cause several files to all be moved to
the same (otherwise vacant) location. Add checking and reporting for such
cases, falling back to no-directory-rename handling for such paths.

Reviewed-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

merge-recursive: check for directory level conflicts

Before trying to apply directory renames to paths within the given
directories, we want to make sure that there aren't conflicts at the
directory level. There will be additional checks at the individual
file level too, which will be added later.

Reviewed-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

merge-recursive: add get_directory_renames()

This populates a set of directory renames for us. The set of directory
renames is not yet used, but will be in subsequent commits.

Note that the use of a string_list for possible_new_dirs in the new
dir_rename_entry struct implies an O(n^2) algorithm; however, in practice
I expect the number of distinct directories that files were renamed into
from a single original directory to be O(1). My guess is that n has a
mode of 1 and a mean of less than 2, so, for now, string_list seems good
enough for possible_new_dirs.

Reviewed-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

The fifth batch for 2.18

Signed-off-by: Junio C Hamano <gitster@pobox.com>

Merge branch 'ma/http-walker-no-partial'

"git http-fetch" (deprecated) had an optional and experimental
"feature" to fetch only commits and/or trees, which nobody used.
This has been removed.

* ma/http-walker-no-partial:
walker: drop fields of `struct walker` which are always 1
http-fetch: make `-a` standard behaviour

Merge branch 'js/runtime-prefix'

* js/runtime-prefix:
Avoid multiple PREFIX definitions
git_setup_gettext: plug memory leak
gettext: avoid initialization if the locale dir is not present

Merge branch 'js/colored-push-errors'

Error messages from "git push" can be painted for more visibility.

* js/colored-push-errors:
config: document the settings to colorize push errors/hints
push: test to verify that push errors are colored
push: colorize errors
color: introduce support for colorizing stderr

Merge branch 'jc/parseopt-expiry-errors'

"git gc --prune=nonsense" spent long time repacking and then
silently failed when underlying "git prune --expire=nonsense"
failed to parse its command line. This has been corrected.

* jc/parseopt-expiry-errors:
parseopt: handle malformed --expire arguments more nicely
gc: do not upcase error message shown with die()

Merge branch 'ma/fast-export-skip-merge-fix'

"git fast-export" had a regression in v2.15.0 era where it skipped
some merge commits in certain cases, which has been corrected.

* ma/fast-export-skip-merge-fix:
fast-export: fix regression skipping some merge-commits

Merge branch 'tz/doc-git-urls-reference'

Doc fix.

* tz/doc-git-urls-reference:
doc/clone: update caption for GIT URLS cross-reference

Merge branch 'tg/demote-stash-save-in-completion'

The command line completion (in contrib/) has been taught that "git
stash save" has been deprecated ("git stash push" is the preferred
spelling in the new world) and does not offer it as a possible
completion candidate when "git stash push" can be.

* tg/demote-stash-save-in-completion:
completion: make stash -p and alias for stash push -p
completion: stop showing 'save' for stash by default

Merge branch 'sa/send-email-dedup-some-headers'

When fed input that already has In-Reply-To: and/or References:
headers and told to add the same information, "git send-email"
added these headers separately, instead of appending to an existing
one, which is a violation of the RFC. This has been corrected.

* sa/send-email-dedup-some-headers:
send-email: avoid duplicate In-Reply-To/References

Merge branch 'nd/submodule-status-fix'

"git submodule status" did not check the symbolic revision name it
computed for the submodule HEAD is not the NULL, and threw it at
printf routines, which has been corrected.

* nd/submodule-status-fix:
submodule--helper: don't print null in 'submodule status'

Merge branch 'js/ident-date-fix'

During a "rebase -i" session, the code could give older timestamp
to commits created by later "pick" than an earlier "reword", which
has been corrected.

* js/ident-date-fix:
sequencer: reset the committer date before commits

Merge branch 'bt/gpg-interface'

What is queued here is only the obviously correct and
uncontroversial code clean-up part, which is an earlier 7 patches,
of a larger series.

The remainder that is not queued introduces a few configuration
variables to deal with e-signature backends with different
signature format.

* bt/gpg-interface:
gpg-interface: find the last gpg signature line
gpg-interface: extract gpg line matching helper
gpg-interface: fix const-correctness of "eol" pointer
gpg-interface: use size_t for signature buffer size
gpg-interface: modernize function declarations
gpg-interface: handle bool user.signingkey
t7004: fix mistaken tag name

Merge branch 'hn/sort-ls-remote'

"git ls-remote" learned an option to allow sorting its output based
on the refnames being shown.

* hn/sort-ls-remote:
ls-remote: create '--sort' option

Merge branch 'ab/git-svn-get-record-typofix'

"git svn" had a minor thinko/typo which has been fixed.

* ab/git-svn-get-record-typofix:
git-svn: avoid warning on undef readline()

Merge branch 'tb/config-default'

"git config --get" learned the "--default" option, to help the
calling script. Building on top of the tb/config-type topic, the
"git config" learns "--type=color" type. Taken together, you can
do things like "git config --get foo.color --default blue" and get
the ANSI color sequence for the color given to foo.color variable,
or "blue" if the variable does not exist.

* tb/config-default:
builtin/config: introduce `color` type specifier
config.c: introduce 'git_config_color' to parse ANSI colors
builtin/config: introduce `--default`

Merge branch 'tb/config-type'

The "git config" command uses separate options e.g. "--int",
"--bool", etc. to specify what type the caller wants the value to
be interpreted as. A new "--type=<typename>" option has been
introduced, which would make it cleaner to define new types.

* tb/config-type:
builtin/config.c: support `--type=<type>` as preferred alias for `--<type>`
builtin/config.c: treat type specifiers singularly

Merge branch 'sg/doc-gc-quote-mismatch-fix'

Doc formatting fix.

* sg/doc-gc-quote-mismatch-fix:
docs/git-gc: fix minor rendering issue

Merge branch 'sg/completion-clear-cached'

The completion script (in contrib/) learned to clear cached list of
command line options upon dot-sourcing it again in a more efficient
way.

* sg/completion-clear-cached:
completion: reduce overhead of clearing cached --options

Merge branch 'sb/worktree-remove-opt-force'

"git worktree remove" learned that "-f" is a shorthand for
"--force" option, just like for "git worktree add".

* sb/worktree-remove-opt-force:
worktree: accept -f as short for --force for removal

Merge branch 'ma/double-dashes-in-docs'

Doc formatting updates.

* ma/double-dashes-in-docs:
git-submodule.txt: quote usage in monospace, drop backslash
git-[short]log.txt: unify quoted standalone --
doc: convert [\--] to [--]
doc: convert \--option to --option

Merge branch 'tq/t1510'

Test cleanup.

* tq/t1510:
t1510-repo-setup.sh: remove useless mkdir

Merge branch 'so/glossary-ancestor'

Docfix.

* so/glossary-ancestor:
glossary: substitute "ancestor" for "direct ancestor" in 'push' description.

Merge branch 'ls/checkout-encoding'

The new "checkout-encoding" attribute can ask Git to convert the
contents to the specified encoding when checking out to the working
tree (and the other way around when checking in).

* ls/checkout-encoding:
convert: add round trip check based on 'core.checkRoundtripEncoding'
convert: add tracing for 'working-tree-encoding' attribute
convert: check for detectable errors in UTF encodings
convert: add 'working-tree-encoding' attribute
utf8: add function to detect a missing UTF-16/32 BOM
utf8: add function to detect prohibited UTF-16/32 BOM
utf8: teach same_encoding() alternative UTF encoding names
strbuf: add a case insensitive starts_with()
strbuf: add xstrdup_toupper()
strbuf: remove unnecessary NUL assignment in xstrdup_tolower()

Merge branch 'ab/nuke-emacs-contrib'

The scripts in contrib/emacs/ have outlived their usefulness and
have been replaced with a stub that errors out and tells the user
there are replacements.

* ab/nuke-emacs-contrib:
git{,-blame}.el: remove old bitrotting Emacs code

Merge branch 'nd/warn-more-for-devs'

The build procedure "make DEVELOPER=YesPlease" learned to enable a
bit more warning options depending on the compiler used to help
developers more. There also is "make DEVOPTS=tokens" knob
available now, for those who want to help fixing warnings we
usually ignore, for example.

* nd/warn-more-for-devs:
Makefile: add a DEVOPTS to get all of -Wextra
Makefile: add a DEVOPTS to suppress -Werror under DEVELOPER
Makefile: detect compiler and enable more warnings in DEVELOPER=1
connect.c: mark die_initial_contact() NORETURN

Merge branch 'sb/object-store-replace'

The effort to pass the repository in-core structure throughout the
API continues. This round deals with the code that implements the
refs/replace/ mechanism.

* sb/object-store-replace:
replace-object: allow lookup_replace_object to handle arbitrary repositories
replace-object: allow do_lookup_replace_object to handle arbitrary repositories
replace-object: allow prepare_replace_object to handle arbitrary repositories
refs: allow for_each_replace_ref to handle arbitrary repositories
refs: store the main ref store inside the repository struct
replace-object: add repository argument to lookup_replace_object
replace-object: add repository argument to do_lookup_replace_object
replace-object: add repository argument to prepare_replace_object
refs: add repository argument to for_each_replace_ref
refs: add repository argument to get_main_ref_store
replace-object: check_replace_refs is safe in multi repo environment
replace-object: eliminate replace objects prepared flag
object-store: move lookup_replace_object to replace-object.h
replace-object: move replace_map to object store
replace_object: use oidmap

Merge branch 'ds/commit-graph'

Precompute and store information necessary for ancestry traversal
in a separate file to optimize graph walking.

* ds/commit-graph:
commit-graph: implement "--append" option
commit-graph: build graph from starting commits
commit-graph: read only from specific pack-indexes
commit: integrate commit graph with commit parsing
commit-graph: close under reachability
commit-graph: add core.commitGraph setting
commit-graph: implement git commit-graph read
commit-graph: implement git-commit-graph write
commit-graph: implement write_commit_graph()
commit-graph: create git-commit-graph builtin
graph: add commit graph design document
commit-graph: add format document
csum-file: refactor finalize_hashfile() method
csum-file: rename hashclose() to finalize_hashfile()

Merge branch 'js/empty-config-section-fix'

"git config --unset a.b", when "a.b" is the last variable in an
otherwise empty section "a", left an empty section "a" behind, and
worse yet, a subsequent "git config a.c value" did not reuse that
empty shell and instead created a new one. These have been
(partially) corrected.

* js/empty-config-section-fix:
git_config_set: reuse empty sections
git config --unset: remove empty sections (in the common case)
git_config_set: make use of the config parser's event stream
git_config_set: do not use a state machine
config_set_store: rename some fields for consistency
config: avoid using the global variable `store`
config: introduce an optional event stream while parsing
t1300: `--unset-all` can leave an empty section behind (bug)
t1300: add a few more hairy examples of sections becoming empty
t1300: remove unreasonable expectation from TODO
t1300: avoid relying on a bug
config --replace-all: avoid extra line breaks
t1300: demonstrate that --replace-all can "invent" newlines
t1300: rename it to reflect that `repo-config` was deprecated
git_config_set: fix off-by-two

Merge branch 'ot/libify-get-ref-atom-value'

Code restructuring, in preparation for further work.

* ot/libify-get-ref-atom-value:
ref-filter: libify get_ref_atom_value()
ref-filter: add return value to parsers
ref-filter: change parsing function error handling
ref-filter: add return value && strbuf to handlers
ref-filter: start adding strbufs with errors
ref-filter: add shortcut to work with strbufs

Merge branch 'sb/submodule-move-nested'

Moving a submodule that itself has submodule in it with "git mv"
forgot to make necessary adjustment to the nested sub-submodules;
now the codepath learned to recurse into the submodules.

* sb/submodule-move-nested:
submodule: fixup nested submodules after moving the submodule
submodule-config: remove submodule_from_cache
submodule-config: add repository argument to submodule_from_{name, path}
submodule-config: allow submodule_free to handle arbitrary repositories
grep: remove "repo" arg from non-supporting funcs
submodule.h: drop declaration of connect_work_tree_and_git_dir

Merge branch 'dj/runtime-prefix'

A build-time option has been added to allow Git to be told to refer
to its associated files relative to the main binary, in the same
way that has been possible on Windows for quite some time, for
Linux, BSDs and Darwin.

* dj/runtime-prefix:
Makefile: quote $INSTLIBDIR when passing it to sed
Makefile: remove unused @@PERLLIBDIR@@ substitution variable
mingw/msvc: use the new-style RUNTIME_PREFIX helper
exec_cmd: provide a new-style RUNTIME_PREFIX helper for Windows
exec_cmd: RUNTIME_PREFIX on some POSIX systems
Makefile: add Perl runtime prefix support
Makefile: generate Perl header from template file

Merge branch 'ab/simplify-perl-makefile'

Recent simplification of build procedure forgot a bit of tweak to
the build procedure of contrib/mw-to-git/

* ab/simplify-perl-makefile:
Makefile: mark perllibdir as a .PHONY target
perl: fix installing modules from contrib

Merge branch 'bw/protocol-v2'

The beginning of the next-gen transfer protocol.

* bw/protocol-v2: (35 commits)
remote-curl: don't request v2 when pushing
remote-curl: implement stateless-connect command
http: eliminate "# service" line when using protocol v2
http: don't always add Git-Protocol header
http: allow providing extra headers for http requests
remote-curl: store the protocol version the server responded with
remote-curl: create copy of the service name
pkt-line: add packet_buf_write_len function
transport-helper: introduce stateless-connect
transport-helper: refactor process_connect_service
transport-helper: remove name parameter
connect: don't request v2 when pushing
connect: refactor git_connect to only get the protocol version once
fetch-pack: support shallow requests
fetch-pack: perform a fetch using v2
upload-pack: introduce fetch server command
push: pass ref prefixes when pushing
fetch: pass ref prefixes when fetching
ls-remote: pass ref prefixes when requesting a remote's refs
transport: convert transport_get_remote_refs to take a list of ref prefixes
...

mailmap: update brian m. carlson's email address

An earlier change, cdb6b5ac (".mailmap: Combine more (name, email) to
individual persons", 2013-08-12), noted that there were two name
spellings and two email addresses and mapped the crustytoothpaste.net
address to the crustytoothpaste.ath.cx address. The latter is an older,
obsolete address, while the former is current, so switch the order of
the addresses so that git log displays the correct address.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

completion: load completion file for external subcommand

Adding external subcommands to Git is as easy as to put an executable
file git-foo into PATH. Packaging such subcommands for a Linux
distribution can be achieved by unpacking the executable into /usr/bin
of the user's system. Adding system-wide completion scripts for new
subcommands, however, can be a bit tricky.

Since bash-completion started to use dynamical loading of completion
scripts since v1.90 (preview of v2.0), it is no longer sufficient to
drop a completion script of a subcommand into the standard completions
path, /usr/share/bash-completion/completions, since this script will not
be loaded if called as a git subcommand.

For example, look at https://bugs.gentoo.org/544722. To give a short
summary: The popular git-flow subcommand provides a completion script,
which gets installed as /usr/share/bash-completion/completions/git-flow.

If you now type into a Bash shell:

git flow <TAB>

You will not get any completions, because bash-completion only loads
completions for git and git has no idea that git-flow is defined in
another file. You have to load this script manually or trigger the
dynamic loader with:

git-flow <TAB> # Please notice the dash instead of whitespace

This will not complete anything either, because it only defines a Bash
function, without generating completions. But now the correct completion
script has been loaded and the first command can use the completions.

So, the goal is now to teach the git completion script to consider the
possibility of external completion scripts for subcommands, but of
course without breaking current workflows.

I think the easiest method is to use a function that was defined by
bash-completion v1.90, namely _completion_loader. It will take care of
loading the correct script if present. Afterwards, the git completion
script behaves as usual.

_completion_loader was introduced in commit 20c05b43 of bash-completion
(https://github.com/scop/bash-completion.git) back in 2011, so it should
be available in even older LTS distributions. This function searches for
external completion scripts not only in the default path
/usr/share/bash-completion/completions, but also in the user's home
directory via $XDG_DATA_HOME and in a user specified directory via
$BASH_COMPLETION_USER_DIR.

The only "drawback" (if it even can be called as such) is, that if
_completion_loader does not find a completion script, it automatically
registers a minimal function for basic path completion. In practice,
however, this will not matter, because in this case the given command is
a git command in its dashed form, e.g. 'git-diff-index', and those have
been deprecated for a long time.

This way we can leverage bash-completion's dynamic loading for git
subcommands and make it easier for developers to distribute custom
completion scripts.

Signed-off-by: Florian Gamböck <mail@floga.de>
Acked-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Documentation: render revisions correctly under Asciidoctor

When creating a literal block from an indented block without any sort of
delimiters, Asciidoctor strips off all leading whitespace, resulting in
a misrendered chart. Use an explicit literal block to indicate to
Asciidoctor that we want to keep the leading whitespace.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Documentation: use 8-space tabs with Asciidoctor

Asciidoctor expands tabs at the beginning of a line. However, it does
not expand them into 8 spaces by default. Since we use 8-space tabs,
tell Asciidoctor that we want 8 spaces by setting the tabsize attribute.
This ensures that our ASCII art renders properly.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Convert remaining die*(BUG) messages

These were not caught by the previous commit, as they did not match the
regular expression.

While at it, remove the localization from one instance: we never want
BUG() messages to be translated, as they target Git developers, not the
end user (hence it would be quite unhelpful to not only burden the
translators, but then even end up with a bug report in a language that
no core Git contributor understands).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Replace all die("BUG: ...") calls by BUG() ones

In d8193743e08 (usage.c: add BUG() function, 2017-05-12), a new macro
was introduced to use for reporting bugs instead of die(). It was then
subsequently used to convert one single caller in 588a538ae55
(setup_git_env: convert die("BUG") to BUG(), 2017-05-12).

The cover letter of the patch series containing this patch
(cf 20170513032414.mfrwabt4hovujde2@sigill.intra.peff.net) is not
terribly clear why only one call site was converted, or what the plan
is for other, similar calls to die() to report bugs.

Let's just convert all remaining ones in one fell swoop.

This trick was performed by this invocation:

sed -i 's/die("BUG: /BUG("/g' $(git grep -l 'die("BUG' \*.c)

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

run-command: use BUG() to report bugs, not die()

The slightly misleading name die_bug() of the function intended to
report a bug is actually called always, and only reports a bug if the
passed-in parameter `err` is non-zero.

It uses die_errno() to report the bug, to helpfully include the error
message corresponding to `err`.

However, as these messages indicate bugs, we really should use BUG().
And as BUG() is a macro to be able to report the exact file and line
number, we need to convert die_bug() to a macro instead of only
replacing the die_errno() by a call to BUG().

While at it, use a name more indicative of the purpose: CHECK_BUG().

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

test-tool: help verifying BUG() code paths

When we call BUG(), we signal via SIGABRT that something bad happened,
dumping cores if so configured. In some setups these coredumps are
redirected to some central place such as /proc/sys/kernel/core_pattern,
which is a good thing.

However, when we try to verify in our test suite that bugs are caught in
certain code paths, we do *not* want to clutter such a central place
with unnecessary coredumps.

So let's special-case the test helpers (which we use to verify such code
paths) so that the BUG() calls will *not* call abort() but exit with a
special-purpose exit code instead.

Helped-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>