t7004: fix embedded single-quotes

This test uses single quotes inside the single-quoted test
snippet, which effectively makes the contents unquoted.
Since they don't need quoted anyway, this isn't a problem,
but let's switch them to double-quotes to make it more
obviously correct.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t0050: appease --chain-lint

Some of the symlink tests check an either-or case using the
"||". This is not wrong, but fools --chain-lint into
thinking the &&-chain is broken (in fact, there is no &&
chain here).

We can solve this by wrapping the "||" inside a {} block.
This is a bit more verbose, but this construct is rare, and
the {} block helps call attention to it.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t9001: use test_when_finished

The confirmation tests in t9001 all save the value of
sendemail.confirm, do something to it, then restore it at
the end, in a way that breaks the &&-chain (they are not
wrong, because they save the $? value, but it fools
--chain-lint).

Instead, they can all use test_when_finished, and we can
even make the code simpler by factoring out the shared
lines.

Note that we can _almost_ use test_config here, except that:

1. We do not restore the config with test_unconfig, but by
setting it back to some prior value.

2. We are not always setting a config variable. Sometimes
the change to be undone is unsetting it entirely.

We could teach test_config to handle these cases, but it's
not worth the complexity for a single call-site.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t4117: use modern test_* helpers

We can use test_must_fail and test_path_* to avoid some
hand-rolled if statements. This makes the code shorter, and
makes it more obvious when we are breaking the &&-chain.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t6034: use modern test_* helpers

These say roughly the same thing as the hand-rolled
messages. We do lose the "merge did not complete" debug
message, but merge and write-tree are prefectly capable of
writing useful error messages when they fail.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t1301: use modern test_* helpers

This shortens the code and fixes some &&-chaining.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t0020: use modern test_* helpers

This test contains a lot of hand-rolled messages to show
when the test fails. We can omit most of these by using
"verbose" and "test_must_fail". A few of them are for
update-index, but we can assume it produces reasonable error
messages when it fails.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t6030: use modern test_* helpers

We can get rid of a lot of hand-rolled error messages by
using test_must_fail and test_expect_code. The existing code
was careful to use "|| return 1" when breaking the
&&-chain, but it did fool --chain-lint; the new code is more
idiomatic.

We also add some uses of test_when_finished, which is less
cryptic and more robust than putting code at the end of a
test. In two cases we run "git bisect reset" from a
subshell, which is a problem for test_when_finished (it
would not run). However, in both of these cases, we are
performing the tests in one-off sub-repos, so we do not need
to clean up at all (and in fact it is nicer not to if the
user wants to inspect the trash directory after a failure).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t9502: fix &&-chain breakage

This script misses a trivial &&-chain in one of its tests,
but it also has a weird reverse: it includes an &&-chain
outside of any test_expect block! This "cat" should never
fail, but if it did, we would not notice, as it would cause
us to skip the follow-on test entirely (which does not
appear intentional; there are many later tests which rely on
this cat).

Let's instead move the setup into its own test_expect_success
block, which is the standard practice nowadays.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t2026: fix broken &&-chain

Signed-off-by: Junio C Hamano <gitster@pobox.com>

t7201: fix &&-chain breakage

One of these breakages is in setup, but one is more severe
and may miss a real test failure. These are pulled out from
the rest, though, because we also clean up a few other
anachronisms. The most interesting is the use of this
here-doc construct:

(cat >... <<EOF
...
EOF
) &&

It looks like an attempt to make the &&-chaining more
natural by letting it come at the end of the here-doc. But
the extra sub-shell is so non-idiomatic (plus the lack of
"<<-") that it ends up confusing.

Since these are just using a single line, we can accomplish
the same thing with a single printf (which also makes the
use of tab more obvious than the verbatim whitespace).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t3600: fix &&-chain breakage for setup commands

As with the earlier patch to fix "trivial" &&-chain
breakage, these missing "&&" operators are not a serious
problem (e.g., we do not expect "echo" to fail).

Ironically, however, inserting them shows that some of the
commands _do_ fail. Specifically, some of the tests start by
making sure we are at a commit with the string "content" in
the file "foo". However, running "git commit" may fail
because the previous test left us in that state already, and
there is nothing to commit.

We could remove these commands entirely, but they serve to
document the test's assumptions, as well as make it robust
when an earlier test has failed. We could use test_might_fail
to handle all cases, but that would miss an unrelated
failure to make the commit. Instead, we can just pass the
--allow-empty flag to git-commit, which means that it will
not complain if our setup is a noop.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t: avoid using ":" for comments

The ":" is not a comment marker, but rather a noop command.
Using it as a comment like:

: do something
cmd1 &&

: something else
cmd2

breaks the &&-chain, and we would fail to notice if "cmd1"
failed in this instance. We can just use regular "#"
comments instead.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t: wrap complicated expect_code users in a block

If we are expecting a command to produce a particular exit
code, we can use test_expect_code. However, some cases are
more complicated, and want to accept one of a range of exit
codes. For these, we end up with something like:

cmd;
case "$?" in
...

That unfortunately breaks the &&-chain and fools
--chain-lint. Since these special cases are so few, we can
wrap them in a block, like this:

{ cmd; ret=$?; } &&
case "$ret" in
...

This accomplishes the same thing, and retains the &&-chain
(the exit status fed to the && is that of the assignment,
which should always be true). It's technically longer, but
it is probably a good thing for unusual code like this to
stand out.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t: use test_expect_code instead of hand-rolled comparison

This makes our output in the event of a failure slightly
nicer, and it means that we do not break the &&-chain.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t: use test_might_fail for diff and grep

Some tests run diff or grep to produce an output, and then
compare the output to an expected value. We know the exit
code we expect these processes to have (e.g., grep yields 0
if it produced output and 1 otherwise), so it would not make
the test wrong to look for it. But the difference between
their output and the expected output (e.g., shown by
test_cmp) is much more useful to somebody debugging the test
than the test just bailing out.

These tests break the &&-chain to skip the exit-code check
of the process. However, we can get the same effect by using
test_might_fail. Note that in some cases the test did use
"|| return 1", which meant the test was not wrong, but it
did fool --chain-lint.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t: fix &&-chaining issues around setup which might fail

Many tests have an initial setup step that might fail based
on whether earlier tests in the script have succeeded or
not. Using a trick like "|| true" breaks the &&-chain,
missing earlier failures (and fooling --chain-lint).

We can use test_might_fail in some cases, which is correct
and makes the intent more obvious. We can also use
test_unconfig for unsetting config (and which is more
robust, as well).

The case in t9500 is an oddball. It wants to run cmd1 _or_
cmd2, and does it like:

cmd1 || cmd2 &&
other_stuff

It's not wrong in this case, but it's a bad habit to get
into, because it breaks the &&-chain if used anywhere except
at the beginning of the test (and we use the correct
solution here, putting it inside a block for precedence).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t: use test_must_fail instead of hand-rolled blocks

These test scripts likely predate test_must_fail, and can be
made simpler by using it (in addition to making them pass
--chain-lint).

The case in t6036 loses some verbosity in the failure case,
but it is so tied to a specific failure mode that it is not
worth keeping around (and the outcome of the test is not
affected at all).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t: use verbose instead of hand-rolled errors

Many tests that predate the "verbose" helper function use a
pattern like:

test ... || {
echo ...
false
}

to give more verbose output. Using the helper, we can do
this with a single line, and avoid a || which interacts
badly with &&-chaining (besides fooling --chain-lint, we hit
the error block no matter which command in the chain failed,
so we may often show useless results).

In most cases, the messages printed by "verbose" are equally
good (in some cases better; t6006 accidentally redirects the
message to a file!). The exception is t7001, whose output
suffers slightly. However, it's still enough to show the
user which part failed, given that we will have just printed
the test script to stderr.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t: assume test_cmp produces verbose output

Some tests call test_cmp, and if it fails show the actual
output generated. This is mostly pointless, as test_cmp will
already show a diff between the expected and actual output.
It also fools --chain-lint by putting an "||" in the middle
of the chain, so we'd rather not use this construct.

Note that these cases actually show a pre-processed version
of the data, rather than exactly what test_cmp would show.
However, test_cmp's output is generally good for pointing
the user in the right direction, and they can then dig in
the trash directory themselves if they want to see more
details.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t: fix trivial &&-chain breakage

These are tests which are missing a link in their &&-chain,
but during a setup phase. We may fail to notice failure in
commands that build the test environment, but these are
typically not expected to fail at all (but it's still good
to double-check that our test environment is what we
expect).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t: fix moderate &&-chain breakage

These are tests which are missing a link in their &&-chain,
but in a way that probably does not effect the outcome of
the test. Most of these are of the form:

some_cmd >actual
test_cmp expect actual

The main point of the test is to verify the output, and a
failure in some_cmd would probably be noticed by bogus
output. But it is good for the tests to also confirm that
"some_cmd" does not die unexpectedly after producing its
output.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t: fix severe &&-chain breakage

These are tests which are missing a link in their &&-chain,
in a location which causes a significant portion of the test
to be missed (e.g., the test effectively does nothing, or
consists of a long string of actions and output comparisons,
and we throw away the exit code of at least one part of the
string).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t/test-lib: introduce --chain-lint option

It's easy to miss an "&&"-chain in a test script, like:

test_expect_success 'check something important' '
cmd1 &&
cmd2
cmd3
'

The test harness will notice if cmd3 fails, but a failure of
cmd1 or cmd2 will go unnoticed, as their exit status is lost
after cmd3 runs.

The toy example above is easy to spot because the "cmds" are
all the same length, but real code is much more complicated.
It's also difficult to detect these situations by statically
analyzing the shell code with regexps (like the
check-non-portable-shell script does); there's too much
context required to know whether a &&-chain is appropriate
on a given line or not.

This patch instead lets the shell check each test by
sticking a command with a specific and unusual return code
at the top of each test, like:

(exit 117) &&
cmd1 &&
cmd2
cmd3

In a well-formed test, the non-zero exit from the first
command prevents any of the rest from being run, and the
test's exit code is 117. In a bad test (like the one above),
the 117 is lost, and cmd3 is run.

When we encounter a failure of this check, we abort the test
script entirely. For one thing, we have no clue which subset
of the commands in the test snippet were actually run.
Running further tests would be pointless, because we're now
in an unknown state. And two, this is not a "test failure"
in the traditional sense. The test script is buggy, not the
code it is testing. We should be able to fix these problems
in the script once, and not have them come back later as a
regression in git's code.

After checking a test snippet for --chain-lint, we do still
run the test itself. We could actually have a pure-lint
mode which just checks each test, but there are a few
reasons not to. One, because the tests are executing
arbitrary code, which could impact the later environment
(e.g., that could impact which set of tests we run at all).
And two, because a pure-lint mode would still be expensive
to run, because a significant amount of code runs outside of
the test_expect_* blocks. Instead, this option is designed
to be used as part of a normal test suite run, where it adds
very little overhead.

Turning on this option detects quite a few problems in
existing tests, which will be fixed in subsequent patches.
However, there are a number of places it cannot reach:

- it cannot find a failure to break out of loops on error,
like:

cmd1 &&
for i in a b c; do
cmd2 $i
done &&
cmd3

which will not notice failures of "cmd2 a" or "cmd b"

- it cannot find a missing &&-chain inside a block or
subfunction, like:

foo () {
cmd1
cmd2
}

foo &&
bar

which will not notice a failure of cmd1.

- it only checks tests that you run; every platform will
have some tests skipped due to missing prequisites,
so it's impossible to say from one run that the test
suite is free of broken &&-chains. However, all tests get
run by _somebody_, so eventually we will notice problems.

- it does not operate on test_when_finished or prerequisite
blocks. It could, but these tends to be much shorter and
less of a problem, so I punted on them in this patch.

This patch was inspired by an earlier patch by Jonathan
Nieder:

http://article.gmane.org/gmane.comp.version-control.git/235913

This implementation and all bugs are mine.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

rev-list: refuse --first-parent combined with --bisect

rev-list --bisect is used by git bisect, but never together with
--first-parent. Because rev-list --bisect together with --first-parent
is not handled currently, and even leads to segfaults, refuse to use
both options together.

Because this is not supported, it makes little sense to use git log
--bisect --first parent either, because refs/heads/bad is not limited to
the first parent chain.

Helped-by: Junio C. Hamano <gitster@pobox.com>
Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Kevin Daudt <me@ikke.info>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

fetch-pack: remove dead assignment to ref->new_sha1

In everything_local(), we used to assign the current ref's value
found in ref->old_sha1 to ref->new_sha1 when we already have all the
necessary objects to complete the history leading to that
commit. This copying was broken at 49bb805e (Do not ask for
objects known to be complete., 2005-10-19) and ever since we
instead stuffed a random bytes in ref->new_sha1 here. No
code complained or failed due to this breakage.

It turns out that no code path that comes after this
assignment even looks at ref->new_sha1 at all.

- The only caller of everything_local(), do_fetch_pack(),
returns this list of refs, whose element has bogus
new_sha1 values, to its caller. It does not look at the
elements itself, but does pass them to find_common, which
looks only at the name and old_sha1 fields.

- The only caller of do_fetch_pack(), fetch_pack(), returns this
list to its caller. It does not look at the elements nor act on
them.

- One of the two callers of fetch_pack() is cmd_fetch_pack(), the
top-level that implements "git fetch-pack". The only thing it
looks at in the elements of the returned ref list is the old_sha1
and name fields.

- The other caller of fetch_pack() is fetch_refs_via_pack() in the
transport layer, which is a helper that implements "git fetch".
It only cares about whether the returned list is empty (i.e.
failed to fetch anything).

Just drop the bogus assignment, that is not even necessary. The
remote-tracking refs are updated based on a different list and not
using the ref list being manipulated by this code path; the caller
do_fetch_pack() created a copy of that real ref list and passed the
copy down to this function, and modifying the elements here does not
affect anything.

Noticed-by: Kyle J. McKay <mackyle@gmail.com>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

fetch_refs_via_pack: free extra copy of refs

When fetch_refs_via_pack calls fetch_pack(), we pass a
list of refs to fetch, and the function returns either a
copy of that list, with the fetched items filled in, or
NULL. We check the return value to see whether the fetch was
successful, but do not otherwise look at the copy, and
simply leak it at the end of the function.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

filter_ref: make a copy of extra "sought" entries

If the server supports allow_tip_sha1_in_want, we add any
unmatched raw-sha1 entries in our "sought" list of refs to
the list of refs we will ask the other side for. We do so by
inserting the original "struct ref" directly into our list,
rather than making a copy. This has several problems.

The most minor problem is that one cannot ever free the
resulting list; it contains structs that are copies of the
remote refs (made earlier by fetch_pack) along with sought
refs that are referenced elsewhere.

But more importantly that we set the ref->next pointer to
NULL, chopping off the remainder of any existing list that
the ref was a part of. We get the set of "sought" refs in
an array rather than a linked list, but that array is often
in turn generated from a list. The test modification in
t5516 demonstrates this. Rather than fetching just an exact
sha1, we fetch that sha1 plus another ref:

- we build a linked list of refs to fetch when do_fetch
calls get_ref_map; the exact sha1 is first, followed by
the named ref ("refs/heads/extra" in this case).

- we pass that linked list to transport_fetch_ref, which
squashes it into an array of pointers

- that array goes to fetch_pack, which calls filter_ref.
There we generate the want list from a mix of what the
remote side has advertised, and the "sought" entry for
the exact sha1. We set the sought entry's "next" pointer
to NULL.

- after we return from transport_fetch_refs, we then try
to update the refs by following the linked list. But our
list is now truncated, and we do not update
refs/heads/extra at all.

We can fix this by making a copy of the ref. There's nothing
that fetch_pack does to it that must be reflected in the
original "sought" list (and indeed, if that were the case we
would have a serious bug, because it is only exact-sha1
entries which are treated this way).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

filter_ref: avoid overwriting ref->old_sha1 with garbage

If the server supports allow_tip_sha1_in_want, then
fetch-pack's filter_refs function tries to check whether a
ref is a request for a straight sha1 by running:

if (get_sha1_hex(ref->name, ref->old_sha1))
...

I.e., we are using get_sha1_hex to ask "is this ref name a
sha1?". If it is true, then the contents of ref->old_sha1
will end up unchanged. But if it is false, then get_sha1_hex
makes no guarantees about what it has written. With a ref
name like "abcdefoo", we would overwrite 3 bytes of
ref->old_sha1 before realizing that it was not a sha1.

This is likely not a problem in practice, as anything in
refs->name (besides a sha1) will start with "refs/", meaning
that we would notice on the first character that there is a
problem. Still, we are making assumptions about the state
left in the output when get_sha1_hex returns an error (e.g.,
it could start from the end of the string, or error check
the values only once they were placed in the output). It's
better to be defensive.

We could just check that we have exactly 40 characters of
sha1. But let's be even more careful and make sure that we
have a 40-char hex refname that matches what is in old_sha1.
This is perhaps overly defensive, but spells out our
assumptions clearly.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

gitweb: fix typo in man page

Signed-off-by: Tony Finch <dot@dotat.at>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

clone: drop period from end of die_errno message

We do not usually end our errors with a full stop, but it
looks especially bad when you use die_errno, which adds a
colon, like:

fatal: could not create work tree dir 'foo'.: No such file or directory

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

clone: initialize atexit cleanup handler earlier

If clone fails, we generally try to clean up any directories
we've created. We do this by installing an atexit handler,
so that we don't have to manually trigger cleanup. However,
since we install this after touching the filesystem, any
errors between our initial mkdir() and our atexit() call
will result in us leaving a crufty directory around.

We can fix this by moving our atexit() call earlier. It's OK
to do it before the junk_work_tree variable is set, because
remove_junk makes sure the variable is initialized. This
means we "activate" the handler by assigning to the
junk_work_tree variable, which we now bump down to just
after we call mkdir(). We probably do not want to do it
before, because a plausible reason for mkdir() to fail is
EEXIST (i.e., we are racing with another "git init"), and we
would not want to remove their work.

OTOH, this is probably not that big a deal; we will allow
cloning into an empty directory (and skip the mkdir), which
is already racy (i.e., one clone may see the other's empty
dir and start writing into it). Still, it does not hurt to
err on the side of caution here.

Note that writing into junk_work_tree and junk_git_dir after
installing the handler is also technically racy, as we call
our handler on an async signal. Depending on the platform,
we could see a sheared write to the variables. Traditionally
we have not worried about this, and indeed we already do
this later in the function. If we want to address that, it
can come as a separate topic.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

sha1fd_check: die when we cannot open the file

Right now we return a NULL "struct sha1file" if we encounter
an error. However, the sole caller (write_idx_file) does not
check the return value, and will segfault if we hit this
case.

One option would be to handle the error in the caller.
However, there's really nothing for it to do but die. This
code path is hit during "git index-pack --verify"; after we
verify the packfile, we check that the ".idx" we would
generate from it is byte-wise identical to what is on disk.
We hit the error (and segfault) if we can't open the .idx
file (a likely cause of this is that somebody else ran "git
repack -ad" while we were verifying). Since we can't
complete the requested verification, we really have no
choice but to die.

Furthermore, the rest of the sha1fd_* functions simply die
on errors. So if were to open the file successfully, for
example, and then hit a read error, sha1write would call
die() for us. So pushing the die() down into sha1fd_check
keeps the interface consistent.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

grep: fix "--quiet" overwriting current output

When grep is called with the --quiet option, the pager is initialized
despite not being used. When the pager is "less", anything output by
previous commands and not ended with a newline is overwritten:

$ echo -n aaa; echo bbb
aaabbb
$ echo -n aaa; git grep -q foo; echo bbb
bbb

This can be worked around, for example, by making sure STDOUT is not a
TTY or more directly by setting git's pager to "cat":

$ echo -n aaa; git grep -q foo > /dev/null; echo bbb
aaabbb
$ echo -n aaa; PAGER=cat git grep -q foo; echo bbb
aaabbb

But prevent calling the pager in the first place, which would also
save an unnecessary fork().

Signed-off-by: Wilhelm Schuermann <wimschuermann@googlemail.com>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

revision: forbid combining --graph and --no-walk

Because "--graph" is about connected history while --no-walk is
about discrete points, it does not make sense to allow these two
options at the same time. [1]

This change makes a few calls to "show --graph" fail in t4052, but
asking to show one commit with graph is a nonsensical thing to do.
Thus, tests on "show --graph" in t4052 have been removed [2,3].
Same tests on "show" without --graph option have already been tested
in 4052.

3 testcases have been added to test this patch.

[1]: http://article.gmane.org/gmane.comp.version-control.git/216083
[2]: http://article.gmane.org/gmane.comp.version-control.git/264950
[3]: http://article.gmane.org/gmane.comp.version-control.git/265107

Helped-By: Eric Sunshine <sunshine@sunshineco.com>
Helped-By: René Scharfe <l.s.r@web.de>
Helped-By: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Dongcan Jiang <dongcan.jiang@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Post 2.3 cyce (batch #10)

Also declare that the next one will be called v2.4 ;-)

Signed-off-by: Junio C Hamano <gitster@pobox.com>

Merge branch 'mg/doc-status-color-slot'

Documentation fixes.

* mg/doc-status-color-slot:
config,completion: add color.status.unmerged

Merge branch 'mg/status-v-v'

"git status" now allows the "-v" to be given twice to show the
differences that are left in the working tree not to be committed.

* mg/status-v-v:
commit/status: show the index-worktree diff with -v -v
t7508: test git status -v
t7508: .gitignore 'expect' and 'output' files

Merge branch 'mg/sequencer-commit-messages-always-verbatim'

"git cherry-pick" used to clean-up the log message even when it is
merely replaying an existing commit. It now replays the message
verbatim unless you are editing the message of resulting commits.

* mg/sequencer-commit-messages-always-verbatim:
sequencer: preserve commit messages

Merge branch 'sg/completion-remote'

Code simplification.

* sg/completion-remote:
completion: simplify __git_remotes()
completion: add a test for __git_remotes() helper function

Merge branch 'es/rebase-i-count-todo'

"git rebase -i" recently started to include the number of
commits in the insn sheet to be processed, but on a platform
that prepends leading whitespaces to "wc -l" output, the numbers
are shown with extra whitespaces that aren't necessary.

* es/rebase-i-count-todo:
rebase-interactive: re-word "item count" comment
rebase-interactive: suppress whitespace preceding item count

Merge branch 'ak/git-done-help-cleanup'

Code simplification.

* ak/git-done-help-cleanup:
git: make was_alias and done_help non-static

Merge branch 'rs/zip-text'

"git archive" can now be told to set the 'text' attribute in the
resulting zip archive.

* rs/zip-text:
archive-zip: mark text files in archives

Merge branch 'rs/deflate-init-cleanup'

Code simplification.

* rs/deflate-init-cleanup:
zlib: initialize git_zstream in git_deflate_init{,_gzip,_raw}

SubmittingPatches: encourage users to use format-patch and send-email

In step "(4) Sending your patches", we instruct users to do an
inline patch, avoid breaking whitespaces, avoid attachments, use
[PATCH v2] for second round, etc., all of which format-patch and
send-email combo know how to do well.

The need was identified by, and the text is based on the work by
Cody Taylor.

Suggested-by: Cody Taylor <cody.taylor@maternityneighborhood.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

git prompt: use toplevel to find untracked files

The __git_ps1() prompt function would not show an untracked state
when all the untracked files are outside the current working
directory.

Signed-off-by: Cody A Taylor <codemister99@yahoo.com>
Helped-by: SZEDER Gábor <szeder@ira.uka.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

gitk: Update .po files

Signed-off-by: Paul Mackerras <paulus@samba.org>

gitk: l10n: Add Catalan translation

Signed-off-by: Alex Henrie <alexhenrie24@gmail.com>
Signed-off-by: Paul Mackerras <paulus@samba.org>

gitk: Fix typo in Russian translation

Signed-off-by: 0xAX <kuleshovmail@gmail.com>
Signed-off-by: Paul Mackerras <paulus@samba.org>

gitk: Remove tcl-format flag from a message that shouldn't have it

xgettext sees "% o" and interprets it as a placeholder for an octal
number preceded by a space. However, in this case it's not actually a
placeholder, and most translations will replace the "% o" sequence with
something else. Removing the tcl-format flag from this string prevents
tools like Poedit from freaking out when "% o" doesn't appear in the
translated string.

The corrected flag will appear in each translation's po file the next time
the translation is updated with `make update-po`.

Signed-off-by: Alex Henrie <alexhenrie24@gmail.com>
Signed-off-by: Paul Mackerras <paulus@samba.org>

gitk: Pass --invert-grep option down to "git log"

"git log --grep=<string>" shows only commits with messages that
match the given string, but sometimes it is useful to be able to
show only commits that do *not* have certain messages (e.g. "show
me ones that are not FIXUP commits").

Now the underlying "git log" learned the "--invert-grep" option.
The option syntactically behaves similar to "--all-match" that
requires that all of the grep strings to match and semantically
behaves the opposite---it requires that none of the grep strings to
match.

Teach "gitk" to allow users to pass it down to underlying "git log"
command by adding it to the known_view_options array.

Signed-off-by: Christoph Junghans <ottxor@gentoo.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Paul Mackerras <paulus@samba.org>

gitk: Synchronize config file writes

If several gitk instances are closed simultaneously, the savestuff
procedure can run at the same time, resulting in a conflict which may
cause losing of some of the instance's changes, failing the saving
operation or even corrupting the configuration file. This can happen,
for example, at user session closing, or at group closing of all
instances of an application which is possible in some desktop
environments.

To avoid this, make sure that only one saving operation is in
progress. It is guarded by existence of the $config_file_tmp
file. Creating the file and moving it to $config_file are both atomic
operations, so it should be reliable.

Reading does not need to be syncronized, because moving is an atomic
operation, and the $config_file always refers to a full and correct file.
But, if there is a stale $config_file_tmp file, report it at gitk start.
If such file is detected when saving, just report it abort the save, as
for other errors in saving.

Signed-off-by: Max Kirillov <max@max630.net>
Signed-off-by: Paul Mackerras <paulus@samba.org>

gitk: Report errors in saving config file

Signed-off-by: Max Kirillov <max@max630.net>
Signed-off-by: Paul Mackerras <paulus@samba.org>

gitk: Only write changed configuration variables

When gitk contains some changed parameter, and there is an existing
instance of gitk where the parameter is still old, it is reverted to
that old value when that instance exits.

Instead, store a parameter in config only if it has been modified in
the exiting instance. Otherwise, preserve the value which currently is in
file. This allows editing the configuration when several instances are
running, without rollback of the modification if some other
instance where the configuration was not edited is closed last.

For scalar variables, use trace(3tcl) to detect their change. Since
`trace` can send bogus events, doublecheck if the value has really
been changed, but once it is marked as changed, do not reset it back
to unchanged ever, because if user has restored the original value,
it's the decision which should be stored as well as modified value.

Treat view list especially: instead of rewriting the whole list, merge
individual views. Place old and updated views in their old places,
add new ones to the end of list. Collect modified views explicitly, in
newviewok{} and delview{}.

Do not merge geometry values. They are almost always changing because
user moves and resises windows, and there is no way to find which one of
the geometries is most desired. Just overwrite them unconditionally,
like earlier.

Signed-off-by: Max Kirillov <max@max630.net>
Signed-off-by: Paul Mackerras <paulus@samba.org>

contrib/completion: escape the forward slash in __git_match_ctag

The current definition results in an incorrect expansion of the term under zsh.
For instance "/^${1////\\/}/" under zsh with the argument "hi" results in:
/^/\/h/\/i/

This results in an output similar to this when trying to complete `git grep
chartab` under zsh:

:: git grep chartabawk: cmd. line:1: /^/\/c/\/h/\/a/\/r/\/t/\/a/\/b/ { print $1 }
awk: cmd. line:1: ^ backslash not last character on line
awk: cmd. line:1: /^/\/c/\/h/\/a/\/r/\/t/\/a/\/b/ { print $1 }
awk: cmd. line:1: ^ syntax error

Leaving the prompt in a goofy state until the user hits a key.

Escaping the literal / in the parameter expansion (using "/^${1//\//\\/}/")
results in:
/^chartab/

allowing the completion to work correctly.

This formulation also works under bash.

Signed-off-by: John Szakmeister <john@szakmeister.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

push: allow --follow-tags to be set by config push.followTags

Signed-off-by: Dave Olszewski <cxreg@pobox.com>
Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Sync with 2.3.3

Git 2.3.3

Signed-off-by: Junio C Hamano <gitster@pobox.com>

Merge branch 'mr/doc-clean-f-f' into maint

Documentation update.

* mr/doc-clean-f-f:
Documentation/git-clean.txt: document that -f may need to be given twice

Merge branch 'ak/t5516-typofix' into maint

* ak/t5516-typofix:
t5516: correct misspelled pushInsteadOf

Merge branch 'jc/diff-test-updates' into maint

Test clean-up.

* jc/diff-test-updates:
test_ln_s_add: refresh stat info of fake symbolic links
t4008: modernise style
t/diff-lib: check exact object names in compare_diff_raw
tests: do not borrow from COPYING and README from the real source
t4010: correct expected object names
t9300: correct expected object names
t4008: correct stale comments

Merge branch 'jk/diffcore-rename-duplicate' into maint

A corrupt input to "git diff -M" can cause us to segfault.

* jk/diffcore-rename-duplicate:
diffcore-rename: avoid processing duplicate destinations
diffcore-rename: split locate_rename_dst into two functions

Merge branch 'bw/kwset-use-unsigned' into maint

The borrowed code in kwset API did not follow our usual convention
to use "unsigned char" to store values that range from 0-255.

* bw/kwset-use-unsigned:
kwset: use unsigned char to store values with high-bit set

Merge branch 'nd/grep-exclude-standard-help-fix' into maint

Description given by "grep -h" for its --exclude-standard option
was phrased poorly.

* nd/grep-exclude-standard-help-fix:
grep: correct help string for --exclude-standard

Merge branch 'mg/doc-remote-tags-or-not' into maint

"git remote add" mentioned "--tags" and "--no-tags" and was not
clear that fetch from the remote in the future will use the default
behaviour when neither is given to override it.

* mg/doc-remote-tags-or-not:
git-remote.txt: describe behavior without --tags and --no-tags

Merge branch 'mk/diff-shortstat-dirstat-fix' into maint

"git diff --shortstat --dirstat=changes" showed a dirstat based on
lines that was never asked by the end user in addition to the
dirstat that the user asked for.

* mk/diff-shortstat-dirstat-fix:
diff --shortstat --dirstat: remove duplicate output

Merge branch 'ms/submodule-update-config-doc' into maint

The interaction between "git submodule update" and the
submodule.*.update configuration was not clearly documented.

* ms/submodule-update-config-doc:
submodule: improve documentation of update subcommand

Merge branch 'jc/apply-beyond-symlink' into maint

"git apply" was not very careful about reading from, removing,
updating and creating paths outside the working tree (under
--index/--cached) or the current directory (when used as a
replacement for GNU patch).

* jc/apply-beyond-symlink:
apply: do not touch a file beyond a symbolic link
apply: do not read from beyond a symbolic link
apply: do not read from the filesystem under --index
apply: reject input that touches outside the working area

Merge branch 'rs/daemon-interpolate' into maint

"git daemon" looked up the hostname even when "%CH" and "%IP"
interpolations are not requested, which was unnecessary.

* rs/daemon-interpolate:
daemon: use callback to build interpolated path
daemon: look up client-supplied hostname lazily

Merge branch 'jk/daemon-interpolate' into maint

The "interpolated-path" option of "git daemon" inserted any string
client declared on the "host=" capability request without checking.
Sanitize and limit %H and %CH to a saner and a valid DNS name.

* jk/daemon-interpolate:
daemon: sanitize incoming virtual hostname
t5570: test git-daemon's --interpolated-path option
git_connect: let user override virtual-host we send to daemon

apply: convert threeway_stage to object_id

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

patch-id: convert to use struct object_id

Convert some magic numbers to the new GIT_SHA1 constants.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

commit: convert parts to struct object_id

Convert struct commit_graft and necessary local parts of commit.c.
Also, convert several constants based on the hex length of an SHA-1 to
use GIT_SHA1_HEXSZ, and move several magic constants into variables for
readability.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

diff: convert struct combine_diff_path to object_id

Also, convert a constant to GIT_SHA1_HEXSZ.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

bulk-checkin.c: convert to use struct object_id

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

zip: use GIT_SHA1_HEXSZ for trailers

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

archive.c: convert to use struct object_id

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

bisect.c: convert leaf functions to use struct object_id

Convert some constants to GIT_SHA1_HEXSZ.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

define utility functions for object IDs

There are several utility functions (hashcmp and friends) that are used
for comparing object IDs (SHA-1 values). Using these functions, which
take pointers to unsigned char, with struct object_id requires tiresome
access to the sha1 member, which bloats code and violates the desired
encapsulation. Provide wrappers around these functions for struct
object_id for neater, more maintainable code. Use the new constants to
avoid the hard-coded 20s and 40s throughout the original functions.

These functions simply call the underlying pointer-to-unsigned-char
versions to ensure that any performance improvements will be passed
through to the new functions.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

define a structure for object IDs

Many places throughout the code use "unsigned char [20]" to store object IDs
(SHA-1 values). This leads to lots of hardcoded numbers throughout the
codebase. It also leads to confusion about the purposes of a buffer.

Introduce a structure for object IDs. This allows us to obtain the benefits
of compile-time checking for misuse. The structure is expected to remain
the same size and have the same alignment requirements on all known
platforms, compared to the array of unsigned char, although this is not
required for correctness.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

*config.txt: stick to camelCase naming convention

This should improve readability. Compare "thislongname" and
"thisLongName". The following keys are left in unchanged. We can
decide what to do with them later.

- am.keepcr
- core.autocrlf .safecrlf .trustctime
- diff.dirstat .noprefix
- gitcvs.usecrlfattr
- gui.blamehistoryctx .trustmtime
- pull.twohead
- receive.autogc
- sendemail.signedoffbycc .smtpsslcertpath .suppresscc

Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t5551: make EXPENSIVE test cheaper

We create 50,000 tags to check that we don't overflow the
command-line of fetch-pack. But by using run_with_cmdline_limit,
we can get the same effect with a much smaller number of
tags. This makes the test fast enough that we can drop the
EXPENSIVE prereq, which means people will actually run it.

It was not documented to do so, but this test was also the
only test of a clone-over-http that requires multiple POSTs
during the conversation. We can continue to test that by
dropping http.postbuffer to its minimum size, and checking
that we get two POSTs.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t5541: move run_with_cmdline_limit to test-lib.sh

We use this to test http pushing with a restricted
commandline. Other scripts (like t5551, which does http
fetching) will want to use it, too.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t: pass GIT_TRACE through Apache

Apache removes GIT_TRACE from the environment before running
git-http-backend. This can make it hard to debug the server
side of an http session. Let's let it through.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t: redirect stderr GIT_TRACE to descriptor 4

If you run a test script like:

GIT_TRACE=1 ./t0061-run-command.sh

you may get test failures, because some tests capture and
check the stderr output from git commands (and with
GIT_TRACE set to 1, the trace output will be included
there).

When we see GIT_TRACE set like this, we print a warning to
the user. However, we can do even better than that by just
pointing it to descriptor 4, which all tests leave connected
to the test script's stderr. That's likely what the user
intended (and any scripts that do want to see GIT_TRACE
output will set GIT_TRACE themselves).

Not only does this avoid false negatives in the tests, but
it means the user will actually see trace output for git
calls that redirect their stderr (whereas before, it was
sometimes confusingly buried in a file).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t: translate SIGINT to an exit

Right now if a test script receives SIGINT (e.g., because a
test was hanging and the user hit ^C), the shell exits
immediately. This can be annoying if the test script did any
global setup, like starting apache or git-daemon, as it will
not have an opportunity to clean up after itself. A
subsequent run of the test won't be able to start its own
daemon, and will either fail or skip the tests.

Instead, let's trap SIGINT to make sure we do a clean
shutdown, and just chain it to a normal exit (which will
trigger any cleanup).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

upload-pack: do not check NULL return of lookup_unknown_object

We check whether the return value of lookup_unknown_object
is NULL, but some code paths dereference it before our
check. This turns out not to be capable of causing a
segfault, though. The lookup_unknown_object function will
never return NULL, since the whole point is to allocate an
object struct if it does not find an existing one. So the
code here is not wrong, it is just confusing. Let's just
drop the NULL check.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

upload-pack: fix transfer.hiderefs over smart-http

When upload-pack advertises the refs (either for a normal,
non-stateless request, or for the initial contact in a
stateless one), we call for_each_ref with the send_ref
function as its callback. send_ref, in turn, calls
mark_our_ref, which checks whether the ref is hidden, and
sets OUR_REF or HIDDEN_REF on the object as appropriate. If
it is hidden, mark_our_ref also returns "1" to signal
send_ref that the ref should not be advertised.

If we are not advertising refs, (i.e., the follow-up
invocation by an http client to send its "want" lines), we
use mark_our_ref directly as a callback to for_each_ref. Its
marking does the right thing, but when it then returns "1"
to for_each_ref, the latter interprets this as an error and
stops iterating. As a result, we skip marking all of the
refs that come lexicographically after it. Any "want" lines
from the client asking for those objects will fail, as they
were not properly marked with OUR_REF.

To solve this, we introduce a wrapper callback around
mark_our_ref which always returns 0 (even if the ref is
hidden, we want to keep iterating). We also tweak the
signature of mark_our_ref to exclude unnecessary parameters
that were present only to conform to the callback interface.
This should make it less likely for somebody to accidentally
use it as a callback in the future.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

untracked cache: guard and disable on system changes

If the user enables untracked cache, then

- move worktree to an unsupported filesystem
- or simply upgrade OS
- or move the whole (portable) disk from one machine to another
- or access a shared fs from another machine

there's no guarantee that untracked cache can still function properly.
Record the worktree location and OS footprint in the cache. If it
changes, err on the safe side and disable the cache. The user can
'update-index --untracked-cache' again to make sure all conditions are
met.

This adds a new requirement that setup_git_directory* must be called
before read_cache() because we need worktree location by then, or the
cache is dropped.

This change does not cover all bases, you can fool it if you try
hard. The point is to stop accidents.

Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Helped-by: brian m. carlson <sandals@crustytoothpaste.net>
Helped-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

git-status.txt: advertisement for untracked cache

When a good user sees the "too long, consider -uno" advice when
running `git status`, they should check out the man page to find out
more. This change suggests they try untracked cache before -uno.

Helped-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

mingw32: add uname()

Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t7063: tests for untracked cache

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

update-index: test the system before enabling untracked cache

Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

update-index: manually enable or disable untracked cache

Overall time saving on "git status" is about 40% in the best case
scenario, removing ..collect_untracked() as the most time consuming
function. read and refresh index operations are now at the top (which
should drop when index-helper and/or watchman support is added). More
numbers and analysis below.

webkit.git
==========

169k files. 6k dirs. Lots of test data (i.e. not touched most of the
time)

Base status
-----------

Index version 4 in split index mode and cache-tree populated. No
untracked cache. It shows how time is consumed by "git status". The
same settings are used for other repos below.

18:28:10.199679 builtin/commit.c:1394 performance: 0.000000451 s: cmd_status:setup
18:28:10.474847 read-cache.c:1407 performance: 0.274873831 s: read_index
18:28:10.475295 read-cache.c:1407 performance: 0.000000656 s: read_index
18:28:10.728443 preload-index.c:131 performance: 0.253147487 s: read_index_preload
18:28:10.741422 read-cache.c:1254 performance: 0.012868340 s: refresh_index
18:28:10.752300 wt-status.c:623 performance: 0.010421357 s: wt_status_collect_changes_worktree
18:28:10.762069 wt-status.c:629 performance: 0.009644748 s: wt_status_collect_changes_index
18:28:11.601019 wt-status.c:632 performance: 0.838859547 s: wt_status_collect_untracked
18:28:11.605939 builtin/commit.c:1421 performance: 0.004835004 s: cmd_status:update_index
18:28:11.606580 trace.c:415 performance: 1.407878388 s: git command: 'git' 'status'

Populating status
-----------------

This is after enabling untracked cache and the cache is still empty.
We see a slight increase in .._collect_untracked() and update_index
(because new cache has to be written to $GIT_DIR/index).

18:28:18.915213 builtin/commit.c:1394 performance: 0.000000326 s: cmd_status:setup
18:28:19.197364 read-cache.c:1407 performance: 0.281901416 s: read_index
18:28:19.197754 read-cache.c:1407 performance: 0.000000546 s: read_index
18:28:19.451355 preload-index.c:131 performance: 0.253599607 s: read_index_preload
18:28:19.464400 read-cache.c:1254 performance: 0.012935336 s: refresh_index
18:28:19.475115 wt-status.c:623 performance: 0.010236920 s: wt_status_collect_changes_worktree
18:28:19.486022 wt-status.c:629 performance: 0.010801685 s: wt_status_collect_changes_index
18:28:20.362660 wt-status.c:632 performance: 0.876551366 s: wt_status_collect_untracked
18:28:20.396199 builtin/commit.c:1421 performance: 0.033447969 s: cmd_status:update_index
18:28:20.396939 trace.c:415 performance: 1.482695902 s: git command: 'git' 'status'

Populated status
----------------

After the cache is populated, wt_status_collect_untracked() drops 82%
from 0.838s to 0.144s. Overall time drops 45%. Top offenders are now
read_index() and read_index_preload().

18:28:20.408605 builtin/commit.c:1394 performance: 0.000000457 s: cmd_status:setup
18:28:20.692864 read-cache.c:1407 performance: 0.283980458 s: read_index
18:28:20.693273 read-cache.c:1407 performance: 0.000000661 s: read_index
18:28:20.958814 preload-index.c:131 performance: 0.265540254 s: read_index_preload
18:28:20.972375 read-cache.c:1254 performance: 0.013437429 s: refresh_index
18:28:20.983959 wt-status.c:623 performance: 0.011146646 s: wt_status_collect_changes_worktree
18:28:20.993948 wt-status.c:629 performance: 0.009879094 s: wt_status_collect_changes_index
18:28:21.138125 wt-status.c:632 performance: 0.144084737 s: wt_status_collect_untracked
18:28:21.173678 builtin/commit.c:1421 performance: 0.035463949 s: cmd_status:update_index
18:28:21.174251 trace.c:415 performance: 0.766707355 s: git command: 'git' 'status'

gentoo-x86.git
==============

This repository is a strange one with a balanced, wide and shallow
worktree (about 100k files and 23k dirs) and no .gitignore in
worktree. .._collect_untracked() time drops 88%, total time drops 56%.

Base status
-----------
18:20:40.828642 builtin/commit.c:1394 performance: 0.000000496 s: cmd_status:setup
18:20:41.027233 read-cache.c:1407 performance: 0.198130532 s: read_index
18:20:41.027670 read-cache.c:1407 performance: 0.000000581 s: read_index
18:20:41.171716 preload-index.c:131 performance: 0.144045594 s: read_index_preload
18:20:41.179171 read-cache.c:1254 performance: 0.007320424 s: refresh_index
18:20:41.185785 wt-status.c:623 performance: 0.006144638 s: wt_status_collect_changes_worktree
18:20:41.192701 wt-status.c:629 performance: 0.006780184 s: wt_status_collect_changes_index
18:20:41.991723 wt-status.c:632 performance: 0.798927029 s: wt_status_collect_untracked
18:20:41.994664 builtin/commit.c:1421 performance: 0.002852772 s: cmd_status:update_index
18:20:41.995458 trace.c:415 performance: 1.168427502 s: git command: 'git' 'status'
Populating status
-----------------
18:20:48.968848 builtin/commit.c:1394 performance: 0.000000380 s: cmd_status:setup
18:20:49.172918 read-cache.c:1407 performance: 0.203734214 s: read_index
18:20:49.173341 read-cache.c:1407 performance: 0.000000562 s: read_index
18:20:49.320013 preload-index.c:131 performance: 0.146671391 s: read_index_preload
18:20:49.328039 read-cache.c:1254 performance: 0.007921957 s: refresh_index
18:20:49.334680 wt-status.c:623 performance: 0.006172020 s: wt_status_collect_changes_worktree
18:20:49.342526 wt-status.c:629 performance: 0.007731746 s: wt_status_collect_changes_index
18:20:50.257510 wt-status.c:632 performance: 0.914864222 s: wt_status_collect_untracked
18:20:50.338371 builtin/commit.c:1421 performance: 0.080776477 s: cmd_status:update_index
18:20:50.338900 trace.c:415 performance: 1.371462446 s: git command: 'git' 'status'
Populated status
----------------
18:20:50.351160 builtin/commit.c:1394 performance: 0.000000571 s: cmd_status:setup
18:20:50.577358 read-cache.c:1407 performance: 0.225917338 s: read_index
18:20:50.577794 read-cache.c:1407 performance: 0.000000617 s: read_index
18:20:50.734140 preload-index.c:131 performance: 0.156345564 s: read_index_preload
18:20:50.745717 read-cache.c:1254 performance: 0.011463075 s: refresh_index
18:20:50.755176 wt-status.c:623 performance: 0.008877929 s: wt_status_collect_changes_worktree
18:20:50.763768 wt-status.c:629 performance: 0.008471633 s: wt_status_collect_changes_index
18:20:50.854885 wt-status.c:632 performance: 0.090988721 s: wt_status_collect_untracked
18:20:50.857765 builtin/commit.c:1421 performance: 0.002789097 s: cmd_status:update_index
18:20:50.858411 trace.c:415 performance: 0.508647673 s: git command: 'git' 'status'

linux-2.6
=========

Reference repo. Not too big. .._collect_status() drops 84%. Total time
drops 42%.

Base status
-----------
18:34:09.870122 builtin/commit.c:1394 performance: 0.000000385 s: cmd_status:setup
18:34:09.943218 read-cache.c:1407 performance: 0.072871177 s: read_index
18:34:09.943614 read-cache.c:1407 performance: 0.000000491 s: read_index
18:34:10.004364 preload-index.c:131 performance: 0.060748102 s: read_index_preload
18:34:10.008190 read-cache.c:1254 performance: 0.003714285 s: refresh_index
18:34:10.012087 wt-status.c:623 performance: 0.002775446 s: wt_status_collect_changes_worktree
18:34:10.016054 wt-status.c:629 performance: 0.003862140 s: wt_status_collect_changes_index
18:34:10.214747 wt-status.c:632 performance: 0.198604837 s: wt_status_collect_untracked
18:34:10.216102 builtin/commit.c:1421 performance: 0.001244166 s: cmd_status:update_index
18:34:10.216817 trace.c:415 performance: 0.347670735 s: git command: 'git' 'status'
Populating status
-----------------
18:34:16.595102 builtin/commit.c:1394 performance: 0.000000456 s: cmd_status:setup
18:34:16.666600 read-cache.c:1407 performance: 0.070992413 s: read_index
18:34:16.667012 read-cache.c:1407 performance: 0.000000606 s: read_index
18:34:16.729375 preload-index.c:131 performance: 0.062362492 s: read_index_preload
18:34:16.732565 read-cache.c:1254 performance: 0.003075517 s: refresh_index
18:34:16.736148 wt-status.c:623 performance: 0.002422201 s: wt_status_collect_changes_worktree
18:34:16.739990 wt-status.c:629 performance: 0.003746618 s: wt_status_collect_changes_index
18:34:16.948505 wt-status.c:632 performance: 0.208426710 s: wt_status_collect_untracked
18:34:16.961744 builtin/commit.c:1421 performance: 0.013151887 s: cmd_status:update_index
18:34:16.962233 trace.c:415 performance: 0.368537535 s: git command: 'git' 'status'
Populated status
----------------
18:34:16.970026 builtin/commit.c:1394 performance: 0.000000631 s: cmd_status:setup
18:34:17.046235 read-cache.c:1407 performance: 0.075904673 s: read_index
18:34:17.046644 read-cache.c:1407 performance: 0.000000681 s: read_index
18:34:17.113564 preload-index.c:131 performance: 0.066920253 s: read_index_preload
18:34:17.117281 read-cache.c:1254 performance: 0.003604055 s: refresh_index
18:34:17.121115 wt-status.c:623 performance: 0.002508345 s: wt_status_collect_changes_worktree
18:34:17.125089 wt-status.c:629 performance: 0.003871636 s: wt_status_collect_changes_index
18:34:17.156089 wt-status.c:632 performance: 0.030895703 s: wt_status_collect_untracked
18:34:17.169861 builtin/commit.c:1421 performance: 0.013686404 s: cmd_status:update_index
18:34:17.170391 trace.c:415 performance: 0.201474531 s: git command: 'git' 'status'

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

status: enable untracked cache

update_index_if_able() is moved down so that the updated untracked
cache could be written out.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

untracked-cache: temporarily disable with $GIT_DISABLE_UNTRACKED_CACHE

This can be used to double check if results with untracked cache are
correctly, compared to vanilla version. Untracked cache remains in
index, but not used.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

untracked cache: mark index dirty if untracked cache is updated

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

untracked cache: print stats with $GIT_TRACE_UNTRACKED_STATS

This could be used to verify correct behavior in tests

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

untracked cache: avoid racy timestamps

When a directory is updated within the same second that its timestamp
is last saved, we cannot realize the directory has been updated by
checking timestamps. Assume the worst (something is update). See
29e4d36 (Racy GIT - 2005-12-20) for more information.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

read-cache.c: split racy stat test to a separate function

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>