fast-import: fix read of uninitialized argv memory

Fast-import shares code between its command-line parser and
the "option" command. To do so, it strips the "--" from any
command-line options and passes them to the option parser.
However, it does not confirm that the option even begins
with "--" before blindly passing "arg + 2".

It does confirm that the option starts with "-", so the only
affected case was:

git fast-import -

which would read uninitialized memory after the argument. We
can fix it by using skip_prefix and checking the result. As
a bonus, this gets rid of some magic numbers.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

apply: use skip_prefix instead of raw addition

A submodule diff generally has content like:

-Subproject commit [0-9a-f]{40}
+Subproject commit [0-9a-f]{40}

When we are using "git apply --index" with a submodule, we
first apply the textual diff, and then parse that result to
figure out the new sha1.

If the diff has bogus input like:

-Subproject commit 1234567890123456789012345678901234567890
+bogus

we will parse the "bogus" portion. Our parser assumes that
the buffer starts with "Subproject commit", and blindly
skips past it using strlen(). This can cause us to read
random memory after the buffer.

This problem was unlikely to have come up in practice (since
it requires a malformed diff), and even when it did, we
likely noticed the problem anyway as the next operation was
to call get_sha1_hex on the random memory.

However, we can easily fix it by using skip_prefix to notice
the parsing error.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

refactor skip_prefix to return a boolean

The skip_prefix() function returns a pointer to the content
past the prefix, or NULL if the prefix was not found. While
this is nice and simple, in practice it makes it hard to use
for two reasons:

1. When you want to conditionally skip or keep the string
as-is, you have to introduce a temporary variable.
For example:

tmp = skip_prefix(buf, "foo");
if (tmp)
buf = tmp;

2. It is verbose to check the outcome in a conditional, as
you need extra parentheses to silence compiler
warnings. For example:

if ((cp = skip_prefix(buf, "foo"))
/* do something with cp */

Both of these make it harder to use for long if-chains, and
we tend to use starts_with() instead. However, the first line
of "do something" is often to then skip forward in buf past
the prefix, either using a magic constant or with an extra
strlen(3) (which is generally computed at compile time, but
means we are repeating ourselves).

This patch refactors skip_prefix() to return a simple boolean,
and to provide the pointer value as an out-parameter. If the
prefix is not found, the out-parameter is untouched. This
lets you write:

if (skip_prefix(arg, "foo ", &arg))
do_foo(arg);
else if (skip_prefix(arg, "bar ", &arg))
do_bar(arg);

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

cleanup duplicate name_compare() functions

We often represent our strings as a counted string, i.e. a pair of
the pointer to the beginning of the string and its length, and the
string may not be NUL terminated to that length.

To compare a pair of such counted strings, unpack-trees.c and
read-cache.c implement their own name_compare() functions
identically. In addition, the cache_name_compare() function in
read-cache.c is nearly identical. The only difference is when one
string is the prefix of the other string, in which case
name_compare() returns -1/+1 to show which one is longer, and
cache_name_compare() returns the difference of the lengths to show
the same information.

Unify these three functions by using the implementation from
cache_name_compare(). This does not make any difference to the
existing and future callers, as they must be paying attention only
to the sign of the returned value (and not the magnitude) because
the original implementations of these two functions return values
returned by memcmp(3) when the one string is not a prefix of the
other string, and the only thing memcmp(3) guarantees its callers is
the sign of the returned value, not the magnitude.

Signed-off-by: Jeremiah Mahler <jmmahler@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

name-hash.c: replace cache_name_compare() with memcmp(3)

The same_name() private function wants a quick-and-exact check to
see if they two names are byte-for-byte identical first and then
fall back to the slow path. Use memcmp(3) for the former to make it
clear that we do not want any "name" specific comparison.

Signed-off-by: Jeremiah Mahler <jmmahler@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

unique_path: fix unlikely heap overflow

When merge-recursive creates a unique filename, it uses a
template like:

path~branch_%d

where the final "_%d" is filled by an incrementing counter
until we find a unique name. We allocate 8 characters for
the counter, but there is no logic to limit the size of the
integer.

Of course, this is extremely unlikely, as you would need a
hundred million collisions to trigger the problem. Even if
an attacker constructed a specialized repo, it is unlikely
that the victim would have the patience to run the merge.

However, we can make it trivially correct (and hopefully
more readable) by using a strbuf.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

walker_fetch: fix minor memory leak

We sometimes allocate "msg" on the heap, but will fail to
free it if we hit the failure code path. We can instead keep
a separate variable that is safe to be freed no matter how
we get to the failure code path.

While we're here, we can also do two readability
improvements:

1. Use xstrfmt instead of a manual malloc/sprintf

2. Due to the "maybe we allocate msg, maybe we don't"
strategy, the logic for deciding which message to show
was split into two parts. Since the deallocation is now
pushed onto a separate variable, this is no longer a
concern, and we can keep all of the logic in the same
place.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

merge: use argv_array when spawning merge strategy

This is shorter, and avoids a rather complicated set of
allocation and free steps.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

sequencer: use argv_array_pushf

This avoids a manual allocation calculation, and is shorter
to boot.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

setup_git_env: use git_pathdup instead of xmalloc + sprintf

This is shorter, harder to get wrong, and more clearly
captures the intent.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

use xstrfmt to replace xmalloc + strcpy/strcat

It's easy to get manual allocation calculations wrong, and
the use of strcpy/strcat raise red flags for people looking
for buffer overflows (though in this case each site was
fine).

It's also shorter to use xstrfmt, and the printf-format
tends to be easier for a reader to see what the final string
will look like.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

use xstrfmt to replace xmalloc + sprintf

This is one line shorter, and makes sure the length in the
malloc and sprintf steps match.

These conversions are very straightforward; we can drop the
malloc entirely, and replace the sprintf with xstrfmt.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

use xstrdup instead of xmalloc + strcpy

This is one line shorter, and makes sure the length in the
malloc and copy steps match.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

git-submodule.sh: avoid "echo" path-like values

SysV-derived implementation of "echo" interprets some backslash
sequences as special instruction, e.g. "echo 'ab\c'" shows an
incomplete line with 'a' and 'b' on it. Avoid using it when showing
a path-like values in the script.

Signed-off-by: Junio C Hamano <gitster@pobox.com>

git-submodule.sh: avoid "test <cond> -a/-o <cond>"

The construct is error-prone; "test" being built-in in most modern
shells, the reason to avoid "test <cond> && test <cond>" spawning
one extra process by using a single "test <cond> -a <cond>" no
longer exists.

Signed-off-by: Elia Pinto <gitter.spiros@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

use xstrfmt in favor of manual size calculations

In many parts of the code, we do an ugly and error-prone
malloc like:

const char *fmt = "something %s";
buf = xmalloc(strlen(foo) + 10 + 1);
sprintf(buf, fmt, foo);

This makes the code brittle, and if we ever get the
allocation wrong, is a potential heap overflow. Let's
instead favor xstrfmt, which handles the allocation
automatically, and makes the code shorter and more readable.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

strbuf: add xstrfmt helper

You can use a strbuf to build up a string from parts, and
then detach it. In the general case, you might use multiple
strbuf_add* functions to do the building. However, in many
cases, a single strbuf_addf is sufficient, and we end up
with:

struct strbuf buf = STRBUF_INIT;
...
strbuf_addf(&buf, fmt, some, args);
str = strbuf_detach(&buf, NULL);

We can make this much more readable (and avoid introducing
an extra variable, which can clutter the code) by
introducing a convenience function:

str = xstrfmt(fmt, some, args);

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

avoid using skip_prefix as a boolean

There's no point in using:

if (skip_prefix(buf, "foo"))

over

if (starts_with(buf, "foo"))

as the point of skip_prefix is to return a pointer to the
data after the prefix. Using starts_with is more readable,
and will make refactoring skip_prefix easier.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

daemon: mark some strings as const

None of these strings is modified; marking them as const
will help later refactoring.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

parse_diff_color_slot: drop ofs parameter

This function originally took a whole config variable name
("var") and an offset ("ofs"). It checked "var+ofs" against
each color slot, but reported errors using the whole "var".

However, since 8b8e862 (ignore unknown color configuration,
2009-12-12), it returns -1 rather than printing its own
error, and therefore only cares about var+ofs. We can drop
the ofs parameter and teach its sole caller to derive the
pointer itself.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

refs.c: SSE2 optimizations for check_refname_component

Optimize check_refname_component using SSE2 on x86_64.

git rev-parse HEAD is a good test-case for this, since it does almost
nothing except parse refs. For one particular repo with about 60k
refs, almost all packed, the timings are:

Look up table: 29 ms
SSE2: 23 ms

This cuts about 20% off of the runtime.

Ondřej Bílka <neleai@seznam.cz> suggested an SSE2 approach to the
substring searches, which netted a speed boost over the SSE4.2 code I
had initially written.

Signed-off-by: David Turner <dturner@twitter.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Update of unicode_width.h to Unicode Version 7.0

Unicode Version 7.0 was released yesterday.
Run ./update_unicode.sh to update the zero_width table.
Note: the double_width is unchanged.

Signed-off-by: Torsten Bögershausen <tboegi@web.de>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

http: fix charset detection of extract_content_type()

extract_content_type() could not extract a charset parameter if the
parameter is not the first one and there is a whitespace and a following
semicolon just before the parameter. For example:

text/plain; format=fixed ;charset=utf-8

And it also could not handle correctly some other cases, such as:

text/plain; charset=utf-8; format=fixed
text/plain; some-param="a long value with ;semicolons;"; charset=utf-8

Thanks-to: Jeff King <peff@peff.net>
Signed-off-by: Yi EungJun <eungjun.yi@navercorp.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

pretty: avoid reading past end-of-string with "%G"

If the user asks for --format=%G with nothing else, we
correctly realize that "%G" is not a valid placeholder (it
should be "%G?", "%GK", etc). But we still tell the
strbuf_expand code that we consumed 2 characters, causing it
to jump over the trailing NUL and output garbage.

This also fixes the case where "%GX" would be consumed (and
produce no output). In other cases, we pass unrecognized
placeholders through to the final string.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t7510: check %G* pretty-format output

We do not check these along with the other pretty-format
placeholders in t6006, because we need signed commits to
make them interesting. t7510 has such commits, and can
easily exercise them in addition to the regular
--show-signature code path.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t7510: test a commit signed by an unknown key

We tested both good and bad signatures, but not ones made
correctly but with a key for which we have no trust.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t7510: use consistent &&-chains in loop

We check multiple commits in a loop. Because we want to
break out of the loop if any single iteration fails, we use
a subshell/exit like:

(
for i in $stuff
do
do-something $i || exit 1
done
)

However, we are inconsistent in our loop body. Some commands
get their own "|| exit 1", and others try to chain to the
next command with "&&", like:

X &&
Y || exit 1
Z || exit 1

This is a little hard to read and follow, because X and Y
are treated differently for no good reason. But much worse,
the second loop follows a similar pattern and gets it wrong.
"Y" is expected to fail, so we use "&& exit 1", giving us:

X &&
Y && exit 1
Z || exit 1

That gets the test for X wrong (we do not exit unless both X
fails and Y unexpectedly succeeds, but we would want to exit
if _either_ is wrong). We can write this clearly and
correctly by consistently using "&&", followed by a single
"|| exit 1", and negating Y with "!" (as we would in a
normal &&-chain). Like:

X &&
! Y &&
Z || exit 1

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t7510: stop referring to master in later tests

Our setup creates a sequence of commits, each with its own
tag. However, we sometimes refer to "seventh-signed" as
"master". This works, since it is at the tip of the created
branch, but is brittle if new tests need to add more
commits. Let's use its tag name to be unambiguous.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

rebase--merge: fix --skip with two conflicts in a row

If git rebase --merge encountered a conflict, --skip would not work if the
next commit also conflicted. The msgnum file would never be updated with
the new patch number, so no patch would actually be skipped, resulting in an
inescapable loop.

Update the msgnum file's value as the first thing in call_merge. This also
avoids an "Already applied" message when skipping a commit. There is no
visible change for the other contexts in which call_merge is invoked, as the
msgnum file's value remains unchanged in those situations.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Third batch for 2.1

Signed-off-by: Junio C Hamano <gitster@pobox.com>

Merge branch 'ib/test-selectively-run'

Allow specifying only certain individual test pieces to be run
using a range notation (e.g. "t1234-test.sh --run='1-4 6 8 9-'").

* ib/test-selectively-run:
t0000-*.sh: fix the GIT_SKIP_TESTS sub-tests
test-lib: '--run' to run only specific tests
test-lib: tests skipped by GIT_SKIP_TESTS say so
test-lib: document short options in t/README

Merge branch 'ta/string-list-init'

* ta/string-list-init:
string-list: spell all values out that are given to a string_list initializer

Merge branch 'jm/dedup-test-config'

* jm/dedup-test-config:
t/t7810-grep.sh: remove duplicate test_config()

Merge branch 'dt/refs-check-refname-component-optim'

* dt/refs-check-refname-component-optim:
refs.c: optimize check_refname_component()

Merge branch 'sk/test-cmp-bin'

* sk/test-cmp-bin:
t5000, t5003: do not use test_cmp to compare binary files

Merge branch 'sh/enable-preloadindex'

* sh/enable-preloadindex:
environment.c: enable core.preloadindex by default

Merge branch 'rs/read-ref-at'

* rs/read-ref-at:
refs.c: change read_ref_at to use the reflog iterators

Merge branch 'jk/error-resolve-conflict-advice'

* jk/error-resolve-conflict-advice:
error_resolve_conflict: drop quotations around operation
error_resolve_conflict: rewrap advice message

Merge branch 'rs/pack-objects-no-unnecessary-realloc'

Avoid unnecessary copy of previous contents when extending the
hashtable used in pack-objects.

* rs/pack-objects-no-unnecessary-realloc:
pack-objects: use free()+xcalloc() instead of xrealloc()+memset()

Merge branch 'lt/log-auto-decorate'

* lt/log-auto-decorate:
git log: support "auto" decorations

Merge branch 'jm/doc-wording-tweaks'

* jm/doc-wording-tweaks:
Documentation: wording fixes in the user manual and glossary

Merge branch 'jm/format-patch-mail-sig'

* jm/format-patch-mail-sig:
format-patch: add "--signature-file=<file>" option
format-patch: make newline after signature conditional

Merge branch 'jk/http-errors'

Propagate the error messages from the webserver better to the
client coming over the HTTP transport.

* jk/http-errors:
http: default text charset to iso-8859-1
remote-curl: reencode http error messages
strbuf: add strbuf_reencode helper
http: optionally extract charset parameter from content-type
http: extract type/subtype portion of content-type
t5550: test display of remote http error messages
t/lib-httpd: use write_script to copy CGI scripts
test-lib: preserve GIT_CURL_VERBOSE from the environment

Merge branch 'ow/config-mailmap-pathname'

mailmap.file configuration names a pathname, hence should honor
~/path and ~user/path as its value.

* ow/config-mailmap-pathname:
config: respect '~' and '~user' in mailmap.file

Merge branch 'fc/remote-helper-refmap'

Allow remote-helper/fast-import based transport to rename the refs
while transferring the history.

* fc/remote-helper-refmap:
transport-helper: remove unnecessary strbuf resets
transport-helper: add support to delete branches
fast-export: add support to delete refs
fast-import: add support to delete refs
transport-helper: add support to push symbolic refs
transport-helper: add support for old:new refspec
fast-export: add new --refspec option
fast-export: improve argument parsing

Merge branch 'nd/daemonize-gc'

"git gc --auto" was recently changed to run in the background to
give control back early to the end-user sitting in front of the
terminal, but it forgot that housekeeping involving reflogs should
be done without other processes competing for accesses to the refs.

* nd/daemonize-gc:
gc --auto: do not lock refs in the background

Merge branch 'jm/t9138-style-fix'

* jm/t9138-style-fix:
t9138-git-svn-authors-prog.sh fixups

Merge branch 'jm/instaweb-apache-24'

* jm/instaweb-apache-24:
git-instaweb: add support for Apache 2.4

Merge branch 'jl/remote-rm-prune'

"git remote rm" and "git remote prune" can involve removing many
refs at once, which is not a very efficient thing to do when very
many refs exist in the packed-refs file.

* jl/remote-rm-prune:
remote prune: optimize "dangling symref" check/warning
remote: repack packed-refs once when deleting multiple refs
remote rm: delete remote configuration as the last

Merge branch 'jk/complete-merge-pull'

The completion code did not know about quite a few options that are
common between "git merge" and "git pull", and a couple of options
unique to "git merge".

* jk/complete-merge-pull:
completion: add missing options for git-merge
completion: add a note that merge options are shared

Merge branch 'bg/xcalloc-nmemb-then-size'

Like calloc(3), xcalloc() takes nmemb and then size.

* bg/xcalloc-nmemb-then-size:
transport-helper.c: rearrange xcalloc arguments
remote.c: rearrange xcalloc arguments
reflog-walk.c: rearrange xcalloc arguments
pack-revindex.c: rearrange xcalloc arguments
notes.c: rearrange xcalloc arguments
imap-send.c: rearrange xcalloc arguments
http-push.c: rearrange xcalloc arguments
diff.c: rearrange xcalloc arguments
config.c: rearrange xcalloc arguments
commit.c: rearrange xcalloc arguments
builtin/remote.c: rearrange xcalloc arguments
builtin/ls-remote.c: rearrange xcalloc arguments

http-protocol.txt: Basic Auth is defined in RFC 2617, not RFC 2616

Signed-off-by: Yi EungJun <eungjun.yi@navercorp.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Win32: Unicode arguments (incoming)

Convert command line arguments from UTF-16 to UTF-8 on startup.

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Stepan Kasal <kasal@ucw.cz>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Win32: Unicode arguments (outgoing)

Convert command line arguments from UTF-8 to UTF-16 when creating other
processes.

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Stepan Kasal <kasal@ucw.cz>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

MinGW: disable CRT command line globbing

MingwRT listens to _CRT_glob to decide if __getmainargs should
perform globbing, with the default being that it should.
Unfortunately, __getmainargs globbing is sub-par; for instance
patterns like "*.c" will only match c-sources in the current
directory.

Disable __getmainargs' command line wildcard expansion, so these
patterns will be left untouched, and handled by Git's superior
built-in globbing instead.

MSVC defaults to no globbing, so we don't need to do anything
in that case.

This fixes t5505 and t7810.

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com>
Signed-off-by: Stepan Kasal <kasal@ucw.cz>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Win32: fix potential multi-threading issue

...by removing a static buffer in do_stat_internal.

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com>
Signed-off-by: Stepan Kasal <kasal@ucw.cz>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Win32: simplify internal mingw_spawn* APIs

The only public spawn function that needs to tweak the environment is
mingw_spawnvpe (called from start_command). Nevertheless, all internal
spawn* functions take an env parameter and needlessly pass the global
char **environ around. Remove the env parameter where it's not needed.

This removes the internal mingw_execve abstraction, which is no longer
needed.

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Stepan Kasal <kasal@ucw.cz>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Win32: let mingw_execve() return an int

This is in the great tradition of POSIX. Original fix by Olivier Refalo.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Stepan Kasal <kasal@ucw.cz>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Win32: reliably detect console pipe handles

As of "Win32: Thread-safe windows console output", child processes may
print to the console even if stdout has been redirected to a file. E.g.:

git config tar.cat.command "cat"
git archive -o test.cat HEAD

Detecting whether stdout / stderr point to our console pipe is currently
based on the assumption that OS HANDLE values are never reused. This is
apparently not true if stdout / stderr is replaced via dup2() (as in
builtin/archive.c:17).

Instead of comparing handle values, check if the file descriptor isatty()
backed by a pipe OS handle. This is only possible by swapping the handles
in MSVCRT's internal data structures, as we do in winansi_init().

Reported-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Merge branch 'jl/status-added-submodule-is-never-ignored'

submodule.*.ignore and diff.ignoresubmodules are used to ignore all
submodule changes in "diff" output, but it can be confusing to
apply these configuration values to status and commit.

This is a backward-incompatible change, but should be so in a good
way (aka bugfix).

* jl/status-added-submodule-is-never-ignored:
commit -m: commit staged submodules regardless of ignore config
status/commit: show staged submodules regardless of ignore config

Merge branch 'cb/byte-order'

Compatibility enhancement for Solaris.

* cb/byte-order:
compat/bswap.h: fix endianness detection
compat/bswap.h: restore preference __BIG_ENDIAN over BIG_ENDIAN
compat/bswap.h: detect endianness on more platforms that don't use BYTE_ORDER

Merge branch 'jk/strbuf-tolower'

* jk/strbuf-tolower:
strbuf: add strbuf_tolower function

Merge branch 'jk/daemon-tolower'

* jk/daemon-tolower:
daemon/config: factor out duplicate xstrdup_tolower

Merge branch 'as/pretty-truncate'

* as/pretty-truncate:
pretty.c: format string with truncate respects logOutputEncoding
t4205, t6006: add tests that fail with i18n.logOutputEncoding set
t4205 (log-pretty-format): use `tformat` rather than `format`
t4041, t4205, t6006, t7102: don't hardcode tested encoding value
t4205 (log-pretty-formats): don't hardcode SHA-1 in expected outputs

Merge branch 'jk/diff-follow-must-take-one-pathspec'

* jk/diff-follow-must-take-one-pathspec:
move "--follow needs one pathspec" rule to diff_setup_done

Merge branch 'sk/windows-unc-path'

* sk/windows-unc-path:
Windows: allow using UNC path for git repository

Merge branch 'rr/rebase-autostash-fix'

* rr/rebase-autostash-fix:
rebase -i: test "Nothing to do" case with autostash
rebase -i: handle "Nothing to do" case with autostash

Merge branch 'jk/report-fail-to-read-objects-better'

* jk/report-fail-to-read-objects-better:
open_sha1_file: report "most interesting" errno

Merge branch 'jk/diff-files-assume-unchanged'

* jk/diff-files-assume-unchanged:
run_diff_files: do not look at uninitialized stat data

Merge branch 'jk/argv-array-for-child-process'

* jk/argv-array-for-child-process:
argv-array: drop "detach" code
get_importer: use run-command's internal argv_array
get_exporter: use argv_array
get_helper: use run-command's internal argv_array
git_connect: use argv_array
run_column_filter: use argv_array
run-command: store an optional argv_array

Merge branch 'sk/wincred'

* sk/wincred:
wincred: avoid overwriting configured variables
wincred: add install target

Merge branch 'jk/do-not-run-httpd-tests-as-root'

* jk/do-not-run-httpd-tests-as-root:
t/lib-httpd: require SANITY prereq

Merge branch 'cc/replace-edit'

"git replace" learns a new "--edit" option.

* cc/replace-edit:
Documentation: replace: describe new --edit option
replace: add --edit to usage string
replace: add tests for --edit
replace: die early if replace ref already exists
replace: refactor checking ref validity
replace: make sure --edit results in a different object
replace: add --edit option
replace: factor object resolution out of replace_object
replace: use OPT_CMDMODE to handle modes
replace: refactor command-mode determination

Merge branch 'mt/patch-id-stable' (early part)

* 'mt/patch-id-stable' (early part):
patch-id-test: test stable and unstable behaviour
patch-id: make it stable against hunk reordering
test doc: test_write_lines does not split its arguments
test: add test_write_lines helper

gitk: Use mktemp -d to avoid predictable temporary directories

gitk uses a predictable ".gitk-tmp.$PID" pattern when generating
a temporary directory.

Use "mktemp -d .gitk-tmp.XXXXXX" to harden gitk against someone
seeding /tmp with files matching the pid pattern.

Signed-off-by: David Aguilar <davvid@gmail.com>
Signed-off-by: Paul Mackerras <paulus@samba.org>

gitk: Show staged submodules regardless of ignore config

Currently setting submodule.<name>.ignore and/or diff.ignoreSubmodules to
"all" suppresses all output of submodule changes for gitk. This is really
confusing, as even when the user chooses to record a new commit for an
ignored submodule by adding it manually this change won't show up under
"Local changes checked in to index but not committed".

Fix that by using the '--ignore-submodules=dirty' option for both callers
of "git diff-index --cached" when the underlying git version supports that
option.

Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Paul Mackerras <paulus@samba.org>

gitk: Honor TMPDIR when viewing external diffs

gitk fails to show diffs when browsing a read-only repository.
This is due to gitk's assumption that the current directory is always
writable.

Teach gitk to honor either the GITK_TMPDIR or TMPDIR environment
variables. This allows users to override the default location
used when writing temporary files.

Signed-off-by: David Aguilar <davvid@gmail.com>
Signed-off-by: Paul Mackerras <paulus@samba.org>

gitk: Allow displaying time zones from author and commit dates timestamps

Now gitk can be configured to display author and commit dates in their
original timezone, by putting %z into datetimeformat in ~/.gitk.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
Signed-off-by: Paul Mackerras <paulus@samba.org>

gitk: Switch to patch mode when searching for line origin

If the "Show origin of this line" is started from tree mode,
it still shows the result in tree mode, which I suppose not
what user expects to see.

Signed-off-by: Paul Mackerras <paulus@samba.org>

gitk: Replace SHA1 entry field on keyboard paste

We already replace old SHA with the clipboard content for the mouse
paste event. It seems reasonable to do the same when pasting from
keyboard.

Signed-off-by: Ilya Bobyr <ilya.bobyr@gmail.com>
Signed-off-by: Paul Mackerras <paulus@samba.org>

submodule: document "sync --recursive"

The "git submodule sync" command supports the --recursive flag, but
the documentation does not mention this. That flag is useful, for
example when a remote is changed in a submodule of a submodule.

Signed-off-by: Matthew Chen <charlesmchen@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

blame: simplify prepare_lines()

Changing get_next_line() to return the end pointer instead of NULL in
case no newline character is found treats allows us to treat complete
and incomplete lines the same, simplifying the code. Switching to
counting lines instead of EOLs allows us to start counting at the
first character, instead of having to call get_next_line() first.

Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

blame: factor out get_next_line()

Move the code for finding the start of the next line into a helper
function in order to reduce duplication.

Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

completion: handle '!f() { ... }; f' and "!sh -c '...' -" aliases

'!f() { ... }; f' and "!sh -c '....' -" are recommended patterns for
declaring more complex aliases (see git wiki [1]). This commit teaches
the completion to handle them.

When determining which completion to use for an alias, an opening brace
or single quote is now skipped, and the search for a git command is
continued. For example, the aliases '!f() { git commit ... }' or "!sh
-c 'git commit ...'" now trigger commit completion. Previously, the
search stopped on the opening brace or quote, and the completion tried
it to determine how to complete, which obviously was useless.

The null command ':' is now skipped, so that it can be used as
a workaround to declare the desired completion style.

For example, the aliases

!f() { : git commit ; if ... } f
!sh -c ': git commit; if ...' -

now trigger commit completion.

Shell function declarations now work with or without space before
the parens, i.e. '!f() ...' and '!f () ...' both work.

[1] https://git.wiki.kernel.org/index.php/Aliases

Signed-off-by: Steffen Prohaska <prohaska@zib.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t0008: do not depend on 'echo' handling backslashes specially

The original used to pass with /bin/dash but not with /bin/bash set
to $SHELL_PATH. The former turns "\\" into "\", but the latter does
not.

Signed-off-by: Junio C Hamano <gitster@pobox.com>

reuse cached commit buffer when parsing signatures

When we call show_signature or show_mergetag, we read the
commit object fresh via read_sha1_file and reparse its
headers. However, in most cases we already have the object
data available, attached to the "struct commit". This is
partially laziness in dealing with the memory allocation
issues, but partially defensive programming, in that we
would always want to verify a clean version of the buffer
(not one that might have been munged by other users of the
commit).

However, we do not currently ever munge the commit buffer,
and not using the already-available buffer carries a fairly
big performance penalty when we are looking at a large
number of commits. Here are timings on linux.git:

[baseline, no signatures]
$ time git log >/dev/null
real 0m4.902s
user 0m4.784s
sys 0m0.120s

[before]
$ time git log --show-signature >/dev/null
real 0m14.735s
user 0m9.964s
sys 0m0.944s

[after]
$ time git log --show-signature >/dev/null
real 0m9.981s
user 0m5.260s
sys 0m0.936s

Note that our user CPU time drops almost in half, close to
the non-signature case, but we do still spend more
wall-clock and system time, presumably from dealing with
gpg.

An alternative to this is to note that most commits do not
have signatures (less than 1% in this repo), yet we pay the
re-parsing cost for every commit just to find out if it has
a mergetag or signature. If we checked that when parsing the
commit initially, we could avoid re-examining most commits
later on. Even if we did pursue that direction, however,
this would still speed up the cases where we _do_ have
signatures. So it's probably worth doing either way.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

commit: record buffer length in cache

Most callsites which use the commit buffer try to use the
cached version attached to the commit, rather than
re-reading from disk. Unfortunately, that interface provides
only a pointer to the NUL-terminated buffer, with no
indication of the original length.

For the most part, this doesn't matter. People do not put
NULs in their commit messages, and the log code is happy to
treat it all as a NUL-terminated string. However, some code
paths do care. For example, when checking signatures, we
want to be very careful that we verify all the bytes to
avoid malicious trickery.

This patch just adds an optional "size" out-pointer to
get_commit_buffer and friends. The existing callers all pass
NULL (there did not seem to be any obvious sites where we
could avoid an immediate strlen() call, though perhaps with
some further refactoring we could).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

commit: convert commit->buffer to a slab

This will make it easier to manage the buffer cache
independently of the "struct commit" objects. It also
shrinks "struct commit" by one pointer, which may be
helpful.

Unfortunately it does not reduce the max memory size of
something like "rev-list", because rev-list uses
get_cached_commit_buffer() to decide not to show each
commit's output (and due to the design of slab_at, accessing
the slab requires us to extend it, allocating exactly the
same number of buffer pointers we dropped from the commit
structs).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

commit-slab: provide a static initializer

Callers currently must use init_foo_slab() at runtime before
accessing a slab. For global slabs, it's much nicer if we
can initialize them in BSS, so that each user does not have
to add code to check-and-initialize.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

use get_commit_buffer everywhere

Each of these sites assumes that commit->buffer is valid.
Since they would segfault if this was not the case, they are
likely to be correct in practice. However, we can
future-proof them by using get_commit_buffer.

And as a side effect, we abstract away the final bare uses
of commit->buffer.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

convert logmsg_reencode to get_commit_buffer

Like the callsites in the previous commit, logmsg_reencode
already falls back to read_sha1_file when necessary.
However, I split its conversion out into its own commit
because it's a bit more complex.

We return either:

1. The original commit->buffer

2. A newly allocated buffer from read_sha1_file

3. A reencoded buffer (based on either 1 or 2 above).

while trying to do as few extra reads/allocations as
possible. Callers currently free the result with
logmsg_free, but we can simplify this by pointing them
straight to unuse_commit_buffer. This is a slight layering
violation, in that we may be passing a buffer from (3).
However, since the end result is to free() anything except
(1), which is unlikely to change, and because this makes the
interface much simpler, it's a reasonable bending of the
rules.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

use get_commit_buffer to avoid duplicate code

For both of these sites, we already do the "fallback to
read_sha1_file" trick. But we can shorten the code by just
using get_commit_buffer.

Note that the error cases are slightly different when
read_sha1_file fails. get_commit_buffer will die() if the
object cannot be loaded, or is a non-commit.

For get_sha1_oneline, this will almost certainly never
happen, as we will have just called parse_object (and if it
does, it's probably worth complaining about).

For record_author_date, the new behavior is probably better;
we notify the user of the error instead of silently ignoring
it. And because it's used only for sorting by author-date,
somebody examining a corrupt repo can fallback to the
regular traversal order.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

use get_cached_commit_buffer where appropriate

Some call sites check commit->buffer to see whether we have
a cached buffer, and if so, do some work with it. In the
long run we may want to switch these code paths to make
their decision on a different boolean flag (because checking
the cache may get a little more expensive in the future).
But for now, we can easily support them by converting the
calls to use get_cached_commit_buffer.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

provide helpers to access the commit buffer

Many sites look at commit->buffer to get more detailed
information than what is in the parsed commit struct.
However, we sometimes drop commit->buffer to save memory,
in which case the caller would need to read the object
afresh. Some callers do this (leading to duplicated code),
and others do not (which opens the possibility of a segfault
if somebody else frees the buffer).

Let's provide a pair of helpers, "get" and "unuse", that let
callers easily get the buffer. They will use the cached
buffer when possible, and otherwise load from disk using
read_sha1_file.

Note that we also need to add a "get_cached" variant which
returns NULL when we do not have a cached buffer. At first
glance this seems to defeat the purpose of "get", which is
to always provide a return value. However, some log code
paths actually use the NULL-ness of commit->buffer as a
boolean flag to decide whether to try printing the
commit. At least for now, we want to continue supporting
that use.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

provide a helper to set the commit buffer

Right now this is just a one-liner, but abstracting it will
make it easier to change later.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

provide a helper to free commit buffer

This converts two lines into one at each caller. But more
importantly, it abstracts the concept of freeing the buffer,
which will make it easier to change later.

Note that we also need to provide a "detach" mechanism for a
tricky case in index-pack. We are passed a buffer for the
object generated by processing the incoming pack. If we are
not using --strict, we just calculate the sha1 on that
buffer and return, leaving the caller to free it. But if we
are using --strict, we actually attach that buffer to an
object, pass the object to the fsck functions, and then
detach the buffer from the object again (so that the caller
can free it as usual). In this case, we don't want to free
the buffer ourselves, but just make sure it is no longer
associated with the commit.

Note that we are making the assumption here that the
attach/detach process does not impact the buffer at all
(e.g., it is never reallocated or modified). That holds true
now, and we have no plans to change that. However, as we
abstract the commit_buffer code, this dependency becomes
less obvious. So when we detach, let's also make sure that
we get back the same buffer that we gave to the
commit_buffer code.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

git-p4: fix submit in non --prepare-p4-only mode

b4073bb3 (git-p4: Do not include diff in spec file when just
preparing p4, 2014-05-24) broke git p4 submit, here is a proper
fix, including proper handling for windows end of lines.

Signed-off-by: Maxime Coste <frrrwww@gmail.com>
Acked-by: Pete Wyckoff <pw@padd.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

git-gui: tolerate major version changes when comparing the git version

Since git 2.0.0 starting git gui in a submodule using a gitfile fails with
the following error:

No working directory ../../../<path>

couldn't change working directory
to "../../../<path>": no such file or
directory

This is because "git rev-parse --show-toplevel" is only run when git gui
sees a git version of at least 1.7.0 (which is the version in which the
--show-toplevel option was introduced). But "package vsatisfies" returns
false when the major version changes, which is not what we want here.

Fix that for both places where the git version is checked using vsatisfies
by appending a '-' to the version number. This tells vsatisfies that a
change of the major version is not considered to be a problem, as long as
the new major version is larger. This is done for both the place that
caused the reported bug and another spot where the git version is tested
for another feature.

Reported-by: Chris Packham <judge.packham@gmail.com>
Reported-by: Yann Dirson <ydirson@free.fr>
Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>

git-gui: show staged submodules regardless of ignore config

Currently setting submodule.<name>.ignore and/or diff.ignoreSubmodules to
"all" suppresses all output of submodule changes for git-gui. This is
really confusing, as even when the user chooses to record a new commit for
an ignored submodule by adding it manually this change won't show up under
"Staged Changes (Will Commit)".

Fix that by using the '--ignore-submodules=dirty' option for both callers
of "git diff-index --cached" when the underlying git version supports that
option.

Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Pat Thoyts <patthoyts@users.sourceforge.net>

t7700: drop explicit --no-pack-kept-objects from .keep test

We want to make sure that the default behavior of git-repack,
without any options, continues to treat .keep files as it
always has. Adding an explicit --no-pack-kept-objects, as
ee34a2b did, is a much less interesting test, and prevented
us from noticing the bug fixed by 64d3dc9 (repack: do not
accidentally pack kept objects by default, 2014-06-10).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>