Simplify the two callback functions that are triggered when the
child process terminates to avoid misuse of the child-process
structure that has already been cleaned up.
* sb/submodule-parallel-fetch:
run-command: do not pass child process data into callbacks
* nd/i18n-2.8.0:
trailer.c: mark strings for translation
ref-filter.c: mark strings for translation
builtin/clone.c: mark strings for translation
builtin/checkout.c: mark strings for translation
The code to read the pack data using the offsets stored in the pack
idx file has been made more carefully check the validity of the
data in the idx.
* jk/pack-idx-corruption-safety:
sha1_file.c: mark strings for translation
use_pack: handle signed off_t overflow
nth_packed_object_offset: bounds-check extended offset
t5313: test bounds-checks of corrupted/malicious pack/idx files
t5510 carefully keeps the cwd at the test root by using either subshells
or explicit cd'ing back to the root. Use a subshell for the last
subtest, too.
Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Commit d53c2c6 (mingw: fix t9700's assumption about
directory separators, 2016-01-27) uses perl's "/r" regex
modifier to do a non-destructive replacement on a string,
leaving the original unmodified and returning the result.
This feature was introduced in perl 5.14, but systems with
older perl are still common (e.g., CentOS 6.5 still has perl
5.10). Let's work around it by providing a helper function
that does the same thing using older syntax.
While we're at it, let's switch to using an alternate regex
separator, which is slightly more readable.
Reported-by: Christian Couder <christian.couder@gmail.com> Helped-by: Dennis Kaarsemaker <dennis@kaarsemaker.net> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
index-pack: add a helper function to derive .idx/.keep filename
These are automatically named by replacing .pack suffix in the
name of the packfile. Add a small helper to do so, as I'll be
adding another one soonish.
When 592ce208 (index-pack: use strip_suffix to avoid magic numbers,
2014-06-30) refactored the code to derive names of .idx and .keep
files from the name of .pack file, a copy-and-paste typo crept in,
mistakingly attempting to create and store the keep message file in
the .idx file we just created, instead of .keep file.
As we create the .keep file with O_CREAT|O_EXCL, and we do so after
we write the .idx file, we luckily do not clobber the .idx file, but
because we deliberately ignored EEXIST when creating .keep file
(which is justifiable because only the existence of .keep file
matters), nobody noticed this mistake so far.
t0001: fix GIT_* environment variable check under --valgrind
When a test case is run without --valgrind, the wrap-for-bin.sh
helper script inserts the environment variable GIT_TEXTDOMAINDIR, but
when run with --valgrind, the variable is missing. A recently
introduced test case expects the presence of the variable, though, and
fails under --valgrind.
Rewrite the test case to strip conditially defined environment variables
from both expected and actual output.
Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
The wording is introduced in c3f0baaca (Documentation: sync git.txt
command list and manual page title, 2007-01-18), but rebase has evolved
since then, capture the modern usage by being more generic about the
rebase command in the summary.
Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
The pthread_exit() function is not expected to return. Ever. On Windows,
we call ExitThread() whose documentation claims: "Ends the calling
thread", i.e. there is no condition in which this function simply
returns: https://msdn.microsoft.com/en-us/library/windows/desktop/ms682659
While at it, fix the return type to be void, as per
http://pubs.opengroup.org/onlinepubs/9699919799/functions/pthread_exit.html
Pointed out by Jeff King, helped by Stefan Naewe, Junio Hamano &
Johannes Sixt.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
git: submodule honor -c credential.* from command line
Due to the way that the git-submodule code works, it clears all local
git environment variables before entering submodules. This is normally
a good thing since we want to clear settings such as GIT_WORKTREE and
other variables which would affect the operation of submodule commands.
However, GIT_CONFIG_PARAMETERS is special, and we actually do want to
preserve these settings. However, we do not want to preserve all
configuration as many things should be left specific to the parent
project.
Add a git submodule--helper function, sanitize-config, which shall be
used to sanitize GIT_CONFIG_PARAMETERS, removing all key/value pairs
except a small subset that are known to be safe and necessary.
Replace all the calls to clear_local_git_env with a wrapped function
that filters GIT_CONFIG_PARAMETERS using the new helper and then
restores it to the filtered subset after clearing the rest of the
environment.
Signed-off-by: Jacob Keller <jacob.keller@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
submodule: fix segmentation fault in submodule--helper clone
The git submodule--helper clone command will fail with a segmentation
fault when given a null url or null path variable. Since these are
required for proper functioning of the submodule--helper clone
subcommand, add checks to prevent running and fail gracefully when
missing.
Update the usage string to reflect the requirement that the --url and
--path "options" are required.
Signed-off-by: Jacob Keller <jacob.keller@gmail.com> Reviewed-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
git submodule--helper clone usage stated that paths were added after the
[--] argument. The actual implementation required use of --path argument
and only supports one path at a time. Update the usage string to match
the current implementation.
Signed-off-by: Jacob Keller <jacob.keller@gmail.com> Reviewed-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
submodule: check argc count for git submodule--helper clone
Extra unused arguments to git submodule--helper clone subcommand were
being silently ignored. Add a check to the argc count after options
handling to ensure that no extra arguments were left on the argv array.
Signed-off-by: Jacob Keller <jacob.keller@gmail.com> Reviewed-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
submodule: don't pass empty string arguments to submodule--helper clone
When --reference or --depth are unused, the current git-submodule.sh
results in empty "" arguments appended to the end of the argv array
inside git submodule--helper clone. This is not caught because the argc
count is not checked today.
Fix git-submodule.sh to only pass an argument when --reference or
--depth are used, preventing the addition of two empty string arguments
on the tail of the argv array.
Signed-off-by: Jacob Keller <jacob.keller@gmail.com> Reviewed-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Expose possible parallelism either via the "--jobs" CLI parameter or
the "submodule.fetchJobs" setting.
By having the variable initialized to -1, we make sure 0 can be passed
into the parallel processing machine, which will then pick as many parallel
workers as there are CPUs.
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
git submodule update: have a dedicated helper for cloning
This introduces a new helper function in git submodule--helper
which takes care of cloning all submodules, which we want to
parallelize eventually.
Some tests (such as empty URL, update_mode=none) are required in the
helper to make the decision for cloning. These checks have been
moved into the C function as well (no need to repeat them in the
shell script).
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
run_processes_parallel: rename parameters for the callbacks
The refs code has a similar pattern of passing around 'struct strbuf *err',
which is strictly used for error reporting. This is not the case here,
as the strbuf is used to accumulate all the output (whether it is error
or not) for the user. Rename it to 'out'.
Suggested-by: Jonathan Nieder <jrnieder@gmail.com> Reviewed-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
run_processes_parallel: treat output of children as byte array
We do not want the output to be interrupted by a NUL byte, so we
cannot use raw fputs. Introduce strbuf_write to avoid having long
arguments in run-command.c.
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
This allows to configure fetching and updating in parallel
without having the command line option.
This moved the responsibility to determine how many parallel processes
to start from builtin/fetch to submodule.c as we need a way to communicate
"The user did not specify the number of parallel processes in the command
line options" in the builtin fetch. The submodule code takes care of
the precedence (CLI > config > default).
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Adhere to the common coding style of Git and not check explicitly
for NULL throughout the file. There are still other occurrences in the
code base but that is usually inside of conditions with side effects.
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Currently submodule.<name>.update is only handled by git-submodule.sh.
C code will start to need to make use of that value as more of the
functionality of git-submodule.sh moves into library code in C.
Add the update field to 'struct submodule' and populate it so it can
be read as sm->update or from sm->update_command.
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Commit 58f2ed0 (remote-curl: pass ref SHA-1 to fetch-pack as well,
2013-12-05) added support for specifying a SHA-1 as well as a ref name.
Add support for specifying just a SHA-1 and set the ref name to the same
value in this case.
Signed-off-by: Gabriel Souza Franco <gabrielfrancosouza@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
The values defined by the macro EXC_FLAG_* (1, 4, 8, 16) are stored
in fields of the structs "pattern" and "exclude", some functions
arguments and a local variable. None of these uses its most
significant bit in any special way and there is no good reason to
use a signed integer for them.
And while we're at it, document "flags" of "exclude" to explicitly
state the values it's supposed to take on.
Signed-off-by: Saurav Sachidanand <sauravsachidanand@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
lockfile: improve error message when lockfile exists
A common mistake leading a user to see this message is to launch "git
commit", let the editor open (and forget about it), and try again to
commit.
The previous message was going too quickly to "a git process crashed"
and to the advice "remove the file manually".
This patch modifies the message in two ways: first, it considers that
"another process is running" is the norm, not the exception, and it
explicitly hints the user to look at text editors.
The message is 2 lines longer, but this is not a problem since
experienced users do not see the message often.
Helped-by: Moritz Neeb <lists@moritzneeb.de> Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
run-command: do not pass child process data into callbacks
The expected way to pass data into the callback is to pass them via
the customizable callback pointer. The error reporting in
default_{start_failure, task_finished} is not user friendly enough, that
we want to encourage using the child data for such purposes.
Furthermore the struct child data is cleaned by the run-command API,
before we access them in the callbacks, leading to use-after-free
situations.
Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Before commit 372370f (http: use credential API to handle proxy auth...),
Environment variable "no_proxy" will take effect if the config variable
"http.proxy" is not set. So the following comamnd won't fail if not
behind a firewall.
But commit 372370f not only read git config variable "http.proxy", but
also read "http_proxy" and "https_proxy" environment variables, and set
the curl option using:
Commit 50a6c8e (use st_add and st_mult for allocation size
computation, 2016-02-22) fixed up many xmalloc call-sites
including ones in compat/mingw.c.
But I screwed up one of them, which was half-converted to
ALLOC_ARRAY, using a very early prototype of the function.
And I never caught it because I don't build on Windows.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
rev-parse: let some options run outside repository
Once upon a time, you could use "--local-env-vars" and
"--resolve-git-dir" outside of any git repository, but they
had to come first on the command line. Commit 68889b4
(rev-parse: remove restrictions on some options, 2013-07-21)
put them into the normal option-parsing loop, fixing the
latter. But it inadvertently broke the former, as we call
setup_git_directory() before starting that loop.
We can note that those options don't care even conditionally
about whether we are in a git repo. So it's fine if we
simply wait to setup the repo until we see an option that
needs it.
However, there is one special exception we should make:
historically, rev-parse will set up the repository and read
config even if there are _no_ options. Some of the
tests in t1300 rely on this to check "git -c $config"
parsing. That's not mirroring real-world use, and we could
tweak the test. But t0002 uses a bare "git rev-parse" to
check "are we in a git repository?". It's plausible that
real-world scripts are relying on this.
So let's cover this case specially, and treat an option-less
"rev-parse" as "see if we're in a repo".
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Gcc under Mac OX 10.6 throws an internal compiler error:
CC combine-diff.o
combine-diff.c: In function ‘diff_tree_combined’:
combine-diff.c:1391: internal compiler error: Segmentation fault
while attempting to build Git at 5b442c4f (tree-diff: catch integer
overflow in combine_diff_path allocation, 2016-02-19).
As clang that ships with the version does not have the same bug,
make Git compile under Mac OS X 10.6 by using clang instead of gcc
to work this around, as it is unlikely that we will see fixed GCC
on that platform.
Later versions of Mac OSX/Xcode only provide clang, and gcc is a
wrapper to it.
Signed-off-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
--get does not fail if a key is multi-valued, it returns the last value
as described in its documentation. Clarify the description of --get-all
to avoid implying that --get does fail in this case.
Signed-off-by: John Keeping <john@keeping.me.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Documentation/git-config: use bulleted list for exit codes
Using a numbered list is confusing because the exit codes are not listed
in order so the numbers at the start of each line do not match the exit
codes described by the following text. Switch to a bulleted list so
that the only number appearing on each line is the exit code described.
Signed-off-by: John Keeping <john@keeping.me.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
The --get, --get-all and --get-regexp options to git-config exit with
status 1 if the key is not found but --get-urlmatch succeeds in this
case.
Change --get-urlmatch to behave in the same way as the other --get*
options so that all four are consistent. --get-color is a special case
because it accepts a default value to return and so should not return an
error if the key is not found.
Also clarify this behaviour in the documentation.
Signed-off-by: John Keeping <john@keeping.me.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
The patch hunk selector of add--interactive knows how ask
git for colorized diffs, and correlate them with the
uncolored diffs we apply. But there's not any way for
somebody who uses a diff-filter tool like contrib's
diff-highlight to see their normal highlighting.
This patch lets users define an arbitrary shell command to
pipe the colorized diff through. The exact output shouldn't
matter (since we just show the result to humans) as long as
it is line-compatible with the original diff (so that
hunk-splitting can split the colorized version, too).
I left two minor issues with the new system that I don't
think are worth fixing right now, but could be done later:
1. We only filter colorized diffs. Theoretically a user
could want to filter a non-colorized diff, but I find
it unlikely in practice. Users who are doing things
like diff-highlighting are likely to want color, too.
2. add--interactive will re-colorize a diff which has been
hand-edited, but it won't have run through the filter.
Fixing this is conceptually easy (just pipe the diff
through the filter), but practically hard to do without
using tempfiles (it would need to feed data to and read
the result from the filter without deadlocking; this
raises portability questions with respect to Windows).
I've punted on both issues for now, and if somebody really
cares later, they can do a patch on top.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
t1515: add tests for rev-parse out-of-repo helpers
The git-rev-parse command is a dumping ground for helpers
that let scripts make various queries of git. Many of these
are conceptually independent of being inside a git
repository.
With the exception of --parseopt, we do not directly test
most of these features in our test suite. Let's give them
some basic sanity checks, which reveals that some of them
have been broken for some time when run from outside a
repository.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
README has been renamed to README.md and its contents got tweaked
slightly to make it easier on the eyes.
* mm/readme-markdown:
README.md: move down historical explanation about the name
README.md: don't call git stupid in the title
README.md: move the link to git-scm.com up
README.md: add hyperlinks on filenames
README: use markdown syntax
"git config section.var value" to set a value in per-repository
configuration file failed when it was run outside any repository,
but didn't say the reason correctly.
* js/config-set-in-non-repository:
git config: report when trying to modify a non-existing repo config
Handling of errors while writing into our internal asynchronous
process has been made more robust, which reduces flakiness in our
tests.
* jk/epipe-in-async:
t5504: handle expected output from SIGPIPE death
test_must_fail: report number of unexpected signal
fetch-pack: ignore SIGPIPE in sideband demuxer
write_or_die: handle EPIPE in async threads
Across the transition at around Git version 2.0, the user used to
get a pretty loud warning when running "git push" without setting
push.default configuration variable. We no longer warn, given that
the transition is over long time ago.
* mm/push-default-warning:
push: remove "push.default is unset" warning message
When "git submodule update" did not result in fetching the commit
object in the submodule that is referenced by the superproject, the
command learned to retry another fetch, specifically asking for
that commit that may not be connected to the refs it usually
fetches.
* sb/submodule-fetch-nontip:
submodule: try harder to fetch needed sha1 by direct fetching sha1
A helper function "git submodule" uses since v2.7.0 to list the
modules that match the pathspec argument given to its subcommands
(e.g. "submodule add <repo> <path>") has been fixed.
Recent versions of GNU grep are pickier when their input contains
arbitrary binary data, which some of our tests uses. Rewrite the
tests to sidestep the problem.
* jk/grep-binary-workaround-in-test:
t9200: avoid grep on non-ASCII data
t8005: avoid grep on non-ASCII data
The "credential-cache" daemon process used to run in whatever
directory it happened to start in, but this made umount(2)ing the
filesystem that houses the repository harder; now the process
chdir()s to the directory that house its own socket on startup.
* jg/credential-cache-chdir-to-sockdir:
credential-cache--daemon: change to the socket dir on startup
credential-cache--daemon: disallow relative socket path
credential-cache--daemon: refactor check_socket_directory
Many codepaths forget to check return value from git_config_set();
the function is made to die() to make sure we do not proceed when
setting a configuration variable failed.
* ps/config-error:
config: rename git_config_set_or_die to git_config_set
config: rename git_config_set to git_config_set_gently
compat: die when unable to set core.precomposeunicode
sequencer: die on config error when saving replay opts
init-db: die on config errors when initializing empty repo
clone: die on config error in cmd_clone
remote: die on config error when manipulating remotes
remote: die on config error when setting/adding branches
remote: die on config error when setting URL
submodule--helper: die on config error when cloning module
submodule: die on config error when linking modules
branch: die on config error when editing branch description
branch: die on config error when unsetting upstream
branch: report errors in tracking branch setup
config: introduce set_or_die wrappers
Traditionally, the tests that try commands that work on the
contents in the working tree were named with "worktree" in their
filenames, but with the recent addition of "git worktree"
subcommand, whose tests are also named similarly, it has become
harder to tell them apart. The traditional tests have been renamed
to use "work-tree" instead in an attempt to differentiate them.
* mg/work-tree-tests:
tests: rename work-tree tests to *work-tree*
The configuration system has been taught to phrase where it found a
bad configuration variable in a better way in its error messages.
"git config" learnt a new "--show-origin" option to indicate where
the values come from.
* ls/config-origin:
config: add '--show-origin' option to print the origin of a config value
config: add 'origin_type' to config_source struct
rename git_config_from_buf to git_config_from_mem
t: do not hide Git's exit code in tests using 'nul_to_q'
Update various codepaths to avoid manually-counted malloc().
* jk/tighten-alloc: (22 commits)
ewah: convert to REALLOC_ARRAY, etc
convert ewah/bitmap code to use xmalloc
diff_populate_gitlink: use a strbuf
transport_anonymize_url: use xstrfmt
git-compat-util: drop mempcpy compat code
sequencer: simplify memory allocation of get_message
test-path-utils: fix normalize_path_copy output buffer size
fetch-pack: simplify add_sought_entry
fast-import: simplify allocation in start_packfile
write_untracked_extension: use FLEX_ALLOC helper
prepare_{git,shell}_cmd: use argv_array
use st_add and st_mult for allocation size computation
convert trivial cases to FLEX_ARRAY macros
use xmallocz to avoid size arithmetic
convert trivial cases to ALLOC_ARRAY
convert manual allocations to argv_array
argv-array: add detach function
add helpers for allocating flex-array structs
harden REALLOC_ARRAY and xcalloc against size_t overflow
tree-diff: catch integer overflow in combine_diff_path allocation
...
"git merge-tree" used to mishandle "both sides added" conflict with
its own "create a fake ancestor file that has the common parts of
what both sides have added and do a 3-way merge" logic; this has
been updated to use the usual "3-way merge with an empty blob as
the fake common ancestor file" approach used in the rest of the
system.
* jk/no-diff-emit-common:
xdiff: drop XDL_EMIT_COMMON
merge-tree: drop generate_common strategy
merge-one-file: use empty blob for add/add base
The internal API to interact with "remote.*" configuration
variables has been streamlined.
* tg/git-remote:
remote: use remote_is_configured() for add and rename
remote: actually check if remote exits
remote: simplify remote_is_configured()
remote: use parse_config_key
credential: let empty credential specs reset helper list
Sine the credential.helper key is a multi-valued config
list, there's no way to "unset" a helper once it's been set.
So if your system /etc/gitconfig sets one, you can never
avoid running it, but only add your own helpers on top.
Since an empty value for credential.helper is nonsensical
(it would just try to run "git-credential-"), we can assume
nobody is using it. Let's define it to reset the helper
list, letting you override lower-priority instances which
have come before.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
In contrast to apache 2.2, apache 2.4 does not load mod_unixd in its
default configuration (because there are choices). Thus, with the
current config, apache 2.4.10 will not be started and the httpd tests
will not run on distros with default apache config (RedHat type).
Enable mod_unixd to make the httpd tests run. This does not affect
distros negatively which have that config already in their default
(Debian type). httpd tests will run on these before and after this patch.
Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Commit 8bf4bec (add "ok=sigpipe" to test_must_fail and use
it to fix flaky tests, 2015-11-27) taught t5504 to handle
"git push" racily exiting with SIGPIPE rather than failing.
However, one of the tests checks the output of the command,
as well. In the SIGPIPE case, we will not have produced any
output. If we want the test to be truly non-flaky, we have
to accept either output.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
test_must_fail: report number of unexpected signal
If a command is marked as test_must_fail but dies with a
signal, we consider that a problem and report the error to
stderr. However, we don't say _which_ signal; knowing that
can make debugging easier. Let's share as much as we know.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
If the other side feeds us a bogus pack, index-pack (or
unpack-objects) may die early, before consuming all of its
input. As a result, the sideband demuxer may get SIGPIPE
(racily, depending on whether our data made it into the pipe
buffer or not). If this happens and we are compiled with
pthread support, it will take down the main thread, too.
This isn't the end of the world, as the main process will
just die() anyway when it sees index-pack failed. But it
does mean we don't get a chance to say "fatal: index-pack
failed" or similar. And it also means that we racily fail
t5504, as we sometimes die() and sometimes are killed by
SIGPIPE.
So let's ignore SIGPIPE while demuxing the sideband. We are
already careful to check the return value of write(), so we
won't waste time writing to a broken pipe. The caller will
notice the error return from the async thread, though in
practice we don't even get that far, as we die() as soon as
we see that index-pack failed.
The non-sideband case is already fine; we let index-pack
read straight from the socket, so there is no SIGPIPE at
all. Technically the non-threaded async case is also OK
without this (the forked async process gets SIGPIPE), but
it's not worth distinguishing from the threaded case here.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
When write_or_die() sees EPIPE, it treats it specially by
converting it into a SIGPIPE death. We obviously cannot
ignore it, as the write has failed and the caller expects us
to die. But likewise, we cannot just call die(), because
printing any message at all would be a nuisance during
normal operations.
However, this is a problem if write_or_die() is called from
a thread. Our raised signal ends up killing the whole
process, when logically we just need to kill the thread
(after all, if we are ignoring SIGPIPE, there is good reason
to think that the main thread is expecting to handle it).
Inside an async thread, the die() code already does the
right thing, because we use our custom die_async() routine,
which calls pthread_join(). So ideally we would piggy-back
on that, and simply call:
die_quietly_with_code(141);
or similar. But refactoring the die code to do this is
surprisingly non-trivial. The die_routines themselves handle
both printing and the decision of the exit code. Every one
of them would have to be modified to take new parameters for
the code, and to tell us to be quiet.
Instead, we can just teach write_or_die() to check for the
async case and handle it specially. We do have to build an
interface to abstract the async exit, but it's simple and
self-contained. If we had many call-sites that wanted to do
this die_quietly_with_code(), this approach wouldn't scale
as well, but we don't. This is the only place where do this
weird exit trick.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
add DEVELOPER makefile knob to check for acknowledged warnings
We assume Git developers have a reasonably modern compiler and recommend
them to enable the DEVELOPER makefile knob to ensure their patches are
clear of all compiler warnings the Git core project cares about.
Enable the DEVELOPER makefile knob in the Travis-CI build.
Suggested-by: Jeff King <peff@peff.net> Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
A v2 pack index file can specify an offset within a packfile
of up to 2^64-1 bytes. On a system with a signed 64-bit
off_t, we can represent only up to 2^63-1. This means that a
corrupted .idx file can end up with a negative offset in the
pack code. Our bounds-checking use_pack function looks for
too-large offsets, but not for ones that have wrapped around
to negative. Let's do so, which fixes an out-of-bounds
access demonstrated in t5313.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
If a pack .idx file has a corrupted offset for an object, we
may try to access an offset in the .idx or .pack file that
is larger than the file's size. For the .pack case, we have
use_pack() to protect us, which realizes the access is out
of bounds. But if the corrupted value asks us to look in the
.idx file's secondary 64-bit offset table, we blindly add it
to the mmap'd index data and access arbitrary memory.
We can fix this with a simple bounds-check compared to the
size we found when we opened the .idx file.
Note that there's similar code in index-pack that is
triggered only during "index-pack --verify". To support
both, we pull the bounds-check into a separate function,
which dies when it sees a corrupted file.
It would be nice if we could return an error, so that the
pack code could try to find a good copy of the object
elsewhere. Currently nth_packed_object_offset doesn't have
any way to return an error, but it could probably use "0" as
a sentinel value (since no object can start there). This is
the minimal fix, and we can improve the resilience later on
top.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>