"git cat-file --batch-all" has been sped up, by taking advantage
of the fact that it does not have to read a list of objects, in two
ways.
* jk/cat-file-buffered-batch-all:
cat-file: default to --buffer when --batch-all-objects is used
cat-file: avoid noop calls to sha1_object_info_extended
"git fast-import --export-marks" would overwrite the existing marks
file even when it makes a dump from its custom die routine.
Prevent it from doing so when we have an import-marks file but
haven't finished reading it.
* fc/fast-import-broken-marks-file:
fast-import: do not truncate exported marks file
Documentation: add instructions to help setup gmail 2FA
For those who use two-factor authentication with gmail, git-send-email
will not work unless it is setup with an app-specific password. The
example for setting up git-send-email for use with gmail will now
include information on generating and storing the app-specific password.
Signed-off-by: Michael Rappazzo <rappazzo@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
On Windows, .git and optionally any files whose name starts with a
dot are now marked as hidden, with a core.hideDotFiles knob to
customize this behaviour.
Merge branch 'kf/gpg-sig-verification-doc' into maint
Documentation for "git merge --verify-signatures" has been updated
to clarify that the signature of only the commit at the tip is
verified. Also the phrasing used for signature and key validity is
adjusted to align with that used by OpenPGP.
* va/i18n-misc-updates:
i18n: unpack-trees: avoid substituting only a verb in sentences
i18n: builtin/pull.c: split strings marked for translation
i18n: builtin/pull.c: mark placeholders for translation
i18n: git-parse-remote.sh: mark strings for translation
i18n: branch: move comment for translators
i18n: branch: unmark string for translation
i18n: builtin/rm.c: remove a comma ',' from string
i18n: unpack-trees: mark strings for translation
i18n: builtin/branch.c: mark option for translation
i18n: index-pack: use plural string instead of normal one
mingw: make isatty() recognize MSYS2's pseudo terminals (/dev/pty*)
MSYS2 emulates pseudo terminals via named pipes, and isatty() returns 0
for such file descriptors. Therefore, some interactive functionality
(such as launching a pager, asking if a failed unlink should be repeated
etc.) doesn't work when run in a terminal emulator that uses MSYS2's
ptys (such as mintty).
However, MSYS2 uses special names for its pty pipes ('msys-*-pty*'),
which allows us to distinguish them from normal piped input / output.
On startup, check if stdin / stdout / stderr are connected to such pipes
using the NtQueryObject API from NTDll.dll. If the names match, adjust
the flags in MSVCRT's ioinfo structure accordingly.
Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Commit f2f0267 (archive-tar: use xsnprintf for trivial
formatting, 2015-09-24) converted cases of "sprintf" to
"xsnprintf", but accidentally left one as just "snprintf".
This meant that we could silently truncate the resulting
buffer instead of flagging an error.
In practice, this is impossible to achieve, as we are
formatting a ustar checksum, which can be at most 7
characters. But the point of xsnprintf is to document and
check for "should be impossible" conditions; this site was
just accidentally mis-converted during f2f0267.
Noticed-by: Paul Green <Paul.Green@stratus.com> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
test_patch_id_file_order shell function uses $name variable to hold
one filename, and calls another shell function calc_patch_id as a
downstream of one pipeline. The called function, however, also uses
the same $name variable. With a shell implementation that runs the
callee in the current shell environment, the caller's $name would
be clobbered by the callee's use of the same variable.
This hasn't been an issue with dash and bash. ksh93 reveals the
breakage in the test script.
Fix it by using a distinct variable name in the callee.
Reported-by: Armin Kunaschik <megabreit@googlemail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
t9xxx series has been updated primarily for readability, while
fixing small bugs in it. A few scripted Porcelains have also been
updated to fix possible bugs around their use of "test -z" and
"test -n".
* jk/test-z-n-unquoted:
always quote shell arguments to test -z/-n
t9103: modernize test style
t9107: switch inverted single/double quotes in test
t9107: use "return 1" instead of "exit 1"
t9100,t3419: enclose all test code in single-quotes
t/lib-git-svn: drop $remote_git_svn and $git_svn_id
Many commands normalize command line arguments from NFD to NFC
variant of UTF-8 on OSX, but commands in the "diff" family did
not, causing "git diff $path" to complain that no such path is
known to Git. They have been taught to do the normalization.
* ar/diff-args-osx-precompose:
diff: run arguments through precompose_argv
Find common mistakes when writing gitlink: in our documentation and
drive the check from "make check-docs".
I am not entirely happy with the way the script chooses what input
file to validate, but it is not worse than not having anything, so
let's move it forward and have the logic improved later when people
care about it deeply.
* jc/doc-lint:
ci: validate "linkgit:" in documentation
t0040 had too many unnecessary repetitions in its test data. Teach
test-parse-options program so that a caller can tell what it
expects in its output, so that these repetitions can be cleaned up.
* jc/test-parse-options-expect:
t0040: convert a few tests to use test-parse-options --expect
t0040: remove unused test helpers
test-parse-options: --expect=<string> option to simplify tests
test-parse-options: fix output when callback option fails
"git commit" learned to pay attention to "commit.verbose"
configuration variable and act as if "--verbose" option was
given from the command line.
* pb/commit-verbose-config:
commit: add a commit.verbose config variable
t7507-commit-verbose: improve test coverage by testing number of diffs
parse-options.c: make OPTION_COUNTUP respect "unspecified" values
t/t7507: improve test coverage
t0040-parse-options: improve test coverage
test-parse-options: print quiet as integer
t0040-test-parse-options.sh: fix style issues
"git format-patch" learned a new "--base" option to record what
(public, well-known) commit the original series was built on in
its output.
* xy/format-patch-base:
format-patch: introduce format.useAutoBase configuration
format-patch: introduce --base=auto option
format-patch: add '--base' option to record base tree info
patch-ids: make commit_patch_id() a public helper function
A couple of bugs around core.autocrlf have been fixed.
* tb/core-eol-fix:
convert.c: ident + core.autocrlf didn't work
t0027: test cases for combined attributes
convert: allow core.autocrlf=input and core.eol=crlf
t0027: make commit_chk_wrnNNO() reliable
The experimental "multiple worktree" feature gains more safety to
forbid operations on a branch that is checked out or being actively
worked on elsewhere, by noticing that e.g. it is being rebased.
* nd/worktree-various-heads:
branch: do not rename a branch under bisect or rebase
worktree.c: check whether branch is bisected in another worktree
wt-status.c: split bisect detection out of wt_status_get_state()
worktree.c: check whether branch is rebased in another worktree
worktree.c: avoid referencing to worktrees[i] multiple times
wt-status.c: make wt_status_check_rebase() work on any worktree
wt-status.c: split rebase detection out of wt_status_get_state()
path.c: refactor and add worktree_git_path()
worktree.c: mark current worktree
worktree.c: make find_shared_symref() return struct worktree *
worktree.c: store "id" instead of "git_dir"
path.c: add git_common_path() and strbuf_git_common_path()
dir.c: rename str(n)cmp_icase to fspath(n)cmp
"git commit --dry-run" reported "No, no, you cannot commit." in one
case where "git commit" would have allowed you to commit, and this
improves it a little bit ("git commit --dry-run --short" still does
not give you the correct answer, for example). This is a stop-gap
measure in that "commit --short --dry-run" still gives an incorrect
result.
* ss/commit-dry-run-resolve-merge-to-no-op:
wt-status.c: set commitable bit if there is a meaningful merge.
and expect that the backslash-dq is passed through literally.
ksh88 eats the backslash and produces a wrong expect file to
compare the actual output with.
Using \\" works this around without breaking other POSIX shells
(which collapse backslash-backslash to a single backslash), and
ksh88 does so, too.
It makes it easier to read, too, because the reason why we are
writing backslash there is *not* because we think dq is special and
want to quote it (if that were the case we would have two more
backslashes on that line). It is simply because we want a single
literal backslash there. Since backslash is treated specially in
unquoted here-document, explicitly doubling it to quote it expresses
our intent better than relying on the character that immediately
comes after it (i.e. '"') not being a special character.
Signed-off-by: Armin Kunaschik <megabreit@googlemail.com> Acked-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* sb/misc-cleanups:
submodule-config: don't shadow `cache`
config.c: drop local variable
credential-cache, send_request: close fd when done
bundle: don't leak an fd in case of early return
abbrev_sha1_in_line: don't leak memory
notes: don't leak memory in git_config_get_notes_strategy
Recent update to Git LFS broke "git p4" by changing the output from
its "lfs pointer" subcommand.
* ls/p4-lfs:
git-p4: fix Git LFS pointer parsing
travis-ci: express Linux/OS X dependency versions more clearly
travis-ci: update Git-LFS and P4 to the latest version
Merge branch 'jk/push-client-deadlock-fix' into HEAD
Some Windows SDK lacks pthread_sigmask() implementation and fails
to compile the recently updated "git push" codepath that uses it.
* jk/push-client-deadlock-fix:
Windows: only add a no-op pthread_sigmask() when needed
Windows: add pthread_sigmask() that does nothing
t5504: drop sigpipe=ok from push tests
fetch-pack: isolate sigpipe in demuxer thread
send-pack: isolate sigpipe in demuxer thread
run-command: teach async threads to ignore SIGPIPE
send-pack: close demux pipe before finishing async process
The "user.useConfigOnly" configuration variable makes it an error
if users do not explicitly set user.name and user.email. However,
its check was not done early enough and allowed another error to
trigger, reporting that the default value we guessed from the
system setting was unusable. This was a suboptimal end-user
experience as we want the users to set user.name/user.email without
relying on the auto-detection at all.
* da/user-useconfigonly:
ident: give "please tell me" message upon useConfigOnly error
ident: check for useConfigOnly before auto-detection of name/email
The test scripts for "git p4" (but not "git p4" implementation
itself) has been updated so that they would work even on a system
where the installed version of Python is python 3.
* ld/p4-test-py3:
git-p4 tests: time_in_seconds should use $PYTHON_PATH
git-p4 tests: work with python3 as well as python2
git-p4 tests: cd to / before running python
cat-file: default to --buffer when --batch-all-objects is used
Traditionally cat-file's batch-mode does not do any output
buffering. The reason is that a caller may have pipes
connected to its input and output, and would want to use
cat-file interactively, getting output immediately for each
input it sends.
This may involve a lot of small write() calls, which can be
slow. So we introduced --buffer to improve this, but we
can't turn it on by default, as it would break the
interactive case above.
However, when --batch-all-objects is used, we do not read
stdin at all. We generate the output ourselves as quickly as
possible, and then exit. In this case buffering is a strict
win, and it is simply a hassle for the user to have to
remember to specify --buffer.
This patch makes --buffer the default when --batch-all-objects
is used. Specifying "--buffer" manually is still OK, and you
can even override it with "--no-buffer" if you're a
masochist (or debugging).
cat-file: avoid noop calls to sha1_object_info_extended
It is not unreasonable to ask cat-file for a batch-check
format of simply "%(objectname)". At first glance this seems
like a noop (you are generally already feeding the object
names on stdin!), but it has a few uses:
1. With --batch-all-objects, you can generate a listing of
the sha1s present in the repository, without any input.
2. You do not have to feed sha1s; you can feed arbitrary
sha1 expressions and have git resolve them en masse.
3. You can even feed a raw sha1, with the result that git
will tell you whether we actually have the object or
not.
In case 3, the call to sha1_object_info is useful; it tells
us whether the object exists or not (technically we could
swap this out for has_sha1_file, but the cost is roughly the
same).
In case 2, the existence check is of debatable value. A
mass-resolution might prefer performance to safety (against
outputting a value for a corrupted ref, for example).
However, the object lookup cost is likely not as noticeable
compared to the resolution cost. And since we have provided
that safety in the past, the conservative choice is to keep
it.
In case 1, though, the object lookup is a definite noop; we
know about the object because we found it in the object
database. There is no new information gained by making the
call.
This patch detects that case and optimizes out the call.
Here are best-of-five timings for linux.git:
[before]
$ time git cat-file --buffer \
--batch-all-objects \
--batch-check='%(objectname)'
real 0m2.117s
user 0m2.044s
sys 0m0.072s
[after]
$ time git cat-file --buffer \
--batch-all-objects \
--batch-check='%(objectname)'
real 0m1.230s
user 0m1.176s
sys 0m0.052s
There are two implementation details to note here.
One is that we detect the noop case by seeing that "struct
object_info" does not request any information. But besides
object existence, there is one other piece of information
which sha1_object_info may fill in: whether the object is
cached, loose, or packed. We don't currently provide that
information in the output, but if we were to do so later,
we'd need to take note and disable the optimization in that
case.
And that leads to the second note. If we were to output
that information, a better implementation would be to
remember where we saw the object in --batch-all-objects in
the first place, and avoid looking it up again by sha1.
In fact, we could probably squeeze out some extra
performance for less-trivial cases, too, by remembering the
pack location where we saw the object, and going directly
there to find its information (like type, size, etc). That
would in theory make this optimization unnecessary.
I didn't pursue that path here for two reasons:
1. It's non-trivial to implement, and has memory
implications. Because we sort and de-dup the list of
output sha1s, we'd have to record the pack information
for each object, too.
2. It doesn't save as much as you might hope. It saves the
find_pack_entry() call, but getting the size and type
for deltified objects requires walking down the delta
chain (for the real type) or reading the delta data
header (for the size). These costs tend to dominate the
non-trivial cases.
By contrast, this optimization is easy and self-contained,
and speeds up a real-world case I've used.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Certain lines of the marks file might be corrupted (or the objects
missing due to a garbage collection), but that's no reason to truncate
the file and essentially destroy the rest of it.
Ideally missing objects should not cause a crash, we could just skip
them, but that's another patch.
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Documentation for "git merge --verify-signatures" has been updated
to clarify that the signature of only the commit at the tip is
verified. Also the phrasing used for signature and key validity is
adjusted to align with that used by OpenPGP.
On Windows, .git and optionally any files whose name starts with a
dot are now marked as hidden, with a core.hideDotFiles knob to
customize this behaviour.
Running tests with '-x' option to trace the individual command
executions is a useful way to debug test scripts, but some tests
that capture the standard error stream and check what the command
said can be broken with the trace output mixed in. When running
our tests under "bash", however, we can redirect the trace output
to another file descriptor to keep the standard error of programs
being tested intact.
* jk/test-send-sh-x-trace-elsewhere:
test-lib: set BASH_XTRACEFD automatically
Update tests for "http.extraHeaders=<header>" to be portable back
to Apache 2.2 (the original depended on <RequireAll/> which is a
more recent feature).
* js/http-custom-headers:
submodule: ensure that -c http.extraheader is heeded
t5551: make the test for extra HTTP headers more robust
tests: adjust the configuration for Apache 2.2
Switching between 'master' and 'next', between which the paths to
test helper binaries have changed, did not update bin-wrappers/*
scripts used in tests, causing false test failures.
* nd/test-helpers:
wrap-for-bin.sh: regenerate bin-wrappers when switching branches
"git rerere" can get confused by conflict markers deliberately left
by the inner merge step, because they are indistinguishable from
the real conflict markers left by the outermost merge which are
what the end user and "rerere" need to look at. This was fixed by
making the conflict markers left by the inner merges a bit longer.
* jc/ll-merge-internal:
t6036: remove pointless test that expects failure
ll-merge: use a longer conflict marker for internal merge
ll-merge: fix typo in comment
The code for warning_errno/die_errno has been refactored and a new
error_errno() reporting helper is introduced.
* nd/error-errno: (41 commits)
wrapper.c: use warning_errno()
vcs-svn: use error_errno()
upload-pack.c: use error_errno()
unpack-trees.c: use error_errno()
transport-helper.c: use error_errno()
sha1_file.c: use {error,die,warning}_errno()
server-info.c: use error_errno()
sequencer.c: use error_errno()
run-command.c: use error_errno()
rerere.c: use error_errno() and warning_errno()
reachable.c: use error_errno()
mailmap.c: use error_errno()
ident.c: use warning_errno()
http.c: use error_errno() and warning_errno()
grep.c: use error_errno()
gpg-interface.c: use error_errno()
fast-import.c: use error_errno()
entry.c: use error_errno()
editor.c: use error_errno()
diff-no-index.c: use error_errno()
...
An earlier addition of "sanitize_submodule_env" with 14111fc4 (git:
submodule honor -c credential.* from command line, 2016-02-29)
turned out to be a convoluted no-op; implement what it wanted to do
correctly, and stop filtering settings given via "git -c var=val".
* jk/submodule-c-credential:
submodule: stop sanitizing config options
submodule: use prepare_submodule_repo_env consistently
submodule--helper: move config-sanitizing to submodule.c
submodule: export sanitized GIT_CONFIG_PARAMETERS
t5550: break submodule config test into multiple sub-tests
t5550: fix typo in $HTTPD_URL