"git notes merge" used to limit the source of the merged notes tree
to somewhere under refs/notes/ hierarchy, which was too limiting
when inventing a workflow to exchange notes with remote
repositories using remote-tracking notes trees (located in e.g.
refs/remote-notes/ or somesuch).
* jk/notes-merge-from-anywhere:
notes: allow merging from arbitrary references
Many commands that read files that are expected to contain text
that is generated (or can be edited) by the end user to control
their behaviour (e.g. "git grep -f <filename>") have been updated
to be more tolerant to lines that are terminated with CRLF (they
used to treat such a line to contain payload that ends with CR,
which is usually not what the users expect).
* jc/peace-with-crlf:
test-sha1-array: read command stream with strbuf_getline()
grep: read -f file with strbuf_getline()
send-pack: read list of refs with strbuf_getline()
column: read lines with strbuf_getline()
cat-file: read batch stream with strbuf_getline()
transport-helper: read helper response with strbuf_getline()
clone/sha1_file: read info/alternates with strbuf_getline()
remote.c: read $GIT_DIR/remotes/* with strbuf_getline()
ident.c: read /etc/mailname with strbuf_getline()
rev-parse: read parseopt spec with strbuf_getline()
revision: read --stdin with strbuf_getline()
hash-object: read --stdin-paths with strbuf_getline()
"git tag" started listing a tag "foo" as "tags/foo" when a branch
named "foo" exists in the same repository; remove this unnecessary
disambiguation, which is a regression introduced in v2.7.0.
* jk/list-tag-2.7-regression:
tag: do not show ambiguous tag names as "tags/foo"
t6300: use test_atom for some un-modern tests
"git shortlog" used to accumulate various pieces of information
regardless of what was asked to be shown in the final output. It
has been optimized by noticing what need not to be collected
(e.g. there is no need to collect the log messages when showing
only the number of changes).
* jk/shortlog:
shortlog: don't warn on empty author
shortlog: optimize out useless string list
shortlog: optimize out useless "<none>" normalization
shortlog: optimize "--summary" mode
shortlog: replace hand-parsing of author with pretty-printer
shortlog: use strbufs to read from stdin
shortlog: match both "Author:" and "author" on stdin
The preliminary clean-up for jc/peace-with-crlf topic.
* jc/strbuf-getline:
strbuf: give strbuf_getline() to the "most text friendly" variant
checkout-index: there are only two possible line terminations
update-index: there are only two possible line terminations
check-ignore: there are only two possible line terminations
check-attr: there are only two possible line terminations
mktree: there are only two possible line terminations
strbuf: introduce strbuf_getline_{lf,nul}()
strbuf: make strbuf_getline_crlf() global
strbuf: miniscule style fix
Beginning of the upstreaming process of Git for Windows effort.
* js/msys2:
mingw: uglify (a, 0) definitions to shut up warnings
mingw: squash another warning about a cast
mingw: avoid warnings when casting HANDLEs to int
mingw: avoid redefining S_* constants
compat/winansi: support compiling with MSys2
compat/mingw: support MSys2-based MinGW build
nedmalloc: allow compiling with MSys2's compiler
config.mak.uname: supporting 64-bit MSys2
config.mak.uname: support MSys2
git-svn: fix auth parameter handling on SVN 1.9.0+
For users with "store-passwords = no" set in the "[auth]" section of
their ~/.subversion/config, SVN 1.9.0+ would fail with the
following message when attempting to call svn_auth_set_parameter:
Value is not a string (or undef) at Git/SVN/Ra.pm
Ironically, this breakage was caused by r1553823 in subversion:
"Make svn_auth_set_parameter() usable from Perl bindings."
Since 2007 (602015e0e6ec), git-svn has used a workaround to make
svn_auth_set_parameter usable internally. However this workaround
breaks under SVN 1.9+, which deals properly with the type mapping
and fails to recognize our workaround.
For pre-1.9.0 SVN, we continue to use the existing workaround for
the lack of proper type mapping in the bindings.
Tested under subversion 1.6.17 and 1.9.3.
I've also verified r1553823 was not backported to SVN 1.8.x:
"git format-patch" learned to notice format.outputDirectory
configuration variable. This allows "-o <dir>" option to be
omitted on the command line if you always use the same directory in
your workflow.
Many codepaths that run "gc --auto" before exiting kept packfiles
mapped and left the file descriptors to them open, which was not
friendly to systems that cannot remove files that are open. They
now close the packs before doing so.
* js/close-packs-before-gc:
receive-pack: release pack files before garbage-collecting
merge: release pack files before garbage-collecting
am: release pack files before garbage-collecting
fetch: release pack files before garbage-collecting
tag: do not show ambiguous tag names as "tags/foo"
Since b7cc53e9 (tag.c: use 'ref-filter' APIs, 2015-07-11),
git-tag has started showing tags with ambiguous names (i.e.,
when both "heads/foo" and "tags/foo" exists) as "tags/foo"
instead of just "foo". This is both:
- pointless; the output of "git tag" includes only
refs/tags, so we know that "foo" means the one in
"refs/tags".
and
- ambiguous; in the original output, we know that the line
"foo" means that "refs/tags/foo" exists. In the new
output, it is unclear whether we mean "refs/tags/foo" or
"refs/tags/tags/foo".
The reason this happens is that commit b7cc53e9 switched
git-tag to use ref-filter's "%(refname:short)" output
formatting, which was adapted from for-each-ref. This more
general code does not know that we care only about tags, and
uses shorten_unambiguous_ref to get the short-name. We need
to tell it that we care only about "refs/tags/", and it
should shorten with respect to that value.
In theory, the ref-filter code could figure this out by us
passing FILTER_REFS_TAGS. But there are two complications
there:
1. The handling of refname:short is deep in formatting
code that does not even have our ref_filter struct, let
alone the arguments to the filter_ref struct.
2. In git v2.7.0, we expose the formatting language to the
user. If we follow this path, it will mean that
"%(refname:short)" behaves differently for "tag" versus
"for-each-ref" (including "for-each-ref refs/tags/"),
which can lead to confusion.
Instead, let's add a new modifier to the formatting
language, "strip", to remove a specific set of prefix
components. This fixes "git tag", and lets users invoke the
same behavior from their own custom formats (for "tag" or
"for-each-ref") while leaving ":short" with its same
consistent meaning in all places.
We introduce a test in t7004 for "git tag", which fails
without this patch. We also add a similar test in t3203 for
"git branch", which does not actually fail. But since it is
likely that "branch" will eventually use the same formatting
code, the test helps defend against future regressions.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Because this script has to test so many formatters, we have
the nice "test_atom" helper, but we don't use it
consistently. Let's do so. This is shorter, gets rid of some
tests that have their "expected" setup outside of a
test_expect_success block, and lets us organize the changes
better (e.g., putting "refname:short" near "refname").
We also expand the "%(push)" tests a little to match the
"%(upstream)" ones.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
A shell script style update to change `command substitution` into
$(command substitution). Coverts contrib/ and much of the t/
directory contents.
* ep/shell-command-substitution-style: (92 commits)
t9901-git-web--browse.sh: use the $( ... ) construct for command substitution
t9501-gitweb-standalone-http-status.sh: use the $( ... ) construct for command substitution
t9350-fast-export.sh: use the $( ... ) construct for command substitution
t9300-fast-import.sh: use the $( ... ) construct for command substitution
t9150-svk-mergetickets.sh: use the $( ... ) construct for command substitution
t9145-git-svn-master-branch.sh: use the $( ... ) construct for command substitution
t9138-git-svn-authors-prog.sh: use the $( ... ) construct for command substitution
t9137-git-svn-dcommit-clobber-series.sh: use the $( ... ) construct for command substitution
t9132-git-svn-broken-symlink.sh: use the $( ... ) construct for command substitution
t9130-git-svn-authors-file.sh: use the $( ... ) construct for command substitution
t9129-git-svn-i18n-commitencoding.sh: use the $( ... ) construct for command substitution
t9119-git-svn-info.sh: use the $( ... ) construct for command substitution
t9118-git-svn-funky-branch-names.sh: use the $( ... ) construct for command substitution
t9114-git-svn-dcommit-merge.sh: use the $( ... ) construct for command substitution
t9110-git-svn-use-svm-props.sh: use the $( ... ) construct for command substitution
t9109-git-svn-multi-glob.sh: use the $( ... ) construct for command substitution
t9108-git-svn-glob.sh: use the $( ... ) construct for command substitution
t9107-git-svn-migrate.sh: use the $( ... ) construct for command substitution
t9105-git-svn-commit-diff.sh: use the $( ... ) construct for command substitution
t9104-git-svn-follow-parent.sh: use the $( ... ) construct for command substitution
...
"git subtree" (in contrib/) records the tag object name in the
commit log message when a subtree is added using a tag, without
peeling it down to the underlying commit. The tag needs to be
peeled when "git subtree split" wants to work on the commit, but
the command forgot to do so.
* rm/subtree-unwrap-tags:
contrib/subtree: unwrap tag refs
"git grep" by default does not fall back to its "--no-index"
behaviour outside a directory under Git's control (otherwise the
user may by mistake end up running a huge recursive search); with a
new configuration (set in $HOME/.gitconfig--by definition this
cannot be set in the config file per project), this safety can be
disabled.
* tg/grep-no-index-fallback:
builtin/grep: add grep.fallbackToNoIndex config
t7810: correct --no-index test
Asking gitweb for a nonexistent commit left a warning in the server
log.
Somebody may want to follow this up with a new test, perhaps?
IIRC, we do test that no Perl warnings are given to the server log,
so this should have been caught if our test coverage were good.
Some codepaths used fopen(3) when opening a fixed path in $GIT_DIR
(e.g. COMMIT_EDITMSG) that is meant to be left after the command is
done. This however did not work well if the repository is set to
be shared with core.sharedRepository and the umask of the previous
user is tighter. They have been made to work better by calling
unlink(2) and retrying after fopen(3) fails with EPERM.
* js/fopen-harder:
Handle more file writes correctly in shared repos
commit: allow editing the commit message even in shared repos
Documentation for "git fetch --depth" has been updated for clarity.
* ss/clone-depth-single-doc:
docs: clarify that --depth for git-fetch works with newly initialized repos
docs: say "commits" in the --depth option wording for git-clone
docs: clarify that passing --depth to git-clone implies --single-branch
The ignore mechanism saw a few regressions around untracked file
listing and sparse checkout selection areas in 2.7.0; the change
that is responsible for the regression has been reverted.
* nd/exclusion-regression-fix:
Revert "dir.c: don't exclude whole dir prematurely if neg pattern may match"
Drop a few old "todo" items by deciding that the change one of them
suggests is not such a good idea, and doing the change the other
one suggested to do.
* ss/user-manual:
user-manual: add addition gitweb information
user-manual: add section documenting shallow clones
glossary: define the term shallow clone
user-manual: remove temporary branch entry from todo list
d95138e6 (setup: set env $GIT_WORK_TREE when work tree is set, like
$GIT_DIR, 2015-06-26) attempted to work around a glitch in alias
handling by overwriting GIT_WORK_TREE environment variable to
affect subprocesses when set_git_work_tree() gets called, which
resulted in a rather unpleasant regression to "clone" and "init".
Try to address the same issue by always restoring the environment
and respawning the real underlying command when handling alias.
* nd/clear-gitenv-upon-use-of-alias:
run-command: don't warn on SIGPIPE deaths
git.c: make sure we do not leak GIT_* to alias scripts
setup.c: re-fix d95138e (setup: set env $GIT_WORK_TREE when ..
git.c: make it clear save_env() is for alias handling only
Paths that have been told the index about with "add -N" are not
quite yet in the index, but a few commands behaved as if they
already are in a harmful way.
* nd/ita-cleanup:
grep: make it clear i-t-a entries are ignored
add and use a convenience macro ce_intent_to_add()
blame: remove obsolete comment
The "exclude_list" structure has the usual "alloc, nr" pair of
fields to be used by ALLOC_GROW(), but clear_exclude_list() forgot
to reset 'alloc' to 0 when it cleared 'nr'to discard the managed
array.
* nd/dir-exclude-cleanup:
dir.c: clean the entire struct in clear_exclude_list()
Some "git notes" operations, e.g. "git log --notes=<note>", should
be able to read notes from any tree-ish that is shaped like a notes
tree, but the notes infrastructure required that the argument must
be a ref under refs/notes/. Loosen it to require a valid ref only
when the operation would update the notes (in which case we must
have a place to store the updated notes tree, iow, a ref).
* mh/notes-allow-reading-treeish:
notes: allow treeish expressions as notes ref
filter-branch: resolve $commit^{tree} in no-index case
Commit 348d4f2 (filter-branch: skip index read/write when
possible, 2015-11-06) taught filter-branch to optimize out
the final "git write-tree" when we know we haven't touched
the tree with any of our filters. It does by simply putting
the literal text "$commit^{tree}" into the "$tree" variable,
avoiding a useless rev-parse call.
However, when we pass this to git_commit_non_empty_tree(),
it gets confused; it resolves "$commit^{tree}" itself, and
compares our string to the 40-hex sha1, which obviously
doesn't match. As a result, "--prune-empty" (or any custom
filter using git_commit_non_empty_tree) will fail to drop
an empty commit (when filter-branch is used without a tree
or index filter).
Let's resolve $tree to the 40-hex ourselves, so that
git_commit_non_empty_tree can work. Unfortunately, this is a
bit slower due to the extra process overhead:
$ cd t/perf && ./run 348d4f2 HEAD p7000-filter-branch.sh
[...]
Test 348d4f2 HEAD
--------------------------------------------------------------
7000.2: noop filter 3.76(0.24+0.26) 4.54(0.28+0.24) +20.7%
We could try to make git_commit_non_empty_tree more clever.
However, the value of $tree here is technically
user-visible. The user can provide arbitrary shell code at
this stage, which could itself have a similar assumption to
what is in git_commit_non_empty_tree. So the conservative
choice to fix this regression is to take the 20% hit and
give the pre-348d4f2 behavior. We still end up much faster
than before the optimization:
$ cd t/perf && ./run 348d4f2^ HEAD p7000-filter-branch.sh
[...]
Test 348d4f2^ HEAD
--------------------------------------------------------------
7000.2: noop filter 9.51(4.32+0.40) 4.51(0.28+0.23) -52.6%
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
f400e51c (test-lib.sh: set prerequisite SANITY by testing what we
really need, 2015-01-27) improved the way SANITY prerequisite was
determined, but made the resulting code (incorrectly) imply that
SANITY is all about effects of permission bits of the containing
directory has on the files contained in it by the comment it added,
its log message and the actual tests.
State what SANITY is about more clearly in the comment, and test
that a file whose permission bits says should be unreadble truly
cannot be read.
Git tries to avoid creating a commit with an empty author
name or email. However, commits created by older, less
strict versions of git may still be in the history. There's
not much point in issuing a warning to stderr for an empty
author. The user can't do anything about it now, and we are
better off to simply include it in the shortlog output as an
empty name/email, and let the caller process it however they
see fit.
Older versions of shortlog differentiated between "author
header not present" (which complained) and "author
name/email are blank" (which included the empty ident in the
output). But since switching to format_commit_message, we
complain to stderr about either case (linux.git has a blank
author deep in its history which triggers this).
We could try to restore the older behavior (complaining only
about the missing header), but in retrospect, there's not
much point in differentiating these cases. A missing
author header is bogus, but as for the "blank" case, the
only useful behavior is to add it to the "empty name"
collection.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
If we are in "--summary" mode, then we do not care about the
actual list of subject onelines associated with each author.
We care only about the number. So rather than store a
string-list for each author full of "<none>", let's just
keep a count.
This drops my best-of-five for "git shortlog -ns HEAD" on
linux.git from:
real 0m5.194s
user 0m5.028s
sys 0m0.168s
to:
real 0m5.057s
user 0m4.916s
sys 0m0.144s
That's about 2.5%.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
shortlog: optimize out useless "<none>" normalization
If we are in --summary mode, we will always pass <none> to
insert_one_record, which will then do some normalization
(e.g., cutting out "[PATCH]"). There's no point in doing so
if we aren't going to use the result anyway.
This drops my best-of-five for "git shortlog -ns HEAD" on
linux.git from:
real 0m5.257s
user 0m5.104s
sys 0m0.156s
to:
real 0m5.194s
user 0m5.028s
sys 0m0.168s
That's only 1%, but arguably the result is clearer to read,
as we're able to group our variable declarations inside the
conditional block. It also opens up further optimization
possibilities for future patches.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
If the user asked us only to show counts for each author,
rather than the individual summary lines, then there is no
point in us generating the summaries only to throw them
away. With this patch, I measured the following speedup for
"git shortlog -ns HEAD" on linux.git (best-of-five):
[before]
real 0m5.644s
user 0m5.472s
sys 0m0.176s
[after]
real 0m5.257s
user 0m5.104s
sys 0m0.156s
That's only ~7%, but it's so easy to do, there's no good
reason not to. We don't have to touch any downstream code,
since we already fill in the magic string "<none>" to handle
commits without a message.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
shortlog: replace hand-parsing of author with pretty-printer
When gathering the author and oneline subject for each
commit, we hand-parse the commit headers to find the
"author" line, and then continue past to the blank line at
the end of the header.
We can replace this tricky hand-parsing by simply asking the
pretty-printer for the relevant items. This also decouples
the author and oneline parsing, opening up some new
optimizations in further commits.
One reason to avoid the pretty-printer is that it might be
less efficient than hand-parsing. However, I measured no
slowdown at all running "git shortlog -ns HEAD" on
linux.git.
As a bonus, we also fix a memory leak in the (uncommon) case
that the author field is blank.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
We currently use fixed-size buffers with fgets(), which
could lead to incorrect results in the unlikely event that a
line had something like "Author:" at exactly its 1024th
character.
But it's easy to convert this to a strbuf, and because we
can reuse the same buffer through the loop, we don't even
pay the extra allocation cost.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
shortlog: match both "Author:" and "author" on stdin
The original git-shortlog could read both the normal "git
log" output as well as "git log --format=raw". However, when
it was converted to C by b8ec592 (Build in shortlog,
2006-10-22), the trailing colon became mandatory, and we no
longer matched the raw output.
Given the amount of intervening time without any bug
reports, it's probable that nobody cares. But it's
relatively easy to fix, and the end result is hopefully more
readable than the original.
Note that this no longer matches "author: ", which we did
before, but that has never been a format generated by git.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Create a new expansion function, expand_loose_notes_ref which will first
check whether the ref can be found using get_sha1. If it can't be found
then it will fallback to using expand_notes_ref. The content of the
strbuf will not be changed if the notes ref can be located using
get_sha1. Otherwise, it may be updated as done by expand_notes_ref.
Since we now support merging from non-notes refs, remove the test case
associated with that behavior. Add a test case for merging from a
non-notes ref.
Signed-off-by: Jacob Keller <jacob.keller@gmail.com> Reviewed-by: Johan Herland <johan@herland.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
MSys2's compiler is correct that casting a "void *" to a "DWORD" loses
precision, but in the case of pthread_exit() we know that the value
fits into a DWORD.
Just like casting handles to DWORDs, let's work around this issue by
casting to "intrptr_t" first, and immediately cast to the final type.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
HANDLE is defined internally as a void *, but in many cases it is
actually guaranteed to be a 32-bit integer. In these cases, GCC should
not warn about a cast of a pointer to an integer of a different type
because we know exactly what we are doing.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
test-sha1-array: read command stream with strbuf_getline()
The input to this command comes from a pipeline in t0064, whose
upstream has bunch of "echo"s. It is not unreasonable to expect
that it may be fed CRLF lines on DOSsy systems.
List of patterns file could come from a DOS editor.
This is iffy; you may actually be trying to find a line with ^M in
it on a system whose line ending is LF. You can of course work it
around by having a line that has "^M^M^J", let the strbuf_getline()
eat the last "^M^J", leaving just the single "^M" as the pattern.
Multiple lines read here are concatenated on a single line to form a
multi-column output line. We do not want to have a CR at the end,
even if the input file consists of CRLF terminated lines.
transport-helper: read helper response with strbuf_getline()
Our implementation of helpers never use CRLF line endings, and they
do not depend on the ability to place a CR as payload at the end of
the line, so this is essentially a no-op for in-tree users. However,
this allows third-party implementation of helpers to give us their
line with CRLF line ending (they cannot expect us to feed CRLF to
them, though).
clone/sha1_file: read info/alternates with strbuf_getline()
$GIT_OBJECT_DIRECTORY/info/alternates is a text file that can be
edited with a DOS editor. We do not want to use the real path with
CR appended at the end.
Just in case /etc/mailname file was edited with a DOS editor,
read it with strbuf_getline() so that a stray CR is not included
as the last character of the mail hostname.
We _might_ want to more aggressively discard whitespace characters
around the line with strbuf_trim(), but that is a bit outside the
scope of this series.
Reading with getwholeline() and manually stripping the terminating
'\n' would leave CR at the end of the line if the input comes from
a DOS editor.
Constrasting this with the other changes around "--stdin" in this
series, one may realize that the way "log" family of commands read
the paths with "--stdin" looks inconsistent and sloppy. It does not
allow us to C-quote a textual input, neither does it accept records
that are NUL-terminated. These are unfortunately way too late to
fix X-<.
strbuf: give strbuf_getline() to the "most text friendly" variant
Now there is no direct caller to strbuf_getline(), we can demote it
to file-scope static that is private to strbuf.c and rename it to
strbuf_getdelim(). Rename strbuf_getline_crlf(), which is designed
to be the most "text friendly" variant, and allow it to take over
this simplest name, strbuf_getline(), so we can add more uses of it
without having to type _crlf over and over again in the coming
steps.
checkout-index: there are only two possible line terminations
The program by default reads LF terminated lines, with an option to
use NUL terminated records. Instead of pretending that there can be
other useful values for line_termination, use a boolean variable,
nul_term_line, to tell if NUL terminated records are used, and
switch between strbuf_getline_{lf,nul} based on it.
update-index: there are only two possible line terminations
The program by default reads LF terminated lines, with an option to
use NUL terminated records. Instead of pretending that there can be
other useful values for line_termination, use a boolean variable,
nul_term_line, to tell if NUL terminated records are used, and
switch between strbuf_getline_{lf,nul} based on it.
check-ignore: there are only two possible line terminations
The program by default reads LF terminated lines, with an option to
use NUL terminated records. Instead of pretending that there can be
other useful values for line_termination, use a boolean variable,
nul_term_line, to tell if NUL terminated records are used, and
switch between strbuf_getline_{lf,nul} based on it.
check-attr: there are only two possible line terminations
The program by default reads LF terminated lines, with an option to
use NUL terminated records. Instead of pretending that there can be
other useful values for line_termination, use a boolean variable,
nul_term_line, to tell if NUL terminated records are used, and
switch between strbuf_getline_{lf,nul} based on it.
mktree: there are only two possible line terminations
The program by default reads LF terminated lines, with an option to
use NUL terminated records. Instead of pretending that there can be
other useful values for line_termination, use a boolean variable,
nul_term_line, to tell if NUL terminated records are used, and
switch between strbuf_getline_{lf,nul} based on it.
The strbuf_getline() interface allows a byte other than LF or NUL as
the line terminator, but this is only because I wrote these
codepaths anticipating that there might be a value other than NUL
and LF that could be useful when I introduced line_termination long
time ago. No useful caller that uses other value has emerged.
By now, it is clear that the interface is overly broad without a
good reason. Many codepaths have hardcoded preference to read
either LF terminated or NUL terminated records from their input, and
then call strbuf_getline() with LF or NUL as the third parameter.
This step introduces two thin wrappers around strbuf_getline(),
namely, strbuf_getline_lf() and strbuf_getline_nul(), and
mechanically rewrites these call sites to call either one of
them. The changes contained in this patch are:
* introduction of these two functions in strbuf.[ch]
* mechanical conversion of all callers to strbuf_getline() with
either '\n' or '\0' as the third parameter to instead call the
respective thin wrapper.
After this step, output from "git grep 'strbuf_getline('" would
become a lot smaller. An interim goal of this series is to make
this an empty set, so that we can have strbuf_getline_crlf() take
over the shorter name strbuf_getline().
Often we read "text" files that are supplied by the end user
(e.g. commit log message that was edited with $GIT_EDITOR upon 'git
commit -e'), and in some environments lines in a text file are
terminated with CRLF. Existing strbuf_getline() knows to read a
single line and then strip the terminating byte from the result, but
it is handy to have a version that is more tailored for a "text"
input that takes both '\n' and '\r\n' as line terminator (aka
<newline> in POSIX lingo) and returns the body of the line after
stripping <newline>.
Recently reimplemented "git am" uses such a function implemented
privately; move it to strbuf.[ch] and make it available for others.
Note that we do not blindly replace calls to strbuf_getline() that
uses LF as the line terminator with calls to strbuf_getline_crlf()
and this is very much deliberate. Some callers may want to treat an
incoming line that ends with CR (and terminated with LF) to have a
payload that includes the final CR, and such a blind replacement
will result in misconversion when done without code audit.
trailer: allow to write to files other than stdout
Use fprintf instead of printf in trailer.c in order to allow printing
to a file other than stdout. This will be needed to support in-place
editing in git interpret-trailers.
Signed-off-by: Tobias Klauser <tklauser@distanz.ch> Signed-off-by: Junio C Hamano <gitster@pobox.com>
The excellent MSys2 project brings a substantially updated MinGW
environment including newer GCC versions and new headers. To support
compiling Git, let's special-case the new MinGW (tell-tale: the
_MINGW64_VERSION_MAJOR constant is defined).
Note: this commit only addresses compile failures, not compile warnings
(that task is left for a future patch).
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
With MSys2's GCC, `ReadWriteBarrier` is already defined, and FORCEINLINE
unfortunately gets defined incorrectly.
Let's work around both problems, using the MSys2-specific
__MINGW64_VERSION_MAJOR constant to guard the FORCEINLINE definition so
as not to affect other platforms.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
remote: handle the config setting branch.*.rebase=interactive
The config variable branch.<branchname>.rebase is not only used by `git
pull`, but also by `git remote` when showing details about a remote.
Therefore, it needs to be taught to accept the newly-introduced
`interactive` value of said variable.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
pull: allow interactive rebase with --rebase=interactive
A couple of years ago, I found the need to collaborate on topic
branches that were rebased all the time, and I really needed to see
what I was rebasing when pulling, so I introduced an
interactively-rebasing pull.
The way builtin pull works, this change also supports the value
'interactive' for the 'branch.<name>.rebase' config variable, which
is a neat thing because users can now configure given branches for
interactively-rebasing pulls without having to type out the complete
`--rebase=interactive` option every time they pull.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
After rebasing, we call "gc --auto" to clean up if we
created a lot of loose objects. However, we do so inside an
&&-chain. If "gc --auto" fails (e.g., because a previous
background gc blocked us by leaving "gc.log" in place),
then:
1. We will fail to clean up the state directory, leaving
the user stuck in the rebase forever (even "git am
--abort" doesn't work, because it calls "gc --auto"!).
2. In some cases, we may return a bogus exit code from
rebase, indicating failure when everything except the
auto-gc succeeded.
We can fix this by ignoring the exit code of "gc --auto".
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
receive-pack: release pack files before garbage-collecting
Before auto-gc'ing, we need to make sure that the pack files are
released in case they need to be repacked and garbage-collected.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Reviewed-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
merge: release pack files before garbage-collecting
Before auto-gc'ing, we need to make sure that the pack files are
released in case they need to be repacked and garbage-collected.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Reviewed-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Before auto-gc'ing, we need to make sure that the pack files are
released in case they need to be repacked and garbage-collected.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Reviewed-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
fetch: release pack files before garbage-collecting
Before auto-gc'ing, we need to make sure that the pack files are
released in case they need to be repacked and garbage-collected.
This fixes https://github.com/git-for-windows/git/issues/500
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Reviewed-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
This just makes things compile, the test suite needs extra tender loving
care in addition to this change. We will address these issues in later
commits.
While at it, also allow building MSys2 Git (i.e. a Git that uses MSys2's
POSIX emulation layer).
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
For a long time, Git for Windows lagged behind Git's 2.x releases because
the Git for Windows developers wanted to let that big jump coincide with
a well-needed jump away from MSys to MSys2.
To understand why this is such a big issue, it needs to be noted that
many parts of Git are not written in portable C, but instead Git relies
on a POSIX shell and Perl to be available.
To support the scripts, Git for Windows has to ship a minimal POSIX
emulation layer with Bash and Perl thrown in, and when the Git for
Windows effort started in August 2007, this developer settled on using
MSys, a stripped down version of Cygwin. Consequently, the original name
of the project was "msysGit" (which, sadly, caused a *lot* of confusion
because few Windows users know about MSys, and even less care).
To compile the C code of Git for Windows, MSys was used, too: it sports
two versions of the GNU C Compiler: one that links implicitly to the
POSIX emulation layer, and another one that targets the plain Win32 API
(with a few convenience functions thrown in). Git for Windows'
executables are built using the latter, and therefore they are really
just Win32 programs. To discern executables requiring the POSIX
emulation layer from the ones that do not, the latter are called MinGW
(Minimal GNU for Windows) when the former are called MSys executables.
This reliance on MSys incurred challenges, too, though: some of our
changes to the MSys runtime -- necessary to support Git for Windows
better -- were not accepted upstream, so we had to maintain our own
fork. Also, the MSys runtime was not developed further to support e.g.
UTF-8 or 64-bit, and apart from lacking a package management system
until much later (when mingw-get was introduced), many packages provided
by the MSys/MinGW project lag behind the respective source code
versions, in particular Bash and OpenSSL. For a while, the Git for
Windows project tried to remedy the situation by trying to build newer
versions of those packages, but the situation quickly became untenable,
especially with problems like the Heartbleed bug requiring swift action
that has nothing to do with developing Git for Windows further.
Happily, in the meantime the MSys2 project (https://msys2.github.io/)
emerged, and was chosen to be the base of the Git for Windows 2.x. Just
like MSys, MSys2 is a stripped down version of Cygwin, but it is
actively kept up-to-date with Cygwin's source code. Thereby, it already
supports Unicode internally, and it also offers the 64-bit support that
we yearned for since the beginning of the Git for Windows project.
MSys2 also ported the Pacman package management system from Arch Linux
and uses it heavily. This brings the same convenience to which Linux
users are used to from `yum` or `apt-get`, and to which MacOSX users are
used to from Homebrew or MacPorts, or BSD users from the Ports system,
to MSys2: a simple `pacman -Syu` will update all installed packages to
the newest versions currently available.
MSys2 is also *very* active, typically providing package updates
multiple times per week.
It still required a two-month effort to bring everything to a state
where Git's test suite passes, many more months until the first official
Git for Windows 2.x was released, and a couple of patches still await
their submission to the respective upstream projects. Yet without MSys2,
the modernization of Git for Windows would simply not have happened.
This commit lays the ground work to supporting MSys2-based Git builds.
Assisted-by: Waldek Maleska <weakcamel@users.github.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
We can pass -o/--output-directory to the format-patch command to store
patches in some place other than the working directory. This patch
introduces format.outputDirectory configuration option for same
purpose.
The case of usage of this configuration option can be convenience
to not pass every time -o/--output-directory if an user has pattern
to store all patches in the /patches directory for example.
The format.outputDirectory has lower priority than command line
option, so if user will set format.outputDirectory and pass the
command line option, a result will be stored in a directory that
passed to command line option.
Signed-off-by: Alexander Kuleshov <kuleshovmail@gmail.com> Signed-off-by: Stephen P. Smith <ischis2@cox.net> Reviewed-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
After changing the type of a file in the git repository, it is not possible to
"git p4 publish" the commit to perforce. This is due to the fact that the git
"T" status is not handled in git-p4.py. This can typically occur when replacing
an existing file with a symbolic link.
The "T" modifier is now supported in git-p4.py. When a file type has changed,
inform perforce with the "p4 edit -f auto" command.
Signed-off-by: Romain Picard <romain.picard@oakbits.com> Acked-by: Luke Diamand <luke@diamand.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
lock_ref_sha1_basic: handle REF_NODEREF with invalid refs
We sometimes call lock_ref_sha1_basic with REF_NODEREF
to operate directly on a symbolic ref. This is used, for
example, to move to a detached HEAD, or when updating
the contents of HEAD via checkout or symbolic-ref.
However, the first step of the function is to resolve the
refname to get the "old" sha1, and we do so without telling
resolve_ref_unsafe() that we are only interested in the
symref. As a result, we may detect a problem there not with
the symref itself, but with something it points to.
The real-world example I found (and what is used in the test
suite) is a HEAD pointing to a ref that cannot exist,
because it would cause a directory/file conflict with other
existing refs. This situation is somewhat broken, of
course, as trying to _commit_ on that HEAD would fail. But
it's not explicitly forbidden, and we should be able to move
away from it. However, neither "git checkout" nor "git
symbolic-ref" can do so. We try to take the lock on HEAD,
which is pointing to a non-existent ref. We bail from
resolve_ref_unsafe() with errno set to EISDIR, and the lock
code thinks we are attempting to create a d/f conflict.
Of course we're not. The problem is that the lock code has
no idea what level we were at when we got EISDIR, so trying
to diagnose or remove empty directories for HEAD is not
useful.
To make things even more complicated, we only get EISDIR in
the loose-ref case. If the refs are packed, the resolution
may "succeed", giving us the pointed-to ref in "refname",
but a null oid. Later, we say "ah, the null oid means we are
creating; let's make sure there is room for it", but
mistakenly check against the _resolved_ refname, not the
original.
We can fix this by making two tweaks:
1. Call resolve_ref_unsafe() with RESOLVE_REF_NO_RECURSE
when REF_NODEREF is set. This means any errors
we get will be from the orig_refname, and we can act
accordingly.
We already do this in the REF_DELETING case, but we
should do it for update, too.
2. If we do get a "refname" return from
resolve_ref_unsafe(), even with RESOLVE_REF_NO_RECURSE
it may be the name of the ref pointed-to by a symref.
We already normalize this back to orig_refname before
taking the lockfile, but we need to do so before the
null_oid check.
While we're rearranging the REF_NODEREF handling, we can
also bump the initialization of lflags to the top of the
function, where we are setting up other flags. This saves us
from having yet another conditional block on REF_NODEREF
just to set it later.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
lock_ref_sha1_basic: always fill old_oid while holding lock
Our basic strategy for taking a ref lock is:
1. Create $ref.lock to take the lock
2. Read the ref again while holding the lock (during which
time we know that nobody else can be updating it).
3. Compare the value we read to the expected "old_sha1"
The value we read in step (2) is returned to the caller via
the lock->old_oid field, who may use it for other purposes
(such as writing a reflog).
If we have no "old_sha1" (i.e., we are unconditionally
taking the lock), then we obviously must omit step 3. But we
_also_ omit step 2. This seems like a nice optimization, but
it means that the caller sees only whatever was left in
lock->old_oid from previous calls to resolve_ref_unsafe(),
which happened outside of the lock.
We can demonstrate this race pretty easily. Imagine you have
three commits, $one, $two, and $three. One script just flips
between $one and $two, without providing an old-sha1:
while true; do
git update-ref -m one refs/heads/foo $one
git update-ref -m two refs/heads/foo $two
done
Meanwhile, another script tries to set the value to $three,
also not using an old-sha1:
while true; do
git update-ref -m three refs/heads/foo $three
done
If these run simultaneously, we'll see a lot of lock
contention, but each of the writes will succeed some of the
time. The reflog may record movements between any of the
three refs, but we would expect it to provide a consistent
log: the "from" field of each log entry should be the same
as the "to" field of the previous one.
But if we check this:
perl -alne '
print "mismatch on line $."
if defined $last && $F[0] ne $last;
$last = $F[1];
' .git/logs/refs/heads/foo
we'll see many mismatches. Why?
Because sometimes, in the time between lock_ref_sha1_basic
filling lock->old_oid via resolve_ref_unsafe() and it taking
the lock, there may be a complete write by another process.
And the "from" field in our reflog entry will be wrong, and
will refer to an older value.
This is probably quite rare in practice. It requires writers
which do not provide an old-sha1 value, and it is a very
quick race. However, it is easy to fix: we simply perform
step (2), the read-under-lock, whether we have an old-sha1
or not. Then the value we hand back to the caller is always
atomic.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git grep" can now be configured (or told from the command line)
how many threads to use when searching in the working tree files.
* vl/grep-configurable-threads:
grep: add --threads=<num> option and grep.threads configuration
grep: slight refactoring to the code that disables threading
grep: allow threading even on a single-core machine