Improve error reporting from the http transfer clients.
* jk/http-error-messages:
http: drop http_error function
remote-curl: die directly with http error messages
http: re-word http error message
http: simplify http_error helper function
remote-curl: consistently report repo url for http errors
remote-curl: always show friendlier 404 message
remote-curl: let servers override http 404 advice
remote-curl: show server content on http errors
http: add HTTP_KEEP_ERROR option
Closing (not redirecting to /dev/null) the standard error stream is
not a very smart thing to do. Later open may return file
descriptor #2 for unrelated purpose, and error reporting code may
write into them.
* tr/perl-keep-stderr-open:
t9700: do not close STDERR
perl: redirect stderr to /dev/null instead of closing
test-bzr: portable shell and utf-8 strings for Mac OS
Make the shell script more portable:
- Split export X=Y into 2 lines
- Use printf instead of echo -e
Use UTF-8 code points which are not decomposed by the filesystem:
Code points like "á" will be decomposed by Mac OS X.
bzr is unable to find the file "á" on disk.
Use code points from unicode which can not be decomposed.
In other words, the precompsed form use the same bytes as decomposed.
Signed-off-by: Torsten Bögershausen <tboegi@web.de> Acked-by: Felipe Contreras <felipe.contreras@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* maint:
Correct common spelling mistakes in comments and tests
kwset: fix spelling in comments
precompose-utf8: fix spelling of "want" in error message
compat/nedmalloc: fix spelling in comments
compat/regex: fix spelling and grammar in comments
obstack: fix spelling of similar
contrib/subtree: fix spelling of accidentally
git-remote-mediawiki: spelling fixes
doc: various spelling fixes
fast-export: fix argument name in error messages
Documentation: distinguish between ref and offset deltas in pack-format
i18n: make the translation of -u advice in one go
Merge branch 'rr/send-email-perl-critique' into maint
* rr/send-email-perl-critique:
send-email: use the three-arg form of open in recipients_cmd
send-email: drop misleading function prototype
send-email: use "return;" not "return undef;" on error codepaths
Most of these were found using Lucas De Marchi's codespell tool.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Documentation: distinguish between ref and offset deltas in pack-format
eb32d236 introduced the OBJ_OFS_DELTA object that uses a relative offset to
identify the base object instead of the 20-byte SHA1 reference. The pack file
documentation only mentions the SHA1 based reference in its description of the
deltified object entry.
Update the pack format documentation to clarify that the deltified object
representation refers to its base using either a relative negative offset or
the absolute SHA1 identifier.
Signed-off-by: Stefan Saasen <ssaasen@atlassian.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
The advice (consider use of -u when read_directory takes too long) is
separated into 3 different status_printf_ln() calls, and which brings
trouble for translators.
Since status_vprintf() called by status_printf_ln() can handle eol in
buffer, we could simply join these lines into one paragraph.
Signed-off-by: Jiang Xin <worldhello.net@gmail.com> Reviewed-by: Eric Sunshine <sunshine@sunshineco.com> Reviewed-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* maint:
Typo fix: replacing it's -> its
t: make PIPE a standard test prerequisite
archive: clarify explanation of --worktree-attributes
t/README: --immediate skips cleanup commands for failed tests
"git help" learned "-g" option to show the list of guides just like
list of commands are given with "-a".
* po/help-guides:
doc: include --guide option description for "git help"
help: mention -a and -g option, and 'git help <concept>' usage.
builtin/help.c: add list_common_guides_help() function
builtin/help.c: add --guide option
builtin/help.c: split "-a" processing into two
archive: clarify explanation of --worktree-attributes
Make it a bit clearer that --worktree-attributes is about files in the
working tree (checked out files, possibly changed) and not the current
working directory ($PWD). Link to the ATTRIBUTES section, which has
more details.
Reported-by: Amit Bakshi <ambakshi@gmail.com> Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx> Signed-off-by: Junio C Hamano <gitster@pobox.com>
A handful of test cases and a corner case bugfix for "git rm".
* jk/rm-removed-paths:
t3600: document failure of rm across symbolic links
t3600: test behavior of reverse-d/f conflict
rm: do not complain about d/f conflicts during deletion
* fc/remote-helpers-test-updates:
remote-hg: fix hg-git test-case
remote-bzr: remove stale check code for tests
remote-helpers: fix the run of all tests
remote-bzr: avoid echo -n
Strip @anchor elements in the texinfo output of the documentation,
as a single document created by concatenating our entire manual set
will produce many duplicates that makes newer texinfo unhappy.
* mg/texinfo-5:
Documentation: Strip texinfo anchors to avoid duplicates
Support "pull from one place, push to another place" workflow
better by introducing remote.pushdefault (overrides the "origin"
thing) and branch.*.pushremote (overrides the branch.*.remote).
* rr/triangle:
remote.c: introduce branch.<name>.pushremote
remote.c: introduce remote.pushdefault
remote.c: introduce a way to have different remotes for fetch/push
t5516 (fetch-push): drop implicit arguments from helper functions
t5516 (fetch-push): update test description
remote.c: simplify a bit of code using git_config_string()
The handing by "git branch --set-upstream-to" against various forms
of errorneous inputs were suboptimal.
* jk/set-upstream-error-cases:
branch: give advice when tracking start-point is missing
branch: mention start_name in set-upstream error messages
branch: improve error message for missing --set-upstream-to ref
branch: factor out "upstream is not a branch" error messages
t3200: test --set-upstream-to with bogus refs
When used with "-d temporary-directory" option, "git filter-branch"
failed to come back to the original working tree to perform the
final clean-up procedure.
* jk/filter-branch-come-back-to-original:
filter-branch: return to original dir after filtering
Merge branch 'jc/directory-attrs-regression-fix' into maint-1.8.1
A pattern "dir" (without trailing slash) in the attributes file
stopped matching a directory "dir" by mistake with an earlier change
that wanted to allow pattern "dir/" to also match.
* jc/directory-attrs-regression-fix:
t: check that a pattern without trailing slash matches a directory
dir.c::match_pathname(): pay attention to the length of string parameters
dir.c::match_pathname(): adjust patternlen when shifting pattern
dir.c::match_basename(): pay attention to the length of string parameters
attr.c::path_matches(): special case paths that end with a slash
attr.c::path_matches(): the basename is part of the pathname
The documentation says that "If no 'refspec' capability is advertised,
there is an implied `refspec *:*`" but this is only the case for the
"import" command.
Since there is a comment in transport-helper.c indicating that this
default is for historical reasons, change the documentation to clarify
that a refspec should always be specified.
Signed-off-by: John Keeping <john@keeping.me.uk> Acked-by: Sverre Rabbelier <srabbelier@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
This might fail if some references have been removed by prune
because some marks will refer to no longer existing commits.
git-fast-export will not need these objects anyway as they were no
longer reachable.
We still need to update last_numid so we don't change the mapping
between marks and objects for remote-helpers.
Unfortunately, the mark file should not be rewritten without lost marks
if no new objects has been exported, as we could lose track of the last
last_numid.
Signed-off-by: Antoine Pelisse <apelisse@gmail.com> Reviewed-by: Felipe Contreras <felipe.contreras@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
remote-curl: die directly with http error messages
When we encounter an unknown http error (e.g., a 403), we
hand the error code to http_error, which then prints it with
error(). After that we die with the redundant message "HTTP
request failed".
Instead, let's just drop http_error entirely, which does
nothing but pass arguments to error(), and instead die
directly with a useful message.
So before:
$ git clone https://example.com/repo.git
Cloning into 'repo'...
error: unable to access 'https://example.com/repo.git': The requested URL returned error: 403 Forbidden
fatal: HTTP request failed
and after:
$ git clone https://example.com/repo.git
Cloning into 'repo'...
fatal: unable to access 'https://example.com/repo.git': The requested URL returned error: 403 Forbidden
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
When we report an http error code, we say something like:
error: The requested URL reported failure: 403 Forbidden while accessing http://example.com/repo.git
Everything between "error:" and "while" is written by curl,
and the resulting sentence is hard to read (especially
because there is no punctuation between curl's sentence and
the remainder of ours). Instead, let's re-order this to give
better flow:
error: unable to access 'http://example.com/repo.git: The requested URL reported failure: 403 Forbidden
This is still annoyingly long, but at least reads more
clearly left to right.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
This helper function should really be a one-liner that
prints an error message, but it has ended up unnecessarily
complicated:
1. We call error() directly when we fail to start the curl
request, so we must later avoid printing a duplicate
error in http_error().
It would be much simpler in this case to just stuff the
error message into our usual curl_errorstr buffer
rather than printing it ourselves. This means that
http_error does not even have to care about curl's exit
value (the interesting part is in the errorstr buffer
already).
2. We return the "ret" value passed in to us, but none of
the callers actually cares about our return value. We
can just drop this entirely.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
remote-curl: consistently report repo url for http errors
When we report http errors in fetching the initial ref
advertisement, we show the full URL we attempted to use,
including "info/refs?service=git-upload-pack". While this
may be useful for debugging a broken server, it is
unnecessarily verbose and confusing for most cases, in which
the client user is not even the same person as the owner of
the repository.
Let's just show the repository URL; debugging can happen
with GIT_CURL_VERBOSE, which shows way more useful
information, anyway.
At the same time, let's also make sure to mention the
repository URL when we report failed authentication
(previously we said only "Authentication failed"). Knowing
the URL can help the user realize why authentication failed
(e.g., they meant to push to remote A, not remote B).
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
When we get an http 404 trying to get the initial list of
refs from the server, we try to be helpful and remind the
user that update-server-info may need to be run. This looks
like:
$ git clone https://github.com/non/existent
Cloning into 'existent'...
fatal: https://github.com/non/existent/info/refs?service=git-upload-pack not found: did you run git update-server-info on the server?
Suggesting update-server-info may be a good suggestion for
users who are in control of the server repo and who are
planning to set up dumb http. But for users of smart http,
and especially users who are not in control of the server
repo, the advice is useless and confusing.
Since most people are expected to use smart http these days,
it does not make sense to keep the update-server-info hint.
We not only drop the mention of update-server-info, but also
show only the main repo URL, not the full "info/refs" and
service parameter. These elements may be useful for
debugging a broken server configuration, but in the majority
of cases, users are not fetching from their own
repositories, but rather from other people's repositories;
they have neither the power nor interest to fix a broken
configuration, and the extra components just make the
message more confusing. Users who do want to debug can and
should use GIT_CURL_VERBOSE to get more complete information
on the actual URLs visited.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
When we get an http 404 trying to get the initial list of
refs from the server, we try to be helpful and remind the
user that update-server-info may need to be run. This looks
like:
$ git clone https://github.com/non/existent
Cloning into 'existent'...
fatal: https://github.com/non/existent/info/refs?service=git-upload-pack not found: did you run git update-server-info on the server?
Suggesting update-server-info may be a good suggestion for
users who are in control of the server repo and who are
planning to set up dumb http. But for users of smart http,
and especially users who are not in control of the server
repo, the advice is useless and confusing.
The previous patch taught remote-curl to show custom advice
from the server when it is available. When we have shown
messages from the server, we can also drop our custom
advice; what the server has to say is likely to be more
accurate and helpful.
We not only drop the mention of update-server-info, but also
show only the main repo URL, not the full "info/refs" and
service parameter. These elements may be useful for
debugging a broken server configuration, but again, anything
the server has provided is likely to be more useful (and one
can still use GIT_CURL_VERBOSE to get much more complete
debugging information).
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
If an http request to a remote git server fails, we show
only the http response code, or sometimes a custom message
for particular codes. This gives the server no opportunity
to offer a more detailed explanation of the reason for the
failure, or to give extra advice.
This patch teaches remote-curl to record and display the
body content of a failed http response. We only display such
responses when the content-type is advertised as text/plain,
as it is the most likely to look presentable on the user's
terminal (and it is hoped to be a good indication that the
message is intended for git clients, and not for a web
browser).
Each line of the new output is prepended with "remote:".
Example output may look like this (assuming the server is
configured to display such a helpful message):
$ GIT_SMART_HTTP=0 git clone https://example.com/some/repo.git
Cloning into 'repo'...
remote: Sorry, fetching via dumb http is forbidden.
remote: Please upgrade your git client to v1.6.6 or greater
remote: and make sure that smart-http is enabled.
error: The requested URL returned error: 403 while accessing http://localhost:5001/some/repo.git/info/refs
fatal: HTTP request failed
For the sake of simplicity, we only record and display these
errors during the initial fetch of the ref list, as that is
the initial contact with the server and where the most
common, interesting errors happen (and there is already
precedent, as that is the only place we currently massage
http error codes into more helpful messages).
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
We currently set curl's FAILONERROR option, which means that
any http failures are reported as curl errors, and the
http body content from the server is thrown away.
This patch introduces a new option to http_get_strbuf which
specifies that the body content from a failed http response
should be placed in the destination strbuf, where it can be
accessed by the caller.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Teach "merge/pull" to optionally verify and reject commits that are
not signed properly.
* sg/gpg-sig:
pretty printing: extend %G? to include 'N' and 'U'
merge/pull Check for untrusted good GPG signatures
merge/pull: verify GPG signatures of commits being merged
commit.c/GPG signature verification: Also look at the first GPG status line
Move commit GPG signature verification to commit.c
Update "git send-email" for issues noticed by PerlCritic.
* rr/send-email-perl-critique:
send-email: use the three-arg form of open in recipients_cmd
send-email: drop misleading function prototype
send-email: use "return;" not "return undef;" on error codepaths
"git merge $(git rev-parse v1.8.2)" behaved quite differently from
"git merge v1.8.2" as if v1.8.2 were written as v1.8.2^0 and did
not pay much attention to the annotated tag payload.
This makes the code notice the type of the tag object, in addition
to the dwim_ref() based classification the current code uses
(i.e. the name appears in refs/tags/) to decide when to special
case merging of tags.
* jc/merge-tag-object:
t6200: test message for merging of an annotated tag
t6200: use test_config/test_unconfig
merge: a random object may not necssarily be a commit
All calls to set_shared_perm() use mode == 0, so simplify the
function.
Because all callers use the macro adjust_shared_perm(path) from
cache.h to call this function, convert it to a proper function,
losing set_shared_perm().
Since path.c has much more functions than just mkpath() these days,
drop the stale comment about it.
Signed-off-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
The logic flow of has_changes() used for "log -S" and diff_grep()
used for "log -G" are essentially the same. See if we have both
sides that could be different in any interesting way, slurp the
contents in core, possibly after applying textconv, inspect the
contents, clean-up and report the result. The only difference
between the two is how "inspect" step works.
Unify this codeflow in a helper, pickaxe_match(), which takes a
callback function that implements the specific "inspect" step.
After removing the common scaffolding code from the existing
has_changes() and diff_grep(), they each becomes such a callback
function suitable for passing to pickaxe_match().
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
diffcore-pickaxe: fix leaks in "log -S<block>" and "log -G<pattern>"
The diff_grep() and has_changes() functions had early return
codepaths for unmerged filepairs, which simply returned 0. When we
taught textconv filter to them, one was ignored and continued to
return early without freeing the result filtered by textconv, and
the other had a failed attempt to fix, which allowed the planned
return value 0 to be overwritten by a bogus call to contains().
diffcore-pickaxe: port optimization from has_changes() to diff_grep()
These two functions are called in the same codeflow to implement
"log -S<block>" and "log -G<pattern>", respectively, but the latter
lacked two obvious optimizations the former implemented, namely:
- When a pickaxe limit is not given at all, they should return
without wasting any cycle;
- When both sides of the filepair are the same, and the same
textconv conversion apply to them, return early, as there will be
no interesting differences between the two anyway.
Also release the filespec data once the processing is done (this is
not about leaking memory--it is about releasing data we finished
looking at as early as possible).
$ echo '*.txt diff=wrong' > .gitattributes
$ git -c diff.wrong.textconv='xxx' log --no-textconv -Sfoo
error: cannot run xxx: No such file or directory
fatal: unable to read files to diff
Reported-by: Matthieu Moy <Matthieu.Moy@grenoble-inp.fr> Signed-off-by: Simon Ruderich <simon@ruderich.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
t3600: document failure of rm across symbolic links
If we have a symlink "d" that points to a directory, we
should not be able to remove "d/f". In the normal case,
where "d/f" does not exist in the index, we already disallow
this, as we only remove things that git knows about in the
index. So for something like:
we will properly produce an error (as there is no index
entry for foo/bar). However, if there is an index entry for
the path (e.g., because the movement is due to working tree
changes that have not yet been reflected in the index), we
will happily delete it, even though the path we delete from the
filesystem is not the same as the path in the index.
This patch documents that failure with a test.
While this is a bug, it should not be possible to cause
serious data loss with it. For any path that does not have
an index entry, we will complain and bail. For a path which
does have an index entry, we will do the usual up-to-date
content check. So even if the deleted path in the filesystem
is not the same as the one we are removing from the index,
we do know that they at least have the same content, and
that the content is included in HEAD.
That means the worst case is not the accidental loss of
content, but rather confusion by the user when a copy of a
file another part of the tree is removed. Which makes this
bug a minor and hard-to-trigger annoyance rather than a
data-loss bug (and hence the fix can be saved for a rainy
day when somebody feels like working on it).
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
fill_one is _almost_ identical to just calling fill_textconv; the
exception is that for the !DIFF_FILE_VALID case, fill_textconv gives us
an empty buffer rather than a NULL one. Since we currently use the NULL
pointer as a signal that the file is not present on one side of the
diff, we must now switch to using DIFF_FILE_VALID to make the same
check.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Simon Ruderich <simon@ruderich.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
diffcore-pickaxe: remove unnecessary call to get_textconv()
The fill_one() function is responsible for finding and filling the
textconv filter as necessary, and is called by diff_grep() function
that implements "git log -G<pattern>".
The has_changes() function that implements "git log -S<block>" calls
get_textconv() for two sides being compared, before it checks to see
if it was asked to perform the pickaxe limiting. Move the code
around to avoid this wastage.
After has_changes() calls get_textconv() to obtain textconv for both
sides, fill_one() is called to use them.
By adding get_textconv() to diff_grep() and relieving fill_one() of
responsibility to find the textconv filter, we can avoid calling
get_textconv() twice in has_changes().
With this change it's also no longer necessary for fill_one() to
modify the textconv argument, therefore pass a pointer instead of a
pointer to a pointer.
Signed-off-by: Simon Ruderich <simon@ruderich.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Much like the previous patch, this triggered an unrelated bug.
Closing STDERR is not worth it anyway, as we risk writing die() and
such to random files that happen to be subsequently opened on FD 2.
Don't do it.
Signed-off-by: Thomas Rast <trast@inf.ethz.ch> Reviewed-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
perl: redirect stderr to /dev/null instead of closing
On my system, t9100.1 triggers the following warning:
==352== Syscall param write(buf) points to uninitialised byte(s)
==352== at 0x57119C0: __write_nocancel (in /lib64/libc-2.17.so)
==352== by 0x56AC1D2: _IO_file_write@@GLIBC_2.2.5 (in /lib64/libc-2.17.so)
==352== by 0x56AC0B1: new_do_write (in /lib64/libc-2.17.so)
==352== by 0x56AD3B4: _IO_do_write@@GLIBC_2.2.5 (in /lib64/libc-2.17.so)
==352== by 0x56AD6FE: _IO_file_overflow@@GLIBC_2.2.5 (in /lib64/libc-2.17.so)
==352== by 0x56AE3D8: _IO_default_xsputn (in /lib64/libc-2.17.so)
==352== by 0x56ACAA2: _IO_file_xsputn@@GLIBC_2.2.5 (in /lib64/libc-2.17.so)
==352== by 0x5682133: buffered_vfprintf (in /lib64/libc-2.17.so)
==352== by 0x567CE9D: vfprintf (in /lib64/libc-2.17.so)
==352== by 0x5687096: fprintf (in /lib64/libc-2.17.so)
==352== by 0x4E7AC5: vreportf (usage.c:15)
==352== by 0x4E7B14: die_builtin (usage.c:38)
The actual complaint appears to be a bug in the underlying
implementation. What's interesting here is that it is apparently
_triggered_ by closing stderr, which results in (from strace)
Closing stderr is a bad idea anyway: there is a very real chance that
we print fatal error messages to some other file that just happens to
be opened on the now-free FD 2. So let's not do that.
As pointed out by Eric Wong (thanks), the initial close needs to go:
die() would again write nowhere if we close STDERR beforehand.
Signed-off-by: Thomas Rast <trast@inf.ethz.ch> Signed-off-by: Eric Wong <normalperson@yhbt.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>