dir.c: git-status --ignored: don't scan the work tree twice
'git-status --ignored' still scans the work tree twice to collect
untracked and ignored files, respectively.
fill_directory / read_directory already supports collecting untracked and
ignored files in a single directory scan. However, the DIR_COLLECT_IGNORED
flag to enable this has some git-add specific side-effects (e.g. it
doesn't recurse into ignored directories, so listing ignored files with
--untracked=all doesn't work).
The DIR_SHOW_IGNORED flag doesn't list untracked files and returns ignored
files in dir_struct.entries[] (instead of dir_struct.ignored[] as
DIR_COLLECT_IGNORED). DIR_SHOW_IGNORED is used all throughout git.
We don't want to break the existing API, so lets introduce a new flag
DIR_SHOW_IGNORED_TOO that lists untracked as well as ignored files similar
to DIR_COLLECT_FILES, but will recurse into sub-directories based on the
other flags as DIR_SHOW_IGNORED does.
In dir.c::read_directory_recursive, add ignored files to either
dir_struct.entries[] or dir_struct.ignored[] based on the flags. Also move
the DIR_COLLECT_IGNORED case here so that filling result lists is in a
common place.
In wt-status.c::wt_status_collect_untracked, use the new flag and read
results from dir_struct.ignored[]. Remove the extra fill_directory call.
builtin/check-ignore.c doesn't call fill_directory, setting the git-add
specific DIR_COLLECT_IGNORED flag has no effect here. Remove for clarity.
Update API documentation to reflect the changes.
Performance: with this patch, 'git-status --ignored' is typically as fast
as 'git-status'.
Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
dir.c: git-status --ignored: don't scan the work tree three times
'git-status --ignored' recursively scans directories up to three times:
1. To collect untracked files.
2. To collect ignored files.
3. When collecting ignored files, to check that an untracked directory
that potentially contains ignored files doesn't also contain untracked
files (i.e. isn't already listed as untracked).
Let's get rid of case 3 first.
Currently, read_directory_recursive returns a boolean whether a directory
contains the requested files or not (actually, it returns the number of
files, but no caller actually needs that), and DIR_SHOW_IGNORED specifies
what we're looking for.
To be able to test for both untracked and ignored files in a single scan,
we need to return a bit more info, and the result must be independent of
the DIR_SHOW_IGNORED flag.
Reuse the path_treatment enum as return value of read_directory_recursive.
Split path_handled in two separate values path_excluded and path_untracked
that don't change their meaning with the DIR_SHOW_IGNORED flag. We don't
need an extra value path_untracked_and_excluded, as directories with both
untracked and ignored files should be listed as untracked.
Rename path_ignored to path_none for clarity (i.e. "don't treat that path"
in contrast to "the path is ignored and should be treated according to
DIR_SHOW_IGNORED").
Replace enum directory_treatment with path_treatment. That's just another
enum with the same meaning, no need to translate back and forth.
In treat_directory, get rid of the extra read_directory_recursive call and
all the DIR_SHOW_IGNORED-specific code.
In read_directory_recursive, decide whether to dir_add_name path_excluded
or path_untracked paths based on the DIR_SHOW_IGNORED flag.
The return value of read_directory_recursive is the maximum path_treatment
of all files and sub-directories. In the check_only case, abort when we've
reached the most significant value (path_untracked).
Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
dir.c: unify is_excluded and is_path_excluded APIs
The is_excluded and is_path_excluded APIs are very similar, except for a
few noteworthy differences:
is_excluded doesn't handle ignored directories, results for paths within
ignored directories are incorrect. This is probably based on the premise
that recursive directory scans should stop at ignored directories, which
is no longer true (in certain cases, read_directory_recursive currently
calls is_excluded *and* is_path_excluded to get correct ignored state).
is_excluded caches parsed .gitignore files of the last directory in struct
dir_struct. If the directory changes, it finds a common parent directory
and is very careful to drop only as much state as necessary. On the other
hand, is_excluded will also read and parse .gitignore files in already
ignored directories, which are completely irrelevant.
is_path_excluded correctly handles ignored directories by checking if any
component in the path is excluded. As it uses is_excluded internally, this
unfortunately forces is_excluded to drop and re-read all .gitignore files,
as there is no common parent directory for the root dir.
is_path_excluded tracks state in a separate struct path_exclude_check,
which is essentially a wrapper of dir_struct with two more fields. However,
as is_path_excluded also modifies dir_struct, it is not possible to e.g.
use multiple path_exclude_check structures with the same dir_struct in
parallel. The additional structure just unnecessarily complicates the API.
Teach is_excluded / prep_exclude about ignored directories: whenever
entering a new directory, first check if the entire directory is excluded.
Remember the excluded state in dir_struct. Don't traverse into already
ignored directories (i.e. don't read irrelevant .gitignore files).
Directories could also be excluded by exclude patterns specified on the
command line or .git/info/exclude, so we cannot simply skip prep_exclude
entirely if there's no .gitignore file name (dir_struct.exclude_per_dir).
Move this check to just before actually reading the file.
is_path_excluded is now equivalent to is_excluded, so we can simply
redirect to it (the public API is cleaned up in the next patch).
The performance impact of the additional ignored check per directory is
hardly noticeable when reading directories recursively (e.g. 'git status').
However, performance of git commands using the is_path_excluded API (e.g.
'git ls-files --cached --ignored --exclude-standard') is greatly improved
as this no longer re-reads .gitignore files on each call.
Here's some performance data from the linux and WebKit repos (best of 10
runs on a Debian Linux on SSD, core.preloadIndex=true):
| ls-files -ci | status | status --ignored
| linux | WebKit | linux | WebKit | linux | WebKit
-------+-------+--------+-------+--------+-------+---------
before | 0.506 | 6.539 | 0.212 | 1.555 | 0.323 | 2.541
after | 0.080 | 1.191 | 0.218 | 1.583 | 0.321 | 2.579
gain | 6.325 | 5.490 | 0.972 | 0.982 | 1.006 | 0.985
Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
- git-status --ignored: lists ignored tracked directories while contained
files may be listed as modified. Paths listed by git-status should be
disjoint (except in long format where a path may be listed in both the
staged and unstaged section).
Additionally, the current behaviour violates documentation in gitignore(5)
("Specifies intentionally *untracked* files to ignore") and Documentation/
technical/api-directory-listing.txt ("DIR_SHOW_OTHER_DIRECTORIES: Include
a directory that is *not tracked*.").
In dir.c::treat_directory, remove the special handling of ignored tracked
directories, so that the DIR_SHOW_OTHER_DIRECTORIES flag only affects
"other" (i.e. untracked) directories. In dir.c::dir_add_name, check that
added paths are untracked even if DIR_SHOW_IGNORED is set.
Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
dir.c: make 'git-status --ignored' work within leading directories
'git-status --ignored path/' doesn't list ignored files and directories
within 'path' if some component of 'path' is classified as untracked.
Disable the DIR_SHOW_OTHER_DIRECTORIES flag while traversing leading
directories. This prevents treat_leading_path() with DIR_SHOW_IGNORED flag
from aborting at the top level untracked directory.
As a side effect, this also eliminates a recursive directory scan per
leading directory level, as treat_directory() can no longer call
read_directory_recursive() when called from treat_leading_path().
Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
dir.c: git-status --ignored: don't list empty directories as ignored
'git-status --ignored' lists empty untracked directories as ignored, even
though they don't have any ignored files.
When checking if a directory is already listed as untracked (i.e. shouldn't
be listed as ignored as well), don't assume that the directory has only
ignored files if it doesn't have untracked files, as the directory may be
empty.
Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
test-bzr: portable shell and utf-8 strings for Mac OS
Make the shell script more portable:
- Split export X=Y into 2 lines
- Use printf instead of echo -e
Use UTF-8 code points which are not decomposed by the filesystem:
Code points like "á" will be decomposed by Mac OS X.
bzr is unable to find the file "á" on disk.
Use code points from unicode which can not be decomposed.
In other words, the precompsed form use the same bytes as decomposed.
Signed-off-by: Torsten Bögershausen <tboegi@web.de> Acked-by: Felipe Contreras <felipe.contreras@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* maint:
Correct common spelling mistakes in comments and tests
kwset: fix spelling in comments
precompose-utf8: fix spelling of "want" in error message
compat/nedmalloc: fix spelling in comments
compat/regex: fix spelling and grammar in comments
obstack: fix spelling of similar
contrib/subtree: fix spelling of accidentally
git-remote-mediawiki: spelling fixes
doc: various spelling fixes
fast-export: fix argument name in error messages
Documentation: distinguish between ref and offset deltas in pack-format
i18n: make the translation of -u advice in one go
Merge branch 'rr/send-email-perl-critique' into maint
* rr/send-email-perl-critique:
send-email: use the three-arg form of open in recipients_cmd
send-email: drop misleading function prototype
send-email: use "return;" not "return undef;" on error codepaths
Most of these were found using Lucas De Marchi's codespell tool.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Documentation: distinguish between ref and offset deltas in pack-format
eb32d236 introduced the OBJ_OFS_DELTA object that uses a relative offset to
identify the base object instead of the 20-byte SHA1 reference. The pack file
documentation only mentions the SHA1 based reference in its description of the
deltified object entry.
Update the pack format documentation to clarify that the deltified object
representation refers to its base using either a relative negative offset or
the absolute SHA1 identifier.
Signed-off-by: Stefan Saasen <ssaasen@atlassian.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
The advice (consider use of -u when read_directory takes too long) is
separated into 3 different status_printf_ln() calls, and which brings
trouble for translators.
Since status_vprintf() called by status_printf_ln() can handle eol in
buffer, we could simply join these lines into one paragraph.
Signed-off-by: Jiang Xin <worldhello.net@gmail.com> Reviewed-by: Eric Sunshine <sunshine@sunshineco.com> Reviewed-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* maint:
Typo fix: replacing it's -> its
t: make PIPE a standard test prerequisite
archive: clarify explanation of --worktree-attributes
t/README: --immediate skips cleanup commands for failed tests
"git help" learned "-g" option to show the list of guides just like
list of commands are given with "-a".
* po/help-guides:
doc: include --guide option description for "git help"
help: mention -a and -g option, and 'git help <concept>' usage.
builtin/help.c: add list_common_guides_help() function
builtin/help.c: add --guide option
builtin/help.c: split "-a" processing into two
archive: clarify explanation of --worktree-attributes
Make it a bit clearer that --worktree-attributes is about files in the
working tree (checked out files, possibly changed) and not the current
working directory ($PWD). Link to the ATTRIBUTES section, which has
more details.
Reported-by: Amit Bakshi <ambakshi@gmail.com> Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx> Signed-off-by: Junio C Hamano <gitster@pobox.com>
A handful of test cases and a corner case bugfix for "git rm".
* jk/rm-removed-paths:
t3600: document failure of rm across symbolic links
t3600: test behavior of reverse-d/f conflict
rm: do not complain about d/f conflicts during deletion
* fc/remote-helpers-test-updates:
remote-hg: fix hg-git test-case
remote-bzr: remove stale check code for tests
remote-helpers: fix the run of all tests
remote-bzr: avoid echo -n
Strip @anchor elements in the texinfo output of the documentation,
as a single document created by concatenating our entire manual set
will produce many duplicates that makes newer texinfo unhappy.
* mg/texinfo-5:
Documentation: Strip texinfo anchors to avoid duplicates
Support "pull from one place, push to another place" workflow
better by introducing remote.pushdefault (overrides the "origin"
thing) and branch.*.pushremote (overrides the branch.*.remote).
* rr/triangle:
remote.c: introduce branch.<name>.pushremote
remote.c: introduce remote.pushdefault
remote.c: introduce a way to have different remotes for fetch/push
t5516 (fetch-push): drop implicit arguments from helper functions
t5516 (fetch-push): update test description
remote.c: simplify a bit of code using git_config_string()
The handing by "git branch --set-upstream-to" against various forms
of errorneous inputs were suboptimal.
* jk/set-upstream-error-cases:
branch: give advice when tracking start-point is missing
branch: mention start_name in set-upstream error messages
branch: improve error message for missing --set-upstream-to ref
branch: factor out "upstream is not a branch" error messages
t3200: test --set-upstream-to with bogus refs
When used with "-d temporary-directory" option, "git filter-branch"
failed to come back to the original working tree to perform the
final clean-up procedure.
* jk/filter-branch-come-back-to-original:
filter-branch: return to original dir after filtering
Merge branch 'jc/directory-attrs-regression-fix' into maint-1.8.1
A pattern "dir" (without trailing slash) in the attributes file
stopped matching a directory "dir" by mistake with an earlier change
that wanted to allow pattern "dir/" to also match.
* jc/directory-attrs-regression-fix:
t: check that a pattern without trailing slash matches a directory
dir.c::match_pathname(): pay attention to the length of string parameters
dir.c::match_pathname(): adjust patternlen when shifting pattern
dir.c::match_basename(): pay attention to the length of string parameters
attr.c::path_matches(): special case paths that end with a slash
attr.c::path_matches(): the basename is part of the pathname
The documentation says that "If no 'refspec' capability is advertised,
there is an implied `refspec *:*`" but this is only the case for the
"import" command.
Since there is a comment in transport-helper.c indicating that this
default is for historical reasons, change the documentation to clarify
that a refspec should always be specified.
Signed-off-by: John Keeping <john@keeping.me.uk> Acked-by: Sverre Rabbelier <srabbelier@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
This might fail if some references have been removed by prune
because some marks will refer to no longer existing commits.
git-fast-export will not need these objects anyway as they were no
longer reachable.
We still need to update last_numid so we don't change the mapping
between marks and objects for remote-helpers.
Unfortunately, the mark file should not be rewritten without lost marks
if no new objects has been exported, as we could lose track of the last
last_numid.
Signed-off-by: Antoine Pelisse <apelisse@gmail.com> Reviewed-by: Felipe Contreras <felipe.contreras@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Teach "merge/pull" to optionally verify and reject commits that are
not signed properly.
* sg/gpg-sig:
pretty printing: extend %G? to include 'N' and 'U'
merge/pull Check for untrusted good GPG signatures
merge/pull: verify GPG signatures of commits being merged
commit.c/GPG signature verification: Also look at the first GPG status line
Move commit GPG signature verification to commit.c
Update "git send-email" for issues noticed by PerlCritic.
* rr/send-email-perl-critique:
send-email: use the three-arg form of open in recipients_cmd
send-email: drop misleading function prototype
send-email: use "return;" not "return undef;" on error codepaths
"git merge $(git rev-parse v1.8.2)" behaved quite differently from
"git merge v1.8.2" as if v1.8.2 were written as v1.8.2^0 and did
not pay much attention to the annotated tag payload.
This makes the code notice the type of the tag object, in addition
to the dwim_ref() based classification the current code uses
(i.e. the name appears in refs/tags/) to decide when to special
case merging of tags.
* jc/merge-tag-object:
t6200: test message for merging of an annotated tag
t6200: use test_config/test_unconfig
merge: a random object may not necssarily be a commit
All calls to set_shared_perm() use mode == 0, so simplify the
function.
Because all callers use the macro adjust_shared_perm(path) from
cache.h to call this function, convert it to a proper function,
losing set_shared_perm().
Since path.c has much more functions than just mkpath() these days,
drop the stale comment about it.
Signed-off-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
The logic flow of has_changes() used for "log -S" and diff_grep()
used for "log -G" are essentially the same. See if we have both
sides that could be different in any interesting way, slurp the
contents in core, possibly after applying textconv, inspect the
contents, clean-up and report the result. The only difference
between the two is how "inspect" step works.
Unify this codeflow in a helper, pickaxe_match(), which takes a
callback function that implements the specific "inspect" step.
After removing the common scaffolding code from the existing
has_changes() and diff_grep(), they each becomes such a callback
function suitable for passing to pickaxe_match().
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
diffcore-pickaxe: fix leaks in "log -S<block>" and "log -G<pattern>"
The diff_grep() and has_changes() functions had early return
codepaths for unmerged filepairs, which simply returned 0. When we
taught textconv filter to them, one was ignored and continued to
return early without freeing the result filtered by textconv, and
the other had a failed attempt to fix, which allowed the planned
return value 0 to be overwritten by a bogus call to contains().
diffcore-pickaxe: port optimization from has_changes() to diff_grep()
These two functions are called in the same codeflow to implement
"log -S<block>" and "log -G<pattern>", respectively, but the latter
lacked two obvious optimizations the former implemented, namely:
- When a pickaxe limit is not given at all, they should return
without wasting any cycle;
- When both sides of the filepair are the same, and the same
textconv conversion apply to them, return early, as there will be
no interesting differences between the two anyway.
Also release the filespec data once the processing is done (this is
not about leaking memory--it is about releasing data we finished
looking at as early as possible).
$ echo '*.txt diff=wrong' > .gitattributes
$ git -c diff.wrong.textconv='xxx' log --no-textconv -Sfoo
error: cannot run xxx: No such file or directory
fatal: unable to read files to diff
Reported-by: Matthieu Moy <Matthieu.Moy@grenoble-inp.fr> Signed-off-by: Simon Ruderich <simon@ruderich.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
t3600: document failure of rm across symbolic links
If we have a symlink "d" that points to a directory, we
should not be able to remove "d/f". In the normal case,
where "d/f" does not exist in the index, we already disallow
this, as we only remove things that git knows about in the
index. So for something like:
we will properly produce an error (as there is no index
entry for foo/bar). However, if there is an index entry for
the path (e.g., because the movement is due to working tree
changes that have not yet been reflected in the index), we
will happily delete it, even though the path we delete from the
filesystem is not the same as the path in the index.
This patch documents that failure with a test.
While this is a bug, it should not be possible to cause
serious data loss with it. For any path that does not have
an index entry, we will complain and bail. For a path which
does have an index entry, we will do the usual up-to-date
content check. So even if the deleted path in the filesystem
is not the same as the one we are removing from the index,
we do know that they at least have the same content, and
that the content is included in HEAD.
That means the worst case is not the accidental loss of
content, but rather confusion by the user when a copy of a
file another part of the tree is removed. Which makes this
bug a minor and hard-to-trigger annoyance rather than a
data-loss bug (and hence the fix can be saved for a rainy
day when somebody feels like working on it).
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
fill_one is _almost_ identical to just calling fill_textconv; the
exception is that for the !DIFF_FILE_VALID case, fill_textconv gives us
an empty buffer rather than a NULL one. Since we currently use the NULL
pointer as a signal that the file is not present on one side of the
diff, we must now switch to using DIFF_FILE_VALID to make the same
check.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Simon Ruderich <simon@ruderich.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
diffcore-pickaxe: remove unnecessary call to get_textconv()
The fill_one() function is responsible for finding and filling the
textconv filter as necessary, and is called by diff_grep() function
that implements "git log -G<pattern>".
The has_changes() function that implements "git log -S<block>" calls
get_textconv() for two sides being compared, before it checks to see
if it was asked to perform the pickaxe limiting. Move the code
around to avoid this wastage.
After has_changes() calls get_textconv() to obtain textconv for both
sides, fill_one() is called to use them.
By adding get_textconv() to diff_grep() and relieving fill_one() of
responsibility to find the textconv filter, we can avoid calling
get_textconv() twice in has_changes().
With this change it's also no longer necessary for fill_one() to
modify the textconv argument, therefore pass a pointer instead of a
pointer to a pointer.
Signed-off-by: Simon Ruderich <simon@ruderich.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>