dir_iterator_advance() is a large function with two nested loops. Let's
improve its readability factoring out three functions and simplifying
its mechanics. The refactored model will no longer depend on
level.initialized and level.dir_state to keep track of the iteration
state and will perform on a single loop.
Also, dir_iterator_begin() currently does not check if the given string
represents a valid directory path. Since the refactored model will have
to stat() the given path at initialization, let's also check for this
kind of error and make dir_iterator_begin() return NULL, on failures,
with errno appropriately set. And add tests for this new behavior.
Improve documentation at dir-iteration.h and code comments at
dir-iterator.c to reflect the changes and eliminate possible
ambiguities.
Finally, adjust refs/files-backend.c to check for now possible
dir_iterator_begin() failures.
Original-patch-by: Daniel Ferreira <bnmvco@gmail.com> Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change warning(..., strerror(errno)) by warning_errno(...). This helps
to unify warning display besides simplifying a bit the code. Also,
improve warning messages by surrounding paths with quotation marks and
using more meaningful statements.
Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Create t/helper/test-dir-iterator.c, which prints relevant information
about a directory tree iterated over with dir-iterator.
Create t/t0066-dir-iterator.sh, which tests that dir-iterator does
iterate through a whole directory tree as expected.
Signed-off-by: Daniel Ferreira <bnmvco@gmail.com>
[matheus.bernardino: update to use test-tool and some minor aesthetics] Helped-by: Matheus Tavares <matheus.bernardino@usp.br> Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br> Signed-off-by: Junio C Hamano <gitster@pobox.com>
clone: better handle symlinked files at .git/objects/
There is currently an odd behaviour when locally cloning a repository
with symlinks at .git/objects: using --no-hardlinks all symlinks are
dereferenced but without it, Git will try to hardlink the files with the
link() function, which has an OS-specific behaviour on symlinks. On OSX
and NetBSD, it creates a hardlink to the file pointed by the symlink
whilst on GNU/Linux, it creates a hardlink to the symlink itself.
On Manjaro GNU/Linux:
$ touch a
$ ln -s a b
$ link b c
$ ls -li a b c
155 [...] a
156 [...] b -> a
156 [...] c -> a
It's not good to have the result of a local clone to be OS-dependent and
besides that, the current behaviour on GNU/Linux may result in broken
symlinks. So let's standardize this by making the hardlinks always point
to dereferenced paths, instead of the symlinks themselves. Also, add
tests for symlinked files at .git/objects/.
Note: Git won't create symlinks at .git/objects itself, but it's better
to handle this case and be friendly with users who manually create them.
Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Co-authored-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
clone: test for our behavior on odd objects/* content
Add tests for what happens when we perform a local clone on a repo
containing odd files at .git/object directory, such as symlinks to other
dirs, or unknown files.
I'm bending over backwards here to avoid a SHA-1 dependency. See [1]
for an earlier and simpler version that hardcoded SHA-1s.
This behavior has been the same for a *long* time, but hasn't been
tested for.
There's a good post-hoc argument to be made for copying over unknown
things, e.g. I'd like a git version that doesn't know about the
commit-graph to copy it under "clone --local" so a newer git version
can make use of it.
In follow-up commits we'll look at changing some of this behavior, but
for now, let's just assert it as-is so we'll notice what we'll change
later.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
[matheus.bernardino: improved and split tests in more than one patch] Helped-by: Matheus Tavares <matheus.bernardino@usp.br> Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br> Signed-off-by: Junio C Hamano <gitster@pobox.com>
diff: munmap() file contents before running external diff
When running an external diff from, say, a diff tool, it is safe to
assume that we want to write the files in question. On Windows, that
means that there cannot be any other process holding an open handle to
said files, or even just a mapped region.
So let's make sure that `git diff` itself is not holding any open handle
to the files in question.
In fact, we will just release the file pair right away, as the external
diff uses the files we just wrote, so we do not need to hold the file
contents in memory anymore.
This fixes https://github.com/git-for-windows/git/issues/1315
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
ref-filter: fix memory leak in `free_array_item()`
We treat the `value` pointer as a pointer to a struct and free its `s`
field. But `value` is in fact an array of structs. As a result, we only
free the first `s` out of `used_atom_cnt`-many and leak the rest. Make
sure we free all items in `value`.
In the caller, `ref_array_clear()`, this means we need to be careful not
to zero `used_atom_cnt` until after we've called `free_array_item()`. We
could move just a single line, but let's keep related things close
together instead, by first handling `array`, then `used_atom`.
Signed-off-by: Martin Ågren <martin.agren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Correct the api-trace2 documentation, which lists "signal" as an
expected field for the signal event type, but which actually outputs
"signo" as the field name.
Signed-off-by: Josh Steadmon <steadmon@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git fetch" and "git pull" reports when a fetch results in
non-fast-forward updates to let the user notice unusual situation.
The commands learned "--no-shown-forced-updates" option to disable
this safety feature.
* ds/fetch-disable-force-notice:
pull: add --[no-]show-forced-updates passthrough
fetch: warn about forced updates in branch listing
fetch: add --[no-]show-forced-updates argument
"git status" can be told a non-standard default value for the
"--[no-]ahead-behind" option with a new configuration variable
status.aheadBehind.
* jh/status-aheadbehind:
status: ignore status.aheadbehind in porcelain formats
status: warn when a/b calculation takes too long
status: add status.aheadbehind setting
"git submodule foreach" did not protect command line options passed
to the command to be run in each submodule correctly, when the
"--recursive" option was in use.
* ms/submodule-foreach-fix:
submodule foreach: fix recursion of options
* jh/msvc:
msvc: ignore .dll and incremental compile output
msvc: avoid debug assertion windows in Debug Mode
msvc: do not pretend to support all signals
msvc: add pragmas for common warnings
msvc: add a compile-time flag to allow detailed heap debugging
msvc: support building Git using MS Visual C++
msvc: update Makefile to allow for spaces in the compiler path
msvc: fix detect_msys_tty()
msvc: define ftello()
msvc: do not re-declare the timespec struct
msvc: mark a variable as non-const
msvc: define O_ACCMODE
msvc: include sigset_t definition
msvc: fix dependencies of compat/msvc.c
mingw: replace mingw_startup() hack
obstack: fix compiler warning
cache-tree/blame: avoid reusing the DEBUG constant
t0001 (mingw): do not expect a specific order of stdout/stderr
Mark .bat files as requiring CR/LF endings
mingw: fix a typo in the msysGit-specific section
Use "Erase in Line" CSI sequence that is already used in the editor
support to clear cruft in the progress output.
* sg/rebase-progress:
progress: use term_clear_line()
rebase: fix garbled progress display with '-x'
pager: add a helper function to clear the last line in the terminal
t3404: make the 'rebase.missingCommitsCheck=ignore' test more focused
t3404: modernize here doc style
Two new commands "git switch" and "git restore" are introduced to
split "checking out a branch to work on advancing its history" and
"checking out paths out of the index and/or a tree-ish to work on
advancing the current history" out of the single "git checkout"
command.
* nd/switch-and-restore: (46 commits)
completion: disable dwim on "git switch -d"
switch: allow to switch in the middle of bisect
t2027: use test_must_be_empty
Declare both git-switch and git-restore experimental
help: move git-diff and git-reset to different groups
doc: promote "git restore"
user-manual.txt: prefer 'merge --abort' over 'reset --hard'
completion: support restore
t: add tests for restore
restore: support --patch
restore: replace --force with --ignore-unmerged
restore: default to --source=HEAD when only --staged is specified
restore: reject invalid combinations with --staged
restore: add --worktree and --staged
checkout: factor out worktree checkout code
restore: disable overlay mode by default
restore: make pathspec mandatory
restore: take tree-ish from --source option instead
checkout: split part of it to new command 'restore'
doc: promote "git switch"
...
Protocol capabilities that go over wire should never be translated,
but it was incorrectly marked for translation, which has been
corrected. The output of protocol capabilities for debugging has
been tweaked a bit.
* nd/fetch-capability-tweak:
fetch-pack: print server version at the top in -v -v
fetch-pack: print all relevant supported capabilities with -v -v
fetch-pack: move capability names out of i18n strings
Code clean-up to remove hardcoded SHA-1 hash from many places.
* jk/oidhash:
hashmap: convert sha1hash() to oidhash()
hash.h: move object_id definition from cache.h
khash: rename oid helper functions
khash: drop sha1-specific map types
pack-bitmap: convert khash_sha1 maps into kh_oid_map
delta-islands: convert island_marks khash to use oids
khash: rename kh_oid_t to kh_oid_set
khash: drop broken oid_map typedef
object: convert create_object() to use object_id
object: convert internal hash_obj() to object_id
object: convert lookup_object() to use object_id
object: convert lookup_unknown_object() to use object_id
pack-objects: convert locate_object_entry_hash() to object_id
pack-objects: convert packlist_find() to use object_id
pack-bitmap-write: convert some helpers to use object_id
upload-pack: rename a "sha1" variable to "oid"
describe: fix accidental oid/hash type-punning
The code to parse scaled numbers out of configuration files has
been made more robust and also easier to follow.
* rs/config-unit-parsing:
config: simplify parsing of unit factors
config: don't multiply in parse_unit_factor()
config: use unsigned_mult_overflows to check for overflows
* js/gcc-8-and-9:
config: avoid calling `labs()` on too-large data type
winansi: simplify loading the GetCurrentConsoleFontEx() function
kwset: allow building with GCC 8
poll (mingw): allow compiling with GCC 8 and DEVELOPER=1
"git rebase --abort" used to leave refs/rewritten/ when concluding
"git rebase -r", which has been corrected.
* pw/rebase-abort-clean-rewritten:
rebase --abort/--quit: cleanup refs/rewritten
sequencer: return errors from sequencer_remove_state()
rebase: warn if state directory cannot be removed
rebase: fix a memory leak
* am/p4-branches-excludes:
git-p4: respect excluded paths when detecting branches
git-p4: add failing test for "git-p4: respect excluded paths when detecting branches"
git-p4: don't exclude other files with same prefix
git-p4: add failing test for "don't exclude other files with same prefix"
git-p4: don't groom exclude path list on every commit
git-p4: match branches case insensitively if configured
git-p4: add failing test for "git-p4: match branches case insensitively if configured"
git-p4: detect/prevent infinite loop in gitCommitByP4Change()
The commit-graph file is now part of the "files that the runtime
may keep open file descriptors on, all of which would need to be
closed when done with the object store", and the file descriptor to
an existing commit-graph file now is closed before "gc" finalizes a
new instance to replace it.
* ds/close-object-store:
packfile: rename close_all_packs to close_object_store
packfile: close commit-graph in close_all_packs
commit-graph: use raw_object_store when closing
An incorrect list of options was cached after command line
completion failed (e.g. trying to complete a command that requires
a repository outside one), which has been corrected.
* nd/completion-no-cache-failure:
completion: do not cache if --git-completion-helper fails
"git mergetool" and its tests now spawn fewer subprocesses.
* js/mergetool-optim:
mergetool: use shell variable magic instead of `awk`
mergetool: dissect strings with shell variable magic instead of `expr`
t7610-mergetool: use test_cmp instead of test $(cat file) = $txt
t7610-mergetool: do not place pipelines headed by `yes` in subshells
Code restructuring during 2.20 period broke fetching tags via
"import" based transports.
* fc/fetch-with-import-fix:
fetch: fix regression with transport helpers
fetch: make the code more understandable
fetch: trivial cleanup
t5801 (remote-helpers): add test to fetch tags
t5801 (remote-helpers): cleanup refspec stuff
"git branch --list" learned to show branches that are checked out
in other worktrees connected to the same repository prefixed with
'+', similar to the way the currently checked out branch is shown
with '*' in front.
* nb/branch-show-other-worktrees-head:
branch: add worktree info on verbose output
branch: update output to include worktree info
ref-filter: add worktreepath atom
builtin/merge.c - cleanup of code in for-cycle that tests strategies
The cmd_merge() function has a loop that tries different
merge strategies in turn, and stops when a strategy gets a
clean merge, while keeping the "best" conflicted merge so
far.
Make the loop easier to follow by moving the code around,
ensuring that there is only one "break" in the loop where
an automerge succeeds. Also group the actions that are
performed after an automerge succeeds together to a single
location, outside and after the loop.
Signed-off-by: Edmundo Carmona Antoranz <eantoranz@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Currently the 'find_name_*()' functions take 'struct apply_state' as
parameter, even though they only need the 'root' member from that
struct.
These functions are in the callchain of 'parse_git_header()', which we
want to make more generally useful in a subsequent commit. To make
that happen we only want to pass in the required data to
'parse_git_header()', and not the whole 'struct apply_state', and thus
we want functions in the callchain of 'parse_git_header()' to only
take arguments they really need.
Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
apply: only pass required data to check_header_line
Currently the 'check_header_line()' function takes 'struct
apply_state' as parameter, even though it only needs the linenr from
that struct.
This function is in the callchain of 'parse_git_header()', which we
want to make more generally useful in a subsequent commit. To make
that happen we only want to pass in the required data to
'parse_git_header()', and not the whole 'struct apply_state', and thus
we want functions in the callchain of 'parse_git_header()' to only
take arguments they really need.
Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Currently the 'git_header_name()' function takes 'struct apply_state'
as parameter, even though it only needs the p_value from that struct.
This function is in the callchain of 'parse_git_header()', which we
want to make more generally useful in a subsequent commit. To make
that happen we only want to pass in the required data to
'parse_git_header()', and not the whole 'struct apply_state', and thus
we want functions in the callchain of 'parse_git_header()' to only
take arguments they really need.
Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
apply: only pass required data to skip_tree_prefix
Currently the 'skip_tree_prefix()' function takes 'struct apply_state'
as parameter, even though it only needs the p_value from that struct.
This function is in the callchain of 'parse_git_header()', which we
want to make more generally useful in a subsequent commit. To make
that happen we only want to pass in the required data to
'parse_git_header()', and not the whole 'struct apply_state', and thus
we want functions in the callchain of 'parse_git_header()' to only
take arguments they really need.
Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
public-inbox.org links include the whole message ID by default. This
means the message can still be found even if the site goes away, which
is not the case with the marc.info link. Replace the marc.info link
with a more future proof one.
Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
t3420: remove progress lines before comparing output
Some of the tests check the output of rebase is what we expect. These
were added after a regression that added unwanted stash output when
using --autostash. They are useful as they prevent unintended changes to
the output of the various rebase commands. However they also include all
the progress output which is less useful as it only tests what would be
written to a dumb terminal which is not the normal use case. The recent
changes to fix clearing the line when printing progress necessarily
meant making an ugly change to these tests. Address this my removing the
progress output before comparing it to the expected output. We do this
by removing everything before the final "\r" on each line as we don't
care about the progress indicator, but we do care about what is printed
immediately after it.
Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
HOME initialization was historically duplicated in many different places,
including /etc/profile, launch scripts such as git-bash.vbs and gitk.cmd,
and (although slightly broken) in the git-wrapper.
Even unrelated projects such as GitExtensions and TortoiseGit need to
implement the same logic to be able to call git directly.
Initialize HOME in git's own startup code so that we can eventually retire
all the duplicate initialization code.
Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
mingw: fix possible buffer overrun when calling `GetUserNameW()`
In 39a98e9b68b8 (mingw: get pw_name in UTF-8 format, 2019-06-27), this
developer missed the fact that the `GetUserNameW()` function takes the
number of characters as `len` parameter, not the number of bytes.
Reported-by: Beat Bolli <dev+git@drbeat.li> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
ci/lib.sh: update a comment about installed P4 and Git-LFS versions
A comment in 'ci/lib.sh' claims that the "OS X build installs the
latest available versions" of P4 and Git-LFS, but since f2f47150
("ci: don't update Homebrew", 2019-07-03) that's no longer the case,
as it will install the versions which were recorded in the image's
Homebrew database when the image was created.
Update this comment accordingly.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Acked-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
On native Windows, Git exclusively uses UTF-8 for console output (both
with MinTTY and native Win32 Console). Gettext uses `setlocale()` to
determine the output encoding for translated text, however, MSVCRT's
`setlocale()` does not support UTF-8. As a result, translated text is
encoded in system encoding (as per `GetAPC()`), and non-ASCII chars are
mangled in console output.
Side note: There is actually a code page for UTF-8: 65001. In practice,
it does not work as expected at least on Windows 7, though, so we cannot
use it in Git. Besides, if we overrode the code page, any process
spawned from Git would inherit that code page (as opposed to the code
page configured for the current user), which would quite possibly break
e.g. diff or merge helpers. So we really cannot override the code page.
In `init_gettext_charset()`, Git calls gettext's
`bind_textdomain_codeset()` with the character set obtained via
`locale_charset()`; Let's override that latter function to force the
encoding to UTF-8 on native Windows.
In Git for Windows' SDK, there is a `libcharset.h` and therefore we
define `HAVE_LIBCHARSET_H` in the MINGW-specific section in
`config.mak.uname`, therefore we need to add the override before that
conditionally-compiled code block.
Rather than simply defining `locale_charset()` to return the string
`"UTF-8"`, though, we are careful not to break `LC_ALL=C`: the
`ab/no-kwset` patch series, for example, needs to have a way to prevent
Git from expecting UTF-8-encoded input.
Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Lately Homebrew learned to automagically clean up information about
outdated packages during other 'brew' commands, which might be useful
for the avarage user, but is a waste of time in CI build jobs, because
the next build jobs will start from the exact same image containing
the same outdated packages anyway.
Export HOMEBREW_NO_INSTALL_CLEANUP=1 to disable this auto cleanup feature,
shaving off about 20-30s from the time needed to install dependencies
in our macOS build jobs on Travis CI.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Lately our GCC macOS build job on Travis CI has been erroring out
while installing dependencies with:
+brew link gcc@8
Error: No such keg: /usr/local/Cellar/gcc@8
The command "ci/install-dependencies.sh" failed and exited with 1 during .
Now, while gcc@8 is still pre-installed (but not linked) and would be
perfectly usable in the Travis CI macOS image we use [1], it's at
version 8.2. However, when installing dependencies we first
explicitly run 'brew update', which spends over two minutes to update
itself and information about the available packages, and it learns
about GCC 8.3. After that point gcc@8 exclusively refers to v8.3,
and, unfortunately, 'brew' is just too dumb to be able to do anything
with the still installed 8.2 package, and the subsequent 'brew link
gcc@8' fails. (Even 'brew uninstall gcc@8' fails with the same
error!)
Don't run 'brew update' to keep the already installed GCC 8.2 'brew
link'-able. Note that in addition we have to 'export
HOMEBREW_NO_AUTO_UPDATE=1' first, because 'brew' is so very helpful
that it would implicitly run update for us on the next 'brew install
<pkg>' otherwise.
Disabling 'brew update' has additional benefits:
- It shaves off 2-3mins from the ~4mins currently spent on
installing dependencies, and the macOS build jobs have always been
prone to exceeding the time limit on Travis CI.
- Our builds won't suddenly break because of the occasional Homebrew
breakages [2].
The drawback is that we'll be stuck with slightly older versions of
the packages that we install via Homebrew (Git-LFS 2.5.2 and Perforce
2018.1; they are currently at 2.7.2 and 2019.1, respectively). We
might want to reconsider this decision as time goes on and/or switch
to a more recent macOS image as they become available.
[1] 2000ac9fbf (travis-ci: switch to Xcode 10.1 macOS image,
2019-01-17)
[2] See e.g. a1ccaedd62 (travis-ci: make the OSX build jobs' 'brew
update' more quiet, 2019-02-02) or
Currenly the data rate in throughput_string(...) method is
output by simple strbuf_humanise_bytes(...) call and '/s' append.
But for proper translation of such string the translator needs
full context.
Add strbuf_humanise_rate(...) method to properly print out
localizable version of data rate ('3.5 MiB/s' etc) with full context.
Strings with the units in strbuf_humanise_bytes(...) are marked
for translation.
Signed-off-by: Dimitriy Ryazantcev <dimitriy.ryazantcev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
List the short form of options (e.g.: '-l') before the long form (e.g.
'--local').
This is to match the doc of git-add, git-commit, git-clean, git-branch...
Signed-off-by: Quentin Nerden <quentin.nerden@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
The previous commit introduced a --skip flag for cherry-pick and
revert. Update the advice messages, to tell users about this less
cumbersome way of skipping commits. Also add tests to ensure
everything is working fine.
Signed-off-by: Rohit Ashiwal <rohit.ashiwal265@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
git am or rebase have a --skip flag to skip the current commit if the
user wishes to do so. During a cherry-pick or revert a user could
likewise skip a commit, but needs to use 'git reset' (or in the case
of conflicts 'git reset --merge'), followed by 'git (cherry-pick |
revert) --continue' to skip the commit. This is more annoying and
sometimes confusing on the users' part. Add a `--skip` option to make
skipping commits easier for the user and to make the commands more
consistent.
In the next commit, we will change the advice messages hence finishing
the process of teaching revert and cherry-pick "how to skip commits".
Signed-off-by: Rohit Ashiwal <rohit.ashiwal265@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Avoid using magic numbers for array size and index under `reset_merge`
function. Use `argv_array` instead. This will make code shorter and
easier to extend.
Signed-off-by: Rohit Ashiwal <rohit.ashiwal265@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
sequencer: rename reset_for_rollback to reset_merge
We are on a path to teach cherry-pick/revert how to skip commits. To
achieve this, we could really make use of existing functions.
reset_for_rollback is one such function, but the name does not
intuitively suggest to use it to reset a merge, which it was born to
perform, see 539047c ("revert: introduce --abort to cancel a failed
cherry-pick", 2011-11-23). Change the name to reset_merge to make
it more intuitive.
Signed-off-by: Rohit Ashiwal <rohit.ashiwal265@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
In the case of merge conflicts, while performing a revert, we are
currently advised to use `git cherry-pick --<sequencer-options>`.
Introduce a separate advice message for `git revert`. Also change
the signature of `create_seq_dir` to handle which advice to display
selectively.
Signed-off-by: Rohit Ashiwal <rohit.ashiwal265@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
There are two loops that create 33 commits each using test_commit. Using
test_commit_bulk speeds this up from:
Benchmark #1: ./t5703-upload-pack-ref-in-want.sh --root=/var/ram/git-tests
Time (mean ± σ): 2.142 s ± 0.161 s [User: 1.136 s, System: 0.974 s]
Range (min … max): 1.903 s … 2.401 s 10 runs
to:
Benchmark #1: ./t5703-upload-pack-ref-in-want.sh --root=/var/ram/git-tests
Time (mean ± σ): 1.440 s ± 0.114 s [User: 737.7 ms, System: 615.4 ms]
Range (min … max): 1.230 s … 1.604 s 10 runs
for an average savings of almost 33%.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
There are two loops that create 32 commits each using test_commit. Using
test_commit_bulk speeds this up from:
Benchmark #1: ./t5702-protocol-v2.sh --root=/var/ram/git-tests
Time (mean ± σ): 5.409 s ± 0.513 s [User: 2.382 s, System: 2.466 s]
Range (min … max): 4.633 s … 5.927 s 10 runs
to:
Benchmark #1: ./t5702-protocol-v2.sh --root=/var/ram/git-tests
Time (mean ± σ): 3.956 s ± 0.242 s [User: 1.775 s, System: 1.627 s]
Range (min … max): 3.449 s … 4.239 s 10 runs
for an average savings of over 25%.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
One of the tests in t3311 creates 300 commits by running "test_commit"
in a loop. This requires 900 processes. Instead, we can use
test_commit_bulk to do it with only four. This improves the runtime of
the script from:
Benchmark #1: ./t3311-notes-merge-fanout.sh --root=/var/ram/git-tests
Time (mean ± σ): 5.821 s ± 0.691 s [User: 3.146 s, System: 2.782 s]
Range (min … max): 4.783 s … 6.841 s 10 runs
to:
Benchmark #1: ./t3311-notes-merge-fanout.sh --root=/var/ram/git-tests
Time (mean ± σ): 1.743 s ± 0.116 s [User: 1.144 s, System: 0.691 s]
Range (min … max): 1.629 s … 1.994 s 10 runs
for an average speedup of over 70%.
Unfortunately we still have to run 300 instances of "git notes add",
since the point is to test the fanout that comes from adding notes one
by one.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
The bitmap index we compute in t5310 has only 20 commits in it. This
gives poor coverage of bitmap_writer_select_commits(), which simply
writes a bitmap for everything when there are fewer than 100 commits.
Let's bump the number of commits in the test to cover the more complex
code paths (this does drop coverage of the individual lines of the
trivial path, but the complex path does everything it does and more).
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Some tests need to create a string of commits. Doing this with
test_commit is very heavy-weight, as it needs at least one process per
commit (and in fact, uses several).
For bulk creation, we can do much better by using fast-import, but it's
often a pain to generate the input. Let's provide a helper to do so.
We'll use t5310 as a guinea pig, as it has three 10-commit loops. Here
are hyperfine results before and after:
[before]
Benchmark #1: ./t5310-pack-bitmaps.sh --root=/var/ram/git-tests
Time (mean ± σ): 2.846 s ± 0.305 s [User: 3.042 s, System: 0.919 s]
Range (min … max): 2.250 s … 3.210 s 10 runs
[after]
Benchmark #1: ./t5310-pack-bitmaps.sh --root=/var/ram/git-tests
Time (mean ± σ): 2.210 s ± 0.174 s [User: 2.570 s, System: 0.604 s]
Range (min … max): 1.999 s … 2.590 s 10 runs
So we're over 20% faster, while making the callers slightly shorter. We
added a lot more lines in test-lib-function.sh, of course, and the
helper is way more featureful than we need here. But my hope is that it
will be flexible enough to use in more places.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
t5319: use 'test-tool path-utils' instead of 'ls -l'
Using 'ls -l' and parsing the columns to find file sizes is
problematic when the platform could report the owner as a name
with spaces. Instead, use the 'test-tool path-utils file-size'
command to list only the sizes.
Reported-by: Johannes Sixt <j6t@kdbg.org> Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de> Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
This test uses several index hashes, which necessarily depend on the
version of the index and the hash algorithm in use. Use test_oid_cache
to provide values for these for both SHA-1 and SHA-256. Also, compute
an object ID and use $EMPTY_BLOB to make the remainder of the tests
independent of the hash algorithm in use.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Update this test to use test_oid_cache to specify the object IDs for
both SHA-1 and SHA-256. Since this test now works with both algorithms,
remove the SHA1 prerequisite.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
One assertion of this test checks for a shrinking cache tree. The
initial index contains a cache tree with two directory names but no
object ID, and the second index contains a cache tree with an object ID
but no directory name.
With SHA-1, the second index is smaller than the first, because the
directory information stored takes more than the 20 bytes of an SHA-1
hash, but with SHA-256, the hash is longer, and the test fails the
assertion that the second index is smaller than the first.
To address this issue, increase the length of the subdirectory name to
ensure that the cache tree does indeed shrink in size regardless of the
algorithm in use.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Several parts of this test generate files that have specific hard-coded
object IDs in them. We don't really care about what the object ID in
question is, so we turn them all to zeros.
However, because some of these values are fixed and some are generated,
they can be of different lengths, which causes problems when running
with SHA-256. Furthermore, some assertions in this test use only fixed
object IDs and some use both fixed and generated ones, so converting
only the expected results fixes some tests while breaking others.
Convert both actual and expected object IDs to the all-zeros object ID
of the appropriate length to ensure that the test passes when using
SHA-256.
The astute observer will notice that both tr and sed are used here.
Converting the tr call to a sed y/// command looks logical at first, but
it isn't possible because POSIX doesn't allow escapes in y/// commands
other than "\\" and "\n".
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Compute several object ID values instead of hard-coding them, and use
test_oid_to_path to cleanly produce a path for an object.
Note that the bisect code which is tested here remains sensitive to the
hash algorithm in use because it uses the object ID to disambiguate
between two equidistant commits. Fortunately, SHA-1 and SHA-256
disambiguate identically in the cases we care about, so there is no need
to modify the test to accommodate this situation. However, if a further
hash algorithm change occurs, this test may require some restructuring.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
This test uses a stub of a very large (64 GB) object to test our
generation of tar archives. In doing so, it uses the object ID of the
object so it can insert it into the database properly. Look up these
values using test_oid. Restructure the test slightly to use
test_oid_in_path.
Since we care about the object, not how it is named in a particular hash
algorithm, rename it to "huge-object", which is shorter and more
descriptive.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Replace several hard-coded full and partial object IDs with variables or
computed values. Create junk data to stuff inside an invalid tree that
can be either 20 or 32 bytes long. Compute a binary all-zeros object ID
instead of hard-coding a 20-byte length.
Additionally, compute various object IDs by using test_oid and
$EMPTY_BLOB so that this test works with multiple hash algorithms.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Instead of parsing object IDs using fixed-length shell patterns, use cut
to extract the first two characters of an object ID in addition to the
test helper for object paths. Update another test to look up an
appropriate object ID fragment from the all-zeros object ID instead of
hardcoding the value.
Although the test for parsing reflogs at BUFSIZ boundaries passes, mark
it with the SHA1 prerequisite, as it doesn't currently usefully test
anything when using a hash longer than 20 bytes.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
There are several places in our testsuite where we want to insert a
slash after an object ID to make it into a path we can reference under
.git/objects, and we have various ways of doing so. Add a helper to
provide a standard way of doing this that works for all size hashes.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
If the user commits or resets a conflict resolution in the middle of a
sequence of cherry-picks or reverts then CHERRY_PICK_HEAD/REVERT_HEAD
will be removed and so in the absence of those files we need to check
.git/sequencer/todo to see if there is a cherry-pick or revert in
progress.
Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
repack: disable bitmaps-by-default if .keep files exist
Bitmaps aren't useful with multiple packs, and users with
.keep files ended up with redundant packs when bitmaps
got enabled by default in bare repos.
So detect when .keep files exist and stop enabling bitmaps
by default in that case.
Wasteful (but otherwise harmless) race conditions with .keep files
documented by Jeff King still apply and there's a chance we'd
still end up with redundant data on the FS:
v2: avoid subshell in test case, be multi-index aware
Fixes: 36eba0323d3288a8 ("repack: enable bitmaps by default on bare repos") Signed-off-by: Eric Wong <e@80x24.org> Helped-by: Jeff King <peff@peff.net> Reported-by: Janos Farkas <chexum@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Testing the 'remove' subcommand was forgotten when t0016
was created. Let's fix that.
Helped-by: Derrick Stolee <stolee@gmail.com> Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
The 'add' subcommand is useless as it is mostly identical
to the 'put' subcommand, so let's remove it.
Helped-by: Derrick Stolee <stolee@gmail.com> Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
check_everything_connected: assume alternate ref tips are valid
When we receive a remote ref update to sha1 "X", we want to check that
we have all of the objects needed by "X". We can assume that our
repository is not currently corrupted, and therefore if we have a ref
pointing at "Y", we have all of its objects. So we can stop our
traversal from "X" as soon as we hit "Y".
If we make the same non-corruption assumption about any repositories we
use to store alternates, then we can also use their ref tips to shorten
the traversal.
This is especially useful when cloning with "--reference", as we
otherwise do not have any local refs to check against, and have to
traverse the whole history, even though the other side may have sent us
few or no objects. Here are results for the included perf test (which
shows off more or less the maximal savings, getting one new commit and
sharing the whole history):
Test HEAD^ HEAD
--------------------------------------------------------------------
[on git.git]
5600.3: clone --reference 2.94(2.86+0.08) 0.09(0.08+0.01) -96.9%
[on linux.git]
5600.3: clone --reference 45.74(45.34+0.41) 0.36(0.30+0.08) -99.2%
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
object-store.h: move for_each_alternate_ref() from transport.h
There's nothing inherently transport-related about enumerating the
alternate ref tips. The code has lived in transport.[ch] because the
only use so far had been advertising available tips during transport.
But it could be used for more, and a future patch will teach rev-list to
access these refs.
Let's move it alongside the other alt-odb code, declaring it in
object-store.h with the implementation in sha1-file.c.
This lets us drop the inclusion of transport.h from receive-pack, which
perhaps shows how it was misplaced (though receive-pack is about
transporting objects, transport.h is mostly about the client side).
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
These unused parameters were introduced recently as part of the
br/blame-ignore topic. I assume they are not indicative of bugs, but are
just leftovers from the development process (they were introduced by the
series but not used in any of its iterations).
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
t7814: do not generate same commits in different repos
t7814 has repo tree like this
initial-repo
submodule
sub
In each repo 'submodule' and 'sub', a commit is made to add the same
initial file 'a' with the same message 'add a'. If tests run fast
enough, the two commits are made in the same second, resulting
identical commits.
There is nothing wrong with that per-se. But it could make the test
flaky. Currently all submodule odbs are merged back in the main
one (because we can't, or couldn't, access separate submodule repos
otherwise). But eventually we need to access objects from the right
repo.
Because the same commit could sometimes be present in both 'submodule'
and 'sub', if there is a bug looking up objects in the wrong repo,
sometimes it will go unnoticed because it finds the needed object in the
wrong repo anyway.
Fix this by changing commit time after every commit. This makes all
commits unique. Of course there are still identical blobs in different
repos, but because we often lookup commit first, then tree and blob,
unique commits are already quite safe.
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Since cfe004a5a9 (ref-filter: limit traversal to prefix, 2017-05-22),
the ref-filter code has sought to limit the traversals to a prefix of
the given patterns.
That code stopped short of handling more than one pattern, because it
means invoking 'for_each_ref_in' multiple times. If we're not careful
about which patterns overlap, we will output the same refs multiple
times.
For instance, consider the set of patterns 'refs/heads/a/*',
'refs/heads/a/b/c', and 'refs/tags/v1.0.0'. If we naïvely ran:
we would see 'refs/heads/a/b/c' (and everything underneath it) twice.
Instead, we want to partition the patterns into disjoint sets, where we
know that no ref will be matched by any two patterns in different sets.
In the above, these are:
- {'refs/heads/a/*', 'refs/heads/a/b/c'}, and
- {'refs/tags/v1.0.0'}
Given one of these disjoint sets, what is a suitable pattern to pass to
'for_each_ref_in'? One approach is to compute the longest common prefix
over all elements in that disjoint set, and let the caller cull out the
refs they didn't want. Computing the longest prefix means that in most
cases, we won't match too many things the caller would like to ignore.
Which provides us with the refs we were looking for with a minimal
amount of extra cruft, but never a duplicate of the ref we asked for.
Implemented here is an algorithm which accomplishes the above, which
works as follows:
1. Lexicographically sort the given list of patterns.
2. Initialize 'prefix' to the empty string, where our goal is to
build each element in the above set of longest common prefixes.
3. Consider each pattern in the given set, and emit 'prefix' if it
reaches the end of a pattern, or touches a wildcard character. The
end of a string is treated as if it precedes a wildcard. (Note that
there is some room for future work to detect that, e.g., 'a?b' and
'abc' are disjoint).
4. Otherwise, recurse on step (3) with the slice of the list
corresponding to our current prefix (i.e., the subset of patterns
that have our prefix as a literal string prefix.)
This algorithm is 'O(kn + n log(n))', where 'k' is max(len(pattern)) for
each pattern in the list, and 'n' is len(patterns).
By discovering this set of interesting patterns, we reduce the runtime
of multi-pattern 'git for-each-ref' (and other ref traversals) from
O(N) to O(n log(N)), where 'N' is the total number of packed references.
Running 'git for-each-ref refs/tags/a refs/tags/b' on a repository with
10,000,000 refs in 'refs/tags/huge-N', my best-of-five times drop from:
real 0m5.805s
user 0m5.188s
sys 0m0.468s
to:
real 0m0.001s
user 0m0.000s
sys 0m0.000s
On linux.git, the times to dig out two of the latest -rc tags drops from
0.002s to 0.001s, so the change on repositories with fewer tags is much
less noticeable.
Co-authored-by: Jeff King <peff@peff.net> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>