From: Junio C Hamano Date: Mon, 30 Sep 2019 04:19:31 +0000 (+0900) Subject: Merge branch 'dl/rebase-i-keep-base' X-Git-Url: https://git.lorimer.id.au/gitweb.git/diff_plain/640f9cd5994ed2061405aa94972b1746bb0c393e?hp=414d924beb41b9f39744b5574231856d5ffbcba8 Merge branch 'dl/rebase-i-keep-base' "git rebase --keep-base " tries to find the original base of the topic being rebased and rebase on top of that same base, which is useful when running the "git rebase -i" (and its limited variant "git rebase -x"). The command also has learned to fast-forward in more cases where it can instead of replaying to recreate identical commits. * dl/rebase-i-keep-base: rebase: teach rebase --keep-base rebase tests: test linear branch topology rebase: fast-forward --fork-point in more cases rebase: fast-forward --onto in more cases rebase: refactor can_fast_forward into goto tower t3432: test for --no-ff's interaction with fast-forward t3432: distinguish "noop-same" v.s. "work-same" in "same head" tests t3432: test rebase fast-forward behavior t3431: add rebase --fork-point tests --- diff --git a/.gitignore b/.gitignore index 521d8f4fb4..fc445edea9 100644 --- a/.gitignore +++ b/.gitignore @@ -231,7 +231,6 @@ *.ipdb *.dll .vs/ -*.manifest Debug/ Release/ /UpgradeLog*.htm diff --git a/Documentation/RelNotes/2.24.0.txt b/Documentation/RelNotes/2.24.0.txt index a95a8b0084..ff48d8582c 100644 --- a/Documentation/RelNotes/2.24.0.txt +++ b/Documentation/RelNotes/2.24.0.txt @@ -11,7 +11,40 @@ Backward compatibility note UI, Workflows & Features - * (no entry yet so far) + * We now have an active interim maintainer for the Git-Gui part of + the system. Praise and thank Pratyush Yadav for volunteering. + + * The command line parser learned "--end-of-options" notation; the + standard convention for scripters to have hardcoded set of options + first on the command line, and force the command to treat end-user + input as non-options, has been to use "--" as the delimiter, but + that would not work for commands that use "--" as a delimiter + between revs and pathspec. + + * A mechanism to affect the default setting for a (related) group of + configuration variables is introduced. + + * "git fetch" learned "--set-upstream" option to help those who first + clone from their private fork they intend to push to, add the true + upstream via "git remote add" and then "git fetch" from it. + + * Device-tree files learned their own userdiff patterns. + (merge 3c81760bc6 sb/userdiff-dts later to maint). + + * "git rebase --rebase-merges" learned to drive different merge + strategies and pass strategy specific options to them. + + * A new "pre-merge-commit" hook has been introduced. + + * Command line completion updates for "git -c var.name=val" have been + added. + + * The lazy clone machinery has been taught that there can be more + than one promisor remote and consult them in order when downloading + missing objects on demand. + + * The list-objects-filter API (used to create a sparse/lazy clone) + learned to take a combined filter specification. Performance, Internal Implementation, Development Support etc. @@ -22,6 +55,22 @@ Performance, Internal Implementation, Development Support etc. * The first line of verbose output from each test piece now carries the test name and number to help scanning with eyeballs. + * Further clean-up of the initialization code. + + * xmalloc() used to have a mechanism to ditch memory and address + space resources as the last resort upon seeing an allocation + failure from the underlying malloc(), which made the code complex + and thread-unsafe with dubious benefit, as major memory resource + users already do limit their uses with various other mechanisms. + It has been simplified away. + + * Unnecessary full-tree diff in "git log -L" machinery has been + optimized away. + + * The http transport lacked some optimization the native transports + learned to avoid unnecessary ref advertisement, which has been + corrected. + Fixes since v2.23 ----------------- @@ -48,5 +97,47 @@ Fixes since v2.23 * Compilation fix. (merge 70597e8386 rs/nedalloc-fixlets later to maint). + * "git gui" learned to call the clean-up procedure before exiting. + (merge 0d88f3d2c5 py/git-gui-do-quit later to maint). + + * We promoted the "indent heuristics" that decides where to split + diff hunks from experimental to the default a few years ago, but + some stale documentation still marked it as experimental, which has + been corrected. + (merge 64e5e1fba1 sg/diff-indent-heuristic-non-experimental later to maint). + + * Fix a mismerge that happened in 2.22 timeframe. + (merge acb7da05ac en/checkout-mismerge-fix later to maint). + + * "git archive" recorded incorrect length in extended pax header in + some corner cases, which has been corrected. + (merge 71d41ff651 rs/pax-extended-header-length-fix later to maint). + + * On-demand object fetching in lazy clone incorrectly tried to fetch + commits from submodule projects, while still working in the + superproject, which has been corrected. + (merge a63694f523 jt/diff-lazy-fetch-submodule-fix later to maint). + + * Prepare get_short_oid() codepath to be thread-safe. + (merge 7cfcb16b0e rs/sort-oid-array-thread-safe later to maint). + + * "for-each-ref" and friends that show refs did not protect themselves + against ancient tags that did not record tagger names when asked to + show "%(taggername)", which have been corrected. + (merge 8b3f33ef11 mp/for-each-ref-missing-name-or-email later to maint). + + * The "git am" based backend of "git rebase" ignored the result of + updating ".gitattributes" done in one step when replaying + subsequent steps. + (merge 2c65d90f75 bc/reread-attributes-during-rebase later to maint). + + * Tell cURL library to use the same malloc() implementation, with the + xmalloc() wrapper, as the rest of the system, for consistency. + (merge 93b980e58f cb/curl-use-xmalloc later to maint). + * Other code cleanup, docfix, build fix, etc. (merge d1387d3895 en/fast-import-merge-doc later to maint). + (merge 1c24a54ea4 bm/repository-layout-typofix later to maint). + (merge 415b770b88 ds/midx-expire-repack later to maint). + (merge 19800bdc3f nd/diff-parseopt later to maint). + (merge 58166c2e9d tg/t0021-racefix later to maint). diff --git a/Documentation/config.txt b/Documentation/config.txt index e3f5bc3396..77f3b1486b 100644 --- a/Documentation/config.txt +++ b/Documentation/config.txt @@ -345,6 +345,8 @@ include::config/difftool.txt[] include::config/fastimport.txt[] +include::config/feature.txt[] + include::config/fetch.txt[] include::config/format.txt[] diff --git a/Documentation/config/core.txt b/Documentation/config/core.txt index 75538d27e7..852d2ba37a 100644 --- a/Documentation/config/core.txt +++ b/Documentation/config/core.txt @@ -86,7 +86,9 @@ core.untrackedCache:: it will automatically be removed, if set to `false`. Before setting it to `true`, you should check that mtime is working properly on your system. - See linkgit:git-update-index[1]. `keep` by default. + See linkgit:git-update-index[1]. `keep` by default, unless + `feature.manyFiles` is enabled which sets this setting to + `true` by default. core.checkStat:: When missing or is set to `default`, many fields in the stat @@ -577,7 +579,7 @@ the `GIT_NOTES_REF` environment variable. See linkgit:git-notes[1]. core.commitGraph:: If true, then git will read the commit-graph file (if it exists) - to parse the graph structure of commits. Defaults to false. See + to parse the graph structure of commits. Defaults to true. See linkgit:git-commit-graph[1] for more information. core.useReplaceRefs:: diff --git a/Documentation/config/diff.txt b/Documentation/config/diff.txt index 5afb5a2cbc..ff09f1cf73 100644 --- a/Documentation/config/diff.txt +++ b/Documentation/config/diff.txt @@ -189,7 +189,7 @@ diff.guitool:: include::../mergetools-diff.txt[] diff.indentHeuristic:: - Set this option to `true` to enable experimental heuristics + Set this option to `false` to disable the default heuristics that shift diff hunk boundaries to make patches easier to read. diff.algorithm:: diff --git a/Documentation/config/feature.txt b/Documentation/config/feature.txt new file mode 100644 index 0000000000..545522f306 --- /dev/null +++ b/Documentation/config/feature.txt @@ -0,0 +1,29 @@ +feature.*:: + The config settings that start with `feature.` modify the defaults of + a group of other config settings. These groups are created by the Git + developer community as recommended defaults and are subject to change. + In particular, new config options may be added with different defaults. + +feature.experimental:: + Enable config options that are new to Git, and are being considered for + future defaults. Config settings included here may be added or removed + with each release, including minor version updates. These settings may + have unintended interactions since they are so new. Please enable this + setting if you are interested in providing feedback on experimental + features. The new default values are: ++ +* `pack.useSparse=true` uses a new algorithm when constructing a pack-file +which can improve `git push` performance in repos with many files. ++ +* `fetch.negotiationAlgorithm=skipping` may improve fetch negotiation times by +skipping more commits at a time, reducing the number of round trips. + +feature.manyFiles:: + Enable config options that optimize for repos with many files in the + working directory. With many files, commands such as `git status` and + `git checkout` may be slow and these new defaults improve performance: ++ +* `index.version=4` enables path-prefix compression in the index. ++ +* `core.untrackedCache=true` enables the untracked cache. This setting assumes +that mtime is working on your machine. diff --git a/Documentation/config/fetch.txt b/Documentation/config/fetch.txt index ba890b5884..d402110638 100644 --- a/Documentation/config/fetch.txt +++ b/Documentation/config/fetch.txt @@ -59,7 +59,8 @@ fetch.negotiationAlgorithm:: effort to converge faster, but may result in a larger-than-necessary packfile; The default is "default" which instructs Git to use the default algorithm that never skips commits (unless the server has acknowledged it or one - of its descendants). + of its descendants). If `feature.experimental` is enabled, then this + setting defaults to "skipping". Unknown values will cause 'git fetch' to error out. + See also the `--negotiation-tip` option for linkgit:git-fetch[1]. diff --git a/Documentation/config/format.txt b/Documentation/config/format.txt index 414a5a8a9d..cb629fa769 100644 --- a/Documentation/config/format.txt +++ b/Documentation/config/format.txt @@ -77,6 +77,7 @@ format.coverLetter:: A boolean that controls whether to generate a cover-letter when format-patch is invoked, but in addition can be set to "auto", to generate a cover-letter only when there's more than one patch. + Default is false. format.outputDirectory:: Set a custom directory to store the resulting files instead of the diff --git a/Documentation/config/gc.txt b/Documentation/config/gc.txt index 02b92b18b5..00ea0a678e 100644 --- a/Documentation/config/gc.txt +++ b/Documentation/config/gc.txt @@ -63,7 +63,7 @@ gc.writeCommitGraph:: If true, then gc will rewrite the commit-graph file when linkgit:git-gc[1] is run. When using `git gc --auto` the commit-graph will be updated if housekeeping is - required. Default is false. See linkgit:git-commit-graph[1] + required. Default is true. See linkgit:git-commit-graph[1] for details. gc.logExpiry:: diff --git a/Documentation/config/index.txt b/Documentation/config/index.txt index f181503041..7cb50b37e9 100644 --- a/Documentation/config/index.txt +++ b/Documentation/config/index.txt @@ -24,3 +24,4 @@ index.threads:: index.version:: Specify the version with which new index files should be initialized. This does not affect existing repositories. + If `feature.manyFiles` is enabled, then the default is 4. diff --git a/Documentation/config/pack.txt b/Documentation/config/pack.txt index 9cdcfa7324..1d66f0c992 100644 --- a/Documentation/config/pack.txt +++ b/Documentation/config/pack.txt @@ -112,7 +112,8 @@ pack.useSparse:: objects. This can have significant performance benefits when computing a pack to send a small change. However, it is possible that extra objects are added to the pack-file if the included - commits contain certain types of direct renames. + commits contain certain types of direct renames. Default is `false` + unless `feature.experimental` is enabled. pack.writeBitmaps (deprecated):: This is a deprecated synonym for `repack.writeBitmaps`. diff --git a/Documentation/config/remote.txt b/Documentation/config/remote.txt index 6c4cad83a2..a8e6437a90 100644 --- a/Documentation/config/remote.txt +++ b/Documentation/config/remote.txt @@ -76,3 +76,11 @@ remote..pruneTags:: + See also `remote..prune` and the PRUNING section of linkgit:git-fetch[1]. + +remote..promisor:: + When set to true, this remote will be used to fetch promisor + objects. + +remote..partialclonefilter:: + The filter that will be applied when fetching from this + promisor remote. diff --git a/Documentation/fetch-options.txt b/Documentation/fetch-options.txt index 3c9b4f9e09..99df1f3d4e 100644 --- a/Documentation/fetch-options.txt +++ b/Documentation/fetch-options.txt @@ -169,6 +169,13 @@ ifndef::git-pull[] Disable recursive fetching of submodules (this has the same effect as using the `--recurse-submodules=no` option). +--set-upstream:: + If the remote is fetched successfully, pull and add upstream + (tracking) reference, used by argument-less + linkgit:git-pull[1] and other commands. For more information, + see `branch..merge` and `branch..remote` in + linkgit:git-config[1]. + --submodule-prefix=:: Prepend to paths printed in informative messages such as "Fetching submodule foo". This option is used diff --git a/Documentation/git-fast-export.txt b/Documentation/git-fast-export.txt index cc940eb9ad..784e934009 100644 --- a/Documentation/git-fast-export.txt +++ b/Documentation/git-fast-export.txt @@ -17,9 +17,9 @@ This program dumps the given revisions in a form suitable to be piped into 'git fast-import'. You can use it as a human-readable bundle replacement (see -linkgit:git-bundle[1]), or as a kind of an interactive -'git filter-branch'. - +linkgit:git-bundle[1]), or as a format that can be edited before being +fed to 'git fast-import' in order to do history rewrites (an ability +relied on by tools like 'git filter-repo'). OPTIONS ------- diff --git a/Documentation/git-filter-branch.txt b/Documentation/git-filter-branch.txt index 6b53dd7e06..5876598852 100644 --- a/Documentation/git-filter-branch.txt +++ b/Documentation/git-filter-branch.txt @@ -16,6 +16,19 @@ SYNOPSIS [--original ] [-d ] [-f | --force] [--state-branch ] [--] [...] +WARNING +------- +'git filter-branch' has a plethora of pitfalls that can produce non-obvious +manglings of the intended history rewrite (and can leave you with little +time to investigate such problems since it has such abysmal performance). +These safety and performance issues cannot be backward compatibly fixed and +as such, its use is not recommended. Please use an alternative history +filtering tool such as https://github.com/newren/git-filter-repo/[git +filter-repo]. If you still need to use 'git filter-branch', please +carefully read <> (and <>) to learn about the land +mines of filter-branch, and then vigilantly avoid as many of the hazards +listed there as reasonably possible. + DESCRIPTION ----------- Lets you rewrite Git revision history by rewriting the branches mentioned @@ -445,36 +458,236 @@ warned. (or if your git-gc is not new enough to support arguments to `--prune`, use `git repack -ad; git prune` instead). -NOTES ------ - -git-filter-branch allows you to make complex shell-scripted rewrites -of your Git history, but you probably don't need this flexibility if -you're simply _removing unwanted data_ like large files or passwords. -For those operations you may want to consider -http://rtyley.github.io/bfg-repo-cleaner/[The BFG Repo-Cleaner], -a JVM-based alternative to git-filter-branch, typically at least -10-50x faster for those use-cases, and with quite different -characteristics: - -* Any particular version of a file is cleaned exactly _once_. The BFG, - unlike git-filter-branch, does not give you the opportunity to - handle a file differently based on where or when it was committed - within your history. This constraint gives the core performance - benefit of The BFG, and is well-suited to the task of cleansing bad - data - you don't care _where_ the bad data is, you just want it - _gone_. - -* By default The BFG takes full advantage of multi-core machines, - cleansing commit file-trees in parallel. git-filter-branch cleans - commits sequentially (i.e. in a single-threaded manner), though it - _is_ possible to write filters that include their own parallelism, - in the scripts executed against each commit. - -* The http://rtyley.github.io/bfg-repo-cleaner/#examples[command options] - are much more restrictive than git-filter branch, and dedicated just - to the tasks of removing unwanted data- e.g: - `--strip-blobs-bigger-than 1M`. +[[PERFORMANCE]] +PERFORMANCE +----------- + +The performance of git-filter-branch is glacially slow; its design makes it +impossible for a backward-compatible implementation to ever be fast: + +* In editing files, git-filter-branch by design checks out each and +every commit as it existed in the original repo. If your repo has 10\^5 +files and 10\^5 commits, but each commit only modifies 5 files, then +git-filter-branch will make you do 10\^10 modifications, despite only +having (at most) 5*10^5 unique blobs. + +* If you try and cheat and try to make git-filter-branch only work on +files modified in a commit, then two things happen + + ** you run into problems with deletions whenever the user is simply + trying to rename files (because attempting to delete files that + don't exist looks like a no-op; it takes some chicanery to remap + deletes across file renames when the renames happen via arbitrary + user-provided shell) + + ** even if you succeed at the map-deletes-for-renames chicanery, you + still technically violate backward compatibility because users are + allowed to filter files in ways that depend upon topology of + commits instead of filtering solely based on file contents or names + (though this has not been observed in the wild). + +* Even if you don't need to edit files but only want to e.g. rename or +remove some and thus can avoid checking out each file (i.e. you can use +--index-filter), you still are passing shell snippets for your filters. +This means that for every commit, you have to have a prepared git repo +where those filters can be run. That's a significant setup. + +* Further, several additional files are created or updated per commit by +git-filter-branch. Some of these are for supporting the convenience +functions provided by git-filter-branch (such as map()), while others +are for keeping track of internal state (but could have also been +accessed by user filters; one of git-filter-branch's regression tests +does so). This essentially amounts to using the filesystem as an IPC +mechanism between git-filter-branch and the user-provided filters. +Disks tend to be a slow IPC mechanism, and writing these files also +effectively represents a forced synchronization point between separate +processes that we hit with every commit. + +* The user-provided shell commands will likely involve a pipeline of +commands, resulting in the creation of many processes per commit. +Creating and running another process takes a widely varying amount of +time between operating systems, but on any platform it is very slow +relative to invoking a function. + +* git-filter-branch itself is written in shell, which is kind of slow. +This is the one performance issue that could be backward-compatibly +fixed, but compared to the above problems that are intrinsic to the +design of git-filter-branch, the language of the tool itself is a +relatively minor issue. + + ** Side note: Unfortunately, people tend to fixate on the + written-in-shell aspect and periodically ask if git-filter-branch + could be rewritten in another language to fix the performance + issues. Not only does that ignore the bigger intrinsic problems + with the design, it'd help less than you'd expect: if + git-filter-branch itself were not shell, then the convenience + functions (map(), skip_commit(), etc) and the `--setup` argument + could no longer be executed once at the beginning of the program + but would instead need to be prepended to every user filter (and + thus re-executed with every commit). + +The https://github.com/newren/git-filter-repo/[git filter-repo] tool is +an alternative to git-filter-branch which does not suffer from these +performance problems or the safety problems (mentioned below). For those +with existing tooling which relies upon git-filter-branch, 'git +repo-filter' also provides +https://github.com/newren/git-filter-repo/blob/master/contrib/filter-repo-demos/filter-lamely[filter-lamely], +a drop-in git-filter-branch replacement (with a few caveats). While +filter-lamely suffers from all the same safety issues as +git-filter-branch, it at least ameloriates the performance issues a +little. + +[[SAFETY]] +SAFETY +------ + +git-filter-branch is riddled with gotchas resulting in various ways to +easily corrupt repos or end up with a mess worse than what you started +with: + +* Someone can have a set of "working and tested filters" which they +document or provide to a coworker, who then runs them on a different OS +where the same commands are not working/tested (some examples in the +git-filter-branch manpage are also affected by this). BSD vs. GNU +userland differences can really bite. If lucky, error messages are +spewed. But just as likely, the commands either don't do the filtering +requested, or silently corrupt by making some unwanted change. The +unwanted change may only affect a few commits, so it's not necessarily +obvious either. (The fact that problems won't necessarily be obvious +means they are likely to go unnoticed until the rewritten history is in +use for quite a while, at which point it's really hard to justify +another flag-day for another rewrite.) + +* Filenames with spaces are often mishandled by shell snippets since +they cause problems for shell pipelines. Not everyone is familiar with +find -print0, xargs -0, git-ls-files -z, etc. Even people who are +familiar with these may assume such flags are not relevant because +someone else renamed any such files in their repo back before the person +doing the filtering joined the project. And often, even those familiar +with handling arguments with spaces may not do so just because they +aren't in the mindset of thinking about everything that could possibly +go wrong. + +* Non-ascii filenames can be silently removed despite being in a desired +directory. Keeping only wanted paths is often done using pipelines like +`git ls-files | grep -v ^WANTED_DIR/ | xargs git rm`. ls-files will +only quote filenames if needed, so folks may not notice that one of the +files didn't match the regex (at least not until it's much too late). +Yes, someone who knows about core.quotePath can avoid this (unless they +have other special characters like \t, \n, or "), and people who use +ls-files -z with something other than grep can avoid this, but that +doesn't mean they will. + +* Similarly, when moving files around, one can find that filenames with +non-ascii or special characters end up in a different directory, one +that includes a double quote character. (This is technically the same +issue as above with quoting, but perhaps an interesting different way +that it can and has manifested as a problem.) + +* It's far too easy to accidentally mix up old and new history. It's +still possible with any tool, but git-filter-branch almost invites it. +If lucky, the only downside is users getting frustrated that they don't +know how to shrink their repo and remove the old stuff. If unlucky, +they merge old and new history and end up with multiple "copies" of each +commit, some of which have unwanted or sensitive files and others which +don't. This comes about in multiple different ways: + + ** the default to only doing a partial history rewrite ('--all' is not + the default and few examples show it) + + ** the fact that there's no automatic post-run cleanup + + ** the fact that --tag-name-filter (when used to rename tags) doesn't + remove the old tags but just adds new ones with the new name + + ** the fact that little educational information is provided to inform + users of the ramifications of a rewrite and how to avoid mixing old + and new history. For example, this man page discusses how users + need to understand that they need to rebase their changes for all + their branches on top of new history (or delete and reclone), but + that's only one of multiple concerns to consider. See the + "DISCUSSION" section of the git filter-repo manual page for more + details. + +* Annotated tags can be accidentally converted to lightweight tags, due +to either of two issues: + + ** Someone can do a history rewrite, realize they messed up, restore + from the backups in refs/original/, and then redo their + git-filter-branch command. (The backup in refs/original/ is not a + real backup; it dereferences tags first.) + + ** Running git-filter-branch with either --tags or --all in your + . In order to retain annotated tags as + annotated, you must use --tag-name-filter (and must not have + restored from refs/original/ in a previously botched rewrite). + +* Any commit messages that specify an encoding will become corrupted +by the rewrite; git-filter-branch ignores the encoding, takes the original +bytes, and feeds it to commit-tree without telling it the proper +encoding. (This happens whether or not --msg-filter is used.) + +* Commit messages (even if they are all UTF-8) by default become +corrupted due to not being updated -- any references to other commit +hashes in commit messages will now refer to no-longer-extant commits. + +* There are no facilities for helping users find what unwanted crud they +should delete, which means they are much more likely to have incomplete +or partial cleanups that sometimes result in confusion and people +wasting time trying to understand. (For example, folks tend to just +look for big files to delete instead of big directories or extensions, +and once they do so, then sometime later folks using the new repository +who are going through history will notice a build artifact directory +that has some files but not others, or a cache of dependencies +(node_modules or similar) which couldn't have ever been functional since +it's missing some files.) + +* If --prune-empty isn't specified, then the filtering process can +create hoards of confusing empty commits + +* If --prune-empty is specified, then intentionally placed empty +commits from before the filtering operation are also pruned instead of +just pruning commits that became empty due to filtering rules. + +* If --prune empty is specified, sometimes empty commits are missed +and left around anyway (a somewhat rare bug, but it happens...) + +* A minor issue, but users who have a goal to update all names and +emails in a repository may be led to --env-filter which will only update +authors and committers, missing taggers. + +* If the user provides a --tag-name-filter that maps multiple tags to +the same name, no warning or error is provided; git-filter-branch simply +overwrites each tag in some undocumented pre-defined order resulting in +only one tag at the end. (A git-filter-branch regression test requires +this surprising behavior.) + +Also, the poor performance of git-filter-branch often leads to safety +issues: + +* Coming up with the correct shell snippet to do the filtering you want +is sometimes difficult unless you're just doing a trivial modification +such as deleting a couple files. Unfortunately, people often learn if +the snippet is right or wrong by trying it out, but the rightness or +wrongness can vary depending on special circumstances (spaces in +filenames, non-ascii filenames, funny author names or emails, invalid +timezones, presence of grafts or replace objects, etc.), meaning they +may have to wait a long time, hit an error, then restart. The +performance of git-filter-branch is so bad that this cycle is painful, +reducing the time available to carefully re-check (to say nothing about +what it does to the patience of the person doing the rewrite even if +they do technically have more time available). This problem is extra +compounded because errors from broken filters may not be shown for a +long time and/or get lost in a sea of output. Even worse, broken +filters often just result in silent incorrect rewrites. + +* To top it all off, even when users finally find working commands, they +naturally want to share them. But they may be unaware that their repo +didn't have some special cases that someone else's does. So, when +someone else with a different repository runs the same commands, they +get hit by the problems above. Or, the user just runs commands that +really were vetted for special cases, but they run it on a different OS +where it doesn't work, as noted above. GIT --- diff --git a/Documentation/git-format-patch.txt b/Documentation/git-format-patch.txt index b9b97e63ae..0ac56f4b70 100644 --- a/Documentation/git-format-patch.txt +++ b/Documentation/git-format-patch.txt @@ -17,9 +17,9 @@ SYNOPSIS [--signature-file=] [-n | --numbered | -N | --no-numbered] [--start-number ] [--numbered-files] - [--in-reply-to=Message-Id] [--suffix=.] + [--in-reply-to=] [--suffix=.] [--ignore-if-in-upstream] - [--rfc] [--subject-prefix=Subject-Prefix] + [--rfc] [--subject-prefix=] [(--reroll-count|-v) ] [--to=] [--cc=] [--[no-]cover-letter] [--quiet] @@ -159,9 +159,9 @@ Beware that the default for 'git send-email' is to thread emails itself. If you want `git format-patch` to take care of threading, you will want to ensure that threading is disabled for `git send-email`. ---in-reply-to=Message-Id:: +--in-reply-to=:: Make the first mail (or all the mails with `--no-thread`) appear as a - reply to the given Message-Id, which avoids breaking threads to + reply to the given , which avoids breaking threads to provide a new patch series. --ignore-if-in-upstream:: @@ -171,9 +171,9 @@ will want to ensure that threading is disabled for `git send-email`. patches being generated, and any patch that matches is ignored. ---subject-prefix=:: +--subject-prefix=:: Instead of the standard '[PATCH]' prefix in the subject - line, instead use '[]'. This + line, instead use '[]'. This allows for useful naming of a patch series, and can be combined with the `--numbered` option. @@ -314,7 +314,8 @@ you can use `--suffix=-patch` to get `0001-description-of-my-change-patch`. --base=:: Record the base tree information to identify the state the patch series applies to. See the BASE TREE INFORMATION section - below for details. + below for details. If is "auto", a base commit is + automatically chosen. --root:: Treat the revision argument as a , even if it @@ -330,8 +331,9 @@ CONFIGURATION ------------- You can specify extra mail header lines to be added to each message, defaults for the subject prefix and file suffix, number patches when -outputting more than one patch, add "To" or "Cc:" headers, configure -attachments, and sign off patches with configuration variables. +outputting more than one patch, add "To:" or "Cc:" headers, configure +attachments, change the patch output directory, and sign off patches +with configuration variables. ------------ [format] @@ -343,7 +345,8 @@ attachments, and sign off patches with configuration variables. cc = attach [ = mime-boundary-string ] signOff = true - coverletter = auto + outputDirectory = + coverLetter = auto ------------ diff --git a/Documentation/git-gc.txt b/Documentation/git-gc.txt index 247f765604..0c114ad1ca 100644 --- a/Documentation/git-gc.txt +++ b/Documentation/git-gc.txt @@ -115,15 +115,14 @@ NOTES ----- 'git gc' tries very hard not to delete objects that are referenced -anywhere in your repository. In -particular, it will keep not only objects referenced by your current set -of branches and tags, but also objects referenced by the index, -remote-tracking branches, refs saved by 'git filter-branch' in -refs/original/, reflogs (which may reference commits in branches -that were later amended or rewound), and anything else in the refs/* namespace. -If you are expecting some objects to be deleted and they aren't, check -all of those locations and decide whether it makes sense in your case to -remove those references. +anywhere in your repository. In particular, it will keep not only +objects referenced by your current set of branches and tags, but also +objects referenced by the index, remote-tracking branches, notes saved +by 'git notes' under refs/notes/, reflogs (which may reference commits +in branches that were later amended or rewound), and anything else in +the refs/* namespace. If you are expecting some objects to be deleted +and they aren't, check all of those locations and decide whether it +makes sense in your case to remove those references. On the other hand, when 'git gc' runs concurrently with another process, there is a risk of it deleting an object that the other process is using diff --git a/Documentation/git-merge.txt b/Documentation/git-merge.txt index 01fd52dc70..092529c619 100644 --- a/Documentation/git-merge.txt +++ b/Documentation/git-merge.txt @@ -10,7 +10,7 @@ SYNOPSIS -------- [verse] 'git merge' [-n] [--stat] [--no-commit] [--squash] [--[no-]edit] - [-s ] [-X ] [-S[]] + [--no-verify] [-s ] [-X ] [-S[]] [--[no-]allow-unrelated-histories] [--[no-]rerere-autoupdate] [-m ] [-F ] [...] 'git merge' (--continue | --abort | --quit) diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt index 3146c1592d..639a4179d1 100644 --- a/Documentation/git-rebase.txt +++ b/Documentation/git-rebase.txt @@ -565,8 +565,6 @@ In addition, the following pairs of options are incompatible: * --preserve-merges and --interactive * --preserve-merges and --signoff * --preserve-merges and --rebase-merges - * --rebase-merges and --strategy - * --rebase-merges and --strategy-option * --keep-base and --onto * --keep-base and --root @@ -856,7 +854,8 @@ Hard case: The changes are not the same.:: This happens if the 'subsystem' rebase had conflicts, or used `--interactive` to omit, edit, squash, or fixup commits; or if the upstream used one of `commit --amend`, `reset`, or - `filter-branch`. + a full history rewriting command like + https://github.com/newren/git-filter-repo[`filter-repo`]. The easy case diff --git a/Documentation/git-replace.txt b/Documentation/git-replace.txt index 246dc9943c..f271d758c3 100644 --- a/Documentation/git-replace.txt +++ b/Documentation/git-replace.txt @@ -123,10 +123,10 @@ The following format are available: CREATING REPLACEMENT OBJECTS ---------------------------- -linkgit:git-filter-branch[1], linkgit:git-hash-object[1] and -linkgit:git-rebase[1], among other git commands, can be used to create -replacement objects from existing objects. The `--edit` option can -also be used with 'git replace' to create a replacement object by +linkgit:git-hash-object[1], linkgit:git-rebase[1], and +https://github.com/newren/git-filter-repo[git-filter-repo], among other git commands, can be used to +create replacement objects from existing objects. The `--edit` option +can also be used with 'git replace' to create a replacement object by editing an existing object. If you want to replace many blobs, trees or commits that are part of a @@ -148,13 +148,13 @@ pending objects. SEE ALSO -------- linkgit:git-hash-object[1] -linkgit:git-filter-branch[1] linkgit:git-rebase[1] linkgit:git-tag[1] linkgit:git-branch[1] linkgit:git-commit[1] linkgit:git-var[1] linkgit:git[1] +https://github.com/newren/git-filter-repo[git-filter-repo] GIT --- diff --git a/Documentation/git-svn.txt b/Documentation/git-svn.txt index 30711625fd..53774f5b64 100644 --- a/Documentation/git-svn.txt +++ b/Documentation/git-svn.txt @@ -769,11 +769,11 @@ option for (hopefully) obvious reasons. + This option is NOT recommended as it makes it difficult to track down old references to SVN revision numbers in existing documentation, bug -reports and archives. If you plan to eventually migrate from SVN to Git -and are certain about dropping SVN history, consider -linkgit:git-filter-branch[1] instead. filter-branch also allows -reformatting of metadata for ease-of-reading and rewriting authorship -info for non-"svn.authorsFile" users. +reports, and archives. If you plan to eventually migrate from SVN to +Git and are certain about dropping SVN history, consider +https://github.com/newren/git-filter-repo[git-filter-repo] instead. +filter-repo also allows reformatting of metadata for ease-of-reading +and rewriting authorship info for non-"svn.authorsFile" users. svn.useSvmProps:: svn-remote..useSvmProps:: diff --git a/Documentation/gitattributes.txt b/Documentation/gitattributes.txt index fb1d188d44..c5a528c667 100644 --- a/Documentation/gitattributes.txt +++ b/Documentation/gitattributes.txt @@ -810,6 +810,8 @@ patterns are available: - `css` suitable for cascading style sheets. +- `dts` suitable for devicetree (DTS) files. + - `fortran` suitable for source code in the Fortran language. - `fountain` suitable for Fountain documents. diff --git a/Documentation/gitcli.txt b/Documentation/gitcli.txt index 1ed3ca33b7..4b32876b6e 100644 --- a/Documentation/gitcli.txt +++ b/Documentation/gitcli.txt @@ -37,6 +37,12 @@ arguments. Here are the rules: file called HEAD in your work tree, `git diff HEAD` is ambiguous, and you have to say either `git diff HEAD --` or `git diff -- HEAD` to disambiguate. + + * Because `--` disambiguates revisions and paths in some commands, it + cannot be used for those commands to separate options and revisions. + You can use `--end-of-options` for this (it also works for commands + that do not distinguish between revisions in paths, in which case it + is simply an alias for `--`). + When writing a script that is expected to handle random user-input, it is a good practice to make it explicit which arguments are which by placing diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt index 82cd573776..50365f2914 100644 --- a/Documentation/githooks.txt +++ b/Documentation/githooks.txt @@ -103,6 +103,28 @@ The default 'pre-commit' hook, when enabled--and with the `hooks.allownonascii` config option unset or set to false--prevents the use of non-ASCII filenames. +pre-merge-commit +~~~~~~~~~~~~~~~~ + +This hook is invoked by linkgit:git-merge[1], and can be bypassed +with the `--no-verify` option. It takes no parameters, and is +invoked after the merge has been carried out successfully and before +obtaining the proposed commit log message to +make a commit. Exiting with a non-zero status from this script +causes the `git merge` command to abort before creating a commit. + +The default 'pre-merge-commit' hook, when enabled, runs the +'pre-commit' hook, if the latter is enabled. + +This hook is invoked with the environment variable +`GIT_EDITOR=:` if the command will not bring up an editor +to modify the commit message. + +If the merge cannot be carried out automatically, the conflicts +need to be resolved and the result committed separately (see +linkgit:git-merge[1]). At that point, this hook will not be executed, +but the 'pre-commit' hook will, if it is enabled. + prepare-commit-msg ~~~~~~~~~~~~~~~~~~ @@ -425,10 +447,12 @@ post-rewrite This hook is invoked by commands that rewrite commits (linkgit:git-commit[1] when called with `--amend` and -linkgit:git-rebase[1]; currently `git filter-branch` does 'not' call -it!). Its first argument denotes the command it was invoked by: -currently one of `amend` or `rebase`. Further command-dependent -arguments may be passed in the future. +linkgit:git-rebase[1]; however, full-history (re)writing tools like +linkgit:git-fast-import[1] or +https://github.com/newren/git-filter-repo[git-filter-repo] typically +do not call it!). Its first argument denotes the command it was +invoked by: currently one of `amend` or `rebase`. Further +command-dependent arguments may be passed in the future. The hook receives a list of the rewritten commits on stdin, in the format diff --git a/Documentation/gitremote-helpers.txt b/Documentation/gitremote-helpers.txt index 43f80c8068..a5c3c04371 100644 --- a/Documentation/gitremote-helpers.txt +++ b/Documentation/gitremote-helpers.txt @@ -297,9 +297,13 @@ Supported if the helper has the "option" capability. same batch are complete. Only objects which were reported in the output of 'list' with a sha1 may be fetched this way. + -Optionally may output a 'lock ' line indicating a file under -GIT_DIR/objects/pack which is keeping a pack until refs can be -suitably updated. +Optionally may output a 'lock ' line indicating the full path of +a file under `$GIT_DIR/objects/pack` which is keeping a pack until +refs can be suitably updated. The path must end with `.keep`. This is +a mechanism to name a tuple by giving only the keep +component. The kept pack will not be deleted by a concurrent repack, +even though its objects may not be referenced until the fetch completes. +The `.keep` file will be deleted at the conclusion of the fetch. + If option 'check-connectivity' is requested, the helper must output 'connectivity-ok' if the clone is self-contained and connected. diff --git a/Documentation/gitrepository-layout.txt b/Documentation/gitrepository-layout.txt index 216b11ee88..d6388f10bb 100644 --- a/Documentation/gitrepository-layout.txt +++ b/Documentation/gitrepository-layout.txt @@ -59,7 +59,7 @@ objects/[0-9a-f][0-9a-f]:: here are often called 'unpacked' (or 'loose') objects. objects/pack:: - Packs (files that store many object in compressed form, + Packs (files that store many objects in compressed form, along with index files to allow them to be randomly accessed) are found in this directory. diff --git a/Documentation/merge-options.txt b/Documentation/merge-options.txt index 79a00d2a4a..59b8ff1e51 100644 --- a/Documentation/merge-options.txt +++ b/Documentation/merge-options.txt @@ -40,20 +40,24 @@ set to `no` at the beginning of them. case of a merge conflict. --ff:: - When the merge resolves as a fast-forward, only update the branch - pointer, without creating a merge commit. This is the default - behavior. - --no-ff:: - Create a merge commit even when the merge resolves as a - fast-forward. This is the default behaviour when merging an - annotated (and possibly signed) tag that is not stored in - its natural place in 'refs/tags/' hierarchy. - --ff-only:: - Refuse to merge and exit with a non-zero status unless the - current `HEAD` is already up to date or the merge can be - resolved as a fast-forward. + Specifies how a merge is handled when the merged-in history is + already a descendant of the current history. `--ff` is the + default unless merging an annotated (and possibly signed) tag + that is not stored in its natural place in the `refs/tags/` + hierarchy, in which case `--no-ff` is assumed. ++ +With `--ff`, when possible resolve the merge as a fast-forward (only +update the branch pointer to match the merged branch; do not create a +merge commit). When not possible (when the merged-in history is not a +descendant of the current history), create a merge commit. ++ +With `--no-ff`, create a merge commit in all cases, even when the merge +could instead be resolved as a fast-forward. ++ +With `--ff-only`, resolve the merge as a fast-forward when possible. +When not possible, refuse to merge and exit with a non-zero status. -S[]:: --gpg-sign[=]:: @@ -105,6 +109,10 @@ option can be used to override --squash. + With --squash, --commit is not allowed, and will fail. +--no-verify:: + This option bypasses the pre-merge and commit-msg hooks. + See also linkgit:githooks[5]. + -s :: --strategy=:: Use the given merge strategy; can be supplied more than diff --git a/Documentation/rev-list-options.txt b/Documentation/rev-list-options.txt index bb1251c036..90ff9e2bea 100644 --- a/Documentation/rev-list-options.txt +++ b/Documentation/rev-list-options.txt @@ -756,6 +756,22 @@ explicitly-given commit or tree. Note that the form '--filter=sparse:path=' that wants to read from an arbitrary path on the filesystem has been dropped for security reasons. ++ +Multiple '--filter=' flags can be specified to combine filters. Only +objects which are accepted by every filter are included. ++ +The form '--filter=combine:++...' can also be +used to combined several filters, but this is harder than just repeating +the '--filter' flag and is usually not necessary. Filters are joined by +'{plus}' and individual filters are %-encoded (i.e. URL-encoded). +Besides the '{plus}' and '%' characters, the following characters are +reserved and also must be encoded: `~!@#$^&*()[]{}\;",<>?`+'`+ +as well as all characters with ASCII code <= `0x20`, which includes +space and newline. ++ +Other arbitrary characters can also be encoded. For instance, +'combine:tree:3+blob:none' and 'combine:tree%3A3+blob%3Anone' are +equivalent. --no-filter:: Turn off any previous `--filter=` argument. diff --git a/Documentation/technical/partial-clone.txt b/Documentation/technical/partial-clone.txt index 896c7b3878..210373e258 100644 --- a/Documentation/technical/partial-clone.txt +++ b/Documentation/technical/partial-clone.txt @@ -30,12 +30,20 @@ advance* during clone and fetch operations and thereby reduce download times and disk usage. Missing objects can later be "demand fetched" if/when needed. +A remote that can later provide the missing objects is called a +promisor remote, as it promises to send the objects when +requested. Initialy Git supported only one promisor remote, the origin +remote from which the user cloned and that was configured in the +"extensions.partialClone" config option. Later support for more than +one promisor remote has been implemented. + Use of partial clone requires that the user be online and the origin -remote be available for on-demand fetching of missing objects. This may -or may not be problematic for the user. For example, if the user can -stay within the pre-selected subset of the source tree, they may not -encounter any missing objects. Alternatively, the user could try to -pre-fetch various objects if they know that they are going offline. +remote or other promisor remotes be available for on-demand fetching +of missing objects. This may or may not be problematic for the user. +For example, if the user can stay within the pre-selected subset of +the source tree, they may not encounter any missing objects. +Alternatively, the user could try to pre-fetch various objects if they +know that they are going offline. Non-Goals @@ -100,18 +108,18 @@ or commits that reference missing trees. Handling Missing Objects ------------------------ -- An object may be missing due to a partial clone or fetch, or missing due - to repository corruption. To differentiate these cases, the local - repository specially indicates such filtered packfiles obtained from the - promisor remote as "promisor packfiles". +- An object may be missing due to a partial clone or fetch, or missing + due to repository corruption. To differentiate these cases, the + local repository specially indicates such filtered packfiles + obtained from promisor remotes as "promisor packfiles". + These promisor packfiles consist of a ".promisor" file with arbitrary contents (like the ".keep" files), in addition to their ".pack" and ".idx" files. - The local repository considers a "promisor object" to be an object that - it knows (to the best of its ability) that the promisor remote has promised - that it has, either because the local repository has that object in one of + it knows (to the best of its ability) that promisor remotes have promised + that they have, either because the local repository has that object in one of its promisor packfiles, or because another promisor object refers to it. + When Git encounters a missing object, Git can see if it is a promisor object @@ -123,12 +131,12 @@ expensive-to-modify list of missing objects.[a] - Since almost all Git code currently expects any referenced object to be present locally and because we do not want to force every command to do a dry-run first, a fallback mechanism is added to allow Git to attempt - to dynamically fetch missing objects from the promisor remote. + to dynamically fetch missing objects from promisor remotes. + When the normal object lookup fails to find an object, Git invokes -fetch-object to try to get the object from the server and then retry -the object lookup. This allows objects to be "faulted in" without -complicated prediction algorithms. +promisor_remote_get_direct() to try to get the object from a promisor +remote and then retry the object lookup. This allows objects to be +"faulted in" without complicated prediction algorithms. + For efficiency reasons, no check as to whether the missing object is actually a promisor object is performed. @@ -157,8 +165,7 @@ and prefetch those objects in bulk. + We are not happy with this global variable and would like to remove it, but that requires significant refactoring of the object code to pass an -additional flag. We hope that concurrent efforts to add an ODB API can -encompass this. +additional flag. Fetching Missing Objects @@ -182,21 +189,63 @@ has been updated to not use any object flags when the corresponding argument though they are not necessary. +Using many promisor remotes +--------------------------- + +Many promisor remotes can be configured and used. + +This allows for example a user to have multiple geographically-close +cache servers for fetching missing blobs while continuing to do +filtered `git-fetch` commands from the central server. + +When fetching objects, promisor remotes are tried one after the other +until all the objects have been fetched. + +Remotes that are considered "promisor" remotes are those specified by +the following configuration variables: + +- `extensions.partialClone = ` + +- `remote..promisor = true` + +- `remote..partialCloneFilter = ...` + +Only one promisor remote can be configured using the +`extensions.partialClone` config variable. This promisor remote will +be the last one tried when fetching objects. + +We decided to make it the last one we try, because it is likely that +someone using many promisor remotes is doing so because the other +promisor remotes are better for some reason (maybe they are closer or +faster for some kind of objects) than the origin, and the origin is +likely to be the remote specified by extensions.partialClone. + +This justification is not very strong, but one choice had to be made, +and anyway the long term plan should be to make the order somehow +fully configurable. + +For now though the other promisor remotes will be tried in the order +they appear in the config file. + Current Limitations ------------------- -- The remote used for a partial clone (or the first partial fetch - following a regular clone) is marked as the "promisor remote". +- It is not possible to specify the order in which the promisor + remotes are tried in other ways than the order in which they appear + in the config file. + -We are currently limited to a single promisor remote and only that -remote may be used for subsequent partial fetches. +It is also not possible to specify an order to be used when fetching +from one remote and a different order when fetching from another +remote. + +- It is not possible to push only specific objects to a promisor + remote. + -We accept this limitation because we believe initial users of this -feature will be using it on repositories with a strong single central -server. +It is not possible to push at the same time to multiple promisor +remote in a specific order. -- Dynamic object fetching will only ask the promisor remote for missing - objects. We assume that the promisor remote has a complete view of the +- Dynamic object fetching will only ask promisor remotes for missing + objects. We assume that promisor remotes have a complete view of the repository and can satisfy all such requests. - Repack essentially treats promisor and non-promisor packfiles as 2 @@ -218,15 +267,17 @@ server. Future Work ----------- -- Allow more than one promisor remote and define a strategy for fetching - missing objects from specific promisor remotes or of iterating over the - set of promisor remotes until a missing object is found. +- Improve the way to specify the order in which promisor remotes are + tried. + -A user might want to have multiple geographically-close cache servers -for fetching missing blobs while continuing to do filtered `git-fetch` -commands from the central server, for example. +For example this could allow to specify explicitly something like: +"When fetching from this remote, I want to use these promisor remotes +in this order, though, when pushing or fetching to that remote, I want +to use those promisor remotes in that order." + +- Allow pushing to promisor remotes. + -Or the user might want to work in a triangular work flow with multiple +The user might want to work in a triangular work flow with multiple promisor remotes that each have an incomplete view of the repository. - Allow repack to work on promisor packfiles (while keeping them distinct diff --git a/Makefile b/Makefile index f9255344ae..f879697ea3 100644 --- a/Makefile +++ b/Makefile @@ -884,7 +884,6 @@ LIB_OBJS += ewah/ewah_io.o LIB_OBJS += ewah/ewah_rlw.o LIB_OBJS += exec-cmd.o LIB_OBJS += fetch-negotiator.o -LIB_OBJS += fetch-object.o LIB_OBJS += fetch-pack.o LIB_OBJS += fsck.o LIB_OBJS += fsmonitor.o @@ -948,6 +947,7 @@ LIB_OBJS += preload-index.o LIB_OBJS += pretty.o LIB_OBJS += prio-queue.o LIB_OBJS += progress.o +LIB_OBJS += promisor-remote.o LIB_OBJS += prompt.o LIB_OBJS += protocol.o LIB_OBJS += quote.o @@ -965,6 +965,7 @@ LIB_OBJS += refspec.o LIB_OBJS += ref-filter.o LIB_OBJS += remote.o LIB_OBJS += replace-object.o +LIB_OBJS += repo-settings.o LIB_OBJS += repository.o LIB_OBJS += rerere.o LIB_OBJS += resolve-undo.o diff --git a/apply.c b/apply.c index cde95369bb..57a61f2881 100644 --- a/apply.c +++ b/apply.c @@ -4643,6 +4643,7 @@ static int apply_patch(struct apply_state *state, struct patch *list = NULL, **listp = &list; int skipped_patch = 0; int res = 0; + int flush_attributes = 0; state->patch_input_file = filename; if (read_patch_file(&buf, fd) < 0) @@ -4670,6 +4671,14 @@ static int apply_patch(struct apply_state *state, patch_stats(state, patch); *listp = patch; listp = &patch->next; + + if ((patch->new_name && + ends_with_path_components(patch->new_name, + GITATTRIBUTES_FILE)) || + (patch->old_name && + ends_with_path_components(patch->old_name, + GITATTRIBUTES_FILE))) + flush_attributes = 1; } else { if (state->apply_verbosity > verbosity_normal) @@ -4746,6 +4755,8 @@ static int apply_patch(struct apply_state *state, if (state->summary && state->apply_verbosity > verbosity_silent) summary_patch_list(list); + if (flush_attributes) + reset_parsed_attributes(); end: free_patch_list(list); strbuf_release(&buf); diff --git a/archive-tar.c b/archive-tar.c index 3e53aac1e6..e16d3f756d 100644 --- a/archive-tar.c +++ b/archive-tar.c @@ -142,19 +142,25 @@ static int stream_blocked(const struct object_id *oid) * string and appends it to a struct strbuf. */ static void strbuf_append_ext_header(struct strbuf *sb, const char *keyword, - const char *value, unsigned int valuelen) + const char *value, size_t valuelen) { - int len, tmp; + size_t orig_len = sb->len; + size_t len, tmp; /* "%u %s=%s\n" */ len = 1 + 1 + strlen(keyword) + 1 + valuelen + 1; - for (tmp = len; tmp > 9; tmp /= 10) + for (tmp = 1; len / 10 >= tmp; tmp *= 10) len++; strbuf_grow(sb, len); - strbuf_addf(sb, "%u %s=", len, keyword); + strbuf_addf(sb, "%"PRIuMAX" %s=", (uintmax_t)len, keyword); strbuf_add(sb, value, valuelen); strbuf_addch(sb, '\n'); + + if (len != sb->len - orig_len) + BUG("pax extended header length miscalculated as %"PRIuMAX + ", should be %"PRIuMAX, + (uintmax_t)len, (uintmax_t)(sb->len - orig_len)); } /* diff --git a/banned.h b/banned.h index 447af24807..60a18d4403 100644 --- a/banned.h +++ b/banned.h @@ -26,7 +26,7 @@ #define vsprintf(...) BANNED(vsprintf) #else #define sprintf(buf,fmt,arg) BANNED(sprintf) -#define vsprintf(buf,fmt,arg) BANNED(sprintf) +#define vsprintf(buf,fmt,arg) BANNED(vsprintf) #endif #endif /* BANNED_H */ diff --git a/builtin/am.c b/builtin/am.c index 1aea657a7f..ee7305eaa6 100644 --- a/builtin/am.c +++ b/builtin/am.c @@ -1272,7 +1272,9 @@ static void get_commit_info(struct am_state *state, struct commit *commit) buffer = logmsg_reencode(commit, NULL, get_commit_output_encoding()); ident_line = find_commit_header(buffer, "author", &ident_len); - + if (!ident_line) + die(_("missing author line in commit %s"), + oid_to_hex(&commit->object.oid)); if (split_ident_line(&id, ident_line, ident_len) < 0) die(_("invalid ident line: %.*s"), (int)ident_len, ident_line); diff --git a/builtin/cat-file.c b/builtin/cat-file.c index 995d47c85a..d6a1aa74cd 100644 --- a/builtin/cat-file.c +++ b/builtin/cat-file.c @@ -15,6 +15,7 @@ #include "sha1-array.h" #include "packfile.h" #include "object-store.h" +#include "promisor-remote.h" struct batch_options { int enabled; @@ -524,8 +525,8 @@ static int batch_objects(struct batch_options *opt) if (opt->all_objects) { struct object_cb_data cb; - if (repository_format_partial_clone) - warning("This repository has extensions.partialClone set. Some objects may not be loaded."); + if (has_promisor_remote()) + warning("This repository uses promisor remotes. Some objects may not be loaded."); cb.opt = opt; cb.expand = &data; diff --git a/builtin/checkout.c b/builtin/checkout.c index 8d3ad7cd9e..1283727761 100644 --- a/builtin/checkout.c +++ b/builtin/checkout.c @@ -731,13 +731,6 @@ static int merge_working_tree(const struct checkout_opts *opts, "the following files:\n%s"), sb.buf); strbuf_release(&sb); - if (repo_index_has_changes(the_repository, - get_commit_tree(old_branch_info->commit), - &sb)) - warning(_("staged changes in the following files may be lost: %s"), - sb.buf); - strbuf_release(&sb); - /* Do more real merge */ /* @@ -1715,6 +1708,15 @@ int cmd_checkout(int argc, const char **argv, const char *prefix) opts.checkout_index = -2; /* default on */ opts.checkout_worktree = -2; /* default on */ + if (argc == 3 && !strcmp(argv[1], "-b")) { + /* + * User ran 'git checkout -b ' and expects + * the same behavior as 'git switch -c '. + */ + opts.switch_branch_doing_nothing_is_ok = 0; + opts.only_merge_on_switching_branches = 1; + } + options = parse_options_dup(checkout_options); options = add_common_options(&opts, options); options = add_common_switch_branch_options(&opts, options); diff --git a/builtin/clone.c b/builtin/clone.c index f665b28ccc..2048b6760a 100644 --- a/builtin/clone.c +++ b/builtin/clone.c @@ -1160,13 +1160,11 @@ int cmd_clone(int argc, const char **argv, const char *prefix) transport->server_options = &server_options; if (filter_options.choice) { - struct strbuf expanded_filter_spec = STRBUF_INIT; - expand_list_objects_filter_spec(&filter_options, - &expanded_filter_spec); + const char *spec = + expand_list_objects_filter_spec(&filter_options); transport_set_option(transport, TRANS_OPT_LIST_OBJECTS_FILTER, - expanded_filter_spec.buf); + spec); transport_set_option(transport, TRANS_OPT_FROM_PROMISOR, "1"); - strbuf_release(&expanded_filter_spec); } if (transport->smart_options && !deepen && !filter_options.choice) diff --git a/builtin/describe.c b/builtin/describe.c index 200154297d..e048f85484 100644 --- a/builtin/describe.c +++ b/builtin/describe.c @@ -313,7 +313,7 @@ static void describe_commit(struct object_id *oid, struct strbuf *dst) */ append_name(n, dst); if (longformat) - append_suffix(0, n->tag ? &n->tag->tagged->oid : oid, dst); + append_suffix(0, n->tag ? get_tagged_oid(n->tag) : oid, dst); if (suffix) strbuf_addstr(dst, suffix); return; diff --git a/builtin/fetch.c b/builtin/fetch.c index 717dd14e89..9b27ae9681 100644 --- a/builtin/fetch.c +++ b/builtin/fetch.c @@ -23,6 +23,8 @@ #include "packfile.h" #include "list-objects-filter-options.h" #include "commit-reach.h" +#include "branch.h" +#include "promisor-remote.h" #define FORCED_UPDATES_DELAY_WARNING_IN_MS (10 * 1000) @@ -50,7 +52,8 @@ static int fetch_prune_tags_config = -1; /* unspecified */ static int prune_tags = -1; /* unspecified */ #define PRUNE_TAGS_BY_DEFAULT 0 /* do we prune tags by default? */ -static int all, append, dry_run, force, keep, multiple, update_head_ok, verbosity, deepen_relative; +static int all, append, dry_run, force, keep, multiple, update_head_ok; +static int verbosity, deepen_relative, set_upstream; static int progress = -1; static int enable_auto_gc = 1; static int tags = TAGS_DEFAULT, unshallow, update_shallow, deepen; @@ -123,6 +126,8 @@ static struct option builtin_fetch_options[] = { OPT__VERBOSITY(&verbosity), OPT_BOOL(0, "all", &all, N_("fetch from all remotes")), + OPT_BOOL(0, "set-upstream", &set_upstream, + N_("set upstream for git pull/fetch")), OPT_BOOL('a', "append", &append, N_("append to .git/FETCH_HEAD instead of overwriting")), OPT_STRING(0, "upload-pack", &upload_pack, N_("path"), @@ -1238,13 +1243,10 @@ static struct transport *prepare_transport(struct remote *remote, int deepen) if (update_shallow) set_option(transport, TRANS_OPT_UPDATE_SHALLOW, "yes"); if (filter_options.choice) { - struct strbuf expanded_filter_spec = STRBUF_INIT; - expand_list_objects_filter_spec(&filter_options, - &expanded_filter_spec); - set_option(transport, TRANS_OPT_LIST_OBJECTS_FILTER, - expanded_filter_spec.buf); + const char *spec = + expand_list_objects_filter_spec(&filter_options); + set_option(transport, TRANS_OPT_LIST_OBJECTS_FILTER, spec); set_option(transport, TRANS_OPT_FROM_PROMISOR, "1"); - strbuf_release(&expanded_filter_spec); } if (negotiation_tip.nr) { if (transport->smart_options) @@ -1367,6 +1369,51 @@ static int do_fetch(struct transport *transport, retcode = 1; goto cleanup; } + + if (set_upstream) { + struct branch *branch = branch_get("HEAD"); + struct ref *rm; + struct ref *source_ref = NULL; + + /* + * We're setting the upstream configuration for the + * current branch. The relevent upstream is the + * fetched branch that is meant to be merged with the + * current one, i.e. the one fetched to FETCH_HEAD. + * + * When there are several such branches, consider the + * request ambiguous and err on the safe side by doing + * nothing and just emit a warning. + */ + for (rm = ref_map; rm; rm = rm->next) { + if (!rm->peer_ref) { + if (source_ref) { + warning(_("multiple branch detected, incompatible with --set-upstream")); + goto skip; + } else { + source_ref = rm; + } + } + } + if (source_ref) { + if (!strcmp(source_ref->name, "HEAD") || + starts_with(source_ref->name, "refs/heads/")) + install_branch_config(0, + branch->name, + transport->remote->name, + source_ref->name); + else if (starts_with(source_ref->name, "refs/remotes/")) + warning(_("not setting upstream for a remote remote-tracking branch")); + else if (starts_with(source_ref->name, "refs/tags/")) + warning(_("not setting upstream for a remote tag")); + else + warning(_("unknown branch type")); + } else { + warning(_("no source branch found.\n" + "you need to specify exactly one branch with the --set-upstream option.")); + } + } + skip: free_refs(ref_map); /* if neither --no-tags nor --tags was specified, do automated tag @@ -1510,37 +1557,27 @@ static inline void fetch_one_setup_partial(struct remote *remote) * If no prior partial clone/fetch and the current fetch DID NOT * request a partial-fetch, do a normal fetch. */ - if (!repository_format_partial_clone && !filter_options.choice) + if (!has_promisor_remote() && !filter_options.choice) return; /* - * If this is the FIRST partial-fetch request, we enable partial - * on this repo and remember the given filter-spec as the default - * for subsequent fetches to this remote. + * If this is a partial-fetch request, we enable partial on + * this repo if not already enabled and remember the given + * filter-spec as the default for subsequent fetches to this + * remote. */ - if (!repository_format_partial_clone && filter_options.choice) { + if (filter_options.choice) { partial_clone_register(remote->name, &filter_options); return; } - /* - * We are currently limited to only ONE promisor remote and only - * allow partial-fetches from the promisor remote. - */ - if (strcmp(remote->name, repository_format_partial_clone)) { - if (filter_options.choice) - die(_("--filter can only be used with the remote " - "configured in extensions.partialClone")); - return; - } - /* * Do a partial-fetch from the promisor remote using either the * explicitly given filter-spec or inherit the filter-spec from * the config. */ if (!filter_options.choice) - partial_clone_get_default_filter_spec(&filter_options); + partial_clone_get_default_filter_spec(&filter_options, remote->name); return; } @@ -1661,7 +1698,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix) if (depth || deepen_since || deepen_not.nr) deepen = 1; - if (filter_options.choice && !repository_format_partial_clone) + if (filter_options.choice && !has_promisor_remote()) die("--filter can only be used when extensions.partialClone is set"); if (all) { @@ -1695,7 +1732,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix) } if (remote) { - if (filter_options.choice || repository_format_partial_clone) + if (filter_options.choice || has_promisor_remote()) fetch_one_setup_partial(remote); result = fetch_one(remote, argc, argv, prune_tags_ok); } else { diff --git a/builtin/gc.c b/builtin/gc.c index 305fb0f45a..fadb45489f 100644 --- a/builtin/gc.c +++ b/builtin/gc.c @@ -27,6 +27,7 @@ #include "pack-objects.h" #include "blob.h" #include "tree.h" +#include "promisor-remote.h" #define FAILED_RUN "failed to run %s" @@ -41,7 +42,6 @@ static int aggressive_depth = 50; static int aggressive_window = 250; static int gc_auto_threshold = 6700; static int gc_auto_pack_limit = 50; -static int gc_write_commit_graph; static int detach_auto = 1; static timestamp_t gc_log_expire_time; static const char *gc_log_expire = "1.day.ago"; @@ -148,7 +148,6 @@ static void gc_config(void) git_config_get_int("gc.aggressivedepth", &aggressive_depth); git_config_get_int("gc.auto", &gc_auto_threshold); git_config_get_int("gc.autopacklimit", &gc_auto_pack_limit); - git_config_get_bool("gc.writecommitgraph", &gc_write_commit_graph); git_config_get_bool("gc.autodetach", &detach_auto); git_config_get_expiry("gc.pruneexpire", &prune_expire); git_config_get_expiry("gc.worktreepruneexpire", &prune_worktrees_expire); @@ -661,7 +660,7 @@ int cmd_gc(int argc, const char **argv, const char *prefix) argv_array_push(&prune, prune_expire); if (quiet) argv_array_push(&prune, "--no-progress"); - if (repository_format_partial_clone) + if (has_promisor_remote()) argv_array_push(&prune, "--exclude-promisor-objects"); if (run_command_v_opt(prune.argv, RUN_GIT_CMD)) @@ -685,11 +684,11 @@ int cmd_gc(int argc, const char **argv, const char *prefix) clean_pack_garbage(); } - if (gc_write_commit_graph && - write_commit_graph_reachable(get_object_directory(), - !quiet && !daemonized ? COMMIT_GRAPH_WRITE_PROGRESS : 0, - NULL)) - return 1; + prepare_repo_settings(the_repository); + if (the_repository->settings.gc_write_commit_graph == 1) + write_commit_graph_reachable(get_object_directory(), + !quiet && !daemonized ? COMMIT_GRAPH_WRITE_PROGRESS : 0, + NULL); if (auto_gc && too_many_loose_objects()) warning(_("There are too many unreachable loose objects; " diff --git a/builtin/grep.c b/builtin/grep.c index 2699001fbd..69ac053acc 100644 --- a/builtin/grep.c +++ b/builtin/grep.c @@ -1110,8 +1110,8 @@ int cmd_grep(int argc, const char **argv, const char *prefix) strbuf_addf(&buf, "+/%s%s", strcmp("less", pager) ? "" : "*", opt.pattern_list->pattern); - string_list_append(&path_list, buf.buf); - strbuf_detach(&buf, NULL); + string_list_append(&path_list, + strbuf_detach(&buf, NULL)); } } diff --git a/builtin/index-pack.c b/builtin/index-pack.c index 0d55f73b0b..a23454da6e 100644 --- a/builtin/index-pack.c +++ b/builtin/index-pack.c @@ -14,7 +14,7 @@ #include "thread-utils.h" #include "packfile.h" #include "object-store.h" -#include "fetch-object.h" +#include "promisor-remote.h" static const char index_pack_usage[] = "git index-pack [-v] [-o ] [--keep | --keep=] [--verify] [--strict] ( | --stdin [--fix-thin] [])"; @@ -1352,7 +1352,7 @@ static void fix_unresolved_deltas(struct hashfile *f) sorted_by_pos[i] = &ref_deltas[i]; QSORT(sorted_by_pos, nr_ref_deltas, delta_pos_compare); - if (repository_format_partial_clone) { + if (has_promisor_remote()) { /* * Prefetch the delta bases. */ @@ -1366,8 +1366,8 @@ static void fix_unresolved_deltas(struct hashfile *f) oid_array_append(&to_fetch, &d->oid); } if (to_fetch.nr) - fetch_objects(repository_format_partial_clone, - to_fetch.oid, to_fetch.nr); + promisor_remote_get_direct(the_repository, + to_fetch.oid, to_fetch.nr); oid_array_clear(&to_fetch); } diff --git a/builtin/log.c b/builtin/log.c index 44b10b3415..c4b35fdaf9 100644 --- a/builtin/log.c +++ b/builtin/log.c @@ -627,6 +627,7 @@ int cmd_show(int argc, const char **argv, const char *prefix) break; case OBJ_TAG: { struct tag *t = (struct tag *)o; + struct object_id *oid = get_tagged_oid(t); if (rev.shown_one) putchar('\n'); @@ -638,10 +639,10 @@ int cmd_show(int argc, const char **argv, const char *prefix) rev.shown_one = 1; if (ret) break; - o = parse_object(the_repository, &t->tagged->oid); + o = parse_object(the_repository, oid); if (!o) ret = error(_("could not read object %s"), - oid_to_hex(&t->tagged->oid)); + oid_to_hex(oid)); objects[i].item = o; i--; break; diff --git a/builtin/merge.c b/builtin/merge.c index e2ccbc44e2..c9746e37b8 100644 --- a/builtin/merge.c +++ b/builtin/merge.c @@ -81,7 +81,7 @@ static int show_progress = -1; static int default_to_upstream = 1; static int signoff; static const char *sign_commit; -static int verify_msg = 1; +static int no_verify; static struct strategy all_strategy[] = { { "recursive", DEFAULT_TWOHEAD | NO_TRIVIAL }, @@ -287,7 +287,7 @@ static struct option builtin_merge_options[] = { N_("GPG sign commit"), PARSE_OPT_OPTARG, NULL, (intptr_t) "" }, OPT_BOOL(0, "overwrite-ignore", &overwrite_ignore, N_("update ignored files (default)")), OPT_BOOL(0, "signoff", &signoff, N_("add Signed-off-by:")), - OPT_BOOL(0, "verify", &verify_msg, N_("verify commit-msg hook")), + OPT_BOOL(0, "no-verify", &no_verify, N_("bypass pre-merge-commit and commit-msg hooks")), OPT_END() }; @@ -816,6 +816,18 @@ static void write_merge_heads(struct commit_list *); static void prepare_to_commit(struct commit_list *remoteheads) { struct strbuf msg = STRBUF_INIT; + const char *index_file = get_index_file(); + + if (!no_verify && run_commit_hook(0 < option_edit, index_file, "pre-merge-commit", NULL)) + abort_commit(remoteheads, NULL); + /* + * Re-read the index as pre-merge-commit hook could have updated it, + * and write it out as a tree. We must do this before we invoke + * the editor and after we invoke run_status above. + */ + if (find_hook("pre-merge-commit")) + discard_cache(); + read_cache_from(index_file); strbuf_addbuf(&msg, &merge_msg); if (squash) BUG("the control must not reach here under --squash"); @@ -842,7 +854,7 @@ static void prepare_to_commit(struct commit_list *remoteheads) abort_commit(remoteheads, NULL); } - if (verify_msg && run_commit_hook(0 < option_edit, get_index_file(), + if (!no_verify && run_commit_hook(0 < option_edit, get_index_file(), "commit-msg", git_path_merge_msg(the_repository), NULL)) abort_commit(remoteheads, NULL); diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c index 76ce906946..5876583220 100644 --- a/builtin/pack-objects.c +++ b/builtin/pack-objects.c @@ -610,12 +610,12 @@ static int mark_tagged(const char *path, const struct object_id *oid, int flag, void *cb_data) { struct object_id peeled; - struct object_entry *entry = packlist_find(&to_pack, oid, NULL); + struct object_entry *entry = packlist_find(&to_pack, oid); if (entry) entry->tagged = 1; if (!peel_ref(path, &peeled)) { - entry = packlist_find(&to_pack, &peeled, NULL); + entry = packlist_find(&to_pack, &peeled); if (entry) entry->tagged = 1; } @@ -996,12 +996,11 @@ static int no_try_delta(const char *path) * few lines later when we want to add the new entry. */ static int have_duplicate_entry(const struct object_id *oid, - int exclude, - uint32_t *index_pos) + int exclude) { struct object_entry *entry; - entry = packlist_find(&to_pack, oid, index_pos); + entry = packlist_find(&to_pack, oid); if (!entry) return 0; @@ -1141,13 +1140,12 @@ static void create_object_entry(const struct object_id *oid, uint32_t hash, int exclude, int no_try_delta, - uint32_t index_pos, struct packed_git *found_pack, off_t found_offset) { struct object_entry *entry; - entry = packlist_alloc(&to_pack, oid->hash, index_pos); + entry = packlist_alloc(&to_pack, oid); entry->hash = hash; oe_set_type(entry, type); if (exclude) @@ -1171,11 +1169,10 @@ static int add_object_entry(const struct object_id *oid, enum object_type type, { struct packed_git *found_pack = NULL; off_t found_offset = 0; - uint32_t index_pos; display_progress(progress_state, ++nr_seen); - if (have_duplicate_entry(oid, exclude, &index_pos)) + if (have_duplicate_entry(oid, exclude)) return 0; if (!want_object_in_pack(oid, exclude, &found_pack, &found_offset)) { @@ -1190,7 +1187,7 @@ static int add_object_entry(const struct object_id *oid, enum object_type type, create_object_entry(oid, type, pack_name_hash(name), exclude, name && no_try_delta(name), - index_pos, found_pack, found_offset); + found_pack, found_offset); return 1; } @@ -1199,17 +1196,15 @@ static int add_object_entry_from_bitmap(const struct object_id *oid, int flags, uint32_t name_hash, struct packed_git *pack, off_t offset) { - uint32_t index_pos; - display_progress(progress_state, ++nr_seen); - if (have_duplicate_entry(oid, 0, &index_pos)) + if (have_duplicate_entry(oid, 0)) return 0; if (!want_object_in_pack(oid, 0, &pack, &offset)) return 0; - create_object_entry(oid, type, name_hash, 0, 0, index_pos, pack, offset); + create_object_entry(oid, type, name_hash, 0, 0, pack, offset); return 1; } @@ -1507,7 +1502,7 @@ static int can_reuse_delta(const unsigned char *base_sha1, * First see if we're already sending the base (or it's explicitly in * our "excluded" list). */ - base = packlist_find(&to_pack, &base_oid, NULL); + base = packlist_find(&to_pack, &base_oid); if (base) { if (!in_same_island(&delta->idx.oid, &base->idx.oid)) return 0; @@ -2342,15 +2337,6 @@ static void find_deltas(struct object_entry **list, unsigned *list_size, free(array); } -static void try_to_free_from_threads(size_t size) -{ - packing_data_lock(&to_pack); - release_pack_memory(size); - packing_data_unlock(&to_pack); -} - -static try_to_free_t old_try_to_free_routine; - /* * The main object list is split into smaller lists, each is handed to * one worker. @@ -2391,12 +2377,10 @@ static void init_threaded_search(void) pthread_mutex_init(&cache_mutex, NULL); pthread_mutex_init(&progress_mutex, NULL); pthread_cond_init(&progress_cond, NULL); - old_try_to_free_routine = set_try_to_free_routine(try_to_free_from_threads); } static void cleanup_threaded_search(void) { - set_try_to_free_routine(old_try_to_free_routine); pthread_cond_destroy(&progress_cond); pthread_mutex_destroy(&cache_mutex); pthread_mutex_destroy(&progress_mutex); @@ -2579,7 +2563,7 @@ static void add_tag_chain(const struct object_id *oid) * it was included via bitmaps, we would not have parsed it * previously). */ - if (packlist_find(&to_pack, oid, NULL)) + if (packlist_find(&to_pack, oid)) return; tag = lookup_tag(the_repository, oid); @@ -2603,7 +2587,7 @@ static int add_ref_tag(const char *path, const struct object_id *oid, int flag, if (starts_with(path, "refs/tags/") && /* is a tag? */ !peel_ref(path, &peeled) && /* peelable? */ - packlist_find(&to_pack, &peeled, NULL)) /* object packed? */ + packlist_find(&to_pack, &peeled)) /* object packed? */ add_tag_chain(oid); return 0; } @@ -2715,10 +2699,6 @@ static int git_pack_config(const char *k, const char *v, void *cb) use_bitmap_index_default = git_config_bool(k, v); return 0; } - if (!strcmp(k, "pack.usesparse")) { - sparse = git_config_bool(k, v); - return 0; - } if (!strcmp(k, "pack.threads")) { delta_search_threads = git_config_int(k, v); if (delta_search_threads < 0) @@ -2803,7 +2783,7 @@ static void show_object(struct object *obj, const char *name, void *data) for (p = strchr(name, '/'); p; p = strchr(p + 1, '/')) depth++; - ent = packlist_find(&to_pack, &obj->oid, NULL); + ent = packlist_find(&to_pack, &obj->oid); if (ent && depth > oe_tree_depth(&to_pack, ent)) oe_set_tree_depth(&to_pack, ent, depth); } @@ -3034,7 +3014,7 @@ static void loosen_unused_packed_objects(void) for (i = 0; i < p->num_objects; i++) { nth_packed_object_oid(&oid, p, i); - if (!packlist_find(&to_pack, &oid, NULL) && + if (!packlist_find(&to_pack, &oid) && !has_sha1_pack_kept_or_nonlocal(&oid) && !loosened_object_can_be_discarded(&oid, p->mtime)) if (force_object_loose(&oid, p->mtime)) @@ -3343,6 +3323,10 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix) read_replace_refs = 0; sparse = git_env_bool("GIT_TEST_PACK_SPARSE", 0); + prepare_repo_settings(the_repository); + if (!sparse && the_repository->settings.pack_use_sparse != -1) + sparse = the_repository->settings.pack_use_sparse; + reset_pack_idx_option(&pack_idx_opts); git_config(git_pack_config, NULL); diff --git a/builtin/pull.c b/builtin/pull.c index f1eaf6e6ed..d25ff13a60 100644 --- a/builtin/pull.c +++ b/builtin/pull.c @@ -129,6 +129,7 @@ static char *opt_refmap; static char *opt_ipv4; static char *opt_ipv6; static int opt_show_forced_updates = -1; +static char *set_upstream; static struct option pull_options[] = { /* Shared options */ @@ -243,6 +244,9 @@ static struct option pull_options[] = { PARSE_OPT_NOARG), OPT_BOOL(0, "show-forced-updates", &opt_show_forced_updates, N_("check for forced-updates on all updated branches")), + OPT_PASSTHRU(0, "set-upstream", &set_upstream, NULL, + N_("set upstream for git pull/fetch"), + PARSE_OPT_NOARG), OPT_END() }; @@ -556,6 +560,8 @@ static int run_fetch(const char *repo, const char **refspecs) argv_array_push(&args, "--show-forced-updates"); else if (opt_show_forced_updates == 0) argv_array_push(&args, "--no-show-forced-updates"); + if (set_upstream) + argv_array_push(&args, set_upstream); if (repo) { argv_array_push(&args, repo); diff --git a/builtin/push.c b/builtin/push.c index 021dd3b1e4..3742daf7b0 100644 --- a/builtin/push.c +++ b/builtin/push.c @@ -385,30 +385,14 @@ static int push_with_options(struct transport *transport, struct refspec *rs, } static int do_push(const char *repo, int flags, - const struct string_list *push_options) + const struct string_list *push_options, + struct remote *remote) { int i, errs; - struct remote *remote = pushremote_get(repo); const char **url; int url_nr; struct refspec *push_refspec = &rs; - if (!remote) { - if (repo) - die(_("bad repository '%s'"), repo); - die(_("No configured push destination.\n" - "Either specify the URL from the command-line or configure a remote repository using\n" - "\n" - " git remote add \n" - "\n" - "and then push using the remote name\n" - "\n" - " git push \n")); - } - - if (remote->mirror) - flags |= (TRANSPORT_PUSH_MIRROR|TRANSPORT_PUSH_FORCE); - if (push_options->nr) flags |= TRANSPORT_PUSH_OPTIONS; @@ -548,6 +532,7 @@ int cmd_push(int argc, const char **argv, const char *prefix) struct string_list push_options_cmdline = STRING_LIST_INIT_DUP; struct string_list *push_options; const struct string_list_item *item; + struct remote *remote; struct option options[] = { OPT__VERBOSITY(&verbosity), @@ -602,20 +587,6 @@ int cmd_push(int argc, const char **argv, const char *prefix) die(_("--delete is incompatible with --all, --mirror and --tags")); if (deleterefs && argc < 2) die(_("--delete doesn't make sense without any refs")); - if (flags & TRANSPORT_PUSH_ALL) { - if (tags) - die(_("--all and --tags are incompatible")); - if (argc >= 2) - die(_("--all can't be combined with refspecs")); - } - if (flags & TRANSPORT_PUSH_MIRROR) { - if (tags) - die(_("--mirror and --tags are incompatible")); - if (argc >= 2) - die(_("--mirror can't be combined with refspecs")); - } - if ((flags & TRANSPORT_PUSH_ALL) && (flags & TRANSPORT_PUSH_MIRROR)) - die(_("--all and --mirror are incompatible")); if (recurse_submodules == RECURSE_SUBMODULES_CHECK) flags |= TRANSPORT_RECURSE_SUBMODULES_CHECK; @@ -632,11 +603,43 @@ int cmd_push(int argc, const char **argv, const char *prefix) set_refspecs(argv + 1, argc - 1, repo); } + remote = pushremote_get(repo); + if (!remote) { + if (repo) + die(_("bad repository '%s'"), repo); + die(_("No configured push destination.\n" + "Either specify the URL from the command-line or configure a remote repository using\n" + "\n" + " git remote add \n" + "\n" + "and then push using the remote name\n" + "\n" + " git push \n")); + } + + if (remote->mirror) + flags |= (TRANSPORT_PUSH_MIRROR|TRANSPORT_PUSH_FORCE); + + if (flags & TRANSPORT_PUSH_ALL) { + if (tags) + die(_("--all and --tags are incompatible")); + if (argc >= 2) + die(_("--all can't be combined with refspecs")); + } + if (flags & TRANSPORT_PUSH_MIRROR) { + if (tags) + die(_("--mirror and --tags are incompatible")); + if (argc >= 2) + die(_("--mirror can't be combined with refspecs")); + } + if ((flags & TRANSPORT_PUSH_ALL) && (flags & TRANSPORT_PUSH_MIRROR)) + die(_("--all and --mirror are incompatible")); + for_each_string_list_item(item, push_options) if (strchr(item->string, '\n')) die(_("push options must not have new line characters")); - rc = do_push(repo, flags, push_options); + rc = do_push(repo, flags, push_options, remote); string_list_clear(&push_options_cmdline, 0); string_list_clear(&push_options_config, 0); if (rc == -1) diff --git a/builtin/rebase.c b/builtin/rebase.c index 27d5c4e19d..f730b15a78 100644 --- a/builtin/rebase.c +++ b/builtin/rebase.c @@ -62,7 +62,7 @@ struct rebase_options { const char *onto_name; const char *revisions; const char *switch_to; - int root; + int root, root_with_onto; struct object_id *squash_onto; struct commit *restrict_revision; int dont_finish_rebase; @@ -374,6 +374,7 @@ static int run_rebase_interactive(struct rebase_options *opts, flags |= abbreviate_commands ? TODO_LIST_ABBREVIATE_CMDS : 0; flags |= opts->rebase_merges ? TODO_LIST_REBASE_MERGES : 0; flags |= opts->rebase_cousins > 0 ? TODO_LIST_REBASE_COUSINS : 0; + flags |= opts->root_with_onto ? TODO_LIST_ROOT_WITH_ONTO : 0; flags |= command == ACTION_SHORTEN_OIDS ? TODO_LIST_SHORTEN_IDS : 0; switch (command) { @@ -1863,15 +1864,6 @@ int cmd_rebase(int argc, const char **argv, const char *prefix) "'--reschedule-failed-exec'")); } - if (options.rebase_merges) { - if (strategy_options.nr) - die(_("cannot combine '--rebase-merges' with " - "'--strategy-option'")); - if (options.strategy) - die(_("cannot combine '--rebase-merges' with " - "'--strategy'")); - } - if (!options.root) { if (argc < 1) { struct branch *branch; @@ -1902,7 +1894,9 @@ int cmd_rebase(int argc, const char **argv, const char *prefix) options.squash_onto = &squash_onto; options.onto_name = squash_onto_name = xstrdup(oid_to_hex(&squash_onto)); - } + } else + options.root_with_onto = 1; + options.upstream_name = NULL; options.upstream = NULL; if (argc > 1) diff --git a/builtin/repack.c b/builtin/repack.c index 632c0c0a79..3b3dd14372 100644 --- a/builtin/repack.c +++ b/builtin/repack.c @@ -11,6 +11,7 @@ #include "midx.h" #include "packfile.h" #include "object-store.h" +#include "promisor-remote.h" static int delta_base_offset = 1; static int pack_kept_objects = -1; @@ -361,7 +362,7 @@ int cmd_repack(int argc, const char **argv, const char *prefix) argv_array_push(&cmd.args, "--all"); argv_array_push(&cmd.args, "--reflog"); argv_array_push(&cmd.args, "--indexed-objects"); - if (repository_format_partial_clone) + if (has_promisor_remote()) argv_array_push(&cmd.args, "--exclude-promisor-objects"); if (write_bitmaps > 0) argv_array_push(&cmd.args, "--write-bitmap-index"); diff --git a/builtin/replace.c b/builtin/replace.c index 644b21ca8d..2a4afb3b93 100644 --- a/builtin/replace.c +++ b/builtin/replace.c @@ -421,7 +421,7 @@ static int check_one_mergetag(struct commit *commit, if (get_oid(mergetag_data->argv[i], &oid) < 0) return error(_("not a valid object name: '%s'"), mergetag_data->argv[i]); - if (oideq(&tag->tagged->oid, &oid)) + if (oideq(get_tagged_oid(tag), &oid)) return 0; /* found */ } diff --git a/builtin/rev-list.c b/builtin/rev-list.c index 301ccb970b..b8dc2e1fba 100644 --- a/builtin/rev-list.c +++ b/builtin/rev-list.c @@ -473,8 +473,10 @@ int cmd_rev_list(int argc, const char **argv, const char *prefix) die(_("object filtering requires --objects")); if (filter_options.choice == LOFC_SPARSE_OID && !filter_options.sparse_oid_value) - die(_("invalid sparse value '%s'"), - filter_options.filter_spec); + die( + _("invalid sparse value '%s'"), + list_objects_filter_spec( + &filter_options)); continue; } if (!strcmp(arg, ("--no-" CL_ARG__FILTER))) { diff --git a/builtin/update-index.c b/builtin/update-index.c index dff2f4b837..49302d98c5 100644 --- a/builtin/update-index.c +++ b/builtin/update-index.c @@ -966,6 +966,7 @@ int cmd_update_index(int argc, const char **argv, const char *prefix) struct parse_opt_ctx_t ctx; strbuf_getline_fn getline_fn; int parseopt_state = PARSE_OPT_UNKNOWN; + struct repository *r = the_repository; struct option options[] = { OPT_BIT('q', NULL, &refresh_args.flags, N_("continue refresh even when index needs update"), @@ -1180,11 +1181,12 @@ int cmd_update_index(int argc, const char **argv, const char *prefix) remove_split_index(&the_index); } + prepare_repo_settings(r); switch (untracked_cache) { case UC_UNSPECIFIED: break; case UC_DISABLE: - if (git_config_get_untracked_cache() == 1) + if (r->settings.core_untracked_cache == UNTRACKED_CACHE_WRITE) warning(_("core.untrackedCache is set to true; " "remove or change it, if you really want to " "disable the untracked cache")); @@ -1196,7 +1198,7 @@ int cmd_update_index(int argc, const char **argv, const char *prefix) return !test_if_untracked_cache_is_supported(); case UC_ENABLE: case UC_FORCE: - if (git_config_get_untracked_cache() == 0) + if (r->settings.core_untracked_cache == UNTRACKED_CACHE_REMOVE) warning(_("core.untrackedCache is set to false; " "remove or change it, if you really want to " "enable the untracked cache")); diff --git a/bulk-checkin.c b/bulk-checkin.c index 39ee7d6107..583aacb9e3 100644 --- a/bulk-checkin.c +++ b/bulk-checkin.c @@ -197,7 +197,7 @@ static int deflate_to_pack(struct bulk_checkin_state *state, git_hash_ctx ctx; unsigned char obuf[16384]; unsigned header_len; - struct hashfile_checkpoint checkpoint; + struct hashfile_checkpoint checkpoint = {0}; struct pack_idx_entry *idx = NULL; seekback = lseek(fd, 0, SEEK_CUR); diff --git a/cache-tree.c b/cache-tree.c index c22161f987..0e5724fad7 100644 --- a/cache-tree.c +++ b/cache-tree.c @@ -5,6 +5,7 @@ #include "cache-tree.h" #include "object-store.h" #include "replace-object.h" +#include "promisor-remote.h" #ifndef DEBUG_CACHE_TREE #define DEBUG_CACHE_TREE 0 @@ -357,7 +358,7 @@ static int update_one(struct cache_tree *it, } ce_missing_ok = mode == S_IFGITLINK || missing_ok || - (repository_format_partial_clone && + (has_promisor_remote() && ce_skip_worktree(ce)); if (is_null_oid(oid) || (!ce_missing_ok && !has_object_file(oid))) { diff --git a/cache.h b/cache.h index b1da1ab08f..5624e6c02d 100644 --- a/cache.h +++ b/cache.h @@ -636,6 +636,9 @@ int daemonize(void); * at least 'nr' entries; the number of entries currently allocated * is 'alloc', using the standard growing factor alloc_nr() macro. * + * Consider using ALLOC_GROW_BY instead of ALLOC_GROW as it has some + * added niceties. + * * DO NOT USE any expression with side-effect for 'x', 'nr', or 'alloc'. */ #define ALLOC_GROW(x, nr, alloc) \ @@ -649,6 +652,25 @@ int daemonize(void); } \ } while (0) +/* + * Similar to ALLOC_GROW but handles updating of the nr value and + * zeroing the bytes of the newly-grown array elements. + * + * DO NOT USE any expression with side-effect for any of the + * arguments. + */ +#define ALLOC_GROW_BY(x, nr, increase, alloc) \ + do { \ + if (increase) { \ + size_t new_nr = nr + (increase); \ + if (new_nr < nr) \ + BUG("negative growth in ALLOC_GROW_BY"); \ + ALLOC_GROW(x, new_nr, alloc); \ + memset((x) + nr, 0, sizeof(*(x)) * (increase)); \ + nr = new_nr; \ + } \ + } while (0) + /* Initialize and use the cache information */ struct lock_file; void preload_index(struct index_state *index, @@ -937,8 +959,6 @@ extern int grafts_replace_parents; #define GIT_REPO_VERSION 0 #define GIT_REPO_VERSION_READ 1 extern int repository_format_precious_objects; -extern char *repository_format_partial_clone; -extern const char *core_partial_clone_filter_default; extern int repository_format_worktree_config; /* diff --git a/ci/lib.sh b/ci/lib.sh index 44db2d5cbb..29dc740d40 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -160,7 +160,7 @@ linux-clang|linux-gcc) export CC=gcc-8 fi - export GIT_TEST_HTTPD=YesPlease + export GIT_TEST_HTTPD=true # The Linux build installs the defined dependency versions below. # The OS X build installs much more recent versions, whichever diff --git a/commit-graph.c b/commit-graph.c index f2888c203b..9b02d2c426 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -467,7 +467,6 @@ static void prepare_commit_graph_one(struct repository *r, const char *obj_dir) static int prepare_commit_graph(struct repository *r) { struct object_directory *odb; - int config_value; if (git_env_bool(GIT_TEST_COMMIT_GRAPH_DIE_ON_LOAD, 0)) die("dying as requested by the '%s' variable on commit-graph load!", @@ -477,9 +476,10 @@ static int prepare_commit_graph(struct repository *r) return !!r->objects->commit_graph; r->objects->commit_graph_attempted = 1; + prepare_repo_settings(r); + if (!git_env_bool(GIT_TEST_COMMIT_GRAPH, 0) && - (repo_config_get_bool(r, "core.commitgraph", &config_value) || - !config_value)) + r->settings.core_commit_graph != 1) /* * This repository is not configured to use commit graphs, so * do not load one. (But report commit_graph_attempted anyway diff --git a/commit.c b/commit.c index a98de16e3d..3fe5f8fa9c 100644 --- a/commit.c +++ b/commit.c @@ -364,8 +364,8 @@ struct object_id *get_commit_tree_oid(const struct commit *commit) void release_commit_memory(struct parsed_object_pool *pool, struct commit *c) { set_commit_tree(c, NULL); - c->index = 0; free_commit_buffer(pool, c); + c->index = 0; free_commit_list(c->parents); c->object.parsed = 0; diff --git a/common-main.c b/common-main.c index 582a7b1886..71e21dd20a 100644 --- a/common-main.c +++ b/common-main.c @@ -39,16 +39,16 @@ int main(int argc, const char **argv) git_resolve_executable_dir(argv[0]); - trace2_initialize(); - trace2_cmd_start(argv); - trace2_collect_process_info(TRACE2_PROCESS_INFO_STARTUP); - git_setup_gettext(); initialize_the_repository(); attr_start(); + trace2_initialize(); + trace2_cmd_start(argv); + trace2_collect_process_info(TRACE2_PROCESS_INFO_STARTUP); + result = cmd_main(argc, argv); trace2_cmd_exit(result); diff --git a/compat/mingw.c b/compat/mingw.c index 738f0a826a..7a0d619fb6 100644 --- a/compat/mingw.c +++ b/compat/mingw.c @@ -1161,14 +1161,21 @@ static char *lookup_prog(const char *dir, int dirlen, const char *cmd, int isexe, int exe_only) { char path[MAX_PATH]; + wchar_t wpath[MAX_PATH]; snprintf(path, sizeof(path), "%.*s\\%s.exe", dirlen, dir, cmd); - if (!isexe && access(path, F_OK) == 0) + if (xutftowcs_path(wpath, path) < 0) + return NULL; + + if (!isexe && _waccess(wpath, F_OK) == 0) return xstrdup(path); - path[strlen(path)-4] = '\0'; - if ((!exe_only || isexe) && access(path, F_OK) == 0) - if (!(GetFileAttributes(path) & FILE_ATTRIBUTE_DIRECTORY)) + wpath[wcslen(wpath)-4] = '\0'; + if ((!exe_only || isexe) && _waccess(wpath, F_OK) == 0) { + if (!(GetFileAttributesW(wpath) & FILE_ATTRIBUTE_DIRECTORY)) { + path[strlen(path)-4] = '\0'; return xstrdup(path); + } + } return NULL; } @@ -1265,7 +1272,7 @@ static wchar_t *make_environment_block(char **deltaenv) } ALLOC_ARRAY(result, size); - memcpy(result, wenv, size * sizeof(*wenv)); + COPY_ARRAY(result, wenv, size); FreeEnvironmentStringsW(wenv); return result; } @@ -1309,7 +1316,7 @@ static wchar_t *make_environment_block(char **deltaenv) continue; size = wcslen(array[i]) + 1; - memcpy(p, array[i], size * sizeof(*p)); + COPY_ARRAY(p, array[i], size); p += size; } *p = L'\0'; diff --git a/compat/mingw.h b/compat/mingw.h index a03e40e6e2..9ad204c57c 100644 --- a/compat/mingw.h +++ b/compat/mingw.h @@ -11,7 +11,7 @@ typedef _sigset_t sigset_t; #undef _POSIX_THREAD_SAFE_FUNCTIONS #endif -extern int mingw_core_config(const char *var, const char *value, void *cb); +int mingw_core_config(const char *var, const char *value, void *cb); #define platform_core_config mingw_core_config /* @@ -443,7 +443,7 @@ static inline void convert_slashes(char *path) *path = '/'; } #define PATH_SEP ';' -extern char *mingw_query_user_email(void); +char *mingw_query_user_email(void); #define query_user_email mingw_query_user_email #if !defined(__MINGW64_VERSION_MAJOR) && (!defined(_MSC_VER) || _MSC_VER < 1800) #define PRIuMAX "I64u" @@ -580,4 +580,4 @@ int main(int argc, const char **argv); /* * Used by Pthread API implementation for Windows */ -extern int err_win_to_posix(DWORD winerr); +int err_win_to_posix(DWORD winerr); diff --git a/compat/win32/pthread.h b/compat/win32/pthread.h index c6cb8dd219..f1cfe73de9 100644 --- a/compat/win32/pthread.h +++ b/compat/win32/pthread.h @@ -50,7 +50,7 @@ typedef struct { DWORD tid; } pthread_t; -extern int pthread_create(pthread_t *thread, const void *unused, +int pthread_create(pthread_t *thread, const void *unused, void *(*start_routine)(void*), void *arg); /* @@ -59,10 +59,10 @@ extern int pthread_create(pthread_t *thread, const void *unused, */ #define pthread_join(a, b) win32_pthread_join(&(a), (b)) -extern int win32_pthread_join(pthread_t *thread, void **value_ptr); +int win32_pthread_join(pthread_t *thread, void **value_ptr); #define pthread_equal(t1, t2) ((t1).tid == (t2).tid) -extern pthread_t pthread_self(void); +pthread_t pthread_self(void); static inline void NORETURN pthread_exit(void *ret) { diff --git a/config.c b/config.c index 3900e4947b..743e4570ee 100644 --- a/config.c +++ b/config.c @@ -275,7 +275,7 @@ static int include_by_branch(const char *cond, size_t cond_len) int flags; int ret; struct strbuf pattern = STRBUF_INIT; - const char *refname = !the_repository || !the_repository->gitdir ? + const char *refname = !the_repository->gitdir ? NULL : resolve_ref_unsafe("HEAD", 0, NULL, &flags); const char *shortname; @@ -1379,11 +1379,6 @@ static int git_default_core_config(const char *var, const char *value, void *cb) return 0; } - if (!strcmp(var, "core.partialclonefilter")) { - return git_config_string(&core_partial_clone_filter_default, - var, value); - } - if (!strcmp(var, "core.usereplacerefs")) { read_replace_refs = git_config_bool(var, value); return 0; @@ -2288,30 +2283,6 @@ int git_config_get_expiry_in_days(const char *key, timestamp_t *expiry, timestam return -1; /* thing exists but cannot be parsed */ } -int git_config_get_untracked_cache(void) -{ - int val = -1; - const char *v; - - /* Hack for test programs like test-dump-untracked-cache */ - if (ignore_untracked_cache_config) - return -1; - - if (!git_config_get_maybe_bool("core.untrackedcache", &val)) - return val; - - if (!git_config_get_value("core.untrackedcache", &v)) { - if (!strcasecmp(v, "keep")) - return -1; - - error(_("unknown core.untrackedCache value '%s'; " - "using 'keep' default value"), v); - return -1; - } - - return -1; /* default value */ -} - int git_config_get_split_index(void) { int val; diff --git a/connect.c b/connect.c index 2778481264..da7daa2b68 100644 --- a/connect.c +++ b/connect.c @@ -915,6 +915,10 @@ static enum protocol parse_connect_url(const char *url_orig, char **ret_host, if (protocol == PROTO_LOCAL) path = end; + else if (protocol == PROTO_FILE && *host != '/' && + !has_dos_drive_prefix(host) && + offset_1st_component(host - 2) > 1) + path = host - 2; /* include the leading "//" */ else if (protocol == PROTO_FILE && has_dos_drive_prefix(end)) path = end; /* "file://$(pwd)" may be "file://C:/projects/repo" */ else diff --git a/connected.c b/connected.c index cd9b324afa..971db009b3 100644 --- a/connected.c +++ b/connected.c @@ -5,6 +5,7 @@ #include "connected.h" #include "transport.h" #include "packfile.h" +#include "promisor-remote.h" /* * If we feed all the commits we want to verify to this command @@ -73,7 +74,7 @@ int check_connected(oid_iterate_fn fn, void *cb_data, argv_array_push(&rev_list.args,"rev-list"); argv_array_push(&rev_list.args, "--objects"); argv_array_push(&rev_list.args, "--stdin"); - if (repository_format_partial_clone) + if (has_promisor_remote()) argv_array_push(&rev_list.args, "--exclude-promisor-objects"); if (!opt->is_deepening_fetch) { argv_array_push(&rev_list.args, "--not"); diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash index 71e7159694..59cd3e8e90 100644 --- a/contrib/completion/git-completion.bash +++ b/contrib/completion/git-completion.bash @@ -340,7 +340,7 @@ __gitcomp () c="$c${4-}" if [[ $c == "$cur_"* ]]; then case $c in - --*=*|*.) ;; + --*=|*.) ;; *) c="$c " ;; esac COMPREPLY[i++]="${2-}$c" @@ -360,7 +360,7 @@ __gitcomp () c="$c${4-}" if [[ $c == "$cur_"* ]]; then case $c in - --*=*|*.) ;; + *=|*.) ;; *) c="$c " ;; esac COMPREPLY[i++]="${2-}$c" @@ -524,7 +524,7 @@ __git_index_files () # Even when a directory name itself does not contain # any special characters, it will still be quoted if # any of its (stripped) trailing path components do. - # Because of this we may have seen the same direcory + # Because of this we may have seen the same directory # both quoted and unquoted. if (p in paths) # We have seen the same directory unquoted, @@ -1361,7 +1361,9 @@ _git_checkout () esac } -__git_cherry_pick_inprogress_options="--continue --quit --abort" +__git_sequencer_inprogress_options="--continue --quit --abort --skip" + +__git_cherry_pick_inprogress_options=$__git_sequencer_inprogress_options _git_cherry_pick () { @@ -1399,7 +1401,18 @@ _git_clean () _git_clone () { + case "$prev" in + -c|--config) + __git_complete_config_variable_name_and_value + return + ;; + esac case "$cur" in + --config=*) + __git_complete_config_variable_name_and_value \ + --cur="${cur##--config=}" + return + ;; --*) __gitcomp_builtin clone return @@ -2225,181 +2238,282 @@ __git_config_vars= __git_compute_config_vars () { test -n "$__git_config_vars" || - __git_config_vars="$(git help --config-for-completion | sort | uniq)" + __git_config_vars="$(git help --config-for-completion | sort -u)" } -_git_config () +# Completes possible values of various configuration variables. +# +# Usage: __git_complete_config_variable_value [