From: Junio C Hamano Date: Tue, 13 Feb 2018 21:39:16 +0000 (-0800) Subject: Merge branch 'ew/svn-branch-segfault-fix' X-Git-Tag: v2.17.0-rc0~105 X-Git-Url: https://git.lorimer.id.au/gitweb.git/diff_plain/9cd5320d3cfb2c2841a76b36b20c18e9299206ab?hp=7f6f75e97acd25f8e95ce431e16d2e1c2093845d Merge branch 'ew/svn-branch-segfault-fix' Workaround for segfault with more recent versions of SVN. * ew/svn-branch-segfault-fix: git-svn: control destruction order to avoid segfault --- diff --git a/.clang-format b/.clang-format index 611ab4750b..12a89f95f9 100644 --- a/.clang-format +++ b/.clang-format @@ -163,7 +163,7 @@ PenaltyBreakComment: 10 PenaltyBreakFirstLessLess: 0 PenaltyBreakString: 10 PenaltyExcessCharacter: 100 -PenaltyReturnTypeOnItsOwnLine: 5 +PenaltyReturnTypeOnItsOwnLine: 60 # Don't sort #include's SortIncludes: false diff --git a/Documentation/Makefile b/Documentation/Makefile index 4ae9ba5c86..6232143cb9 100644 --- a/Documentation/Makefile +++ b/Documentation/Makefile @@ -72,6 +72,7 @@ TECH_DOCS += SubmittingPatches TECH_DOCS += technical/hash-function-transition TECH_DOCS += technical/http-protocol TECH_DOCS += technical/index-format +TECH_DOCS += technical/long-running-process-protocol TECH_DOCS += technical/pack-format TECH_DOCS += technical/pack-heuristics TECH_DOCS += technical/pack-protocol diff --git a/Documentation/RelNotes/2.17.0.txt b/Documentation/RelNotes/2.17.0.txt new file mode 100644 index 0000000000..759e75fbde --- /dev/null +++ b/Documentation/RelNotes/2.17.0.txt @@ -0,0 +1,84 @@ +Git 2.17 Release Notes +====================== + +Updates since v2.16 +------------------- + +UI, Workflows & Features + + * "diff" family of commands learned "--find-object=" option + to limit the findings to changes that involve the named object. + (merge 4d8c51aa19 sb/diff-blobfind-pickaxe later to maint). + + +Performance, Internal Implementation, Development Support etc. + + * More perf tests for threaded grep + (merge 7b31b55db1 ab/perf-grep-threads later to maint). + + * "perf" test output can be sent to codespeed server. + (merge 19cf57a92e cc/codespeed later to maint). + + +Also contains various documentation updates and code clean-ups. + + +Fixes since v2.16 +----------------- + + * An old regression in "git describe --all $annotated_tag^0" has been + fixed. + (merge 1bba00130a dk/describe-all-output-fix later to maint). + + * "git status" after moving a path in the working tree (hence making + it appear "removed") and then adding with the -N option (hence + making that appear "added") detected it as a rename, but did not + report the old and new pathnames correctly. + (merge 176ea74793 nd/ita-wt-renames-in-status later to maint). + + * "git svn dcommit" did not take into account the fact that a + svn+ssh:// URL with a username@ (typically used for pushing) refers + to the same SVN repository without the username@ and failed when + svn.pushmergeinfo option is set. + (merge 8aaed892fd jm/svn-pushmergeinfo-fix later to maint). + + * API clean-up around revision traversal. + (merge 6fcec2f9ae rs/lose-leak-pending later to maint). + + * "git merge -Xours/-Xtheirs" learned to use our/their version when + resolving a conflicting updates to a symbolic link. + (merge fd48b46474 jc/merge-symlink-ours-theirs later to maint). + + * "git clone $there $here" is allowed even when here directory exists + as long as it is an empty directory, but the command incorrectly + removed it upon a failure of the operation. + (merge d45420c1c8 jk/abort-clone-with-existing-dest later to maint). + + * "git commit --fixup" did not allow "-m" option to be used + at the same time; allow it to annotate resulting commit with more + text. + (merge 30884c9afc ab/commit-m-with-fixup later to maint). + + * When resetting the working tree files recursively, the working tree + of submodules are now also reset to match. + (merge 7dcc1f4df8 sb/submodule-update-reset-fix later to maint). + + * "git stash -- " incorrectly blew away untracked files in + the directory that matched the pathspec, which has been corrected. + (merge bba067d2fa tg/stash-with-pathspec-fix later to maint). + + * Instead of maintaining home-grown email address parsing code, ship + a copy of reasonably recent Mail::Address to be used as a fallback + in 'git send-email' when the platform lacks it. + (merge d60be8acab mm/send-email-fallback-to-local-mail-address later to maint). + + * "git add -p" was taught to ignore local changes to submodules as + they do not interfere with the partial addition of regular changes + anyway. + (merge 12434efc1d nd/add-i-ignore-submodules later to maint). + + * Other minor doc, test and build updates and code cleanups. + (merge e2a5a028c7 bw/oidmap-autoinit later to maint). + (merge f0a6068a9f ys/bisect-object-id-missing-conversion-fix later to maint). + (merge 30221a3389 as/read-tree-prefix-doc-fix later to maint). + (merge 9bd2ce5432 ab/doc-cat-file-e-still-shows-errors later to maint). diff --git a/Documentation/config.txt b/Documentation/config.txt index 0e25b2c92b..f57e9cf10c 100644 --- a/Documentation/config.txt +++ b/Documentation/config.txt @@ -3343,6 +3343,10 @@ uploadpack.packObjectsHook:: was run. I.e., `upload-pack` will feed input intended for `pack-objects` to the hook, and expects a completed packfile on stdout. + +uploadpack.allowFilter:: + If this option is set, `upload-pack` will advertise partial + clone and partial fetch object filtering. + Note that this configuration variable is ignored if it is seen in the repository-level config (this is a safety measure against fetching from diff --git a/Documentation/diff-options.txt b/Documentation/diff-options.txt index 743af97b06..c330c01ff0 100644 --- a/Documentation/diff-options.txt +++ b/Documentation/diff-options.txt @@ -508,6 +508,15 @@ occurrences of that string did not change). See the 'pickaxe' entry in linkgit:gitdiffcore[7] for more information. +--find-object=:: + Look for differences that change the number of occurrences of + the specified object. Similar to `-S`, just the argument is different + in that it doesn't search for a specific string but for a specific + object id. ++ +The object can be a blob or a submodule commit. It implies the `-t` option in +`git-log` to also find trees. + --pickaxe-all:: When `-S` or `-G` finds a change, show all the changes in that changeset, not just the files that contain the change @@ -516,6 +525,7 @@ information. --pickaxe-regex:: Treat the given to `-S` as an extended POSIX regular expression to match. + endif::git-format-patch[] -O:: diff --git a/Documentation/git-cat-file.txt b/Documentation/git-cat-file.txt index fb09cd69d6..f90f09b03f 100644 --- a/Documentation/git-cat-file.txt +++ b/Documentation/git-cat-file.txt @@ -42,8 +42,9 @@ OPTIONS . -e:: - Suppress all output; instead exit with zero status if - exists and is a valid object. + Exit with zero status if exists and is a valid + object. If is of an invalid format exit with non-zero and + emits an error on stderr. -p:: Pretty-print the contents of based on its type. @@ -168,7 +169,7 @@ If `-t` is specified, one of the . If `-s` is specified, the size of the in bytes. -If `-e` is specified, no output. +If `-e` is specified, no output, unless the is malformed. If `-p` is specified, the contents of are pretty-printed. diff --git a/Documentation/git-commit.txt b/Documentation/git-commit.txt index 8c74a2ca03..f970a43422 100644 --- a/Documentation/git-commit.txt +++ b/Documentation/git-commit.txt @@ -144,6 +144,8 @@ OPTIONS Use the given as the commit message. If multiple `-m` options are given, their values are concatenated as separate paragraphs. ++ +The `-m` option is mutually exclusive with `-c`, `-C`, and `-F`. -t :: --template=:: diff --git a/Documentation/git-pack-objects.txt b/Documentation/git-pack-objects.txt index aa403d02f3..81bc490ac5 100644 --- a/Documentation/git-pack-objects.txt +++ b/Documentation/git-pack-objects.txt @@ -255,6 +255,17 @@ a missing object is encountered. This is the default action. The form '--missing=allow-any' will allow object traversal to continue if a missing object is encountered. Missing objects will silently be omitted from the results. ++ +The form '--missing=allow-promisor' is like 'allow-any', but will only +allow object traversal to continue for EXPECTED promisor missing objects. +Unexpected missing object will raise an error. + +--exclude-promisor-objects:: + Omit objects that are known to be in the promisor remote. (This + option has the purpose of operating only on locally created objects, + so that when we repack, we still maintain a distinction between + locally created objects [without .promisor] and objects from the + promisor remote [with .promisor].) This is used with partial clone. SEE ALSO -------- diff --git a/Documentation/git-read-tree.txt b/Documentation/git-read-tree.txt index 72bd809fb8..f2a07d54d6 100644 --- a/Documentation/git-read-tree.txt +++ b/Documentation/git-read-tree.txt @@ -81,12 +81,11 @@ OPTIONS * when both sides add a path identically. The resolution is to add that path. ---prefix=/:: +--prefix=:: Keep the current index contents, and read the contents of the named tree-ish under the directory at ``. The command will refuse to overwrite entries that already - existed in the original index file. Note that the `/` - value must end with a slash. + existed in the original index file. --exclude-per-directory=:: When running the command with `-u` and `-m` options, the diff --git a/Documentation/git-status.txt b/Documentation/git-status.txt index 81cab9aefb..72bfb87f66 100644 --- a/Documentation/git-status.txt +++ b/Documentation/git-status.txt @@ -149,14 +149,15 @@ the status.relativePaths config option below. Short Format ~~~~~~~~~~~~ -In the short-format, the status of each path is shown as +In the short-format, the status of each path is shown as one of these +forms - XY PATH1 -> PATH2 + XY PATH + XY ORIG_PATH -> PATH -where `PATH1` is the path in the `HEAD`, and the " `-> PATH2`" part is -shown only when `PATH1` corresponds to a different path in the -index/worktree (i.e. the file is renamed). The `XY` is a two-letter -status code. +where `ORIG_PATH` is where the renamed/copied contents came +from. `ORIG_PATH` is only shown when the entry is renamed or +copied. The `XY` is a two-letter status code. The fields (including the `->`) are separated from each other by a single space. If a filename contains whitespace or other nonprintable @@ -192,6 +193,8 @@ in which case `XY` are `!!`. [MARC] index and work tree matches [ MARC] M work tree changed since index [ MARC] D deleted in work tree + [ D] R renamed in work tree + [ D] C copied in work tree ------------------------------------------------- D D unmerged, both deleted A U unmerged, added by us @@ -309,13 +312,13 @@ Renamed or copied entries have the following format: of similarity between the source and target of the move or copy). For example "R100" or "C75". The pathname. In a renamed/copied entry, this - is the path in the index and in the working tree. + is the target path. When the `-z` option is used, the 2 pathnames are separated with a NUL (ASCII 0x00) byte; otherwise, a tab (ASCII 0x09) byte separates them. - The pathname in the commit at HEAD. This is only - present in a renamed/copied entry, and tells - where the renamed/copied contents came from. + The pathname in the commit at HEAD or in the index. + This is only present in a renamed/copied entry, and + tells where the renamed/copied contents came from. -------------------------------------------------------- Unmerged entries have the following format; the first character is diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt index ff612001d2..71c5618e82 100644 --- a/Documentation/git-submodule.txt +++ b/Documentation/git-submodule.txt @@ -70,8 +70,8 @@ status [--cached] [--recursive] [--] [...]:: Show the status of the submodules. This will print the SHA-1 of the currently checked out commit for each submodule, along with the submodule path and the output of 'git describe' for the - SHA-1. Each SHA-1 will be prefixed with `-` if the submodule is not - initialized, `+` if the currently checked out submodule commit + SHA-1. Each SHA-1 will possibly be prefixed with `-` if the submodule is + not initialized, `+` if the currently checked out submodule commit does not match the SHA-1 found in the index of the containing repository and `U` if the submodule has merge conflicts. + @@ -132,15 +132,15 @@ expects by cloning missing submodules and updating the working tree of the submodules. The "updating" can be done in several ways depending on command line options and the value of `submodule..update` configuration variable. The command line option takes precedence over -the configuration variable. if neither is given, a checkout is performed. -update procedures supported both from the command line as well as setting -`submodule..update`: +the configuration variable. If neither is given, a 'checkout' is performed. +The 'update' procedures supported both from the command line as well as +through the `submodule..update` configuration are: checkout;; the commit recorded in the superproject will be checked out in the submodule on a detached HEAD. + If `--force` is specified, the submodule will be checked out (using -`git checkout --force` if appropriate), even if the commit specified +`git checkout --force`), even if the commit specified in the index of the containing repository already matches the commit checked out in the submodule. @@ -150,8 +150,8 @@ checked out in the submodule. merge;; the commit recorded in the superproject will be merged into the current branch in the submodule. -The following procedures are only available via the `submodule..update` -configuration variable: +The following 'update' procedures are only available via the +`submodule..update` configuration variable: custom command;; arbitrary shell command that takes a single argument (the sha1 of the commit recorded in the diff --git a/Documentation/git.txt b/Documentation/git.txt index 3f4161a799..8163b5796b 100644 --- a/Documentation/git.txt +++ b/Documentation/git.txt @@ -646,6 +646,16 @@ of clones and fetches. variable. See `GIT_TRACE` for available trace output options. +`GIT_TRACE_CURL_NO_DATA`:: + When a curl trace is enabled (see `GIT_TRACE_CURL` above), do not dump + data (that is, only dump info lines and headers). + +`GIT_REDACT_COOKIES`:: + This can be set to a comma-separated list of strings. When a curl trace + is enabled (see `GIT_TRACE_CURL` above), whenever a "Cookies:" header + sent by the client is dumped, values of cookies whose key is in that + list (case-sensitive) are redacted. + `GIT_LITERAL_PATHSPECS`:: Setting this variable to `1` will cause Git to treat all pathspecs literally, rather than as glob patterns. For example, diff --git a/Documentation/gitattributes.txt b/Documentation/gitattributes.txt index 30687de81a..c21f5ca109 100644 --- a/Documentation/gitattributes.txt +++ b/Documentation/gitattributes.txt @@ -392,46 +392,14 @@ Long Running Filter Process If the filter command (a string value) is defined via `filter..process` then Git can process all blobs with a single filter invocation for the entire life of a single Git -command. This is achieved by using a packet format (pkt-line, -see technical/protocol-common.txt) based protocol over standard -input and standard output as follows. All packets, except for the -"*CONTENT" packets and the "0000" flush packet, are considered -text and therefore are terminated by a LF. - -Git starts the filter when it encounters the first file -that needs to be cleaned or smudged. After the filter started -Git sends a welcome message ("git-filter-client"), a list of supported -protocol version numbers, and a flush packet. Git expects to read a welcome -response message ("git-filter-server"), exactly one protocol version number -from the previously sent list, and a flush packet. All further -communication will be based on the selected version. The remaining -protocol description below documents "version=2". Please note that -"version=42" in the example below does not exist and is only there -to illustrate how the protocol would look like with more than one -version. - -After the version negotiation Git sends a list of all capabilities that -it supports and a flush packet. Git expects to read a list of desired -capabilities, which must be a subset of the supported capabilities list, -and a flush packet as response: ------------------------- -packet: git> git-filter-client -packet: git> version=2 -packet: git> version=42 -packet: git> 0000 -packet: git< git-filter-server -packet: git< version=2 -packet: git< 0000 -packet: git> capability=clean -packet: git> capability=smudge -packet: git> capability=not-yet-invented -packet: git> 0000 -packet: git< capability=clean -packet: git< capability=smudge -packet: git< 0000 ------------------------- -Supported filter capabilities in version 2 are "clean", "smudge", -and "delay". +command. This is achieved by using the long-running process protocol +(described in technical/long-running-process-protocol.txt). + +When Git encounters the first file that needs to be cleaned or smudged, +it starts the filter and performs the handshake. In the handshake, the +welcome message sent by Git is "git-filter-client", only version 2 is +suppported, and the supported capabilities are "clean", "smudge", and +"delay". Afterwards Git sends a list of "key=value" pairs terminated with a flush packet. The list will contain at least the filter command @@ -517,12 +485,6 @@ the protocol then Git will stop the filter process and restart it with the next file that needs to be processed. Depending on the `filter..required` flag Git will interpret that as error. -After the filter has processed a command it is expected to wait for -a "key=value" list containing the next command. Git will close -the command pipe on exit. The filter is expected to detect EOF -and exit gracefully on its own. Git will wait until the filter -process has stopped. - Delay ^^^^^ diff --git a/Documentation/gitremote-helpers.txt b/Documentation/gitremote-helpers.txt index 4a584f3c5d..4b8c93ec59 100644 --- a/Documentation/gitremote-helpers.txt +++ b/Documentation/gitremote-helpers.txt @@ -466,6 +466,13 @@ set by Git if the remote helper has the 'option' capability. Transmit as a push option. As the push option must not contain LF or NUL characters, the string is not encoded. +'option from-promisor' {'true'|'false'}:: + Indicate that these objects are being fetched from a promisor. + +'option no-dependents' {'true'|'false'}:: + Indicate that only the objects wanted need to be fetched, not + their dependents. + SEE ALSO -------- linkgit:git-remote[1] diff --git a/Documentation/gitsubmodules.txt b/Documentation/gitsubmodules.txt index 46cf120f66..4d6c17782f 100644 --- a/Documentation/gitsubmodules.txt +++ b/Documentation/gitsubmodules.txt @@ -36,8 +36,8 @@ The `gitlink` entry contains the object name of the commit that the superproject expects the submodule’s working directory to be at. The section `submodule.foo.*` in the `.gitmodules` file gives additional -hints to Gits porcelain layer such as where to obtain the submodule via -the `submodule.foo.url` setting. +hints to Git's porcelain layer. For example, the `submodule.foo.url` +setting specifies where to obtain the submodule. Submodules can be used for at least two different use cases: @@ -51,18 +51,21 @@ Submodules can be used for at least two different use cases: 2. Splitting a (logically single) project into multiple repositories and tying them back together. This can be used to - overcome current limitations of Gits implementation to have + overcome current limitations of Git's implementation to have finer grained access: - * Size of the git repository: + * Size of the Git repository: In its current form Git scales up poorly for large repositories containing content that is not compressed by delta computation between trees. - However you can also use submodules to e.g. hold large binary assets - and these repositories are then shallowly cloned such that you do not + For example, you can use submodules to hold large binary assets + and these repositories can be shallowly cloned such that you do not have a large history locally. * Transfer size: In its current form Git requires the whole working tree present. It does not allow partial trees to be transferred in fetch or clone. + If the project you work on consists of multiple repositories tied + together as submodules in a superproject, you can avoid fetching the + working trees of the repositories you are not interested in. * Access control: By restricting user access to submodules, this can be used to implement read/write policies for different users. @@ -73,9 +76,10 @@ The configuration of submodules Submodule operations can be configured using the following mechanisms (from highest to lowest precedence): - * The command line for those commands that support taking submodule specs. - Most commands have a boolean flag '--recurse-submodules' whether to - recurse into submodules. Examples are `ls-files` or `checkout`. + * The command line for those commands that support taking submodules + as part of their pathspecs. Most commands have a boolean flag + `--recurse-submodules` which specify whether to recurse into submodules. + Examples are `grep` and `checkout`. Some commands take enums, such as `fetch` and `push`, where you can specify how submodules are affected. @@ -87,8 +91,8 @@ Submodule operations can be configured using the following mechanisms For example an effect from the submodule's `.gitignore` file would be observed when you run `git status --ignore-submodules=none` in the superproject. This collects information from the submodule's working -directory by running `status` in the submodule, which does pay attention -to its `.gitignore` file. +directory by running `status` in the submodule while paying attention +to the `.gitignore` file of the submodule. + The submodule's `$GIT_DIR/config` file would come into play when running `git push --recurse-submodules=check` in the superproject, as this would @@ -97,20 +101,20 @@ remotes are configured in the submodule as usual in the `$GIT_DIR/config` file. * The configuration file `$GIT_DIR/config` in the superproject. - Typical configuration at this place is controlling if a submodule - is recursed into at all via the `active` flag for example. + Git only recurses into active submodules (see "ACTIVE SUBMODULES" + section below). + If the submodule is not yet initialized, then the configuration -inside the submodule does not exist yet, so configuration where to +inside the submodule does not exist yet, so where to obtain the submodule from is configured here for example. - * the `.gitmodules` file inside the superproject. Additionally to the - required mapping between submodule's name and path, a project usually + * The `.gitmodules` file inside the superproject. A project usually uses this file to suggest defaults for the upstream collection - of repositories. + of repositories for the mapping that is required between a + submodule's name and its path. + -This file mainly serves as the mapping between name and path in -the superproject, such that the submodule's git directory can be +This file mainly serves as the mapping between the name and path of submodules +in the superproject, such that the submodule's Git directory can be located. + If the submodule has never been initialized, this is the only place @@ -137,8 +141,8 @@ directory is automatically moved to `$GIT_DIR/modules//` of the superproject. * Deinitialized submodule: A `gitlink`, and a `.gitmodules` entry, -but no submodule working directory. The submodule’s git directory -may be there as after deinitializing the git directory is kept around. +but no submodule working directory. The submodule’s Git directory +may be there as after deinitializing the Git directory is kept around. The directory which is supposed to be the working directory is empty instead. + A submodule can be deinitialized by running `git submodule deinit`. @@ -160,6 +164,60 @@ from another repository. To completely remove a submodule, manually delete `$GIT_DIR/modules//`. +ACTIVE SUBMODULES +----------------- + +A submodule is considered active, + + (a) if `submodule..active` is set to `true` + or + (b) if the submodule's path matches the pathspec in `submodule.active` + or + (c) if `submodule..url` is set. + +and these are evaluated in this order. + +For example: + + [submodule "foo"] + active = false + url = https://example.org/foo + [submodule "bar"] + active = true + url = https://example.org/bar + [submodule "baz"] + url = https://example.org/baz + +In the above config only the submodule 'bar' and 'baz' are active, +'bar' due to (a) and 'baz' due to (c). 'foo' is inactive because +(a) takes precedence over (c) + +Note that (c) is a historical artefact and will be ignored if the +(a) and (b) specify that the submodule is not active. In other words, +if we have an `submodule..active` set to `false` or if the +submodule's path is excluded in the pathspec in `submodule.active`, the +url doesn't matter whether it is present or not. This is illustrated in +the example that follows. + + [submodule "foo"] + active = true + url = https://example.org/foo + [submodule "bar"] + url = https://example.org/bar + [submodule "baz"] + url = https://example.org/baz + [submodule "bob"] + ignore = true + [submodule] + active = b* + active = :(exclude) baz + +In here all submodules except 'baz' (foo, bar, bob) are active. +'foo' due to its own active flag and all the others due to the +submodule active pathspec, which specifies that any submodule +starting with 'b' except 'baz' are also active, regardless of the +presence of the .url field. + Workflow for a third party library ---------------------------------- diff --git a/Documentation/rev-list-options.txt b/Documentation/rev-list-options.txt index 22f5c9b43d..7b273635de 100644 --- a/Documentation/rev-list-options.txt +++ b/Documentation/rev-list-options.txt @@ -750,10 +750,21 @@ The form '--missing=allow-any' will allow object traversal to continue if a missing object is encountered. Missing objects will silently be omitted from the results. + +The form '--missing=allow-promisor' is like 'allow-any', but will only +allow object traversal to continue for EXPECTED promisor missing objects. +Unexpected missing objects will raise an error. ++ The form '--missing=print' is like 'allow-any', but will also print a list of the missing objects. Object IDs are prefixed with a ``?'' character. endif::git-rev-list[] +--exclude-promisor-objects:: + (For internal use only.) Prefilter object traversal at + promisor boundary. This is used with partial clone. This is + stronger than `--missing=allow-promisor` because it limits the + traversal, rather than just silencing errors about missing + objects. + --no-walk[=(sorted|unsorted)]:: Only show the given commits, but do not traverse their ancestors. This has no effect if a range is specified. If the argument diff --git a/Documentation/technical/long-running-process-protocol.txt b/Documentation/technical/long-running-process-protocol.txt new file mode 100644 index 0000000000..aa0aa9af1c --- /dev/null +++ b/Documentation/technical/long-running-process-protocol.txt @@ -0,0 +1,50 @@ +Long-running process protocol +============================= + +This protocol is used when Git needs to communicate with an external +process throughout the entire life of a single Git command. All +communication is in pkt-line format (see technical/protocol-common.txt) +over standard input and standard output. + +Handshake +--------- + +Git starts by sending a welcome message (for example, +"git-filter-client"), a list of supported protocol version numbers, and +a flush packet. Git expects to read the welcome message with "server" +instead of "client" (for example, "git-filter-server"), exactly one +protocol version number from the previously sent list, and a flush +packet. All further communication will be based on the selected version. +The remaining protocol description below documents "version=2". Please +note that "version=42" in the example below does not exist and is only +there to illustrate how the protocol would look like with more than one +version. + +After the version negotiation Git sends a list of all capabilities that +it supports and a flush packet. Git expects to read a list of desired +capabilities, which must be a subset of the supported capabilities list, +and a flush packet as response: +------------------------ +packet: git> git-filter-client +packet: git> version=2 +packet: git> version=42 +packet: git> 0000 +packet: git< git-filter-server +packet: git< version=2 +packet: git< 0000 +packet: git> capability=clean +packet: git> capability=smudge +packet: git> capability=not-yet-invented +packet: git> 0000 +packet: git< capability=clean +packet: git< capability=smudge +packet: git< 0000 +------------------------ + +Shutdown +-------- + +Git will close +the command pipe on exit. The filter is expected to detect EOF +and exit gracefully on its own. Git will wait until the filter +process has stopped. diff --git a/Documentation/technical/pack-protocol.txt b/Documentation/technical/pack-protocol.txt index cd31edc91e..7fee6b780a 100644 --- a/Documentation/technical/pack-protocol.txt +++ b/Documentation/technical/pack-protocol.txt @@ -241,6 +241,7 @@ out of what the server said it could do with the first 'want' line. upload-request = want-list *shallow-line *1depth-request + [filter-request] flush-pkt want-list = first-want @@ -256,6 +257,8 @@ out of what the server said it could do with the first 'want' line. additional-want = PKT-LINE("want" SP obj-id) depth = 1*DIGIT + + filter-request = PKT-LINE("filter" SP filter-spec) ---- Clients MUST send all the obj-ids it wants from the reference @@ -278,6 +281,11 @@ complete those commits. Commits whose parents are not received as a result are defined as shallow and marked as such in the server. This information is sent back to the client in the next step. +The client can optionally request that pack-objects omit various +objects from the packfile using one of several filtering techniques. +These are intended for use with partial clone and partial fetch +operations. See `rev-list` for possible "filter-spec" values. + Once all the 'want's and 'shallow's (and optional 'deepen') are transferred, clients MUST send a flush-pkt, to tell the server side that it is done sending the list. diff --git a/Documentation/technical/protocol-capabilities.txt b/Documentation/technical/protocol-capabilities.txt index 26dcc6f502..332d209b58 100644 --- a/Documentation/technical/protocol-capabilities.txt +++ b/Documentation/technical/protocol-capabilities.txt @@ -309,3 +309,11 @@ to accept a signed push certificate, and asks the to be included in the push certificate. A send-pack client MUST NOT send a push-cert packet unless the receive-pack server advertises this capability. + +filter +------ + +If the upload-pack server advertises the 'filter' capability, +fetch-pack may send "filter" commands to request a partial clone +or partial fetch and request that the server omit various objects +from the packfile. diff --git a/Documentation/technical/repository-version.txt b/Documentation/technical/repository-version.txt index 00ad37986e..e03eaccebc 100644 --- a/Documentation/technical/repository-version.txt +++ b/Documentation/technical/repository-version.txt @@ -86,3 +86,15 @@ for testing format-1 compatibility. When the config key `extensions.preciousObjects` is set to `true`, objects in the repository MUST NOT be deleted (e.g., by `git-prune` or `git repack -d`). + +`partialclone` +~~~~~~~~~~~~~~ + +When the config key `extensions.partialclone` is set, it indicates +that the repo was created with a partial clone (or later performed +a partial fetch) and that the remote may have omitted sending +certain unwanted objects. Such a remote is called a "promisor remote" +and it promises that all such omitted objects can be fetched from it +in the future. + +The value of this key is the name of the promisor remote. diff --git a/GIT-VERSION-GEN b/GIT-VERSION-GEN index 470f7b7b83..0c86f1c768 100755 --- a/GIT-VERSION-GEN +++ b/GIT-VERSION-GEN @@ -1,7 +1,7 @@ #!/bin/sh GVF=GIT-VERSION-FILE -DEF_VER=v2.16.1 +DEF_VER=v2.16.GIT LF=' ' diff --git a/INSTALL b/INSTALL index ffb071e9f0..808e07b659 100644 --- a/INSTALL +++ b/INSTALL @@ -84,9 +84,24 @@ Issues of note: GIT_EXEC_PATH=`pwd` PATH=`pwd`:$PATH - GITPERLLIB=`pwd`/perl/blib/lib + GITPERLLIB=`pwd`/perl/build/lib export GIT_EXEC_PATH PATH GITPERLLIB + - By default (unless NO_PERL is provided) Git will ship various perl + scripts & libraries it needs. However, for simplicity it doesn't + use the ExtUtils::MakeMaker toolchain to decide where to place the + perl libraries. Depending on the system this can result in the perl + libraries not being where you'd like them if they're expected to be + used by things other than Git itself. + + Manually supplying a perllibdir prefix should fix this, if this is + a problem you care about, e.g.: + + prefix=/usr perllibdir=/usr/$(/usr/bin/perl -MConfig -wle 'print substr $Config{installsitelib}, 1 + length $Config{siteprefixexp}') + + Will result in e.g. perllibdir=/usr/share/perl/5.26.1 on Debian, + perllibdir=/usr/share/perl5 (which we'd use by default) on CentOS. + - Git is reasonably self-sufficient, but does depend on a few external programs and libraries. Git can be used without most of them by adding the approriate "NO_=YesPlease" to the make command line or diff --git a/Makefile b/Makefile index 1a9b23b679..5bcd83ddf3 100644 --- a/Makefile +++ b/Makefile @@ -294,9 +294,6 @@ all:: # # Define PERL_PATH to the path of your Perl binary (usually /usr/bin/perl). # -# Define NO_PERL_MAKEMAKER if you cannot use Makefiles generated by perl's -# MakeMaker (e.g. using ActiveState under Cygwin). -# # Define NO_PERL if you do not want Perl scripts or libraries at all. # # Define PYTHON_PATH to the path of your Python binary (often /usr/bin/python @@ -479,6 +476,7 @@ gitexecdir = libexec/git-core mergetoolsdir = $(gitexecdir)/mergetools sharedir = $(prefix)/share gitwebdir = $(sharedir)/gitweb +perllibdir = $(sharedir)/perl5 localedir = $(sharedir)/locale template_dir = share/git-core/templates htmldir = $(prefix)/share/doc/git-doc @@ -492,7 +490,7 @@ mandir_relative = $(patsubst $(prefix)/%,%,$(mandir)) infodir_relative = $(patsubst $(prefix)/%,%,$(infodir)) htmldir_relative = $(patsubst $(prefix)/%,%,$(htmldir)) -export prefix bindir sharedir sysconfdir gitwebdir localedir +export prefix bindir sharedir sysconfdir gitwebdir perllibdir localedir CC = cc AR = ar @@ -804,6 +802,7 @@ LIB_OBJS += ewah/ewah_bitmap.o LIB_OBJS += ewah/ewah_io.o LIB_OBJS += ewah/ewah_rlw.o LIB_OBJS += exec_cmd.o +LIB_OBJS += fetch-object.o LIB_OBJS += fetch-pack.o LIB_OBJS += fsck.o LIB_OBJS += fsmonitor.o @@ -832,7 +831,6 @@ LIB_OBJS += merge.o LIB_OBJS += merge-blobs.o LIB_OBJS += merge-recursive.o LIB_OBJS += mergesort.o -LIB_OBJS += mru.o LIB_OBJS += name-hash.o LIB_OBJS += notes.o LIB_OBJS += notes-cache.o @@ -1543,9 +1541,6 @@ ifdef SHA1_MAX_BLOCK_SIZE LIB_OBJS += compat/sha1-chunked.o BASIC_CFLAGS += -DSHA1_MAX_BLOCK_SIZE="$(SHA1_MAX_BLOCK_SIZE)" endif -ifdef NO_PERL_MAKEMAKER - export NO_PERL_MAKEMAKER -endif ifdef NO_HSTRERROR COMPAT_CFLAGS += -DNO_HSTRERROR COMPAT_OBJS += compat/hstrerror.o @@ -1732,8 +1727,10 @@ ETC_GITATTRIBUTES_SQ = $(subst ','\'',$(ETC_GITATTRIBUTES)) DESTDIR_SQ = $(subst ','\'',$(DESTDIR)) bindir_SQ = $(subst ','\'',$(bindir)) bindir_relative_SQ = $(subst ','\'',$(bindir_relative)) +mandir_SQ = $(subst ','\'',$(mandir)) mandir_relative_SQ = $(subst ','\'',$(mandir_relative)) infodir_relative_SQ = $(subst ','\'',$(infodir_relative)) +perllibdir_SQ = $(subst ','\'',$(perllibdir)) localedir_SQ = $(subst ','\'',$(localedir)) gitexecdir_SQ = $(subst ','\'',$(gitexecdir)) template_dir_SQ = $(subst ','\'',$(template_dir)) @@ -1843,9 +1840,6 @@ all:: ifndef NO_TCLTK $(QUIET_SUBDIR0)git-gui $(QUIET_SUBDIR1) gitexecdir='$(gitexec_instdir_SQ)' all $(QUIET_SUBDIR0)gitk-git $(QUIET_SUBDIR1) all -endif -ifndef NO_PERL - $(QUIET_SUBDIR0)perl $(QUIET_SUBDIR1) PERL_PATH='$(PERL_PATH_SQ)' prefix='$(prefix_SQ)' localedir='$(localedir_SQ)' all endif $(QUIET_SUBDIR0)templates $(QUIET_SUBDIR1) SHELL_PATH='$(SHELL_PATH_SQ)' PERL_PATH='$(PERL_PATH_SQ)' @@ -1928,7 +1922,8 @@ common-cmds.h: $(wildcard Documentation/git-*.txt) SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\ $(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\ - $(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV) + $(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\ + $(perllibdir_SQ) define cmd_munge_script $(RM) $@ $@+ && \ sed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \ @@ -1972,23 +1967,12 @@ git.res: git.rc GIT-VERSION-FILE $(SCRIPT_PERL_GEN): GIT-BUILD-OPTIONS ifndef NO_PERL -$(SCRIPT_PERL_GEN): perl/perl.mak - -perl/perl.mak: perl/PM.stamp - -perl/PM.stamp: FORCE - @$(FIND) perl -type f -name '*.pm' | sort >$@+ && \ - $(PERL_PATH) -V >>$@+ && \ - { cmp $@+ $@ >/dev/null 2>/dev/null || mv $@+ $@; } && \ - $(RM) $@+ - -perl/perl.mak: GIT-CFLAGS GIT-PREFIX perl/Makefile perl/Makefile.PL - $(QUIET_SUBDIR0)perl $(QUIET_SUBDIR1) PERL_PATH='$(PERL_PATH_SQ)' prefix='$(prefix_SQ)' $(@F) +$(SCRIPT_PERL_GEN): -PERL_DEFINES = $(PERL_PATH_SQ):$(PERLLIB_EXTRA_SQ) -$(SCRIPT_PERL_GEN): % : %.perl perl/perl.mak GIT-PERL-DEFINES GIT-VERSION-FILE +PERL_DEFINES = $(PERL_PATH_SQ):$(PERLLIB_EXTRA_SQ):$(perllibdir_SQ) +$(SCRIPT_PERL_GEN): % : %.perl GIT-PERL-DEFINES GIT-VERSION-FILE $(QUIET_GEN)$(RM) $@ $@+ && \ - INSTLIBDIR=`MAKEFLAGS= $(MAKE) -C perl -s --no-print-directory instlibdir` && \ + INSTLIBDIR='$(perllibdir_SQ)' && \ INSTLIBDIR_EXTRA='$(PERLLIB_EXTRA_SQ)' && \ INSTLIBDIR="$$INSTLIBDIR$${INSTLIBDIR_EXTRA:+:$$INSTLIBDIR_EXTRA}" && \ sed -e '1{' \ @@ -2312,6 +2296,21 @@ endif po/build/locale/%/LC_MESSAGES/git.mo: po/%.po $(QUIET_MSGFMT)mkdir -p $(dir $@) && $(MSGFMT) -o $@ $< +LIB_PERL := $(wildcard perl/Git.pm perl/Git/*.pm perl/Git/*/*.pm perl/Git/*/*/*.pm) +LIB_PERL_GEN := $(patsubst perl/%.pm,perl/build/lib/%.pm,$(LIB_PERL)) + +ifndef NO_PERL +all:: $(LIB_PERL_GEN) +endif + +perl/build/lib/%.pm: perl/%.pm + $(QUIET_GEN)mkdir -p $(dir $@) && \ + sed -e 's|@@LOCALEDIR@@|$(localedir_SQ)|g' < $< > $@ + +perl/build/man/man3/Git.3pm: perl/Git.pm + $(QUIET_GEN)mkdir -p $(dir $@) && \ + pod2man $< $@ + FIND_SOURCE_FILES = ( \ git ls-files \ '*.[hcS]' \ @@ -2572,7 +2571,9 @@ ifndef NO_GETTEXT (cd '$(DESTDIR_SQ)$(localedir_SQ)' && umask 022 && $(TAR) xof -) endif ifndef NO_PERL - $(MAKE) -C perl prefix='$(prefix_SQ)' DESTDIR='$(DESTDIR_SQ)' install + $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perllibdir_SQ)' + (cd perl/build/lib && $(TAR) cf - .) | \ + (cd '$(DESTDIR_SQ)$(perllibdir_SQ)' && umask 022 && $(TAR) xof -) $(MAKE) -C gitweb install endif ifndef NO_TCLTK @@ -2622,12 +2623,17 @@ endif install-gitweb: $(MAKE) -C gitweb install -install-doc: +install-doc: install-man-perl $(MAKE) -C Documentation install -install-man: +install-man: install-man-perl $(MAKE) -C Documentation install-man +install-man-perl: perl/build/man/man3/Git.3pm + $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(mandir_SQ)/man3' + (cd perl/build/man/man3 && $(TAR) cf - .) | \ + (cd '$(DESTDIR_SQ)$(mandir_SQ)/man3' && umask 022 && $(TAR) xof -) + install-html: $(MAKE) -C Documentation install-html @@ -2719,7 +2725,7 @@ clean: profile-clean coverage-clean $(MAKE) -C Documentation/ clean ifndef NO_PERL $(MAKE) -C gitweb clean - $(MAKE) -C perl clean + $(RM) -r perl/build/ endif $(MAKE) -C templates/ clean $(MAKE) -C t/ clean diff --git a/RelNotes b/RelNotes index 0172894cc2..7a6dc0603b 120000 --- a/RelNotes +++ b/RelNotes @@ -1 +1 @@ -Documentation/RelNotes/2.16.1.txt \ No newline at end of file +Documentation/RelNotes/2.17.0.txt \ No newline at end of file diff --git a/apply.c b/apply.c index 321a9fa68d..f8b67bfee2 100644 --- a/apply.c +++ b/apply.c @@ -2263,8 +2263,8 @@ static void show_stats(struct apply_state *state, struct patch *patch) static int read_old_data(struct stat *st, struct patch *patch, const char *path, struct strbuf *buf) { - enum safe_crlf safe_crlf = patch->crlf_in_old ? - SAFE_CRLF_KEEP_CRLF : SAFE_CRLF_RENORMALIZE; + int conv_flags = patch->crlf_in_old ? + CONV_EOL_KEEP_CRLF : CONV_EOL_RENORMALIZE; switch (st->st_mode & S_IFMT) { case S_IFLNK: if (strbuf_readlink(buf, path, st->st_size) < 0) @@ -2281,7 +2281,7 @@ static int read_old_data(struct stat *st, struct patch *patch, * should never look at the index when explicit crlf option * is given. */ - convert_to_git(NULL, path, buf->buf, buf->len, buf, safe_crlf); + convert_to_git(NULL, path, buf->buf, buf->len, buf, conv_flags); return 0; default: return -1; diff --git a/bisect.c b/bisect.c index 2f3008b078..f6d05bd66f 100644 --- a/bisect.c +++ b/bisect.c @@ -132,7 +132,7 @@ static void show_list(const char *debug, int counted, int nr, unsigned flags = commit->object.flags; enum object_type type; unsigned long size; - char *buf = read_sha1_file(commit->object.sha1, &type, &size); + char *buf = read_sha1_file(commit->object.oid.hash, &type, &size); const char *subject_start; int subject_len; @@ -144,10 +144,10 @@ static void show_list(const char *debug, int counted, int nr, fprintf(stderr, "%3d", weight(p)); else fprintf(stderr, "---"); - fprintf(stderr, " %.*s", 8, sha1_to_hex(commit->object.sha1)); + fprintf(stderr, " %.*s", 8, sha1_to_hex(commit->object.oid.hash)); for (pp = commit->parents; pp; pp = pp->next) fprintf(stderr, " %.*s", 8, - sha1_to_hex(pp->item->object.sha1)); + sha1_to_hex(pp->item->object.oid.hash)); subject_len = find_commit_subject(buf, &subject_start); if (subject_len) @@ -792,11 +792,9 @@ static void handle_skipped_merge_base(const struct object_id *mb) * - If one is "skipped", we can't know but we should warn. * - If we don't know, we should check it out and ask the user to test. */ -static void check_merge_bases(int no_checkout) +static void check_merge_bases(int rev_nr, struct commit **rev, int no_checkout) { struct commit_list *result; - int rev_nr; - struct commit **rev = get_bad_and_good_commits(&rev_nr); result = get_merge_bases_many(rev[0], rev_nr - 1, rev + 1); @@ -814,34 +812,21 @@ static void check_merge_bases(int no_checkout) } } - free(rev); free_commit_list(result); } -static int check_ancestors(const char *prefix) +static int check_ancestors(int rev_nr, struct commit **rev, const char *prefix) { struct rev_info revs; - struct object_array pending_copy; int res; bisect_rev_setup(&revs, prefix, "^%s", "%s", 0); - /* Save pending objects, so they can be cleaned up later. */ - pending_copy = revs.pending; - revs.leak_pending = 1; - - /* - * bisect_common calls prepare_revision_walk right away, which - * (together with .leak_pending = 1) makes us the sole owner of - * the list of pending objects. - */ bisect_common(&revs); res = (revs.commits != NULL); /* Clean up objects used, as they will be reused. */ - clear_commit_marks_for_object_array(&pending_copy, ALL_REV_FLAGS); - - object_array_clear(&pending_copy); + clear_commit_marks_many(rev_nr, rev, ALL_REV_FLAGS); return res; } @@ -858,7 +843,8 @@ static void check_good_are_ancestors_of_bad(const char *prefix, int no_checkout) { char *filename = git_pathdup("BISECT_ANCESTORS_OK"); struct stat st; - int fd; + int fd, rev_nr; + struct commit **rev; if (!current_bad_oid) die(_("a %s revision is needed"), term_bad); @@ -872,8 +858,10 @@ static void check_good_are_ancestors_of_bad(const char *prefix, int no_checkout) goto done; /* Check if all good revs are ancestor of the bad rev. */ - if (check_ancestors(prefix)) - check_merge_bases(no_checkout); + rev = get_bad_and_good_commits(&rev_nr); + if (check_ancestors(rev_nr, rev, prefix)) + check_merge_bases(rev_nr, rev, no_checkout); + free(rev); /* Create file BISECT_ANCESTORS_OK. */ fd = open(filename, O_CREAT | O_TRUNC | O_WRONLY, 0600); diff --git a/builtin/am.c b/builtin/am.c index acfe9d3c8c..5bdd2d7578 100644 --- a/builtin/am.c +++ b/builtin/am.c @@ -1061,7 +1061,7 @@ static void am_setup(struct am_state *state, enum patch_format patch_format, } write_state_text(state, "scissors", str); - sq_quote_argv(&sb, state->git_apply_opts.argv, 0); + sq_quote_argv(&sb, state->git_apply_opts.argv); write_state_text(state, "apply-opt", sb.buf); if (state->rebasing) diff --git a/builtin/cat-file.c b/builtin/cat-file.c index f5fa4fd75a..cf9ea5c796 100644 --- a/builtin/cat-file.c +++ b/builtin/cat-file.c @@ -475,6 +475,8 @@ static int batch_objects(struct batch_options *opt) for_each_loose_object(batch_loose_object, &sa, 0); for_each_packed_object(batch_packed_object, &sa, 0); + if (repository_format_partial_clone) + warning("This repository has extensions.partialClone set. Some objects may not be loaded."); cb.opt = opt; cb.expand = &data; diff --git a/builtin/checkout.c b/builtin/checkout.c index 8bdc927d3f..c54c78df54 100644 --- a/builtin/checkout.c +++ b/builtin/checkout.c @@ -791,7 +791,6 @@ static void orphaned_commit_warning(struct commit *old, struct commit *new) { struct rev_info revs; struct object *object = &old->object; - struct object_array refs; init_revisions(&revs, NULL); setup_revisions(0, NULL, &revs, NULL); @@ -802,14 +801,6 @@ static void orphaned_commit_warning(struct commit *old, struct commit *new) for_each_ref(add_pending_uninteresting_ref, &revs); add_pending_oid(&revs, "HEAD", &new->object.oid, UNINTERESTING); - /* Save pending objects, so they can be cleaned up later. */ - refs = revs.pending; - revs.leak_pending = 1; - - /* - * prepare_revision_walk (together with .leak_pending = 1) makes us - * the sole owner of the list of pending objects. - */ if (prepare_revision_walk(&revs)) die(_("internal error in revision walk")); if (!(old->object.flags & UNINTERESTING)) @@ -818,9 +809,7 @@ static void orphaned_commit_warning(struct commit *old, struct commit *new) describe_detached_head(_("Previous HEAD position was"), old); /* Clean up objects used, as they will be reused. */ - clear_commit_marks_for_object_array(&refs, ALL_REV_FLAGS); - - object_array_clear(&refs); + clear_commit_marks_all(ALL_REV_FLAGS); } static int switch_branches(const struct checkout_opts *opts, diff --git a/builtin/clone.c b/builtin/clone.c index 2da71db107..101c27a593 100644 --- a/builtin/clone.c +++ b/builtin/clone.c @@ -26,6 +26,7 @@ #include "run-command.h" #include "connected.h" #include "packfile.h" +#include "list-objects-filter-options.h" /* * Overall FIXMEs: @@ -60,6 +61,7 @@ static struct string_list option_optional_reference = STRING_LIST_INIT_NODUP; static int option_dissociate; static int max_jobs = -1; static struct string_list option_recurse_submodules = STRING_LIST_INIT_NODUP; +static struct list_objects_filter_options filter_options; static int recurse_submodules_cb(const struct option *opt, const char *arg, int unset) @@ -135,6 +137,7 @@ static struct option builtin_clone_options[] = { TRANSPORT_FAMILY_IPV4), OPT_SET_INT('6', "ipv6", &family, N_("use IPv6 addresses only"), TRANSPORT_FAMILY_IPV6), + OPT_PARSE_LIST_OBJECTS_FILTER(&filter_options), OPT_END() }; @@ -473,7 +476,9 @@ static void clone_local(const char *src_repo, const char *dest_repo) } static const char *junk_work_tree; +static int junk_work_tree_flags; static const char *junk_git_dir; +static int junk_git_dir_flags; static enum { JUNK_LEAVE_NONE, JUNK_LEAVE_REPO, @@ -502,12 +507,12 @@ static void remove_junk(void) if (junk_git_dir) { strbuf_addstr(&sb, junk_git_dir); - remove_dir_recursively(&sb, 0); + remove_dir_recursively(&sb, junk_git_dir_flags); strbuf_reset(&sb); } if (junk_work_tree) { strbuf_addstr(&sb, junk_work_tree); - remove_dir_recursively(&sb, 0); + remove_dir_recursively(&sb, junk_work_tree_flags); } strbuf_release(&sb); } @@ -863,10 +868,15 @@ static void dissociate_from_references(void) free(alternates); } +static int dir_exists(const char *path) +{ + struct stat sb; + return !stat(path, &sb); +} + int cmd_clone(int argc, const char **argv, const char *prefix) { int is_bundle = 0, is_local; - struct stat buf; const char *repo_name, *repo, *work_tree, *git_dir; char *path, *dir; int dest_exists; @@ -886,6 +896,8 @@ int cmd_clone(int argc, const char **argv, const char *prefix) struct refspec *refspec; const char *fetch_pattern; + fetch_if_missing = 0; + packet_trace_identity("clone"); argc = parse_options(argc, argv, prefix, builtin_clone_options, builtin_clone_usage, 0); @@ -938,7 +950,7 @@ int cmd_clone(int argc, const char **argv, const char *prefix) dir = guess_dir_name(repo_name, is_bundle, option_bare); strip_trailing_slashes(dir); - dest_exists = !stat(dir, &buf); + dest_exists = dir_exists(dir); if (dest_exists && !is_empty_dir(dir)) die(_("destination path '%s' already exists and is not " "an empty directory."), dir); @@ -949,7 +961,7 @@ int cmd_clone(int argc, const char **argv, const char *prefix) work_tree = NULL; else { work_tree = getenv("GIT_WORK_TREE"); - if (work_tree && !stat(work_tree, &buf)) + if (work_tree && dir_exists(work_tree)) die(_("working tree '%s' already exists."), work_tree); } @@ -967,14 +979,24 @@ int cmd_clone(int argc, const char **argv, const char *prefix) if (safe_create_leading_directories_const(work_tree) < 0) die_errno(_("could not create leading directories of '%s'"), work_tree); - if (!dest_exists && mkdir(work_tree, 0777)) + if (dest_exists) + junk_work_tree_flags |= REMOVE_DIR_KEEP_TOPLEVEL; + else if (mkdir(work_tree, 0777)) die_errno(_("could not create work tree dir '%s'"), work_tree); junk_work_tree = work_tree; set_git_work_tree(work_tree); } - junk_git_dir = real_git_dir ? real_git_dir : git_dir; + if (real_git_dir) { + if (dir_exists(real_git_dir)) + junk_git_dir_flags |= REMOVE_DIR_KEEP_TOPLEVEL; + junk_git_dir = real_git_dir; + } else { + if (dest_exists) + junk_git_dir_flags |= REMOVE_DIR_KEEP_TOPLEVEL; + junk_git_dir = git_dir; + } if (safe_create_leading_directories_const(git_dir) < 0) die(_("could not create leading directories of '%s'"), git_dir); @@ -1073,6 +1095,8 @@ int cmd_clone(int argc, const char **argv, const char *prefix) warning(_("--shallow-since is ignored in local clones; use file:// instead.")); if (option_not.nr) warning(_("--shallow-exclude is ignored in local clones; use file:// instead.")); + if (filter_options.choice) + warning(_("--filter is ignored in local clones; use file:// instead.")); if (!access(mkpath("%s/shallow", path), F_OK)) { if (option_local > 0) warning(_("source repository is shallow, ignoring --local")); @@ -1101,7 +1125,13 @@ int cmd_clone(int argc, const char **argv, const char *prefix) transport_set_option(transport, TRANS_OPT_UPLOADPACK, option_upload_pack); - if (transport->smart_options && !deepen) + if (filter_options.choice) { + transport_set_option(transport, TRANS_OPT_LIST_OBJECTS_FILTER, + filter_options.filter_spec); + transport_set_option(transport, TRANS_OPT_FROM_PROMISOR, "1"); + } + + if (transport->smart_options && !deepen && !filter_options.choice) transport->smart_options->check_self_contained_and_connected = 1; refs = transport_get_remote_refs(transport); @@ -1161,13 +1191,17 @@ int cmd_clone(int argc, const char **argv, const char *prefix) write_refspec_config(src_ref_prefix, our_head_points_at, remote_head_points_at, &branch_top); + if (filter_options.choice) + partial_clone_register("origin", &filter_options); + if (is_local) clone_local(path, git_dir); else if (refs && complete_refs_before_fetch) transport_fetch_refs(transport, mapped_refs); update_remote_refs(refs, mapped_refs, remote_head_points_at, - branch_top.buf, reflog_msg.buf, transport, !is_local); + branch_top.buf, reflog_msg.buf, transport, + !is_local && !filter_options.choice); update_head(our_head_points_at, remote_head, reflog_msg.buf); @@ -1188,6 +1222,7 @@ int cmd_clone(int argc, const char **argv, const char *prefix) } junk_mode = JUNK_LEAVE_REPO; + fetch_if_missing = 1; err = checkout(submodule_progress); strbuf_release(&reflog_msg); diff --git a/builtin/commit.c b/builtin/commit.c index 8a87701414..5dd766af28 100644 --- a/builtin/commit.c +++ b/builtin/commit.c @@ -31,9 +31,7 @@ #include "gpg-interface.h" #include "column.h" #include "sequencer.h" -#include "notes-utils.h" #include "mailmap.h" -#include "sigchain.h" static const char * const builtin_commit_usage[] = { N_("git commit [] [--] ..."), @@ -45,31 +43,6 @@ static const char * const builtin_status_usage[] = { NULL }; -static const char implicit_ident_advice_noconfig[] = -N_("Your name and email address were configured automatically based\n" -"on your username and hostname. Please check that they are accurate.\n" -"You can suppress this message by setting them explicitly. Run the\n" -"following command and follow the instructions in your editor to edit\n" -"your configuration file:\n" -"\n" -" git config --global --edit\n" -"\n" -"After doing this, you may fix the identity used for this commit with:\n" -"\n" -" git commit --amend --reset-author\n"); - -static const char implicit_ident_advice_config[] = -N_("Your name and email address were configured automatically based\n" -"on your username and hostname. Please check that they are accurate.\n" -"You can suppress this message by setting them explicitly:\n" -"\n" -" git config --global user.name \"Your Name\"\n" -" git config --global user.email you@example.com\n" -"\n" -"After doing this, you may fix the identity used for this commit with:\n" -"\n" -" git commit --amend --reset-author\n"); - static const char empty_amend_advice[] = N_("You asked to amend the most recent commit, but doing so would make\n" "it empty. You can repeat your command with --allow-empty, or you can\n" @@ -93,8 +66,6 @@ N_("If you wish to skip this commit, use:\n" "Then \"git cherry-pick --continue\" will resume cherry-picking\n" "the remaining commits.\n"); -static GIT_PATH_FUNC(git_path_commit_editmsg, "COMMIT_EDITMSG") - static const char *use_message_buffer; static struct lock_file index_lock; /* real index */ static struct lock_file false_lock; /* used only for partial commits */ @@ -128,12 +99,7 @@ static char *sign_commit; * if editor is used, and only the whitespaces if the message * is specified explicitly. */ -static enum { - CLEANUP_SPACE, - CLEANUP_NONE, - CLEANUP_SCISSORS, - CLEANUP_ALL -} cleanup_mode; +static enum commit_msg_cleanup_mode cleanup_mode; static const char *cleanup_arg; static enum commit_whence whence; @@ -673,7 +639,7 @@ static int prepare_to_commit(const char *index_file, const char *prefix, struct strbuf sb = STRBUF_INIT; const char *hook_arg1 = NULL; const char *hook_arg2 = NULL; - int clean_message_contents = (cleanup_mode != CLEANUP_NONE); + int clean_message_contents = (cleanup_mode != COMMIT_MSG_CLEANUP_NONE); int old_display_comment_prefix; /* This checks and barfs if author is badly specified */ @@ -701,7 +667,7 @@ static int prepare_to_commit(const char *index_file, const char *prefix, } } - if (have_option_m) { + if (have_option_m && !fixup_message) { strbuf_addbuf(&sb, &message); hook_arg1 = "message"; } else if (logfile && !strcmp(logfile, "-")) { @@ -731,6 +697,8 @@ static int prepare_to_commit(const char *index_file, const char *prefix, ctx.output_encoding = get_commit_output_encoding(); format_commit_message(commit, "fixup! %s\n\n", &sb, &ctx); + if (have_option_m) + strbuf_addbuf(&sb, &message); hook_arg1 = "message"; } else if (!stat(git_path_merge_msg(), &statbuf)) { /* @@ -812,7 +780,7 @@ static int prepare_to_commit(const char *index_file, const char *prefix, struct ident_split ci, ai; if (whence != FROM_COMMIT) { - if (cleanup_mode == CLEANUP_SCISSORS) + if (cleanup_mode == COMMIT_MSG_CLEANUP_SCISSORS) wt_status_add_cut_line(s->fp); status_printf_ln(s, GIT_COLOR_NORMAL, whence == FROM_MERGE @@ -832,14 +800,15 @@ static int prepare_to_commit(const char *index_file, const char *prefix, } fprintf(s->fp, "\n"); - if (cleanup_mode == CLEANUP_ALL) + if (cleanup_mode == COMMIT_MSG_CLEANUP_ALL) status_printf(s, GIT_COLOR_NORMAL, _("Please enter the commit message for your changes." " Lines starting\nwith '%c' will be ignored, and an empty" " message aborts the commit.\n"), comment_line_char); - else if (cleanup_mode == CLEANUP_SCISSORS && whence == FROM_COMMIT) + else if (cleanup_mode == COMMIT_MSG_CLEANUP_SCISSORS && + whence == FROM_COMMIT) wt_status_add_cut_line(s->fp); - else /* CLEANUP_SPACE, that is. */ + else /* COMMIT_MSG_CLEANUP_SPACE, that is. */ status_printf(s, GIT_COLOR_NORMAL, _("Please enter the commit message for your changes." " Lines starting\n" @@ -984,65 +953,6 @@ static int prepare_to_commit(const char *index_file, const char *prefix, return 1; } -static int rest_is_empty(struct strbuf *sb, int start) -{ - int i, eol; - const char *nl; - - /* Check if the rest is just whitespace and Signed-off-by's. */ - for (i = start; i < sb->len; i++) { - nl = memchr(sb->buf + i, '\n', sb->len - i); - if (nl) - eol = nl - sb->buf; - else - eol = sb->len; - - if (strlen(sign_off_header) <= eol - i && - starts_with(sb->buf + i, sign_off_header)) { - i = eol; - continue; - } - while (i < eol) - if (!isspace(sb->buf[i++])) - return 0; - } - - return 1; -} - -/* - * Find out if the message in the strbuf contains only whitespace and - * Signed-off-by lines. - */ -static int message_is_empty(struct strbuf *sb) -{ - if (cleanup_mode == CLEANUP_NONE && sb->len) - return 0; - return rest_is_empty(sb, 0); -} - -/* - * See if the user edited the message in the editor or left what - * was in the template intact - */ -static int template_untouched(struct strbuf *sb) -{ - struct strbuf tmpl = STRBUF_INIT; - const char *start; - - if (cleanup_mode == CLEANUP_NONE && sb->len) - return 0; - - if (!template_file || strbuf_read_file(&tmpl, template_file, 0) <= 0) - return 0; - - strbuf_stripspace(&tmpl, cleanup_mode == CLEANUP_ALL); - if (!skip_prefix(sb->buf, tmpl.buf, &start)) - start = sb->buf; - strbuf_release(&tmpl); - return rest_is_empty(sb, start - sb->buf); -} - static const char *find_author_by_nickname(const char *name) { struct rev_info revs; @@ -1197,8 +1107,8 @@ static int parse_and_validate_options(int argc, const char *argv[], f++; if (f > 1) die(_("Only one of -c/-C/-F/--fixup can be used.")); - if (have_option_m && f > 0) - die((_("Option -m cannot be combined with -c/-C/-F/--fixup."))); + if (have_option_m && (edit_message || use_message || logfile)) + die((_("Option -m cannot be combined with -c/-C/-F."))); if (f || have_option_m) template_file = NULL; if (edit_message) @@ -1227,15 +1137,17 @@ static int parse_and_validate_options(int argc, const char *argv[], if (argc == 0 && (also || (only && !amend && !allow_empty))) die(_("No paths with --include/--only does not make sense.")); if (!cleanup_arg || !strcmp(cleanup_arg, "default")) - cleanup_mode = use_editor ? CLEANUP_ALL : CLEANUP_SPACE; + cleanup_mode = use_editor ? COMMIT_MSG_CLEANUP_ALL : + COMMIT_MSG_CLEANUP_SPACE; else if (!strcmp(cleanup_arg, "verbatim")) - cleanup_mode = CLEANUP_NONE; + cleanup_mode = COMMIT_MSG_CLEANUP_NONE; else if (!strcmp(cleanup_arg, "whitespace")) - cleanup_mode = CLEANUP_SPACE; + cleanup_mode = COMMIT_MSG_CLEANUP_SPACE; else if (!strcmp(cleanup_arg, "strip")) - cleanup_mode = CLEANUP_ALL; + cleanup_mode = COMMIT_MSG_CLEANUP_ALL; else if (!strcmp(cleanup_arg, "scissors")) - cleanup_mode = use_editor ? CLEANUP_SCISSORS : CLEANUP_SPACE; + cleanup_mode = use_editor ? COMMIT_MSG_CLEANUP_SCISSORS : + COMMIT_MSG_CLEANUP_SPACE; else die(_("Invalid cleanup mode %s"), cleanup_arg); @@ -1437,98 +1349,6 @@ int cmd_status(int argc, const char **argv, const char *prefix) return 0; } -static const char *implicit_ident_advice(void) -{ - char *user_config = expand_user_path("~/.gitconfig", 0); - char *xdg_config = xdg_config_home("config"); - int config_exists = file_exists(user_config) || file_exists(xdg_config); - - free(user_config); - free(xdg_config); - - if (config_exists) - return _(implicit_ident_advice_config); - else - return _(implicit_ident_advice_noconfig); - -} - -static void print_summary(const char *prefix, const struct object_id *oid, - int initial_commit) -{ - struct rev_info rev; - struct commit *commit; - struct strbuf format = STRBUF_INIT; - const char *head; - struct pretty_print_context pctx = {0}; - struct strbuf author_ident = STRBUF_INIT; - struct strbuf committer_ident = STRBUF_INIT; - - commit = lookup_commit(oid); - if (!commit) - die(_("couldn't look up newly created commit")); - if (parse_commit(commit)) - die(_("could not parse newly created commit")); - - strbuf_addstr(&format, "format:%h] %s"); - - format_commit_message(commit, "%an <%ae>", &author_ident, &pctx); - format_commit_message(commit, "%cn <%ce>", &committer_ident, &pctx); - if (strbuf_cmp(&author_ident, &committer_ident)) { - strbuf_addstr(&format, "\n Author: "); - strbuf_addbuf_percentquote(&format, &author_ident); - } - if (author_date_is_interesting()) { - struct strbuf date = STRBUF_INIT; - format_commit_message(commit, "%ad", &date, &pctx); - strbuf_addstr(&format, "\n Date: "); - strbuf_addbuf_percentquote(&format, &date); - strbuf_release(&date); - } - if (!committer_ident_sufficiently_given()) { - strbuf_addstr(&format, "\n Committer: "); - strbuf_addbuf_percentquote(&format, &committer_ident); - if (advice_implicit_identity) { - strbuf_addch(&format, '\n'); - strbuf_addstr(&format, implicit_ident_advice()); - } - } - strbuf_release(&author_ident); - strbuf_release(&committer_ident); - - init_revisions(&rev, prefix); - setup_revisions(0, NULL, &rev, NULL); - - rev.diff = 1; - rev.diffopt.output_format = - DIFF_FORMAT_SHORTSTAT | DIFF_FORMAT_SUMMARY; - - rev.verbose_header = 1; - rev.show_root_diff = 1; - get_commit_format(format.buf, &rev); - rev.always_show_header = 0; - rev.diffopt.detect_rename = 1; - rev.diffopt.break_opt = 0; - diff_setup_done(&rev.diffopt); - - head = resolve_ref_unsafe("HEAD", 0, NULL, NULL); - if (!head) - die_errno(_("unable to resolve HEAD after creating commit")); - if (!strcmp(head, "HEAD")) - head = _("detached HEAD"); - else - skip_prefix(head, "refs/heads/", &head); - printf("[%s%s ", head, initial_commit ? _(" (root-commit)") : ""); - - if (!log_tree_commit(&rev, commit)) { - rev.always_show_header = 1; - rev.use_terminator = 1; - log_tree_commit(&rev, commit); - } - - strbuf_release(&format); -} - static int git_commit_config(const char *k, const char *v, void *cb) { struct wt_status *s = cb; @@ -1558,37 +1378,6 @@ static int git_commit_config(const char *k, const char *v, void *cb) return git_status_config(k, v, s); } -static int run_rewrite_hook(const struct object_id *oldoid, - const struct object_id *newoid) -{ - struct child_process proc = CHILD_PROCESS_INIT; - const char *argv[3]; - int code; - struct strbuf sb = STRBUF_INIT; - - argv[0] = find_hook("post-rewrite"); - if (!argv[0]) - return 0; - - argv[1] = "amend"; - argv[2] = NULL; - - proc.argv = argv; - proc.in = -1; - proc.stdout_to_stderr = 1; - - code = start_command(&proc); - if (code) - return code; - strbuf_addf(&sb, "%s %s\n", oid_to_hex(oldoid), oid_to_hex(newoid)); - sigchain_push(SIGPIPE, SIG_IGN); - write_in_full(proc.in, sb.buf, sb.len); - close(proc.in); - strbuf_release(&sb); - sigchain_pop(SIGPIPE); - return finish_command(&proc); -} - int run_commit_hook(int editor_is_used, const char *index_file, const char *name, ...) { struct argv_array hook_env = ARGV_ARRAY_INIT; @@ -1671,13 +1460,11 @@ int cmd_commit(int argc, const char **argv, const char *prefix) struct strbuf sb = STRBUF_INIT; struct strbuf author_ident = STRBUF_INIT; const char *index_file, *reflog_msg; - char *nl; struct object_id oid; struct commit_list *parents = NULL; struct stat statbuf; struct commit *current_head = NULL; struct commit_extra_header *extra = NULL; - struct ref_transaction *transaction; struct strbuf err = STRBUF_INIT; if (argc == 2 && !strcmp(argv[1], "-h")) @@ -1768,17 +1555,17 @@ int cmd_commit(int argc, const char **argv, const char *prefix) } if (verbose || /* Truncate the message just before the diff, if any. */ - cleanup_mode == CLEANUP_SCISSORS) + cleanup_mode == COMMIT_MSG_CLEANUP_SCISSORS) strbuf_setlen(&sb, wt_status_locate_end(sb.buf, sb.len)); - if (cleanup_mode != CLEANUP_NONE) - strbuf_stripspace(&sb, cleanup_mode == CLEANUP_ALL); + if (cleanup_mode != COMMIT_MSG_CLEANUP_NONE) + strbuf_stripspace(&sb, cleanup_mode == COMMIT_MSG_CLEANUP_ALL); - if (message_is_empty(&sb) && !allow_empty_message) { + if (message_is_empty(&sb, cleanup_mode) && !allow_empty_message) { rollback_index_files(); fprintf(stderr, _("Aborting commit due to empty commit message.\n")); exit(1); } - if (template_untouched(&sb) && !allow_empty_message) { + if (template_untouched(&sb, template_file, cleanup_mode) && !allow_empty_message) { rollback_index_files(); fprintf(stderr, _("Aborting commit; you did not edit the message.\n")); exit(1); @@ -1800,25 +1587,11 @@ int cmd_commit(int argc, const char **argv, const char *prefix) strbuf_release(&author_ident); free_commit_extra_headers(extra); - nl = strchr(sb.buf, '\n'); - if (nl) - strbuf_setlen(&sb, nl + 1 - sb.buf); - else - strbuf_addch(&sb, '\n'); - strbuf_insert(&sb, 0, reflog_msg, strlen(reflog_msg)); - strbuf_insert(&sb, strlen(reflog_msg), ": ", 2); - - transaction = ref_transaction_begin(&err); - if (!transaction || - ref_transaction_update(transaction, "HEAD", &oid, - current_head - ? ¤t_head->object.oid : &null_oid, - 0, sb.buf, &err) || - ref_transaction_commit(transaction, &err)) { + if (update_head_with_reflog(current_head, &oid, reflog_msg, &sb, + &err)) { rollback_index_files(); die("%s", err.buf); } - ref_transaction_free(transaction); unlink(git_path_cherry_pick_head()); unlink(git_path_revert_head()); @@ -1835,17 +1608,17 @@ int cmd_commit(int argc, const char **argv, const char *prefix) rerere(0); run_commit_hook(use_editor, get_index_file(), "post-commit", NULL); if (amend && !no_post_rewrite) { - struct notes_rewrite_cfg *cfg; - cfg = init_copy_notes_for_rewrite("amend"); - if (cfg) { - /* we are amending, so current_head is not NULL */ - copy_note_for_rewrite(cfg, ¤t_head->object.oid, &oid); - finish_copy_notes_for_rewrite(cfg, "Notes added by 'git commit --amend'"); - } - run_rewrite_hook(¤t_head->object.oid, &oid); + commit_post_rewrite(current_head, &oid); + } + if (!quiet) { + unsigned int flags = 0; + + if (!current_head) + flags |= SUMMARY_INITIAL_COMMIT; + if (author_date_is_interesting()) + flags |= SUMMARY_SHOW_AUTHOR_DATE; + print_commit_summary(prefix, &oid, flags); } - if (!quiet) - print_summary(prefix, &oid, !current_head); UNLEAK(err); UNLEAK(sb); diff --git a/builtin/describe.c b/builtin/describe.c index 3b0b204b1e..c428984706 100644 --- a/builtin/describe.c +++ b/builtin/describe.c @@ -274,10 +274,13 @@ static void append_name(struct commit_name *n, struct strbuf *dst) n->name_checked = 1; } - if (n->tag) + if (n->tag) { + if (all) + strbuf_addstr(dst, "tags/"); strbuf_addstr(dst, n->tag->tag); - else + } else { strbuf_addstr(dst, n->path); + } } static void append_suffix(int depth, const struct object_id *oid, struct strbuf *dst) @@ -380,7 +383,7 @@ static void describe_commit(struct object_id *oid, struct strbuf *dst) if (!match_cnt) { struct object_id *cmit_oid = &cmit->object.oid; if (always) { - strbuf_addstr(dst, find_unique_abbrev(cmit_oid->hash, abbrev)); + strbuf_add_unique_abbrev(dst, cmit_oid->hash, abbrev); if (suffix) strbuf_addstr(dst, suffix); return; diff --git a/builtin/fetch-pack.c b/builtin/fetch-pack.c index 366b9d13f9..a7bc1366ab 100644 --- a/builtin/fetch-pack.c +++ b/builtin/fetch-pack.c @@ -53,6 +53,8 @@ int cmd_fetch_pack(int argc, const char **argv, const char *prefix) struct oid_array shallow = OID_ARRAY_INIT; struct string_list deepen_not = STRING_LIST_INIT_DUP; + fetch_if_missing = 0; + packet_trace_identity("fetch-pack"); memset(&args, 0, sizeof(args)); @@ -143,6 +145,22 @@ int cmd_fetch_pack(int argc, const char **argv, const char *prefix) args.update_shallow = 1; continue; } + if (!strcmp("--from-promisor", arg)) { + args.from_promisor = 1; + continue; + } + if (!strcmp("--no-dependents", arg)) { + args.no_dependents = 1; + continue; + } + if (skip_prefix(arg, ("--" CL_ARG__FILTER "="), &arg)) { + parse_list_objects_filter(&args.filter_options, arg); + continue; + } + if (!strcmp(arg, ("--no-" CL_ARG__FILTER))) { + list_objects_filter_set_no_filter(&args.filter_options); + continue; + } usage(fetch_pack_usage); } if (deepen_not.nr) diff --git a/builtin/fetch.c b/builtin/fetch.c index 7bbcd26faf..8ee998ea2e 100644 --- a/builtin/fetch.c +++ b/builtin/fetch.c @@ -19,6 +19,7 @@ #include "argv-array.h" #include "utf8.h" #include "packfile.h" +#include "list-objects-filter-options.h" static const char * const builtin_fetch_usage[] = { N_("git fetch [] [ [...]]"), @@ -56,6 +57,7 @@ static int recurse_submodules_default = RECURSE_SUBMODULES_ON_DEMAND; static int shown_url = 0; static int refmap_alloc, refmap_nr; static const char **refmap_array; +static struct list_objects_filter_options filter_options; static int git_fetch_config(const char *k, const char *v, void *cb) { @@ -161,6 +163,7 @@ static struct option builtin_fetch_options[] = { TRANSPORT_FAMILY_IPV4), OPT_SET_INT('6', "ipv6", &family, N_("use IPv6 addresses only"), TRANSPORT_FAMILY_IPV6), + OPT_PARSE_LIST_OBJECTS_FILTER(&filter_options), OPT_END() }; @@ -1045,6 +1048,11 @@ static struct transport *prepare_transport(struct remote *remote, int deepen) set_option(transport, TRANS_OPT_DEEPEN_RELATIVE, "yes"); if (update_shallow) set_option(transport, TRANS_OPT_UPDATE_SHALLOW, "yes"); + if (filter_options.choice) { + set_option(transport, TRANS_OPT_LIST_OBJECTS_FILTER, + filter_options.filter_spec); + set_option(transport, TRANS_OPT_FROM_PROMISOR, "1"); + } return transport; } @@ -1265,6 +1273,56 @@ static int fetch_multiple(struct string_list *list) return result; } +/* + * Fetching from the promisor remote should use the given filter-spec + * or inherit the default filter-spec from the config. + */ +static inline void fetch_one_setup_partial(struct remote *remote) +{ + /* + * Explicit --no-filter argument overrides everything, regardless + * of any prior partial clones and fetches. + */ + if (filter_options.no_filter) + return; + + /* + * If no prior partial clone/fetch and the current fetch DID NOT + * request a partial-fetch, do a normal fetch. + */ + if (!repository_format_partial_clone && !filter_options.choice) + return; + + /* + * If this is the FIRST partial-fetch request, we enable partial + * on this repo and remember the given filter-spec as the default + * for subsequent fetches to this remote. + */ + if (!repository_format_partial_clone && filter_options.choice) { + partial_clone_register(remote->name, &filter_options); + return; + } + + /* + * We are currently limited to only ONE promisor remote and only + * allow partial-fetches from the promisor remote. + */ + if (strcmp(remote->name, repository_format_partial_clone)) { + if (filter_options.choice) + die(_("--filter can only be used with the remote configured in core.partialClone")); + return; + } + + /* + * Do a partial-fetch from the promisor remote using either the + * explicitly given filter-spec or inherit the filter-spec from + * the config. + */ + if (!filter_options.choice) + partial_clone_get_default_filter_spec(&filter_options); + return; +} + static int fetch_one(struct remote *remote, int argc, const char **argv) { static const char **refs = NULL; @@ -1320,12 +1378,14 @@ int cmd_fetch(int argc, const char **argv, const char *prefix) { int i; struct string_list list = STRING_LIST_INIT_DUP; - struct remote *remote; + struct remote *remote = NULL; int result = 0; struct argv_array argv_gc_auto = ARGV_ARRAY_INIT; packet_trace_identity("fetch"); + fetch_if_missing = 0; + /* Record the command line for the reflog */ strbuf_addstr(&default_rla, "fetch"); for (i = 1; i < argc; i++) @@ -1359,23 +1419,23 @@ int cmd_fetch(int argc, const char **argv, const char *prefix) if (depth || deepen_since || deepen_not.nr) deepen = 1; + if (filter_options.choice && !repository_format_partial_clone) + die("--filter can only be used when extensions.partialClone is set"); + if (all) { if (argc == 1) die(_("fetch --all does not take a repository argument")); else if (argc > 1) die(_("fetch --all does not make sense with refspecs")); (void) for_each_remote(get_one_remote_for_fetch, &list); - result = fetch_multiple(&list); } else if (argc == 0) { /* No arguments -- use default remote */ remote = remote_get(NULL); - result = fetch_one(remote, argc, argv); } else if (multiple) { /* All arguments are assumed to be remotes or groups */ for (i = 0; i < argc; i++) if (!add_remote_or_group(argv[i], &list)) die(_("No such remote or remote group: %s"), argv[i]); - result = fetch_multiple(&list); } else { /* Single remote or group */ (void) add_remote_or_group(argv[0], &list); @@ -1383,14 +1443,25 @@ int cmd_fetch(int argc, const char **argv, const char *prefix) /* More than one remote */ if (argc > 1) die(_("Fetching a group and specifying refspecs does not make sense")); - result = fetch_multiple(&list); } else { /* Zero or one remotes */ remote = remote_get(argv[0]); - result = fetch_one(remote, argc-1, argv+1); + argc--; + argv++; } } + if (remote) { + if (filter_options.choice || repository_format_partial_clone) + fetch_one_setup_partial(remote); + result = fetch_one(remote, argc, argv); + } else { + if (filter_options.choice) + die(_("--filter can only be used with the remote configured in core.partialClone")); + /* TODO should this also die if we have a previous partial-clone? */ + result = fetch_multiple(&list); + } + if (!result && (recurse_submodules != RECURSE_SUBMODULES_OFF)) { struct argv_array options = ARGV_ARRAY_INIT; diff --git a/builtin/fsck.c b/builtin/fsck.c index 04846d46f9..7a8a679d4f 100644 --- a/builtin/fsck.c +++ b/builtin/fsck.c @@ -149,6 +149,15 @@ static int mark_object(struct object *obj, int type, void *data, struct fsck_opt if (obj->flags & REACHABLE) return 0; obj->flags |= REACHABLE; + + if (is_promisor_object(&obj->oid)) + /* + * Further recursion does not need to be performed on this + * object since it is a promisor object (so it does not need to + * be added to "pending"). + */ + return 0; + if (!(obj->flags & HAS_OBJ)) { if (parent && !has_object_file(&obj->oid)) { printf("broken link from %7s %s\n", @@ -208,6 +217,8 @@ static void check_reachable_object(struct object *obj) * do a full fsck */ if (!(obj->flags & HAS_OBJ)) { + if (is_promisor_object(&obj->oid)) + return; if (has_sha1_pack(obj->oid.hash)) return; /* it is in pack - forget about it */ printf("missing %s %s\n", printable_type(obj), @@ -398,7 +409,7 @@ static void fsck_handle_reflog_oid(const char *refname, struct object_id *oid, xstrfmt("%s@{%"PRItime"}", refname, timestamp)); obj->flags |= USED; mark_object_reachable(obj); - } else { + } else if (!is_promisor_object(oid)) { error("%s: invalid reflog entry %s", refname, oid_to_hex(oid)); errors_found |= ERROR_REACHABLE; } @@ -434,6 +445,14 @@ static int fsck_handle_ref(const char *refname, const struct object_id *oid, obj = parse_object(oid); if (!obj) { + if (is_promisor_object(oid)) { + /* + * Increment default_refs anyway, because this is a + * valid ref. + */ + default_refs++; + return 0; + } error("%s: invalid sha1 pointer %s", refname, oid_to_hex(oid)); errors_found |= ERROR_REACHABLE; /* We'll continue with the rest despite the error.. */ @@ -659,6 +678,9 @@ int cmd_fsck(int argc, const char **argv, const char *prefix) int i; struct alternate_object_database *alt; + /* fsck knows how to handle missing promisor objects */ + fetch_if_missing = 0; + errors_found = 0; check_replace_refs = 0; @@ -731,6 +753,8 @@ int cmd_fsck(int argc, const char **argv, const char *prefix) struct object *obj = lookup_object(oid.hash); if (!obj || !(obj->flags & HAS_OBJ)) { + if (is_promisor_object(&oid)) + continue; error("%s: object missing", oid_to_hex(&oid)); errors_found |= ERROR_OBJECT; continue; diff --git a/builtin/gc.c b/builtin/gc.c index 3c5eae0edf..77fa720bd0 100644 --- a/builtin/gc.c +++ b/builtin/gc.c @@ -458,6 +458,9 @@ int cmd_gc(int argc, const char **argv, const char *prefix) argv_array_push(&prune, prune_expire); if (quiet) argv_array_push(&prune, "--no-progress"); + if (repository_format_partial_clone) + argv_array_push(&prune, + "--exclude-promisor-objects"); if (run_command_v_opt(prune.argv, RUN_GIT_CMD)) return error(FAILED_RUN, prune.argv[0]); } diff --git a/builtin/index-pack.c b/builtin/index-pack.c index 4c51aec81f..5ebd370c56 100644 --- a/builtin/index-pack.c +++ b/builtin/index-pack.c @@ -1389,15 +1389,60 @@ static void fix_unresolved_deltas(struct sha1file *f) free(sorted_by_pos); } +static const char *derive_filename(const char *pack_name, const char *suffix, + struct strbuf *buf) +{ + size_t len; + if (!strip_suffix(pack_name, ".pack", &len)) + die(_("packfile name '%s' does not end with '.pack'"), + pack_name); + strbuf_add(buf, pack_name, len); + strbuf_addch(buf, '.'); + strbuf_addstr(buf, suffix); + return buf->buf; +} + +static void write_special_file(const char *suffix, const char *msg, + const char *pack_name, const unsigned char *sha1, + const char **report) +{ + struct strbuf name_buf = STRBUF_INIT; + const char *filename; + int fd; + int msg_len = strlen(msg); + + if (pack_name) + filename = derive_filename(pack_name, suffix, &name_buf); + else + filename = odb_pack_name(&name_buf, sha1, suffix); + + fd = odb_pack_keep(filename); + if (fd < 0) { + if (errno != EEXIST) + die_errno(_("cannot write %s file '%s'"), + suffix, filename); + } else { + if (msg_len > 0) { + write_or_die(fd, msg, msg_len); + write_or_die(fd, "\n", 1); + } + if (close(fd) != 0) + die_errno(_("cannot close written %s file '%s'"), + suffix, filename); + if (report) + *report = suffix; + } + strbuf_release(&name_buf); +} + static void final(const char *final_pack_name, const char *curr_pack_name, const char *final_index_name, const char *curr_index_name, - const char *keep_name, const char *keep_msg, + const char *keep_msg, const char *promisor_msg, unsigned char *sha1) { const char *report = "pack"; struct strbuf pack_name = STRBUF_INIT; struct strbuf index_name = STRBUF_INIT; - struct strbuf keep_name_buf = STRBUF_INIT; int err; if (!from_stdin) { @@ -1409,28 +1454,12 @@ static void final(const char *final_pack_name, const char *curr_pack_name, die_errno(_("error while closing pack file")); } - if (keep_msg) { - int keep_fd, keep_msg_len = strlen(keep_msg); - - if (!keep_name) - keep_name = odb_pack_name(&keep_name_buf, sha1, "keep"); - - keep_fd = odb_pack_keep(keep_name); - if (keep_fd < 0) { - if (errno != EEXIST) - die_errno(_("cannot write keep file '%s'"), - keep_name); - } else { - if (keep_msg_len > 0) { - write_or_die(keep_fd, keep_msg, keep_msg_len); - write_or_die(keep_fd, "\n", 1); - } - if (close(keep_fd) != 0) - die_errno(_("cannot close written keep file '%s'"), - keep_name); - report = "keep"; - } - } + if (keep_msg) + write_special_file("keep", keep_msg, final_pack_name, sha1, + &report); + if (promisor_msg) + write_special_file("promisor", promisor_msg, final_pack_name, + sha1, NULL); if (final_pack_name != curr_pack_name) { if (!final_pack_name) @@ -1472,7 +1501,6 @@ static void final(const char *final_pack_name, const char *curr_pack_name, strbuf_release(&index_name); strbuf_release(&pack_name); - strbuf_release(&keep_name_buf); } static int git_index_pack_config(const char *k, const char *v, void *cb) @@ -1615,32 +1643,26 @@ static void show_pack_info(int stat_only) } } -static const char *derive_filename(const char *pack_name, const char *suffix, - struct strbuf *buf) -{ - size_t len; - if (!strip_suffix(pack_name, ".pack", &len)) - die(_("packfile name '%s' does not end with '.pack'"), - pack_name); - strbuf_add(buf, pack_name, len); - strbuf_addstr(buf, suffix); - return buf->buf; -} - int cmd_index_pack(int argc, const char **argv, const char *prefix) { int i, fix_thin_pack = 0, verify = 0, stat_only = 0; const char *curr_index; const char *index_name = NULL, *pack_name = NULL; - const char *keep_name = NULL, *keep_msg = NULL; - struct strbuf index_name_buf = STRBUF_INIT, - keep_name_buf = STRBUF_INIT; + const char *keep_msg = NULL; + const char *promisor_msg = NULL; + struct strbuf index_name_buf = STRBUF_INIT; struct pack_idx_entry **idx_objects; struct pack_idx_option opts; unsigned char pack_sha1[20]; unsigned foreign_nr = 1; /* zero is a "good" value, assume bad */ int report_end_of_input = 0; + /* + * index-pack never needs to fetch missing objects, since it only + * accesses the repo to do hash collision checks + */ + fetch_if_missing = 0; + if (argc == 2 && !strcmp(argv[1], "-h")) usage(index_pack_usage); @@ -1678,6 +1700,8 @@ int cmd_index_pack(int argc, const char **argv, const char *prefix) stat_only = 1; } else if (skip_to_optional_arg(arg, "--keep", &keep_msg)) { ; /* nothing to do */ + } else if (skip_to_optional_arg(arg, "--promisor", &promisor_msg)) { + ; /* already parsed */ } else if (starts_with(arg, "--threads=")) { char *end; nr_threads = strtoul(arg+10, &end, 0); @@ -1740,9 +1764,7 @@ int cmd_index_pack(int argc, const char **argv, const char *prefix) if (from_stdin && !startup_info->have_repository) die(_("--stdin requires a git repository")); if (!index_name && pack_name) - index_name = derive_filename(pack_name, ".idx", &index_name_buf); - if (keep_msg && !keep_name && pack_name) - keep_name = derive_filename(pack_name, ".keep", &keep_name_buf); + index_name = derive_filename(pack_name, "idx", &index_name_buf); if (verify) { if (!index_name) @@ -1790,13 +1812,12 @@ int cmd_index_pack(int argc, const char **argv, const char *prefix) if (!verify) final(pack_name, curr_pack, index_name, curr_index, - keep_name, keep_msg, + keep_msg, promisor_msg, pack_sha1); else close(input_fd); free(objects); strbuf_release(&index_name_buf); - strbuf_release(&keep_name_buf); if (pack_name == NULL) free((void *) curr_pack); if (index_name == NULL) diff --git a/builtin/log.c b/builtin/log.c index 14fdf39165..46b4ca13e5 100644 --- a/builtin/log.c +++ b/builtin/log.c @@ -188,8 +188,8 @@ static void cmd_log_init_finish(int argc, const char **argv, const char *prefix, if (rev->show_notes) init_display_notes(&rev->notes_opt); - if (rev->diffopt.pickaxe || rev->diffopt.filter || - rev->diffopt.flags.follow_renames) + if ((rev->diffopt.pickaxe_opts & DIFF_PICKAXE_KINDS_MASK) || + rev->diffopt.filter || rev->diffopt.flags.follow_renames) rev->always_show_header = 0; if (source) diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c index 6b9cfc289d..83dcbc9773 100644 --- a/builtin/pack-objects.c +++ b/builtin/pack-objects.c @@ -26,7 +26,7 @@ #include "reachable.h" #include "sha1-array.h" #include "argv-array.h" -#include "mru.h" +#include "list.h" #include "packfile.h" static const char *pack_usage[] = { @@ -75,6 +75,8 @@ static int use_bitmap_index = -1; static int write_bitmap_index; static uint16_t write_bitmap_options; +static int exclude_promisor_objects; + static unsigned long delta_cache_size = 0; static unsigned long max_delta_cache_size = 256 * 1024 * 1024; static unsigned long cache_max_small_delta_size = 1000; @@ -84,8 +86,9 @@ static unsigned long window_memory_limit = 0; static struct list_objects_filter_options filter_options; enum missing_action { - MA_ERROR = 0, /* fail if any missing objects are encountered */ - MA_ALLOW_ANY, /* silently allow ALL missing objects */ + MA_ERROR = 0, /* fail if any missing objects are encountered */ + MA_ALLOW_ANY, /* silently allow ALL missing objects */ + MA_ALLOW_PROMISOR, /* silently allow all missing PROMISOR objects */ }; static enum missing_action arg_missing_action; static show_object_fn fn_show_object; @@ -1006,8 +1009,8 @@ static int want_object_in_pack(const struct object_id *oid, struct packed_git **found_pack, off_t *found_offset) { - struct mru_entry *entry; int want; + struct list_head *pos; if (!exclude && local && has_loose_object_nonlocal(oid->hash)) return 0; @@ -1023,8 +1026,8 @@ static int want_object_in_pack(const struct object_id *oid, return want; } - for (entry = packed_git_mru.head; entry; entry = entry->next) { - struct packed_git *p = entry->item; + list_for_each(pos, &packed_git_mru) { + struct packed_git *p = list_entry(pos, struct packed_git, mru); off_t offset; if (p == *found_pack) @@ -1041,7 +1044,7 @@ static int want_object_in_pack(const struct object_id *oid, } want = want_found_object(exclude, p); if (!exclude && want > 0) - mru_mark(&packed_git_mru, entry); + list_move(&p->mru, &packed_git_mru); if (want != -1) return want; } @@ -2578,6 +2581,20 @@ static void show_object__ma_allow_any(struct object *obj, const char *name, void show_object(obj, name, data); } +static void show_object__ma_allow_promisor(struct object *obj, const char *name, void *data) +{ + assert(arg_missing_action == MA_ALLOW_PROMISOR); + + /* + * Quietly ignore EXPECTED missing objects. This avoids problems with + * staging them now and getting an odd error later. + */ + if (!has_object_file(&obj->oid) && is_promisor_object(&obj->oid)) + return; + + show_object(obj, name, data); +} + static int option_parse_missing_action(const struct option *opt, const char *arg, int unset) { @@ -2592,10 +2609,18 @@ static int option_parse_missing_action(const struct option *opt, if (!strcmp(arg, "allow-any")) { arg_missing_action = MA_ALLOW_ANY; + fetch_if_missing = 0; fn_show_object = show_object__ma_allow_any; return 0; } + if (!strcmp(arg, "allow-promisor")) { + arg_missing_action = MA_ALLOW_PROMISOR; + fetch_if_missing = 0; + fn_show_object = show_object__ma_allow_promisor; + return 0; + } + die(_("invalid value for --missing")); return 0; } @@ -3009,6 +3034,8 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix) { OPTION_CALLBACK, 0, "missing", NULL, N_("action"), N_("handling for missing objects"), PARSE_OPT_NONEG, option_parse_missing_action }, + OPT_BOOL(0, "exclude-promisor-objects", &exclude_promisor_objects, + N_("do not pack objects in promisor packfiles")), OPT_END(), }; @@ -3054,6 +3081,12 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix) argv_array_push(&rp, "--unpacked"); } + if (exclude_promisor_objects) { + use_internal_rev_list = 1; + fetch_if_missing = 0; + argv_array_push(&rp, "--exclude-promisor-objects"); + } + if (!reuse_object) reuse_delta = 0; if (pack_compression_level == -1) diff --git a/builtin/prune.c b/builtin/prune.c index d2fdae680a..4cfec82f40 100644 --- a/builtin/prune.c +++ b/builtin/prune.c @@ -101,12 +101,15 @@ int cmd_prune(int argc, const char **argv, const char *prefix) { struct rev_info revs; struct progress *progress = NULL; + int exclude_promisor_objects = 0; const struct option options[] = { OPT__DRY_RUN(&show_only, N_("do not remove, show only")), OPT__VERBOSE(&verbose, N_("report pruned objects")), OPT_BOOL(0, "progress", &show_progress, N_("show progress")), OPT_EXPIRY_DATE(0, "expire", &expire, N_("expire objects older than