Ralf Thielow <ralf.thielow@gmail.com> <ralf.thielow@googlemail.com>
Ramsay Jones <ramsay@ramsayjones.plus.com> <ramsay@ramsay1.demon.co.uk>
René Scharfe <l.s.r@web.de> <rene.scharfe@lsrfire.ath.cx>
+René Scharfe <l.s.r@web.de> Rene Scharfe
Richard Hansen <rhansen@rhansen.org> <hansenr@google.com>
Richard Hansen <rhansen@rhansen.org> <rhansen@bbn.com>
Robert Fitzsimons <robfitz@273k.net>
API_DOCS = $(patsubst %.txt,%,$(filter-out technical/api-index-skel.txt technical/api-index.txt, $(wildcard technical/api-*.txt)))
SP_ARTICLES += $(API_DOCS)
+TECH_DOCS += technical/hash-function-transition
TECH_DOCS += technical/http-protocol
TECH_DOCS += technical/index-format
TECH_DOCS += technical/pack-format
more explicit '.' for that instead. The hope is that existing
users will not mind this change, and eventually the warning can be
turned into a hard error, upgrading the deprecation into removal of
- this (mis)feature. That is now scheduled to happen in the upcoming
- release.
+ this (mis)feature. That is now scheduled to happen in Git v2.16,
+ the next major release after this one.
* Git now avoids blindly falling back to ".git" when the setup
sequence said we are _not_ in Git repository. A corner case that
- happens to work right now may be broken by a call to die("BUG").
+ happens to work right now may be broken by a call to BUG().
We've tried hard to locate such cases and fixed them, but there
might still be cases that need to be addressed--bug reports are
greatly appreciated.
other options to make it easier for scripts to grab existing
trailer lines from a commit log message.
+ * The "--format=%(trailers)" option "git log" and its friends take
+ learned to take the 'unfold' and 'only' modifiers to normalize its
+ output, e.g. "git log --format=%(trailers:only,unfold)".
+
* "gitweb" shows a link to visit the 'raw' contents of blbos in the
history overview page.
* Add comment to clarify that the style file is meant to be used with
clang-5 and the rules are still work in progress.
+ * Many variables that points at a region of memory that will live
+ throughout the life of the program have been marked with UNLEAK
+ marker to help the leak checkers concentrate on real leaks..
+
+
Also contains various documentation updates and code clean-ups.
request-pull script.
(merge e66d7c37a5 ar/request-pull-phrasofix later to maint).
+ * Fixes for a handful memory access issues identified by valgrind.
+ (merge 2944a94c6b tg/memfixes later to maint).
+
+ * Backports a moral equivalent of 2015 fix to the poll() emulation
+ from the upstream gnulib to fix occasional breakages on HPE NonStop.
+ (merge 61b2a1acaa rb/compat-poll-fix later to maint).
+
+ * Users with "color.ui = always" in their configuration were broken
+ by a recent change that made plumbing commands to pay attention to
+ them as the patch created internally by "git add -p" were colored
+ (heh) and made unusable. Fix this regression by redefining
+ 'always' to mean the same thing as 'auto'.
+ (merge 6be4595edb jk/ui-color-always-to-auto-maint later to maint).
+
+ * In the "--format=..." option of the "git for-each-ref" command (and
+ its friends, i.e. the listing mode of "git branch/tag"), "%(atom:)"
+ (e.g. "%(refname:)", "%(body:)" used to error out. Instead, treat
+ them as if the colon and an empty string that follows it were not
+ there.
+ (merge bea4dbeafd tb/ref-filter-empty-modifier later to maint).
+
* Other minor doc, test and build updates and code cleanups.
(merge f094b89a4d ma/parse-maybe-bool later to maint).
(merge 39b00fa4d4 jk/drop-sha1-entry-pos later to maint).
(merge 9ca356fa8b rs/cocci-de-paren-call-params later to maint).
(merge 7099153e8d rs/tag-null-pointer-arith-fix later to maint).
(merge 0e187d758c rs/run-command-use-alloc-array later to maint).
+ (merge e0222159fa jn/strbuf-doc-re-reuse later to maint).
+ (merge 97487ea11a rs/qsort-s later to maint).
+ (merge a9155c50bd sb/branch-avoid-repeated-strbuf-release later to maint).
+ (merge f777623514 ks/branch-tweak-error-message-for-extra-args later to maint).
+ (merge 33f3c683ec ks/verify-filename-non-option-error-message-tweak later to maint).
color.branch::
A boolean to enable/disable color in the output of
- linkgit:git-branch[1]. May be set to `always`,
- `false` (or `never`) or `auto` (or `true`), in which case colors are used
- only when the output is to a terminal. If unset, then the
- value of `color.ui` is used (`auto` by default).
+ linkgit:git-branch[1]. May be set to `false` (or `never`) to
+ disable color entirely, `auto` (or `true` or `always`) in which
+ case colors are used only when the output is to a terminal. If
+ unset, then the value of `color.ui` is used (`auto` by default).
color.branch.<slot>::
Use customized color for branch coloration. `<slot>` is one of
color.diff::
Whether to use ANSI escape sequences to add color to patches.
- If this is set to `always`, linkgit:git-diff[1],
+ If this is set to `true` or `auto`, linkgit:git-diff[1],
linkgit:git-log[1], and linkgit:git-show[1] will use color
- for all patches. If it is set to `true` or `auto`, those
- commands will only use color when output is to the terminal.
- If unset, then the value of `color.ui` is used (`auto` by
- default).
+ when output is to the terminal. The value `always` is a
+ historical synonym for `auto`. If unset, then the value of
+ `color.ui` is used (`auto` by default).
+
This does not affect linkgit:git-format-patch[1] or the
'git-diff-{asterisk}' plumbing commands. Can be overridden on the
--
color.interactive::
- When set to `always`, always use colors for interactive prompts
+ When set to `true` or `auto`, use colors for interactive prompts
and displays (such as those used by "git-add --interactive" and
- "git-clean --interactive"). When false (or `never`), never.
- When set to `true` or `auto`, use colors only when the output is
- to the terminal. If unset, then the value of `color.ui` is
- used (`auto` by default).
+ "git-clean --interactive") when the output is to the terminal.
+ When false (or `never`), never show colors. The value `always`
+ is a historical synonym for `auto`. If unset, then the value of
+ `color.ui` is used (`auto` by default).
color.interactive.<slot>::
Use customized color for 'git add --interactive' and 'git clean
configuration to set a default for the `--color` option. Set it
to `false` or `never` if you prefer Git commands not to use
color unless enabled explicitly with some other configuration
- or the `--color` option. Set it to `always` if you want all
- output not intended for machine consumption to use color, to
- `true` or `auto` (this is the default since Git 1.8.4) if you
- want such output to use color when written to the terminal.
+ or the `--color` option. Set it to `true` or `auto` to enable
+ color when output is written to the terminal (this is also the
+ default since Git 1.8.4). The value `always` is a historical
+ synonym for `auto`.
column.ui::
Specify whether supported commands should output in columns.
`xx`; for example `%00` interpolates to `\0` (NUL),
`%09` to `\t` (TAB) and `%0a` to `\n` (LF).
+--color[=<when>]:
+ Respect any colors specified in the `--format` option. The
+ `<when>` field must be one of `always`, `never`, or `auto` (if
+ `<when>` is absent, behave as if `always` was given).
+
--shell::
--perl::
--python::
The complete message in a commit and tag object is `contents`.
Its first line is `contents:subject`, where subject is the concatenation
of all lines of the commit message up to the first blank line. The next
-line is 'contents:body', where body is all of the lines after the first
+line is `contents:body`, where body is all of the lines after the first
blank line. The optional GPG signature is `contents:signature`. The
first `N` lines of the message is obtained using `contents:lines=N`.
Additionally, the trailers as interpreted by linkgit:git-interpret-trailers[1]
-are obtained as 'contents:trailers'.
+are obtained as `trailers` (or by using the historical alias
+`contents:trailers`). Non-trailer lines from the trailer block can be omitted
+with `trailers:only`. Whitespace-continuations can be removed from trailers so
+that each trailer appears on a line by itself with its full content with
+`trailers:unfold`. Both can be used together as `trailers:unfold,only`.
For sorting purposes, fields with numeric values sort in numeric order
(`objectsize`, `authordate`, `committerdate`, `creatordate`, `taggerdate`).
variable if it exists, or lexicographic order otherwise. See
linkgit:git-config[1].
+--color[=<when>]:
+ Respect any colors specified in the `--format` option. The
+ `<when>` field must be one of `always`, `never`, or `auto` (if
+ `<when>` is absent, behave as if `always` was given).
+
-i::
--ignore-case::
Sorting and filtering tags are case insensitive.
makes the result suitable for passing to functions expecting to receive
argv from main(), or the link:api-run-command.html[run-command API].
-The link:api-string-list.html[string-list API] is similar, but cannot be
+The string-list API (documented in string-list.h) is similar, but cannot be
used for these purposes; instead of storing a straight string pointer,
it contains an item structure with a `util` field that is not compatible
with the traditional argv interface.
--- /dev/null
+Git hash function transition
+============================
+
+Objective
+---------
+Migrate Git from SHA-1 to a stronger hash function.
+
+Background
+----------
+At its core, the Git version control system is a content addressable
+filesystem. It uses the SHA-1 hash function to name content. For
+example, files, directories, and revisions are referred to by hash
+values unlike in other traditional version control systems where files
+or versions are referred to via sequential numbers. The use of a hash
+function to address its content delivers a few advantages:
+
+* Integrity checking is easy. Bit flips, for example, are easily
+ detected, as the hash of corrupted content does not match its name.
+* Lookup of objects is fast.
+
+Using a cryptographically secure hash function brings additional
+advantages:
+
+* Object names can be signed and third parties can trust the hash to
+ address the signed object and all objects it references.
+* Communication using Git protocol and out of band communication
+ methods have a short reliable string that can be used to reliably
+ address stored content.
+
+Over time some flaws in SHA-1 have been discovered by security
+researchers. https://shattered.io demonstrated a practical SHA-1 hash
+collision. As a result, SHA-1 cannot be considered cryptographically
+secure any more. This impacts the communication of hash values because
+we cannot trust that a given hash value represents the known good
+version of content that the speaker intended.
+
+SHA-1 still possesses the other properties such as fast object lookup
+and safe error checking, but other hash functions are equally suitable
+that are believed to be cryptographically secure.
+
+Goals
+-----
+Where NewHash is a strong 256-bit hash function to replace SHA-1 (see
+"Selection of a New Hash", below):
+
+1. The transition to NewHash can be done one local repository at a time.
+ a. Requiring no action by any other party.
+ b. A NewHash repository can communicate with SHA-1 Git servers
+ (push/fetch).
+ c. Users can use SHA-1 and NewHash identifiers for objects
+ interchangeably (see "Object names on the command line", below).
+ d. New signed objects make use of a stronger hash function than
+ SHA-1 for their security guarantees.
+2. Allow a complete transition away from SHA-1.
+ a. Local metadata for SHA-1 compatibility can be removed from a
+ repository if compatibility with SHA-1 is no longer needed.
+3. Maintainability throughout the process.
+ a. The object format is kept simple and consistent.
+ b. Creation of a generalized repository conversion tool.
+
+Non-Goals
+---------
+1. Add NewHash support to Git protocol. This is valuable and the
+ logical next step but it is out of scope for this initial design.
+2. Transparently improving the security of existing SHA-1 signed
+ objects.
+3. Intermixing objects using multiple hash functions in a single
+ repository.
+4. Taking the opportunity to fix other bugs in Git's formats and
+ protocols.
+5. Shallow clones and fetches into a NewHash repository. (This will
+ change when we add NewHash support to Git protocol.)
+6. Skip fetching some submodules of a project into a NewHash
+ repository. (This also depends on NewHash support in Git
+ protocol.)
+
+Overview
+--------
+We introduce a new repository format extension. Repositories with this
+extension enabled use NewHash instead of SHA-1 to name their objects.
+This affects both object names and object content --- both the names
+of objects and all references to other objects within an object are
+switched to the new hash function.
+
+NewHash repositories cannot be read by older versions of Git.
+
+Alongside the packfile, a NewHash repository stores a bidirectional
+mapping between NewHash and SHA-1 object names. The mapping is generated
+locally and can be verified using "git fsck". Object lookups use this
+mapping to allow naming objects using either their SHA-1 and NewHash names
+interchangeably.
+
+"git cat-file" and "git hash-object" gain options to display an object
+in its sha1 form and write an object given its sha1 form. This
+requires all objects referenced by that object to be present in the
+object database so that they can be named using the appropriate name
+(using the bidirectional hash mapping).
+
+Fetches from a SHA-1 based server convert the fetched objects into
+NewHash form and record the mapping in the bidirectional mapping table
+(see below for details). Pushes to a SHA-1 based server convert the
+objects being pushed into sha1 form so the server does not have to be
+aware of the hash function the client is using.
+
+Detailed Design
+---------------
+Repository format extension
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+A NewHash repository uses repository format version `1` (see
+Documentation/technical/repository-version.txt) with extensions
+`objectFormat` and `compatObjectFormat`:
+
+ [core]
+ repositoryFormatVersion = 1
+ [extensions]
+ objectFormat = newhash
+ compatObjectFormat = sha1
+
+Specifying a repository format extension ensures that versions of Git
+not aware of NewHash do not try to operate on these repositories,
+instead producing an error message:
+
+ $ git status
+ fatal: unknown repository extensions found:
+ objectformat
+ compatobjectformat
+
+See the "Transition plan" section below for more details on these
+repository extensions.
+
+Object names
+~~~~~~~~~~~~
+Objects can be named by their 40 hexadecimal digit sha1-name or 64
+hexadecimal digit newhash-name, plus names derived from those (see
+gitrevisions(7)).
+
+The sha1-name of an object is the SHA-1 of the concatenation of its
+type, length, a nul byte, and the object's sha1-content. This is the
+traditional <sha1> used in Git to name objects.
+
+The newhash-name of an object is the NewHash of the concatenation of its
+type, length, a nul byte, and the object's newhash-content.
+
+Object format
+~~~~~~~~~~~~~
+The content as a byte sequence of a tag, commit, or tree object named
+by sha1 and newhash differ because an object named by newhash-name refers to
+other objects by their newhash-names and an object named by sha1-name
+refers to other objects by their sha1-names.
+
+The newhash-content of an object is the same as its sha1-content, except
+that objects referenced by the object are named using their newhash-names
+instead of sha1-names. Because a blob object does not refer to any
+other object, its sha1-content and newhash-content are the same.
+
+The format allows round-trip conversion between newhash-content and
+sha1-content.
+
+Object storage
+~~~~~~~~~~~~~~
+Loose objects use zlib compression and packed objects use the packed
+format described in Documentation/technical/pack-format.txt, just like
+today. The content that is compressed and stored uses newhash-content
+instead of sha1-content.
+
+Pack index
+~~~~~~~~~~
+Pack index (.idx) files use a new v3 format that supports multiple
+hash functions. They have the following format (all integers are in
+network byte order):
+
+- A header appears at the beginning and consists of the following:
+ - The 4-byte pack index signature: '\377t0c'
+ - 4-byte version number: 3
+ - 4-byte length of the header section, including the signature and
+ version number
+ - 4-byte number of objects contained in the pack
+ - 4-byte number of object formats in this pack index: 2
+ - For each object format:
+ - 4-byte format identifier (e.g., 'sha1' for SHA-1)
+ - 4-byte length in bytes of shortened object names. This is the
+ shortest possible length needed to make names in the shortened
+ object name table unambiguous.
+ - 4-byte integer, recording where tables relating to this format
+ are stored in this index file, as an offset from the beginning.
+ - 4-byte offset to the trailer from the beginning of this file.
+ - Zero or more additional key/value pairs (4-byte key, 4-byte
+ value). Only one key is supported: 'PSRC'. See the "Loose objects
+ and unreachable objects" section for supported values and how this
+ is used. All other keys are reserved. Readers must ignore
+ unrecognized keys.
+- Zero or more NUL bytes. This can optionally be used to improve the
+ alignment of the full object name table below.
+- Tables for the first object format:
+ - A sorted table of shortened object names. These are prefixes of
+ the names of all objects in this pack file, packed together
+ without offset values to reduce the cache footprint of the binary
+ search for a specific object name.
+
+ - A table of full object names in pack order. This allows resolving
+ a reference to "the nth object in the pack file" (from a
+ reachability bitmap or from the next table of another object
+ format) to its object name.
+
+ - A table of 4-byte values mapping object name order to pack order.
+ For an object in the table of sorted shortened object names, the
+ value at the corresponding index in this table is the index in the
+ previous table for that same object.
+
+ This can be used to look up the object in reachability bitmaps or
+ to look up its name in another object format.
+
+ - A table of 4-byte CRC32 values of the packed object data, in the
+ order that the objects appear in the pack file. This is to allow
+ compressed data to be copied directly from pack to pack during
+ repacking without undetected data corruption.
+
+ - A table of 4-byte offset values. For an object in the table of
+ sorted shortened object names, the value at the corresponding
+ index in this table indicates where that object can be found in
+ the pack file. These are usually 31-bit pack file offsets, but
+ large offsets are encoded as an index into the next table with the
+ most significant bit set.
+
+ - A table of 8-byte offset entries (empty for pack files less than
+ 2 GiB). Pack files are organized with heavily used objects toward
+ the front, so most object references should not need to refer to
+ this table.
+- Zero or more NUL bytes.
+- Tables for the second object format, with the same layout as above,
+ up to and not including the table of CRC32 values.
+- Zero or more NUL bytes.
+- The trailer consists of the following:
+ - A copy of the 20-byte NewHash checksum at the end of the
+ corresponding packfile.
+
+ - 20-byte NewHash checksum of all of the above.
+
+Loose object index
+~~~~~~~~~~~~~~~~~~
+A new file $GIT_OBJECT_DIR/loose-object-idx contains information about
+all loose objects. Its format is
+
+ # loose-object-idx
+ (newhash-name SP sha1-name LF)*
+
+where the object names are in hexadecimal format. The file is not
+sorted.
+
+The loose object index is protected against concurrent writes by a
+lock file $GIT_OBJECT_DIR/loose-object-idx.lock. To add a new loose
+object:
+
+1. Write the loose object to a temporary file, like today.
+2. Open loose-object-idx.lock with O_CREAT | O_EXCL to acquire the lock.
+3. Rename the loose object into place.
+4. Open loose-object-idx with O_APPEND and write the new object
+5. Unlink loose-object-idx.lock to release the lock.
+
+To remove entries (e.g. in "git pack-refs" or "git-prune"):
+
+1. Open loose-object-idx.lock with O_CREAT | O_EXCL to acquire the
+ lock.
+2. Write the new content to loose-object-idx.lock.
+3. Unlink any loose objects being removed.
+4. Rename to replace loose-object-idx, releasing the lock.
+
+Translation table
+~~~~~~~~~~~~~~~~~
+The index files support a bidirectional mapping between sha1-names
+and newhash-names. The lookup proceeds similarly to ordinary object
+lookups. For example, to convert a sha1-name to a newhash-name:
+
+ 1. Look for the object in idx files. If a match is present in the
+ idx's sorted list of truncated sha1-names, then:
+ a. Read the corresponding entry in the sha1-name order to pack
+ name order mapping.
+ b. Read the corresponding entry in the full sha1-name table to
+ verify we found the right object. If it is, then
+ c. Read the corresponding entry in the full newhash-name table.
+ That is the object's newhash-name.
+ 2. Check for a loose object. Read lines from loose-object-idx until
+ we find a match.
+
+Step (1) takes the same amount of time as an ordinary object lookup:
+O(number of packs * log(objects per pack)). Step (2) takes O(number of
+loose objects) time. To maintain good performance it will be necessary
+to keep the number of loose objects low. See the "Loose objects and
+unreachable objects" section below for more details.
+
+Since all operations that make new objects (e.g., "git commit") add
+the new objects to the corresponding index, this mapping is possible
+for all objects in the object store.
+
+Reading an object's sha1-content
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+The sha1-content of an object can be read by converting all newhash-names
+its newhash-content references to sha1-names using the translation table.
+
+Fetch
+~~~~~
+Fetching from a SHA-1 based server requires translating between SHA-1
+and NewHash based representations on the fly.
+
+SHA-1s named in the ref advertisement that are present on the client
+can be translated to NewHash and looked up as local objects using the
+translation table.
+
+Negotiation proceeds as today. Any "have"s generated locally are
+converted to SHA-1 before being sent to the server, and SHA-1s
+mentioned by the server are converted to NewHash when looking them up
+locally.
+
+After negotiation, the server sends a packfile containing the
+requested objects. We convert the packfile to NewHash format using
+the following steps:
+
+1. index-pack: inflate each object in the packfile and compute its
+ SHA-1. Objects can contain deltas in OBJ_REF_DELTA format against
+ objects the client has locally. These objects can be looked up
+ using the translation table and their sha1-content read as
+ described above to resolve the deltas.
+2. topological sort: starting at the "want"s from the negotiation
+ phase, walk through objects in the pack and emit a list of them,
+ excluding blobs, in reverse topologically sorted order, with each
+ object coming later in the list than all objects it references.
+ (This list only contains objects reachable from the "wants". If the
+ pack from the server contained additional extraneous objects, then
+ they will be discarded.)
+3. convert to newhash: open a new (newhash) packfile. Read the topologically
+ sorted list just generated. For each object, inflate its
+ sha1-content, convert to newhash-content, and write it to the newhash
+ pack. Record the new sha1<->newhash mapping entry for use in the idx.
+4. sort: reorder entries in the new pack to match the order of objects
+ in the pack the server generated and include blobs. Write a newhash idx
+ file
+5. clean up: remove the SHA-1 based pack file, index, and
+ topologically sorted list obtained from the server in steps 1
+ and 2.
+
+Step 3 requires every object referenced by the new object to be in the
+translation table. This is why the topological sort step is necessary.
+
+As an optimization, step 1 could write a file describing what non-blob
+objects each object it has inflated from the packfile references. This
+makes the topological sort in step 2 possible without inflating the
+objects in the packfile for a second time. The objects need to be
+inflated again in step 3, for a total of two inflations.
+
+Step 4 is probably necessary for good read-time performance. "git
+pack-objects" on the server optimizes the pack file for good data
+locality (see Documentation/technical/pack-heuristics.txt).
+
+Details of this process are likely to change. It will take some
+experimenting to get this to perform well.
+
+Push
+~~~~
+Push is simpler than fetch because the objects referenced by the
+pushed objects are already in the translation table. The sha1-content
+of each object being pushed can be read as described in the "Reading
+an object's sha1-content" section to generate the pack written by git
+send-pack.
+
+Signed Commits
+~~~~~~~~~~~~~~
+We add a new field "gpgsig-newhash" to the commit object format to allow
+signing commits without relying on SHA-1. It is similar to the
+existing "gpgsig" field. Its signed payload is the newhash-content of the
+commit object with any "gpgsig" and "gpgsig-newhash" fields removed.
+
+This means commits can be signed
+1. using SHA-1 only, as in existing signed commit objects
+2. using both SHA-1 and NewHash, by using both gpgsig-newhash and gpgsig
+ fields.
+3. using only NewHash, by only using the gpgsig-newhash field.
+
+Old versions of "git verify-commit" can verify the gpgsig signature in
+cases (1) and (2) without modifications and view case (3) as an
+ordinary unsigned commit.
+
+Signed Tags
+~~~~~~~~~~~
+We add a new field "gpgsig-newhash" to the tag object format to allow
+signing tags without relying on SHA-1. Its signed payload is the
+newhash-content of the tag with its gpgsig-newhash field and "-----BEGIN PGP
+SIGNATURE-----" delimited in-body signature removed.
+
+This means tags can be signed
+1. using SHA-1 only, as in existing signed tag objects
+2. using both SHA-1 and NewHash, by using gpgsig-newhash and an in-body
+ signature.
+3. using only NewHash, by only using the gpgsig-newhash field.
+
+Mergetag embedding
+~~~~~~~~~~~~~~~~~~
+The mergetag field in the sha1-content of a commit contains the
+sha1-content of a tag that was merged by that commit.
+
+The mergetag field in the newhash-content of the same commit contains the
+newhash-content of the same tag.
+
+Submodules
+~~~~~~~~~~
+To convert recorded submodule pointers, you need to have the converted
+submodule repository in place. The translation table of the submodule
+can be used to look up the new hash.
+
+Loose objects and unreachable objects
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Fast lookups in the loose-object-idx require that the number of loose
+objects not grow too high.
+
+"git gc --auto" currently waits for there to be 6700 loose objects
+present before consolidating them into a packfile. We will need to
+measure to find a more appropriate threshold for it to use.
+
+"git gc --auto" currently waits for there to be 50 packs present
+before combining packfiles. Packing loose objects more aggressively
+may cause the number of pack files to grow too quickly. This can be
+mitigated by using a strategy similar to Martin Fick's exponential
+rolling garbage collection script:
+https://gerrit-review.googlesource.com/c/gerrit/+/35215
+
+"git gc" currently expels any unreachable objects it encounters in
+pack files to loose objects in an attempt to prevent a race when
+pruning them (in case another process is simultaneously writing a new
+object that refers to the about-to-be-deleted object). This leads to
+an explosion in the number of loose objects present and disk space
+usage due to the objects in delta form being replaced with independent
+loose objects. Worse, the race is still present for loose objects.
+
+Instead, "git gc" will need to move unreachable objects to a new
+packfile marked as UNREACHABLE_GARBAGE (using the PSRC field; see
+below). To avoid the race when writing new objects referring to an
+about-to-be-deleted object, code paths that write new objects will
+need to copy any objects from UNREACHABLE_GARBAGE packs that they
+refer to to new, non-UNREACHABLE_GARBAGE packs (or loose objects).
+UNREACHABLE_GARBAGE are then safe to delete if their creation time (as
+indicated by the file's mtime) is long enough ago.
+
+To avoid a proliferation of UNREACHABLE_GARBAGE packs, they can be
+combined under certain circumstances. If "gc.garbageTtl" is set to
+greater than one day, then packs created within a single calendar day,
+UTC, can be coalesced together. The resulting packfile would have an
+mtime before midnight on that day, so this makes the effective maximum
+ttl the garbageTtl + 1 day. If "gc.garbageTtl" is less than one day,
+then we divide the calendar day into intervals one-third of that ttl
+in duration. Packs created within the same interval can be coalesced
+together. The resulting packfile would have an mtime before the end of
+the interval, so this makes the effective maximum ttl equal to the
+garbageTtl * 4/3.
+
+This rule comes from Thirumala Reddy Mutchukota's JGit change
+https://git.eclipse.org/r/90465.
+
+The UNREACHABLE_GARBAGE setting goes in the PSRC field of the pack
+index. More generally, that field indicates where a pack came from:
+
+ - 1 (PACK_SOURCE_RECEIVE) for a pack received over the network
+ - 2 (PACK_SOURCE_AUTO) for a pack created by a lightweight
+ "gc --auto" operation
+ - 3 (PACK_SOURCE_GC) for a pack created by a full gc
+ - 4 (PACK_SOURCE_UNREACHABLE_GARBAGE) for potential garbage
+ discovered by gc
+ - 5 (PACK_SOURCE_INSERT) for locally created objects that were
+ written directly to a pack file, e.g. from "git add ."
+
+This information can be useful for debugging and for "gc --auto" to
+make appropriate choices about which packs to coalesce.
+
+Caveats
+-------
+Invalid objects
+~~~~~~~~~~~~~~~
+The conversion from sha1-content to newhash-content retains any
+brokenness in the original object (e.g., tree entry modes encoded with
+leading 0, tree objects whose paths are not sorted correctly, and
+commit objects without an author or committer). This is a deliberate
+feature of the design to allow the conversion to round-trip.
+
+More profoundly broken objects (e.g., a commit with a truncated "tree"
+header line) cannot be converted but were not usable by current Git
+anyway.
+
+Shallow clone and submodules
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Because it requires all referenced objects to be available in the
+locally generated translation table, this design does not support
+shallow clone or unfetched submodules. Protocol improvements might
+allow lifting this restriction.
+
+Alternates
+~~~~~~~~~~
+For the same reason, a newhash repository cannot borrow objects from a
+sha1 repository using objects/info/alternates or
+$GIT_ALTERNATE_OBJECT_REPOSITORIES.
+
+git notes
+~~~~~~~~~
+The "git notes" tool annotates objects using their sha1-name as key.
+This design does not describe a way to migrate notes trees to use
+newhash-names. That migration is expected to happen separately (for
+example using a file at the root of the notes tree to describe which
+hash it uses).
+
+Server-side cost
+~~~~~~~~~~~~~~~~
+Until Git protocol gains NewHash support, using NewHash based storage
+on public-facing Git servers is strongly discouraged. Once Git
+protocol gains NewHash support, NewHash based servers are likely not
+to support SHA-1 compatibility, to avoid what may be a very expensive
+hash reencode during clone and to encourage peers to modernize.
+
+The design described here allows fetches by SHA-1 clients of a
+personal NewHash repository because it's not much more difficult than
+allowing pushes from that repository. This support needs to be guarded
+by a configuration option --- servers like git.kernel.org that serve a
+large number of clients would not be expected to bear that cost.
+
+Meaning of signatures
+~~~~~~~~~~~~~~~~~~~~~
+The signed payload for signed commits and tags does not explicitly
+name the hash used to identify objects. If some day Git adopts a new
+hash function with the same length as the current SHA-1 (40
+hexadecimal digit) or NewHash (64 hexadecimal digit) objects then the
+intent behind the PGP signed payload in an object signature is
+unclear:
+
+ object e7e07d5a4fcc2a203d9873968ad3e6bd4d7419d7
+ type commit
+ tag v2.12.0
+ tagger Junio C Hamano <gitster@pobox.com> 1487962205 -0800
+
+ Git 2.12
+
+Does this mean Git v2.12.0 is the commit with sha1-name
+e7e07d5a4fcc2a203d9873968ad3e6bd4d7419d7 or the commit with
+new-40-digit-hash-name e7e07d5a4fcc2a203d9873968ad3e6bd4d7419d7?
+
+Fortunately NewHash and SHA-1 have different lengths. If Git starts
+using another hash with the same length to name objects, then it will
+need to change the format of signed payloads using that hash to
+address this issue.
+
+Object names on the command line
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+To support the transition (see Transition plan below), this design
+supports four different modes of operation:
+
+ 1. ("dark launch") Treat object names input by the user as SHA-1 and
+ convert any object names written to output to SHA-1, but store
+ objects using NewHash. This allows users to test the code with no
+ visible behavior change except for performance. This allows
+ allows running even tests that assume the SHA-1 hash function, to
+ sanity-check the behavior of the new mode.
+
+ 2. ("early transition") Allow both SHA-1 and NewHash object names in
+ input. Any object names written to output use SHA-1. This allows
+ users to continue to make use of SHA-1 to communicate with peers
+ (e.g. by email) that have not migrated yet and prepares for mode 3.
+
+ 3. ("late transition") Allow both SHA-1 and NewHash object names in
+ input. Any object names written to output use NewHash. In this
+ mode, users are using a more secure object naming method by
+ default. The disruption is minimal as long as most of their peers
+ are in mode 2 or mode 3.
+
+ 4. ("post-transition") Treat object names input by the user as
+ NewHash and write output using NewHash. This is safer than mode 3
+ because there is less risk that input is incorrectly interpreted
+ using the wrong hash function.
+
+The mode is specified in configuration.
+
+The user can also explicitly specify which format to use for a
+particular revision specifier and for output, overriding the mode. For
+example:
+
+git --output-format=sha1 log abac87a^{sha1}..f787cac^{newhash}
+
+Selection of a New Hash
+-----------------------
+In early 2005, around the time that Git was written, Xiaoyun Wang,
+Yiqun Lisa Yin, and Hongbo Yu announced an attack finding SHA-1
+collisions in 2^69 operations. In August they published details.
+Luckily, no practical demonstrations of a collision in full SHA-1 were
+published until 10 years later, in 2017.
+
+The hash function NewHash to replace SHA-1 should be stronger than
+SHA-1 was: we would like it to be trustworthy and useful in practice
+for at least 10 years.
+
+Some other relevant properties:
+
+1. A 256-bit hash (long enough to match common security practice; not
+ excessively long to hurt performance and disk usage).
+
+2. High quality implementations should be widely available (e.g. in
+ OpenSSL).
+
+3. The hash function's properties should match Git's needs (e.g. Git
+ requires collision and 2nd preimage resistance and does not require
+ length extension resistance).
+
+4. As a tiebreaker, the hash should be fast to compute (fortunately
+ many contenders are faster than SHA-1).
+
+Some hashes under consideration are SHA-256, SHA-512/256, SHA-256x16,
+K12, and BLAKE2bp-256.
+
+Transition plan
+---------------
+Some initial steps can be implemented independently of one another:
+- adding a hash function API (vtable)
+- teaching fsck to tolerate the gpgsig-newhash field
+- excluding gpgsig-* from the fields copied by "git commit --amend"
+- annotating tests that depend on SHA-1 values with a SHA1 test
+ prerequisite
+- using "struct object_id", GIT_MAX_RAWSZ, and GIT_MAX_HEXSZ
+ consistently instead of "unsigned char *" and the hardcoded
+ constants 20 and 40.
+- introducing index v3
+- adding support for the PSRC field and safer object pruning
+
+
+The first user-visible change is the introduction of the objectFormat
+extension (without compatObjectFormat). This requires:
+- implementing the loose-object-idx
+- teaching fsck about this mode of operation
+- using the hash function API (vtable) when computing object names
+- signing objects and verifying signatures
+- rejecting attempts to fetch from or push to an incompatible
+ repository
+
+Next comes introduction of compatObjectFormat:
+- translating object names between object formats
+- translating object content between object formats
+- generating and verifying signatures in the compat format
+- adding appropriate index entries when adding a new object to the
+ object store
+- --output-format option
+- ^{sha1} and ^{newhash} revision notation
+- configuration to specify default input and output format (see
+ "Object names on the command line" above)
+
+The next step is supporting fetches and pushes to SHA-1 repositories:
+- allow pushes to a repository using the compat format
+- generate a topologically sorted list of the SHA-1 names of fetched
+ objects
+- convert the fetched packfile to newhash format and generate an idx
+ file
+- re-sort to match the order of objects in the fetched packfile
+
+The infrastructure supporting fetch also allows converting an existing
+repository. In converted repositories and new clones, end users can
+gain support for the new hash function without any visible change in
+behavior (see "dark launch" in the "Object names on the command line"
+section). In particular this allows users to verify NewHash signatures
+on objects in the repository, and it should ensure the transition code
+is stable in production in preparation for using it more widely.
+
+Over time projects would encourage their users to adopt the "early
+transition" and then "late transition" modes to take advantage of the
+new, more futureproof NewHash object names.
+
+When objectFormat and compatObjectFormat are both set, commands
+generating signatures would generate both SHA-1 and NewHash signatures
+by default to support both new and old users.
+
+In projects using NewHash heavily, users could be encouraged to adopt
+the "post-transition" mode to avoid accidentally making implicit use
+of SHA-1 object names.
+
+Once a critical mass of users have upgraded to a version of Git that
+can verify NewHash signatures and have converted their existing
+repositories to support verifying them, we can add support for a
+setting to generate only NewHash signatures. This is expected to be at
+least a year later.
+
+That is also a good moment to advertise the ability to convert
+repositories to use NewHash only, stripping out all SHA-1 related
+metadata. This improves performance by eliminating translation
+overhead and security by avoiding the possibility of accidentally
+relying on the safety of SHA-1.
+
+Updating Git's protocols to allow a server to specify which hash
+functions it supports is also an important part of this transition. It
+is not discussed in detail in this document but this transition plan
+assumes it happens. :)
+
+Alternatives considered
+-----------------------
+Upgrading everyone working on a particular project on a flag day
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Projects like the Linux kernel are large and complex enough that
+flipping the switch for all projects based on the repository at once
+is infeasible.
+
+Not only would all developers and server operators supporting
+developers have to switch on the same flag day, but supporting tooling
+(continuous integration, code review, bug trackers, etc) would have to
+be adapted as well. This also makes it difficult to get early feedback
+from some project participants testing before it is time for mass
+adoption.
+
+Using hash functions in parallel
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+(e.g. https://public-inbox.org/git/22708.8913.864049.452252@chiark.greenend.org.uk/ )
+Objects newly created would be addressed by the new hash, but inside
+such an object (e.g. commit) it is still possible to address objects
+using the old hash function.
+* You cannot trust its history (needed for bisectability) in the
+ future without further work
+* Maintenance burden as the number of supported hash functions grows
+ (they will never go away, so they accumulate). In this proposal, by
+ comparison, converted objects lose all references to SHA-1.
+
+Signed objects with multiple hashes
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Instead of introducing the gpgsig-newhash field in commit and tag objects
+for newhash-content based signatures, an earlier version of this design
+added "hash newhash <newhash-name>" fields to strengthen the existing
+sha1-content based signatures.
+
+In other words, a single signature was used to attest to the object
+content using both hash functions. This had some advantages:
+* Using one signature instead of two speeds up the signing process.
+* Having one signed payload with both hashes allows the signer to
+ attest to the sha1-name and newhash-name referring to the same object.
+* All users consume the same signature. Broken signatures are likely
+ to be detected quickly using current versions of git.
+
+However, it also came with disadvantages:
+* Verifying a signed object requires access to the sha1-names of all
+ objects it references, even after the transition is complete and
+ translation table is no longer needed for anything else. To support
+ this, the design added fields such as "hash sha1 tree <sha1-name>"
+ and "hash sha1 parent <sha1-name>" to the newhash-content of a signed
+ commit, complicating the conversion process.
+* Allowing signed objects without a sha1 (for after the transition is
+ complete) complicated the design further, requiring a "nohash sha1"
+ field to suppress including "hash sha1" fields in the newhash-content
+ and signed payload.
+
+Lazily populated translation table
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Some of the work of building the translation table could be deferred to
+push time, but that would significantly complicate and slow down pushes.
+Calculating the sha1-name at object creation time at the same time it is
+being streamed to disk and having its newhash-name calculated should be
+an acceptable cost.
+
+Document History
+----------------
+
+2017-03-03
+bmwill@google.com, jonathantanmy@google.com, jrnieder@gmail.com,
+sbeller@google.com
+
+Initial version sent to
+http://public-inbox.org/git/20170304011251.GA26789@aiede.mtv.corp.google.com
+
+2017-03-03 jrnieder@gmail.com
+Incorporated suggestions from jonathantanmy and sbeller:
+* describe purpose of signed objects with each hash type
+* redefine signed object verification using object content under the
+ first hash function
+
+2017-03-06 jrnieder@gmail.com
+* Use SHA3-256 instead of SHA2 (thanks, Linus and brian m. carlson).[1][2]
+* Make sha3-based signatures a separate field, avoiding the need for
+ "hash" and "nohash" fields (thanks to peff[3]).
+* Add a sorting phase to fetch (thanks to Junio for noticing the need
+ for this).
+* Omit blobs from the topological sort during fetch (thanks to peff).
+* Discuss alternates, git notes, and git servers in the caveats
+ section (thanks to Junio Hamano, brian m. carlson[4], and Shawn
+ Pearce).
+* Clarify language throughout (thanks to various commenters,
+ especially Junio).
+
+2017-09-27 jrnieder@gmail.com, sbeller@google.com
+* use placeholder NewHash instead of SHA3-256
+* describe criteria for picking a hash function.
+* include a transition plan (thanks especially to Brandon Williams
+ for fleshing these ideas out)
+* define the translation table (thanks, Shawn Pearce[5], Jonathan
+ Tan, and Masaya Suzuki)
+* avoid loose object overhead by packing more aggressively in
+ "git gc --auto"
+
+[1] http://public-inbox.org/git/CA+55aFzJtejiCjV0e43+9oR3QuJK2PiFiLQemytoLpyJWe6P9w@mail.gmail.com/
+[2] http://public-inbox.org/git/CA+55aFz+gkAsDZ24zmePQuEs1XPS9BP_s8O7Q4wQ7LV7X5-oDA@mail.gmail.com/
+[3] http://public-inbox.org/git/20170306084353.nrns455dvkdsfgo5@sigill.intra.peff.net/
+[4] http://public-inbox.org/git/20170304224936.rqqtkdvfjgyezsht@genre.crustytoothpaste.net
+[5] https://public-inbox.org/git/CAJo=hJtoX9=AyLHHpUJS7fueV9ciZ_MNpnEPHUz8Whui6g9F0A@mail.gmail.com/
LIB_OBJS += notes-merge.o
LIB_OBJS += notes-utils.o
LIB_OBJS += object.o
+LIB_OBJS += oidmap.o
LIB_OBJS += oidset.o
LIB_OBJS += packfile.o
LIB_OBJS += pack-bitmap.o
if (!head_rev)
die(_("Couldn't look up commit object for HEAD"));
}
- for (i = 0; i < argc; i++, strbuf_release(&bname)) {
+ for (i = 0; i < argc; i++, strbuf_reset(&bname)) {
char *target = NULL;
int flags = 0;
}
free(name);
+ strbuf_release(&bname);
- return(ret);
+ return ret;
}
static int calc_maxwidth(struct ref_array *refs, int remote_bonus)
else if (argc == 2)
copy_or_rename_branch(argv[0], argv[1], 0, rename > 1);
else
- die(_("too many branches for a rename operation"));
+ die(_("too many arguments for a rename operation"));
} else if (new_upstream) {
struct branch *branch = branch_get(argv[0]);
if (argc > 1)
- die(_("too many branches to set new upstream"));
+ die(_("too many arguments to set new upstream"));
if (!branch) {
if (!argc || !strcmp(argv[0], "HEAD"))
struct strbuf buf = STRBUF_INIT;
if (argc > 1)
- die(_("too many branches to unset upstream"));
+ die(_("too many arguments to unset upstream"));
if (!branch) {
if (!argc || !strcmp(argv[0], "HEAD"))
strbuf_release(&buf);
}
+ UNLEAK(opts);
if (opts.patch_mode || opts.pathspec.nr)
return checkout_paths(&opts, new.name);
else
return -1;
}
result = run_diff_index(&rev, cached);
+ UNLEAK(rev);
return diff_result_code(&rev.diffopt, result);
}
result = diff_result_code(&rev.diffopt, result);
if (1 < rev.diffopt.skip_stat_unmatch)
refresh_index_quietly();
+ UNLEAK(rev);
+ UNLEAK(ent);
+ UNLEAK(blob);
return result;
}
OPT_GROUP(""),
OPT_INTEGER( 0 , "count", &maxcount, N_("show only <n> matched refs")),
OPT_STRING( 0 , "format", &format.format, N_("format"), N_("format to use for the output")),
+ OPT__COLOR(&format.use_color, N_("respect format colors")),
OPT_CALLBACK(0 , "sort", sorting_tail, N_("key"),
N_("field name to sort on"), &parse_opt_ref_sorting),
OPT_CALLBACK(0, "points-at", &filter.points_at,
always, allow_undefined, data.name_only);
}
+ UNLEAK(revs);
return 0;
}
},
OPT_STRING( 0 , "format", &format.format, N_("format"),
N_("format to use for the output")),
+ OPT__COLOR(&format.use_color, N_("respect format colors")),
OPT_BOOL('i', "ignore-case", &icase, N_("sorting and filtering are case insensitive")),
OPT_END()
};
if (force && !is_null_oid(&prev) && oidcmp(&prev, &object))
printf(_("Updated tag '%s' (was %s)\n"), tag, find_unique_abbrev(prev.hash, DEFAULT_ABBREV));
- strbuf_release(&err);
- strbuf_release(&buf);
- strbuf_release(&ref);
- strbuf_release(&reflog_msg);
+ UNLEAK(buf);
+ UNLEAK(ref);
+ UNLEAK(reflog_msg);
+ UNLEAK(msg);
+ UNLEAK(err);
return 0;
}
if (!strcasecmp(value, "never"))
return 0;
if (!strcasecmp(value, "always"))
- return 1;
+ return var ? GIT_COLOR_AUTO : 1;
if (!strcasecmp(value, "auto"))
return GIT_COLOR_AUTO;
}
pfd[i].revents = happened;
rc++;
}
+ else
+ {
+ pfd[i].revents = 0;
+ }
}
return rc;
* add to "newlist" between calls, the additions will always be for
* oids that are already in the set.
*/
- if (!tip_oids->map.tablesize) {
+ if (!tip_oids->map.map.tablesize) {
add_refs_to_oidset(tip_oids, unmatched);
add_refs_to_oidset(tip_oids, newlist);
}
continue;
if (S_ISDIR(entry.mode)) {
- obj = &lookup_tree(entry.oid)->object;
- if (name)
+ obj = (struct object *)lookup_tree(entry.oid);
+ if (name && obj)
put_object_name(options, obj, "%s%s/", name,
entry.path);
result = options->walk(obj, OBJ_TREE, data, options);
}
else if (S_ISREG(entry.mode) || S_ISLNK(entry.mode)) {
- obj = &lookup_blob(entry.oid)->object;
- if (name)
+ obj = (struct object *)lookup_blob(entry.oid);
+ if (name && obj)
put_object_name(options, obj, "%s%s", name,
entry.path);
result = options->walk(obj, OBJ_BLOB, data, options);
memcpy(hex, path, 2);
path += 2;
path++; /* skip '/' */
- memcpy(hex, path, GIT_SHA1_HEXSZ - 2);
+ memcpy(hex + 2, path, GIT_SHA1_HEXSZ - 2);
return get_oid_hex(hex, oid);
}
--- /dev/null
+#include "cache.h"
+#include "oidmap.h"
+
+static int cmpfn(const void *hashmap_cmp_fn_data,
+ const void *entry, const void *entry_or_key,
+ const void *keydata)
+{
+ const struct oidmap_entry *entry_ = entry;
+ if (keydata)
+ return oidcmp(&entry_->oid, (const struct object_id *) keydata);
+ return oidcmp(&entry_->oid,
+ &((const struct oidmap_entry *) entry_or_key)->oid);
+}
+
+static int hash(const struct object_id *oid)
+{
+ int hash;
+ memcpy(&hash, oid->hash, sizeof(hash));
+ return hash;
+}
+
+void oidmap_init(struct oidmap *map, size_t initial_size)
+{
+ hashmap_init(&map->map, cmpfn, NULL, initial_size);
+}
+
+void oidmap_free(struct oidmap *map, int free_entries)
+{
+ if (!map)
+ return;
+ hashmap_free(&map->map, free_entries);
+}
+
+void *oidmap_get(const struct oidmap *map, const struct object_id *key)
+{
+ return hashmap_get_from_hash(&map->map, hash(key), key);
+}
+
+void *oidmap_remove(struct oidmap *map, const struct object_id *key)
+{
+ struct hashmap_entry entry;
+ hashmap_entry_init(&entry, hash(key));
+ return hashmap_remove(&map->map, &entry, key);
+}
+
+void *oidmap_put(struct oidmap *map, void *entry)
+{
+ struct oidmap_entry *to_put = entry;
+ hashmap_entry_init(&to_put->internal_entry, hash(&to_put->oid));
+ return hashmap_put(&map->map, to_put);
+}
--- /dev/null
+#ifndef OIDMAP_H
+#define OIDMAP_H
+
+#include "hashmap.h"
+
+/*
+ * struct oidmap_entry is a structure representing an entry in the hash table,
+ * which must be used as first member of user data structures.
+ *
+ * Users should set the oid field. oidmap_put() will populate the
+ * internal_entry field.
+ */
+struct oidmap_entry {
+ /* For internal use only */
+ struct hashmap_entry internal_entry;
+
+ struct object_id oid;
+};
+
+struct oidmap {
+ struct hashmap map;
+};
+
+#define OIDMAP_INIT { { NULL } }
+
+/*
+ * Initializes an oidmap structure.
+ *
+ * `map` is the oidmap to initialize.
+ *
+ * If the total number of entries is known in advance, the `initial_size`
+ * parameter may be used to preallocate a sufficiently large table and thus
+ * prevent expensive resizing. If 0, the table is dynamically resized.
+ */
+extern void oidmap_init(struct oidmap *map, size_t initial_size);
+
+/*
+ * Frees an oidmap structure and allocated memory.
+ *
+ * If `free_entries` is true, each oidmap_entry in the map is freed as well
+ * using stdlibs free().
+ */
+extern void oidmap_free(struct oidmap *map, int free_entries);
+
+/*
+ * Returns the oidmap entry for the specified oid, or NULL if not found.
+ */
+extern void *oidmap_get(const struct oidmap *map,
+ const struct object_id *key);
+
+/*
+ * Adds or replaces an oidmap entry.
+ *
+ * ((struct oidmap_entry *) entry)->internal_entry will be populated by this
+ * function.
+ *
+ * Returns the replaced entry, or NULL if not found (i.e. the entry was added).
+ */
+extern void *oidmap_put(struct oidmap *map, void *entry);
+
+/*
+ * Removes an oidmap entry matching the specified oid.
+ *
+ * Returns the removed entry, or NULL if not found.
+ */
+extern void *oidmap_remove(struct oidmap *map, const struct object_id *key);
+
+#endif
#include "cache.h"
#include "oidset.h"
-struct oidset_entry {
- struct hashmap_entry hash;
- struct object_id oid;
-};
-
-static int oidset_hashcmp(const void *unused_cmp_data,
- const void *va, const void *vb,
- const void *vkey)
-{
- const struct oidset_entry *a = va, *b = vb;
- const struct object_id *key = vkey;
- return oidcmp(&a->oid, key ? key : &b->oid);
-}
-
int oidset_contains(const struct oidset *set, const struct object_id *oid)
{
- struct hashmap_entry key;
-
- if (!set->map.cmpfn)
+ if (!set->map.map.tablesize)
return 0;
-
- hashmap_entry_init(&key, sha1hash(oid->hash));
- return !!hashmap_get(&set->map, &key, oid);
+ return !!oidmap_get(&set->map, oid);
}
int oidset_insert(struct oidset *set, const struct object_id *oid)
{
- struct oidset_entry *entry;
-
- if (!set->map.cmpfn)
- hashmap_init(&set->map, oidset_hashcmp, NULL, 0);
+ struct oidmap_entry *entry;
- if (oidset_contains(set, oid))
+ if (!set->map.map.tablesize)
+ oidmap_init(&set->map, 0);
+ else if (oidset_contains(set, oid))
return 1;
entry = xmalloc(sizeof(*entry));
- hashmap_entry_init(&entry->hash, sha1hash(oid->hash));
oidcpy(&entry->oid, oid);
- hashmap_add(&set->map, entry);
+ oidmap_put(&set->map, entry);
return 0;
}
void oidset_clear(struct oidset *set)
{
- hashmap_free(&set->map, 1);
+ oidmap_free(&set->map, 1);
}
#ifndef OIDSET_H
#define OIDSET_H
+#include "oidmap.h"
+
/**
* This API is similar to sha1-array, in that it maintains a set of object ids
* in a memory-efficient way. The major differences are:
* A single oidset; should be zero-initialized (or use OIDSET_INIT).
*/
struct oidset {
- struct hashmap map;
+ struct oidmap map;
};
-#define OIDSET_INIT { { NULL } }
+#define OIDSET_INIT { OIDMAP_INIT }
/**
* Returns true iff `set` contains `oid`.
return sb;
}
-static char *cleanup_path(char *path)
+static const char *cleanup_path(const char *path)
{
/* Clean it up */
- if (!memcmp(path, "./", 2)) {
- path += 2;
+ if (skip_prefix(path, "./", &path)) {
while (*path == '/')
path++;
}
static void strbuf_cleanup_path(struct strbuf *sb)
{
- char *path = cleanup_path(sb->buf);
+ const char *path = cleanup_path(sb->buf);
if (path > sb->buf)
strbuf_remove(sb, 0, path - sb->buf);
}
strlcpy(buf, bad_path, n);
return buf;
}
- return cleanup_path(buf);
+ return (char *)cleanup_path(buf);
}
static int dir_prefix(const char *buf, const char *dir)
return 0;
}
+static int match_placeholder_arg(const char *to_parse, const char *candidate,
+ const char **end)
+{
+ const char *p;
+
+ if (!(skip_prefix(to_parse, candidate, &p)))
+ return 0;
+ if (*p == ',') {
+ *end = p + 1;
+ return 1;
+ }
+ if (*p == ')') {
+ *end = p;
+ return 1;
+ }
+ return 0;
+}
+
static size_t format_commit_one(struct strbuf *sb, /* in UTF-8 */
const char *placeholder,
void *context)
if (skip_prefix(placeholder, "(trailers", &arg)) {
struct process_trailer_options opts = PROCESS_TRAILER_OPTIONS_INIT;
- while (*arg == ':') {
- if (skip_prefix(arg, ":only", &arg))
- opts.only_trailers = 1;
- else if (skip_prefix(arg, ":unfold", &arg))
- opts.unfold = 1;
+ if (*arg == ':') {
+ arg++;
+ for (;;) {
+ if (match_placeholder_arg(arg, "only", &arg))
+ opts.only_trailers = 1;
+ else if (match_placeholder_arg(arg, "unfold", &arg))
+ opts.unfold = 1;
+ else
+ break;
+ }
}
if (*arg == ')') {
format_trailers_from_commit(sb, msg + c->subject_off, &opts);
} remote_ref;
struct {
enum { C_BARE, C_BODY, C_BODY_DEP, C_LINES, C_SIG, C_SUB, C_TRAILERS } option;
+ struct process_trailer_options trailer_opts;
unsigned int nlines;
} contents;
struct {
static void trailers_atom_parser(const struct ref_format *format, struct used_atom *atom, const char *arg)
{
- if (arg)
- die(_("%%(trailers) does not take arguments"));
+ struct string_list params = STRING_LIST_INIT_DUP;
+ int i;
+
+ if (arg) {
+ string_list_split(¶ms, arg, ',', -1);
+ for (i = 0; i < params.nr; i++) {
+ const char *s = params.items[i].string;
+ if (!strcmp(s, "unfold"))
+ atom->u.contents.trailer_opts.unfold = 1;
+ else if (!strcmp(s, "only"))
+ atom->u.contents.trailer_opts.only_trailers = 1;
+ else
+ die(_("unknown %%(trailers) argument: %s"), s);
+ }
+ }
atom->u.contents.option = C_TRAILERS;
+ string_list_clear(¶ms, 0);
}
static void contents_atom_parser(const struct ref_format *format, struct used_atom *atom, const char *arg)
atom->u.contents.option = C_SIG;
else if (!strcmp(arg, "subject"))
atom->u.contents.option = C_SUB;
- else if (!strcmp(arg, "trailers"))
- atom->u.contents.option = C_TRAILERS;
- else if (skip_prefix(arg, "lines=", &arg)) {
+ else if (skip_prefix(arg, "trailers", &arg)) {
+ skip_prefix(arg, ":", &arg);
+ trailers_atom_parser(format, atom, *arg ? arg : NULL);
+ } else if (skip_prefix(arg, "lines=", &arg)) {
atom->u.contents.option = C_LINES;
if (strtoul_ui(arg, 10, &atom->u.contents.nlines))
die(_("positive value expected contents:lines=%s"), arg);
REALLOC_ARRAY(used_atom, used_atom_cnt);
used_atom[at].name = xmemdupz(atom, ep - atom);
used_atom[at].type = valid_atom[i].cmp_type;
- if (arg)
+ if (arg) {
arg = used_atom[at].name + (arg - atom) + 1;
+ if (!*arg) {
+ /*
+ * Treat empty sub-arguments list as NULL (i.e.,
+ * "%(atom:)" is equivalent to "%(atom)").
+ */
+ arg = NULL;
+ }
+ }
memset(&used_atom[at].u, 0, sizeof(used_atom[at].u));
if (valid_atom[i].parser)
valid_atom[i].parser(format, &used_atom[at], arg);
name++;
if (strcmp(name, "subject") &&
strcmp(name, "body") &&
- strcmp(name, "trailers") &&
+ !starts_with(name, "trailers") &&
!starts_with(name, "contents"))
continue;
if (!subpos)
append_lines(&s, subpos, contents_end - subpos, atom->u.contents.nlines);
v->s = strbuf_detach(&s, NULL);
} else if (atom->u.contents.option == C_TRAILERS) {
- struct trailer_info info;
+ struct strbuf s = STRBUF_INIT;
- /* Search for trailer info */
- trailer_info_get(&info, subpos);
- v->s = xmemdupz(info.trailer_start,
- info.trailer_end - info.trailer_start);
- trailer_info_release(&info);
+ /* Format the trailer info according to the trailer_opts given */
+ format_trailers_from_commit(&s, subpos, &atom->u.contents.trailer_opts);
+
+ v->s = strbuf_detach(&s, NULL);
} else if (atom->u.contents.option == C_BARE)
v->s = xstrdup(subpos);
}
return CHECK_WARN;
if (!strcasecmp("error", value))
return CHECK_ERROR;
- warning(_("unrecognized setting %s for option"
+ warning(_("unrecognized setting %s for option "
"rebase.missingCommitsCheck. Ignoring."), value);
return CHECK_IGNORE;
}
int diagnose_misspelt_rev)
{
if (*arg == '-')
- die("bad flag '%s' used after filename", arg);
+ die("option '%s' must come before non-option arguments", arg);
if (looks_like_pathspec(arg) || check_filename(prefix, arg))
return;
die_verify_filename(prefix, arg, diagnose_misspelt_rev);
} else if ((status = parse_sha1_header_extended(hdr, oi, flags)) < 0)
status = error("unable to parse %s header", sha1_to_hex(sha1));
- if (status >= 0 && oi->contentp)
+ if (status >= 0 && oi->contentp) {
*oi->contentp = unpack_sha1_rest(&stream, hdr,
*oi->sizep, sha1);
- else
+ if (!*oi->contentp) {
+ git_inflate_end(&stream);
+ status = -1;
+ }
+ } else
git_inflate_end(&stream);
munmap(map, mapsize);
extern void strbuf_init(struct strbuf *, size_t);
/**
- * Release a string buffer and the memory it used. You should not use the
- * string buffer after using this function, unless you initialize it again.
+ * Release a string buffer and the memory it used. After this call, the
+ * strbuf points to an empty string that does not need to be free()ed, as
+ * if it had been set to `STRBUF_INIT` and never modified.
+ *
+ * To clear a strbuf in preparation for further use without the overhead
+ * of free()ing and malloc()ing again, use strbuf_reset() instead.
*/
extern void strbuf_release(struct strbuf *);
* Detach the string from the strbuf and returns it; you now own the
* storage the string occupies and it is your responsibility from then on
* to release it with `free(3)` when you are done with it.
+ *
+ * The strbuf that previously held the string is reset to `STRBUF_INIT` so
+ * it can be reused after calling this function.
*/
extern char *strbuf_detach(struct strbuf *, size_t *);
{
int err;
struct child_process *process;
- const char *argv[] = { cmd, NULL };
entry->cmd = cmd;
process = &entry->process;
child_process_init(process);
- process->argv = argv;
+ argv_array_push(&process->args, cmd);
process->use_shell = 1;
process->in = -1;
process->out = -1;
* Split by newline, but don't create a string_list item
* for the empty string after the last separator.
*/
- if (sb.buf[sb.len - 1] == '\n')
+ if (sb.len && sb.buf[sb.len - 1] == '\n')
strbuf_setlen(&sb, sb.len - 1);
string_list_split_in_place(&list, sb.buf, '\n', -1);
test_i18ngrep ! "fatal: empty filename in tree entry" out
'
+hex2oct() {
+ perl -ne 'printf "\\%03o", hex for /../g'
+}
+
+test_expect_success 'tree entry with type mismatch' '
+ test_when_finished "remove_object \$blob" &&
+ test_when_finished "remove_object \$tree" &&
+ test_when_finished "remove_object \$commit" &&
+ test_when_finished "git update-ref -d refs/heads/type_mismatch" &&
+ blob=$(echo blob | git hash-object -w --stdin) &&
+ blob_bin=$(echo $blob | hex2oct) &&
+ tree=$(
+ printf "40000 dir\0${blob_bin}100644 file\0${blob_bin}" |
+ git hash-object -t tree --stdin -w --literally
+ ) &&
+ commit=$(git commit-tree $tree) &&
+ git update-ref refs/heads/type_mismatch $commit &&
+ test_must_fail git fsck >out 2>&1 &&
+ test_i18ngrep "is a blob, not a tree" out &&
+ test_i18ngrep ! "dangling blob" out
+'
+
test_expect_success 'tag pointing to nonexistent' '
cat >invalid-tag <<-\EOF &&
object ffffffffffffffffffffffffffffffffffffffff
'
test_expect_success TTY '%(color) present with tty' '
- test_terminal env TERM=vt100 git branch $color_args >actual.raw &&
- test_decode_color <actual.raw >actual &&
- test_cmp expect.color actual
-'
-
-test_expect_success 'color.branch=always overrides auto-color' '
- git -c color.branch=always branch $color_args >actual.raw &&
+ test_terminal git branch $color_args >actual.raw &&
test_decode_color <actual.raw >actual &&
test_cmp expect.color actual
'
# choose non-default colors to make sure config
# is taking effect
test_expect_success 'set up some color config' '
- git config color.branch always &&
git config color.branch.local blue &&
git config color.branch.remote yellow &&
git config color.branch.current cyan
<BLUE>other<RESET>
<YELLOW>remotes/origin/master<RESET>
EOF
- git branch -a >actual.raw &&
+ git branch --color -a >actual.raw &&
test_decode_color <actual.raw >actual &&
test_cmp expect actual
'
<BLUE>other <RESET> $oid foo
<YELLOW>remotes/origin/master<RESET> $oid foo
EOF
- git branch -v -a >actual.raw &&
+ git branch --color -v -a >actual.raw &&
test_decode_color <actual.raw >actual &&
test_cmp expect actual
'
test_description='add -i basic tests'
. ./test-lib.sh
+. "$TEST_DIRECTORY"/lib-terminal.sh
if ! test_have_prereq PERL
then
test_cmp expected diff
'
-test_expect_success 'diffs can be colorized' '
+test_expect_success TTY 'diffs can be colorized' '
git reset --hard &&
- # force color even though the test script has no terminal
- test_config color.ui always &&
-
echo content >test &&
- printf y | git add -p >output 2>&1 &&
+ printf y | test_terminal git add -p >output 2>&1 &&
# We do not want to depend on the exact coloring scheme
# git uses for diffs, so just check that we saw some kind of color.
git diff --exit-code
'
+test_expect_success 'add -p works even with color.ui=always' '
+ git reset --hard &&
+ echo change >>file &&
+ test_config color.ui always &&
+ echo y | git add -p &&
+ echo file >expect &&
+ git diff --cached --name-only >actual &&
+ test_cmp expect actual
+'
+
test_done
# Start testing the colored format for whitespace checks
test_expect_success 'setup diff colors' '
- git config color.diff always &&
git config color.diff.plain normal &&
git config color.diff.meta bold &&
git config color.diff.frag cyan &&
echo "test" >x &&
git commit -m "initial" x &&
echo "{NTN}" | tr "NT" "\n\t" >>x &&
- git -c color.diff=always diff | test_decode_color >current &&
+ git diff --color | test_decode_color >current &&
cat >expected <<-\EOF &&
<BOLD>diff --git a/x b/x<RESET>
echo "2. and a new line "
} >x &&
- git -c color.diff=always diff |
+ git diff --color |
test_decode_color >current &&
cat >expected <<-\EOF &&
test_expect_success 'test --ws-error-highlight option' '
- git -c color.diff=always diff --ws-error-highlight=default,old |
+ git diff --color --ws-error-highlight=default,old |
test_decode_color >current &&
test_cmp expect.default-old current &&
- git -c color.diff=always diff --ws-error-highlight=all |
+ git diff --color --ws-error-highlight=all |
test_decode_color >current &&
test_cmp expect.all current &&
- git -c color.diff=always diff --ws-error-highlight=none |
+ git diff --color --ws-error-highlight=none |
test_decode_color >current &&
test_cmp expect.none current
test_expect_success 'test diff.wsErrorHighlight config' '
- git -c color.diff=always -c diff.wsErrorHighlight=default,old diff |
+ git -c diff.wsErrorHighlight=default,old diff --color |
test_decode_color >current &&
test_cmp expect.default-old current &&
- git -c color.diff=always -c diff.wsErrorHighlight=all diff |
+ git -c diff.wsErrorHighlight=all diff --color |
test_decode_color >current &&
test_cmp expect.all current &&
- git -c color.diff=always -c diff.wsErrorHighlight=none diff |
+ git -c diff.wsErrorHighlight=none diff --color |
test_decode_color >current &&
test_cmp expect.none current
test_expect_success 'option overrides diff.wsErrorHighlight' '
- git -c color.diff=always -c diff.wsErrorHighlight=none \
- diff --ws-error-highlight=default,old |
+ git -c diff.wsErrorHighlight=none \
+ diff --color --ws-error-highlight=default,old |
test_decode_color >current &&
test_cmp expect.default-old current &&
- git -c color.diff=always -c diff.wsErrorHighlight=default \
- diff --ws-error-highlight=all |
+ git -c diff.wsErrorHighlight=default \
+ diff --color --ws-error-highlight=all |
test_decode_color >current &&
test_cmp expect.all current &&
- git -c color.diff=always -c diff.wsErrorHighlight=all \
- diff --ws-error-highlight=none |
+ git -c diff.wsErrorHighlight=all \
+ diff --color --ws-error-highlight=none |
test_decode_color >current &&
test_cmp expect.none current
git mv test.c main.c &&
test_config color.diff.oldMoved "normal red" &&
test_config color.diff.newMoved "normal green" &&
- git diff HEAD --color-moved=zebra --no-renames | test_decode_color >actual &&
+ git diff HEAD --color-moved=zebra --color --no-renames | test_decode_color >actual &&
cat >expected <<-\EOF &&
<BOLD>diff --git a/main.c b/main.c<RESET>
<BOLD>new file mode 100644<RESET>
bar();
}
EOF
- git diff HEAD --no-renames --color-moved=zebra| test_decode_color >actual &&
+ git diff HEAD --no-renames --color-moved=zebra --color | test_decode_color >actual &&
cat <<-\EOF >expected &&
<BOLD>diff --git a/main.c b/main.c<RESET>
<BOLD>index 27a619c..7cf9336 100644<RESET>
test_config color.diff.oldMovedAlternative "blue" &&
test_config color.diff.newMovedAlternative "yellow" &&
# needs previous test as setup
- git diff HEAD --no-renames --color-moved=plain| test_decode_color >actual &&
+ git diff HEAD --no-renames --color-moved=plain --color | test_decode_color >actual &&
cat <<-\EOF >expected &&
<BOLD>diff --git a/main.c b/main.c<RESET>
<BOLD>index 27a619c..7cf9336 100644<RESET>
test_config color.diff.newMovedDimmed "normal cyan" &&
test_config color.diff.oldMovedAlternativeDimmed "normal blue" &&
test_config color.diff.newMovedAlternativeDimmed "normal yellow" &&
- git diff HEAD --no-renames --color-moved=dimmed_zebra |
+ git diff HEAD --no-renames --color-moved=dimmed_zebra --color |
grep -v "index" |
test_decode_color >actual &&
cat <<-\EOF >expected &&
test_config color.diff.oldMovedAlternativeDimmed "normal blue" &&
test_config color.diff.newMovedAlternativeDimmed "normal yellow" &&
test_config diff.colorMoved zebra &&
- git diff HEAD --no-renames --color-moved |
+ git diff HEAD --no-renames --color-moved --color |
grep -v "index" |
test_decode_color >actual &&
cat <<-\EOF >expected &&
EOF
test_config color.diff.oldMoved "magenta" &&
test_config color.diff.newMoved "cyan" &&
- git diff HEAD --no-renames --color-moved |
+ git diff HEAD --no-renames --color-moved --color |
grep -v "index" |
test_decode_color >actual &&
cat <<-\EOF >expected &&
EOF
test_cmp expected actual &&
- git diff HEAD --no-renames -w --color-moved |
+ git diff HEAD --no-renames -w --color-moved --color |
grep -v "index" |
test_decode_color >actual &&
cat <<-\EOF >expected &&
irrelevant_line
EOF
- git diff HEAD --color-moved=zebra --no-renames |
+ git diff HEAD --color-moved=zebra --color --no-renames |
grep -v "index" |
test_decode_color >actual &&
cat >expected <<-\EOF &&
nineteen chars 456789
EOF
- git diff HEAD --color-moved=zebra --no-renames |
+ git diff HEAD --color-moved=zebra --color --no-renames |
grep -v "index" |
test_decode_color >actual &&
cat >expected <<-\EOF &&
7charsA
EOF
- git diff HEAD --color-moved=zebra --no-renames | grep -v "index" | test_decode_color >actual &&
+ git diff HEAD --color-moved=zebra --color --no-renames | grep -v "index" | test_decode_color >actual &&
cat >expected <<-\EOF &&
<BOLD>diff --git a/bar b/bar<RESET>
<BOLD>--- a/bar<RESET>
echo foul >bananas/recipe &&
echo ripe >fruit.t &&
- git diff --submodule=diff --color-moved >actual &&
+ git diff --submodule=diff --color-moved --color >actual &&
# no move detection as the moved line is across repository boundaries.
test_decode_color <actual >decoded_actual &&
! grep BRED decoded_actual &&
# nor did we mess with it another way
- git diff --submodule=diff | test_decode_color >expect &&
+ git diff --submodule=diff --color | test_decode_color >expect &&
test_cmp expect decoded_actual
'
'
test_expect_success TTY 'log output on a TTY' '
- git log --oneline --decorate >expect.short &&
+ git log --color --oneline --decorate >expect.short &&
test_terminal git log --oneline >actual &&
test_cmp expect.short actual
EOF
unfold () {
- perl -0pe 's/\n\s+/ /'
+ perl -0pe 's/\n\s+/ /g'
}
test_expect_success 'set up trailer tests' '
'
test_expect_success ':only and :unfold work together' '
- git log --no-walk --pretty="%(trailers:only:unfold)" >actual &&
- git log --no-walk --pretty="%(trailers:unfold:only)" >reverse &&
+ git log --no-walk --pretty="%(trailers:only,unfold)" >actual &&
+ git log --no-walk --pretty="%(trailers:unfold,only)" >reverse &&
test_cmp actual reverse &&
{
grep -v patch.description <trailers | unfold &&
has_no_color actual
'
- test_expect_success "$desc enables colors for color.diff" '
- git -c color.diff=always log --format=$color -1 >actual &&
- has_color actual
- '
-
- test_expect_success "$desc enables colors for color.ui" '
- git -c color.ui=always log --format=$color -1 >actual &&
- has_color actual
- '
-
test_expect_success "$desc respects --color" '
git log --format=$color -1 --color >actual &&
has_color actual
'
- test_expect_success "$desc respects --no-color" '
- git -c color.ui=always log --format=$color -1 --no-color >actual &&
- has_no_color actual
- '
-
test_expect_success TTY "$desc respects --color=auto (stdout is tty)" '
- test_terminal env TERM=vt100 \
- git log --format=$color -1 --color=auto >actual &&
+ test_terminal git log --format=$color -1 --color=auto >actual &&
has_color actual
'
has_no_color actual
)
'
+
+ test_expect_success TTY "$desc respects --no-color" '
+ test_terminal git log --format=$color -1 --no-color >actual &&
+ has_no_color actual
+ '
done
test_expect_success '%C(always,...) enables color even without tty' '
}
test_atom head refname refs/heads/master
+test_atom head refname: refs/heads/master
test_atom head refname:short master
test_atom head refname:lstrip=1 heads/master
test_atom head refname:lstrip=2 master
'
test_expect_success TTY '%(color) shows color with a tty' '
- test_terminal env TERM=vt100 \
- git for-each-ref --format="$color_format" >actual.raw &&
+ test_terminal git for-each-ref --format="$color_format" >actual.raw &&
test_decode_color <actual.raw >actual &&
test_cmp expected.color actual
'
test_cmp expected.bare actual
'
-test_expect_success 'color.ui=always can override tty check' '
- git -c color.ui=always for-each-ref --format="$color_format" >actual.raw &&
+test_expect_success '--color can override tty check' '
+ git for-each-ref --color --format="$color_format" >actual.raw &&
test_decode_color <actual.raw >actual &&
test_cmp expected.color actual
'
cat >trailers <<EOF
Reviewed-by: A U Thor <author@example.com>
Signed-off-by: A U Thor <author@example.com>
+[ v2 updated patch description ]
+Acked-by: A U Thor
+ <author@example.com>
EOF
-test_expect_success 'basic atom: head contents:trailers' '
+unfold () {
+ perl -0pe 's/\n\s+/ /g'
+}
+
+test_expect_success 'set up trailers for next test' '
echo "Some contents" > two &&
git add two &&
- git commit -F - <<-EOF &&
+ git commit -F - <<-EOF
trailers: this commit message has trailers
Some message contents
$(cat trailers)
EOF
+'
+
+test_expect_success '%(trailers:unfold) unfolds trailers' '
+ git for-each-ref --format="%(trailers:unfold)" refs/heads/master >actual &&
+ {
+ unfold <trailers
+ echo
+ } >expect &&
+ test_cmp expect actual
+'
+
+test_expect_success '%(trailers:only) shows only "key: value" trailers' '
+ git for-each-ref --format="%(trailers:only)" refs/heads/master >actual &&
+ {
+ grep -v patch.description <trailers &&
+ echo
+ } >expect &&
+ test_cmp expect actual
+'
+
+test_expect_success '%(trailers:only) and %(trailers:unfold) work together' '
+ git for-each-ref --format="%(trailers:only,unfold)" refs/heads/master >actual &&
+ git for-each-ref --format="%(trailers:unfold,only)" refs/heads/master >reverse &&
+ test_cmp actual reverse &&
+ {
+ grep -v patch.description <trailers | unfold &&
+ echo
+ } >expect &&
+ test_cmp expect actual
+'
+
+test_expect_success '%(contents:trailers:unfold) unfolds trailers' '
+ git for-each-ref --format="%(contents:trailers:unfold)" refs/heads/master >actual &&
+ {
+ unfold <trailers
+ echo
+ } >expect &&
+ test_cmp expect actual
+'
+
+test_expect_success '%(contents:trailers:only) shows only "key: value" trailers' '
+ git for-each-ref --format="%(contents:trailers:only)" refs/heads/master >actual &&
+ {
+ grep -v patch.description <trailers &&
+ echo
+ } >expect &&
+ test_cmp expect actual
+'
+
+test_expect_success '%(contents:trailers:only) and %(contents:trailers:unfold) work together' '
+ git for-each-ref --format="%(contents:trailers:only,unfold)" refs/heads/master >actual &&
+ git for-each-ref --format="%(contents:trailers:unfold,only)" refs/heads/master >reverse &&
+ test_cmp actual reverse &&
+ {
+ grep -v patch.description <trailers | unfold &&
+ echo
+ } >expect &&
+ test_cmp expect actual
+'
+
+test_expect_success '%(trailers) rejects unknown trailers arguments' '
+ # error message cannot be checked under i18n
+ cat >expect <<-EOF &&
+ fatal: unknown %(trailers) argument: unsupported
+ EOF
+ test_must_fail git for-each-ref --format="%(trailers:unsupported)" 2>actual &&
+ test_i18ncmp expect actual
+'
+
+test_expect_success '%(contents:trailers) rejects unknown trailers arguments' '
+ # error message cannot be checked under i18n
+ cat >expect <<-EOF &&
+ fatal: unknown %(trailers) argument: unsupported
+ EOF
+ test_must_fail git for-each-ref --format="%(contents:trailers:unsupported)" 2>actual &&
+ test_i18ncmp expect actual
+'
+
+test_expect_success 'basic atom: head contents:trailers' '
git for-each-ref --format="%(contents:trailers)" refs/heads/master >actual &&
sanitize_pgp <actual >actual.clean &&
# git for-each-ref ends with a blank line
'
test_expect_success TTY '%(color) present with tty' '
- test_terminal env TERM=vt100 git tag $color_args >actual.raw &&
+ test_terminal git tag $color_args >actual.raw &&
test_decode_color <actual.raw >actual &&
test_cmp expect.color actual
'
-test_expect_success 'color.ui=always overrides auto-color' '
- git -c color.ui=always tag $color_args >actual.raw &&
+test_expect_success '--color overrides auto-color' '
+ git tag --color $color_args >actual.raw &&
test_decode_color <actual.raw >actual &&
test_cmp expect.color actual
'
test_expect_success TTY 'color when writing to a pager' '
rm -f paginated.out &&
test_config color.ui auto &&
- test_terminal env TERM=vt100 git log &&
+ test_terminal git log &&
colorful paginated.out
'
rm -f paginated.out &&
test_config color.ui auto &&
test_config color.pager false &&
- test_terminal env TERM=vt100 git log &&
+ test_terminal git log &&
! colorful paginated.out
'
test_expect_success TTY 'colors are sent to pager for external commands' '
test_config alias.externallog "!git log" &&
test_config color.ui auto &&
- test_terminal env TERM=vt100 git -p externallog &&
+ test_terminal git -p externallog &&
colorful paginated.out
'
test_description='git clean -i basic tests'
. ./test-lib.sh
+. "$TEST_DIRECTORY"/lib-terminal.sh
test_expect_success 'setup' '
'
-test_expect_success 'git clean -i paints the header in HEADER color' '
+test_expect_success TTY 'git clean -i paints the header in HEADER color' '
>a.out &&
echo q |
- git -c color.ui=always clean -i |
+ test_terminal git clean -i |
test_decode_color |
head -n 1 >header &&
# not i18ngrep
test_expect_success 'verbose respects diff config' '
- test_config color.diff always &&
+ test_config diff.noprefix true &&
git status -v >actual &&
- grep "\[1mdiff --git" actual
+ grep "diff --git negative negative" actual
'
mesg_with_comment_and_newlines='
test_description='git status'
. ./test-lib.sh
+. "$TEST_DIRECTORY"/lib-terminal.sh
test_expect_success 'status -h in broken repository' '
git config --global advice.statusuoption false &&
'
-test_expect_success 'status with color.ui' '
+test_expect_success TTY 'status with color.ui' '
cat >expect <<\EOF &&
On branch <GREEN>master<RESET>
Your branch and '\''upstream'\'' have diverged,
<BLUE>untracked<RESET>
EOF
- test_config color.ui always &&
- git status | test_decode_color >output &&
+ test_config color.ui auto &&
+ test_terminal git status | test_decode_color >output &&
test_i18ncmp expect output
'
-test_expect_success 'status with color.status' '
- test_config color.status always &&
- git status | test_decode_color >output &&
+test_expect_success TTY 'status with color.status' '
+ test_config color.status auto &&
+ test_terminal git status | test_decode_color >output &&
test_i18ncmp expect output
'
<BLUE>??<RESET> untracked
EOF
-test_expect_success 'status -s with color.ui' '
+test_expect_success TTY 'status -s with color.ui' '
- git config color.ui always &&
- git status -s | test_decode_color >output &&
+ git config color.ui auto &&
+ test_terminal git status -s | test_decode_color >output &&
test_cmp expect output
'
-test_expect_success 'status -s with color.status' '
+test_expect_success TTY 'status -s with color.status' '
git config --unset color.ui &&
- git config color.status always &&
- git status -s | test_decode_color >output &&
+ git config color.status auto &&
+ test_terminal git status -s | test_decode_color >output &&
test_cmp expect output
'
<BLUE>??<RESET> untracked
EOF
-test_expect_success 'status -s -b with color.status' '
+test_expect_success TTY 'status -s -b with color.status' '
- git status -s -b | test_decode_color >output &&
+ test_terminal git status -s -b | test_decode_color >output &&
test_i18ncmp expect output
'
?? untracked
EOF
-test_expect_success 'status --porcelain ignores color.ui' '
+test_expect_success TTY 'status --porcelain ignores color.ui' '
git config --unset color.status &&
- git config color.ui always &&
- git status --porcelain | test_decode_color >output &&
+ git config color.ui auto &&
+ test_terminal git status --porcelain | test_decode_color >output &&
test_cmp expect output
'
-test_expect_success 'status --porcelain ignores color.status' '
+test_expect_success TTY 'status --porcelain ignores color.status' '
git config --unset color.ui &&
- git config color.status always &&
- git status --porcelain | test_decode_color >output &&
+ git config color.status auto &&
+ test_terminal git status --porcelain | test_decode_color >output &&
test_cmp expect output
'
if ($#ARGV < 1) {
die "usage: test-terminal program args";
}
+$ENV{TERM} = 'vt100';
my $master_in = new IO::Pty;
my $master_out = new IO::Pty;
my $master_err = new IO::Pty;