gitweb.git
update-index: stat updated files earlierJeff King Mon, 14 May 2018 15:00:56 +0000 (11:00 -0400)

update-index: stat updated files earlier

In the update_one(), we check verify_path() on the proposed
path before doing anything else. In preparation for having
verify_path() look at the file mode, let's stat the file
earlier, so we can check the mode accurately.

This is made a bit trickier by the fact that this function
only does an lstat in a few code paths (the ones that flow
down through process_path()). So we can speculatively do the
lstat() here and pass the results down, and just use a dummy
mode for cases where we won't actually be updating the index
from the filesystem.

Signed-off-by: Jeff King <peff@peff.net>

verify_dotfile: mention case-insensitivity in commentJeff King Tue, 15 May 2018 13:56:50 +0000 (09:56 -0400)

verify_dotfile: mention case-insensitivity in comment

We're more restrictive than we need to be in matching ".GIT"
on case-sensitive filesystems; let's make a note that this
is intentional.

Signed-off-by: Jeff King <peff@peff.net>

verify_path: drop clever fallthroughJeff King Sun, 13 May 2018 17:00:23 +0000 (13:00 -0400)

verify_path: drop clever fallthrough

We check ".git" and ".." in the same switch statement, and
fall through the cases to share the end-of-component check.
While this saves us a line or two, it makes modifying the
function much harder. Let's just write it out.

Signed-off-by: Jeff King <peff@peff.net>

skip_prefix: add case-insensitive variantJeff King Sun, 13 May 2018 16:57:14 +0000 (12:57 -0400)

skip_prefix: add case-insensitive variant

We have the convenient skip_prefix() helper, but if you want
to do case-insensitive matching, you're stuck doing it by
hand. We could add an extra parameter to the function to
let callers ask for this, but the function is small and
somewhat performance-critical. Let's just re-implement it
for the case-insensitive version.

Signed-off-by: Jeff King <peff@peff.net>

is_{hfs,ntfs}_dotgitmodules: add testsJohannes Schindelin Sat, 12 May 2018 20:16:51 +0000 (22:16 +0200)

is_{hfs,ntfs}_dotgitmodules: add tests

This tests primarily for NTFS issues, but also adds one example of an
HFS+ issue.

Thanks go to Congyi Wu for coming up with the list of examples where
NTFS would possibly equate the filename with `.gitmodules`.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Jeff King <peff@peff.net>

is_ntfs_dotgit: match other .git filesJohannes Schindelin Fri, 11 May 2018 14:03:54 +0000 (16:03 +0200)

is_ntfs_dotgit: match other .git files

When we started to catch NTFS short names that clash with .git, we only
looked for GIT~1. This is sufficient because we only ever clone into an
empty directory, so .git is guaranteed to be the first subdirectory or
file in that directory.

However, even with a fresh clone, .gitmodules is *not* necessarily the
first file to be written that would want the NTFS short name GITMOD~1: a
malicious repository can add .gitmodul0000 and friends, which sorts
before `.gitmodules` and is therefore checked out *first*. For that
reason, we have to test not only for ~1 short names, but for others,
too.

It's hard to just adapt the existing checks in is_ntfs_dotgit(): since
Windows 2000 (i.e., in all Windows versions still supported by Git),
NTFS short names are only generated in the <prefix>~<number> form up to
number 4. After that, a *different* prefix is used, calculated from the
long file name using an undocumented, but stable algorithm.

For example, the short name of .gitmodules would be GITMOD~1, but if it
is taken, and all of ~2, ~3 and ~4 are taken, too, the short name
GI7EBA~1 will be used. From there, collisions are handled by
incrementing the number, shortening the prefix as needed (until ~9999999
is reached, in which case NTFS will not allow the file to be created).

We'd also want to handle .gitignore and .gitattributes, which suffer
from a similar problem, using the fall-back short names GI250A~1 and
GI7D29~1, respectively.

To accommodate for that, we could reimplement the hashing algorithm, but
it is just safer and simpler to provide the known prefixes. This
algorithm has been reverse-engineered and described at
https://usn.pw/blog/gen/2015/06/09/filenames/, which is defunct but
still available via https://web.archive.org/.

These can be recomputed by running the following Perl script:

-- snip --
use warnings;
use strict;

sub compute_short_name_hash ($) {
my $checksum = 0;
foreach (split('', $_[0])) {
$checksum = ($checksum * 0x25 + ord($_)) & 0xffff;
}

$checksum = ($checksum * 314159269) & 0xffffffff;
$checksum = 1 + (~$checksum & 0x7fffffff) if ($checksum & 0x80000000);
$checksum -= (($checksum * 1152921497) >> 60) * 1000000007;

return scalar reverse sprintf("%x", $checksum & 0xffff);
}

print compute_short_name_hash($ARGV[0]);
-- snap --

E.g., running that with the argument ".gitignore" will
result in "250a" (which then becomes "gi250a" in the code).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Jeff King <peff@peff.net>

is_hfs_dotgit: match other .git filesJeff King Wed, 2 May 2018 19:23:45 +0000 (15:23 -0400)

is_hfs_dotgit: match other .git files

Both verify_path() and fsck match ".git", ".GIT", and other
variants specific to HFS+. Let's allow matching other
special files like ".gitmodules", which we'll later use to
enforce extra restrictions via verify_path() and fsck.

Signed-off-by: Jeff King <peff@peff.net>

is_ntfs_dotgit: use a size_t for traversing stringJeff King Sun, 13 May 2018 16:09:42 +0000 (12:09 -0400)

is_ntfs_dotgit: use a size_t for traversing string

We walk through the "name" string using an int, which can
wrap to a negative value and cause us to read random memory
before our array (e.g., by creating a tree with a name >2GB,
since "int" is still 32 bits even on most 64-bit platforms).
Worse, this is easy to trigger during the fsck_tree() check,
which is supposed to be protecting us from malicious
garbage.

Note one bit of trickiness in the existing code: we
sometimes assign -1 to "len" at the end of the loop, and
then rely on the "len++" in the for-loop's increment to take
it back to 0. This is still legal with a size_t, since
assigning -1 will turn into SIZE_MAX, which then wraps
around to 0 on increment.

Signed-off-by: Jeff King <peff@peff.net>

submodule-config: verify submodule names as pathsJeff King Mon, 30 Apr 2018 07:25:25 +0000 (03:25 -0400)

submodule-config: verify submodule names as paths

Submodule "names" come from the untrusted .gitmodules file,
but we blindly append them to $GIT_DIR/modules to create our
on-disk repo paths. This means you can do bad things by
putting "../" into the name (among other things).

Let's sanity-check these names to avoid building a path that
can be exploited. There are two main decisions:

1. What should the allowed syntax be?

It's tempting to reuse verify_path(), since submodule
names typically come from in-repo paths. But there are
two reasons not to:

a. It's technically more strict than what we need, as
we really care only about breaking out of the
$GIT_DIR/modules/ hierarchy. E.g., having a
submodule named "foo/.git" isn't actually
dangerous, and it's possible that somebody has
manually given such a funny name.

b. Since we'll eventually use this checking logic in
fsck to prevent downstream repositories, it should
be consistent across platforms. Because
verify_path() relies on is_dir_sep(), it wouldn't
block "foo\..\bar" on a non-Windows machine.

2. Where should we enforce it? These days most of the
.gitmodules reads go through submodule-config.c, so
I've put it there in the reading step. That should
cover all of the C code.

We also construct the name for "git submodule add"
inside the git-submodule.sh script. This is probably
not a big deal for security since the name is coming
from the user anyway, but it would be polite to remind
them if the name they pick is invalid (and we need to
expose the name-checker to the shell anyway for our
test scripts).

This patch issues a warning when reading .gitmodules
and just ignores the related config entry completely.
This will generally end up producing a sensible error,
as it works the same as a .gitmodules file which is
missing a submodule entry (so "submodule update" will
barf, but "git clone --recurse-submodules" will print
an error but not abort the clone.

There is one minor oddity, which is that we print the
warning once per malformed config key (since that's how
the config subsystem gives us the entries). So in the
new test, for example, the user would see three
warnings. That's OK, since the intent is that this case
should never come up outside of malicious repositories
(and then it might even benefit the user to see the
message multiple times).

Credit for finding this vulnerability and the proof of
concept from which the test script was adapted goes to
Etienne Stalmans.

Signed-off-by: Jeff King <peff@peff.net>

commit-graph: fix UX issue when .lock file existsDerrick Stolee Thu, 10 May 2018 17:42:52 +0000 (17:42 +0000)

commit-graph: fix UX issue when .lock file exists

We use the lockfile API to avoid multiple Git processes from writing to
the commit-graph file in the .git/objects/info directory. In some cases,
this directory may not exist, so we check for its existence.

The existing code does the following when acquiring the lock:

1. Try to acquire the lock.
2. If it fails, try to create the .git/object/info directory.
3. Try to acquire the lock, failing if necessary.

The problem is that if the lockfile exists, then the mkdir fails, giving
an error that doesn't help the user:

"fatal: cannot mkdir .git/objects/info: File exists"

While technically this honors the lockfile, it does not help the user.

Instead, do the following:

1. Check for existence of .git/objects/info; create if necessary.
2. Try to acquire the lock, failing if necessary.

The new output looks like:

fatal: Unable to create
'<dir>/.git/objects/info/commit-graph.lock': File exists.

Another git process seems to be running in this repository, e.g.
an editor opened by 'git commit'. Please make sure all processes
are terminated then try again. If it still fails, a git process
may have crashed in this repository earlier:
remove the file manually to continue.

Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

commit-graph.txt: update design documentDerrick Stolee Tue, 1 May 2018 12:47:25 +0000 (12:47 +0000)

commit-graph.txt: update design document

We now calculate generation numbers in the commit-graph file and use
them in paint_down_to_common().

Expand the section on generation numbers to discuss how the three
special generation numbers GENERATION_NUMBER_INFINITY, _ZERO, and
_MAX interact with other generation numbers.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

merge: check config before loading commitsDerrick Stolee Tue, 1 May 2018 12:47:23 +0000 (12:47 +0000)

merge: check config before loading commits

Now that we use generation numbers from the commit-graph, we must
ensure that all commits that exist in the commit-graph are loaded
from that file instead of from the object database. Since the
commit-graph file is only checked if core.commitGraph is true, we
must check the default config before we load any commits.

In the merge builtin, the config was checked after loading the HEAD
commit. This was due to the use of the global 'branch' when checking
merge-specific config settings.

Move the config load to be between the initialization of 'branch' and
the commit lookup.

Without this change, a fast-forward merge would hit a BUG("bad
generation skip") statement in commit.c during paint_down_to_common().
This is because the HEAD commit would be loaded with "infinite"
generation but then reached by commits with "finite" generation
numbers.

Add a test to t5318-commit-graph.sh that exercises this code path to
prevent a regression.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

commit: use generation number in remove_redundant()Derrick Stolee Tue, 1 May 2018 12:47:21 +0000 (12:47 +0000)

commit: use generation number in remove_redundant()

The static remove_redundant() method is used to filter a list
of commits by removing those that are reachable from another
commit in the list. This is used to remove all possible merge-
bases except a maximal, mutually independent set.

To determine these commits are independent, we use a number of
paint_down_to_common() walks and use the PARENT1, PARENT2 flags
to determine reachability. Since we only care about reachability
and not the full set of merge-bases between 'one' and 'twos', we
can use the 'min_generation' parameter to short-circuit the walk.

When no commit-graph exists, there is no change in behavior.

For a copy of the Linux repository, we measured the following
performance improvements:

git merge-base v3.3 v4.5

Before: 234 ms
After: 208 ms
Rel %: -11%

git merge-base v4.3 v4.5

Before: 102 ms
After: 83 ms
Rel %: -19%

The experiments above were chosen to demonstrate that we are
improving the filtering of the merge-base set. In the first
example, more time is spent walking the history to find the
set of merge bases before the remove_redundant() call. The
starting commits are closer together in the second example,
therefore more time is spent in remove_redundant(). The relative
change in performance differs as expected.

Reported-by: Jakub Narebski <jnareb@gmail.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

commit: add short-circuit to paint_down_to_common()Derrick Stolee Tue, 1 May 2018 12:47:19 +0000 (12:47 +0000)

commit: add short-circuit to paint_down_to_common()

When running 'git branch --contains', the in_merge_bases_many()
method calls paint_down_to_common() to discover if a specific
commit is reachable from a set of branches. Commits with lower
generation number are not needed to correctly answer the
containment query of in_merge_bases_many().

Add a new parameter, min_generation, to paint_down_to_common() that
prevents walking commits with generation number strictly less than
min_generation. If 0 is given, then there is no functional change.

For in_merge_bases_many(), we can pass commit->generation as the
cutoff, and this saves time during 'git branch --contains' queries
that would otherwise walk "around" the commit we are inspecting.

For a copy of the Linux repository, where HEAD is checked out at
v4.13~100, we get the following performance improvement for
'git branch --contains' over the previous commit:

Before: 0.21s
After: 0.13s
Rel %: -38%

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

commit: use generation numbers for in_merge_bases()Derrick Stolee Tue, 1 May 2018 12:47:17 +0000 (12:47 +0000)

commit: use generation numbers for in_merge_bases()

The containment algorithm for 'git branch --contains' is different
from that for 'git tag --contains' in that it uses is_descendant_of()
instead of contains_tag_algo(). The expensive portion of the branch
algorithm is computing merge bases.

When a commit-graph file exists with generation numbers computed,
we can avoid this merge-base calculation when the target commit has
a larger generation number than the initial commits.

Performance tests were run on a copy of the Linux repository where
HEAD is contained in v4.13 but no earlier tag. Also, all tags were
copied to branches and 'git branch --contains' was tested:

Before: 60.0s
After: 0.4s
Rel %: -99.3%

Reported-by: Jeff King <peff@peff.net>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

ref-filter: use generation number for --containsDerrick Stolee Tue, 1 May 2018 12:47:15 +0000 (12:47 +0000)

ref-filter: use generation number for --contains

A commit A can reach a commit B only if the generation number of A
is strictly larger than the generation number of B. This condition
allows significantly short-circuiting commit-graph walks.

Use generation number for '--contains' type queries.

On a copy of the Linux repository where HEAD is contained in v4.13
but no earlier tag, the command 'git tag --contains HEAD' had the
following peformance improvement:

Before: 0.81s
After: 0.04s
Rel %: -95%

Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

commit-graph: always load commit-graph informationDerrick Stolee Tue, 1 May 2018 12:47:13 +0000 (12:47 +0000)

commit-graph: always load commit-graph information

Most code paths load commits using lookup_commit() and then
parse_commit(). In some cases, including some branch lookups, the commit
is parsed using parse_object_buffer() which side-steps parse_commit() in
favor of parse_commit_buffer().

With generation numbers in the commit-graph, we need to ensure that any
commit that exists in the commit-graph file has its generation number
loaded.

Create new load_commit_graph_info() method to fill in the information
for a commit that exists only in the commit-graph file. Call it from
parse_commit_buffer() after loading the other commit information from
the given buffer. Only fill this information when specified by the
'check_graph' parameter.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

commit: use generations in paint_down_to_common()Derrick Stolee Tue, 1 May 2018 12:47:11 +0000 (12:47 +0000)

commit: use generations in paint_down_to_common()

Define compare_commits_by_gen_then_commit_date(), which uses generation
numbers as a primary comparison and commit date to break ties (or as a
comparison when both commits do not have computed generation numbers).

Since the commit-graph file is closed under reachability, we know that
all commits in the file have generation at most GENERATION_NUMBER_MAX
which is less than GENERATION_NUMBER_INFINITY.

This change does not affect the number of commits that are walked during
the execution of paint_down_to_common(), only the order that those
commits are inspected. In the case that commit dates violate topological
order (i.e. a parent is "newer" than a child), the previous code could
walk a commit twice: if a commit is reached with the PARENT1 bit, but
later is re-visited with the PARENT2 bit, then that PARENT2 bit must be
propagated to its parents. Using generation numbers avoids this extra
effort, even if it is somewhat rare.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

commit-graph: compute generation numbersDerrick Stolee Tue, 1 May 2018 12:47:09 +0000 (12:47 +0000)

commit-graph: compute generation numbers

While preparing commits to be written into a commit-graph file, compute
the generation numbers using a depth-first strategy.

The only commits that are walked in this depth-first search are those
without a precomputed generation number. Thus, computation time will be
relative to the number of new commits to the commit-graph file.

If a computed generation number would exceed GENERATION_NUMBER_MAX, then
use GENERATION_NUMBER_MAX instead.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

commit: add generation number to struct commitDerrick Stolee Wed, 25 Apr 2018 14:37:55 +0000 (14:37 +0000)

commit: add generation number to struct commit

The generation number of a commit is defined recursively as follows:

* If a commit A has no parents, then the generation number of A is one.
* If a commit A has parents, then the generation number of A is one
more than the maximum generation number among the parents of A.

Add a uint32_t generation field to struct commit so we can pass this
information to revision walks. We use three special values to signal
the generation number is invalid:

GENERATION_NUMBER_INFINITY 0xFFFFFFFF
GENERATION_NUMBER_MAX 0x3FFFFFFF
GENERATION_NUMBER_ZERO 0

The first (_INFINITY) means the generation number has not been loaded or
computed. The second (_MAX) means the generation number is too large to
store in the commit-graph file. The third (_ZERO) means the generation
number was loaded from a commit graph file that was written by a version
of git that did not support generation numbers.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

submodule: add --dissociate option to add/update commandsCasey Fitzpatrick Thu, 3 May 2018 10:53:46 +0000 (06:53 -0400)

submodule: add --dissociate option to add/update commands

Add --dissociate option to add and update commands, both clone helper commands
that already have the --reference option --dissociate pairs with.

Signed-off-by: Casey Fitzpatrick <kcghost@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

submodule: add --progress option to add commandCasey Fitzpatrick Thu, 3 May 2018 10:53:45 +0000 (06:53 -0400)

submodule: add --progress option to add command

The '--progress' was introduced in 72c5f88311d (clone: pass --progress
decision to recursive submodules, 2016-09-22) to fix the progress reporting
of the clone command. Also add the progress option to the 'submodule add'
command. The update command already supports the progress flag, but it
is not documented.

Signed-off-by: Casey Fitzpatrick <kcghost@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

submodule: clean up substitutions in scriptCasey Fitzpatrick Thu, 3 May 2018 10:53:44 +0000 (06:53 -0400)

submodule: clean up substitutions in script

'recommend_shallow' and 'jobs' variables do not need quotes. They only hold a
single token value, and even if they were multi-token it is likely we would want
them split at IFS rather than pass a single string.

'progress' is a boolean value. Treat it like the other boolean values in the
script by using a substitution.

Signed-off-by: Casey Fitzpatrick <kcghost@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

unpack_trees_options: free messages when doneMartin Ågren Mon, 21 May 2018 14:54:28 +0000 (16:54 +0200)

unpack_trees_options: free messages when done

The strings allocated in `setup_unpack_trees_porcelain()` are never
freed. Provide a function `clear_unpack_trees_porcelain()` to do so and
call it where we use `setup_unpack_trees_porcelain()`. The only
non-trivial user is `unpack_trees_start()`, where we should place the
new call in `unpack_trees_finish()`.

We keep the string pointers in an array, mixing pointers to static
memory and memory that we allocate on the heap. We also keep several
copies of the individual pointers. So we need to make sure that we do
not free what we must not free and that we do not double-free. Let a
separate argv_array take ownership of all the strings we create so that
we can easily free them.

Zero the whole array of string pointers to make sure that we do not
leave any dangling pointers.

Note that we only take responsibility for the memory allocated in
`setup_unpack_trees_porcelain()` and not any other members of the
`struct unpack_trees_options`.

Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

argv-array: return the pushed string from argv_push*()Junio C Hamano Mon, 21 May 2018 14:54:27 +0000 (16:54 +0200)

argv-array: return the pushed string from argv_push*()

Such an API change allows us to use an argv_array this way:

struct argv_array to_free = ARGV_ARRAY_INIT;
const char *msg;

if (some condition) {
msg = "constant string message";
... other logic ...
} else {
msg = argv_array_pushf(&to_free, "format %s", var);
}
... use "msg" ...
... do other things ...
argv_array_clear(&to_free);

Note that argv_array_pushl() and argv_array_pushv() are used to push
one or more strings with a single call, so we do not return any one
of these strings from these two functions in order to reduce the
chance to misuse the API.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

travis-ci: run gcc-8 on linux-gcc jobsNguyễn Thái Ngọc Duy Sat, 19 May 2018 04:32:34 +0000 (06:32 +0200)

travis-ci: run gcc-8 on linux-gcc jobs

Switch from gcc-4.8 to gcc-8. Newer compilers come with more warning
checks (usually in -Wextra). Since -Wextra is enabled in developer
mode (which is also enabled in travis), this lets travis report more
warnings before other people do it.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

commit.h: delete 'util' field in struct commitNguyễn Thái Ngọc Duy Sat, 19 May 2018 05:28:31 +0000 (07:28 +0200)

commit.h: delete 'util' field in struct commit

If you have come this far, you probably have seen that this 'util'
pointer is used for many different purposes. Some are not even
contained in a command code, but buried deep in common code with no
clue who will use it and how.

The move to using commit-slab gives us a much better picture of how
some piece of data is associated with a commit and what for. Since
nobody uses 'util' pointer anymore, we can retire so that nobody will
abuse it again. commit-slab will be the way forward for associating
data to a commit.

As a side benefit, this shrinks struct commit by 8 bytes (on 64-bit
architecture) which should help reduce memory usage for reachability
test a bit. This is also what commit-slab is invented for [1].

[1] 96c4f4a370 (commit: allow associating auxiliary info on-demand -
2013-04-09)

Signed-off-by: Junio C Hamano <gitster@pobox.com>

merge: use commit-slab in merge remote desc instead... Nguyễn Thái Ngọc Duy Sat, 19 May 2018 05:28:30 +0000 (07:28 +0200)

merge: use commit-slab in merge remote desc instead of commit->util

It's done so that commit->util can be removed. See more explanation in
the commit that removes commit->util.

Signed-off-by: Junio C Hamano <gitster@pobox.com>

log: use commit-slab in prepare_bases() instead of... Nguyễn Thái Ngọc Duy Sat, 19 May 2018 05:28:29 +0000 (07:28 +0200)

log: use commit-slab in prepare_bases() instead of commit->util

It's done so that commit->util can be removed. See more explanation in
the commit that removes commit->util.

Signed-off-by: Junio C Hamano <gitster@pobox.com>

show-branch: note about its object flags usageNguyễn Thái Ngọc Duy Sat, 19 May 2018 05:28:28 +0000 (07:28 +0200)

show-branch: note about its object flags usage

This is another candidate for commit-slab. Keep Junio's observation in
code so we can search it later on when somebody wants to improve the
code.

Signed-off-by: Junio C Hamano <gitster@pobox.com>

show-branch: use commit-slab for commit-name instead... Nguyễn Thái Ngọc Duy Sat, 19 May 2018 05:28:27 +0000 (07:28 +0200)

show-branch: use commit-slab for commit-name instead of commit->util

It's done so that commit->util can be removed. See more explanation in
the commit that removes commit->util.

Signed-off-by: Junio C Hamano <gitster@pobox.com>

name-rev: use commit-slab for rev-name instead of commi... Nguyễn Thái Ngọc Duy Sat, 19 May 2018 05:28:26 +0000 (07:28 +0200)

name-rev: use commit-slab for rev-name instead of commit->util

It's done so that commit->util can be removed. See more explanation in
the commit that removes commit->util.

Signed-off-by: Junio C Hamano <gitster@pobox.com>

bisect.c: use commit-slab for commit weight instead... Nguyễn Thái Ngọc Duy Sat, 19 May 2018 05:28:25 +0000 (07:28 +0200)

bisect.c: use commit-slab for commit weight instead of commit->util

It's done so that commit->util can be removed. See more explanation in
the commit that removes commit->util.

Signed-off-by: Junio C Hamano <gitster@pobox.com>

revision.c: use commit-slab for show_sourceNguyễn Thái Ngọc Duy Sat, 19 May 2018 05:28:24 +0000 (07:28 +0200)

revision.c: use commit-slab for show_source

Instead of relying on commit->util to store the source string, let the
user provide a commit-slab to store the source strings in.

It's done so that commit->util can be removed. See more explanation in
the commit that removes commit->util.

Signed-off-by: Junio C Hamano <gitster@pobox.com>

sequencer.c: use commit-slab to associate todo items... Nguyễn Thái Ngọc Duy Sat, 19 May 2018 05:28:23 +0000 (07:28 +0200)

sequencer.c: use commit-slab to associate todo items to commits

It's done so that commit->util can be removed. See more explanation in
the commit that removes commit->util.

Signed-off-by: Junio C Hamano <gitster@pobox.com>

sequencer.c: use commit-slab to mark seen commitsNguyễn Thái Ngọc Duy Sat, 19 May 2018 05:28:22 +0000 (07:28 +0200)

sequencer.c: use commit-slab to mark seen commits

It's done so that commit->util can be removed. See more explanation in
the commit that removes commit->util.

Signed-off-by: Junio C Hamano <gitster@pobox.com>

shallow.c: use commit-slab for commit depth instead... Nguyễn Thái Ngọc Duy Sat, 19 May 2018 05:28:21 +0000 (07:28 +0200)

shallow.c: use commit-slab for commit depth instead of commit->util

It's done so that commit->util can be removed. See more explanation in
the commit that removes commit->util.

While at there, plug a leak for keeping track of depth in this code.

Signed-off-by: Junio C Hamano <gitster@pobox.com>

describe: use commit-slab for commit names instead... Nguyễn Thái Ngọc Duy Sat, 19 May 2018 05:28:20 +0000 (07:28 +0200)

describe: use commit-slab for commit names instead of commit->util

It's done so that commit->util can be removed. See more explanation in
the commit that removes commit->util.

Signed-off-by: Junio C Hamano <gitster@pobox.com>

blame: use commit-slab for blame suspects instead of... Nguyễn Thái Ngọc Duy Sat, 19 May 2018 05:28:19 +0000 (07:28 +0200)

blame: use commit-slab for blame suspects instead of commit->util

It's done so that commit->util can be removed. See more explanation in
the commit that removes commit->util.

Signed-off-by: Junio C Hamano <gitster@pobox.com>

commit-slab: support shared commit-slabNguyễn Thái Ngọc Duy Sat, 19 May 2018 05:28:18 +0000 (07:28 +0200)

commit-slab: support shared commit-slab

define_shared_commit_slab() could be used in a header file to define a
commit-slab. One of these C files must include commit-slab-impl.h and
"call" implement_shared_commit_slab().

Signed-off-by: Junio C Hamano <gitster@pobox.com>

commit-slab.h: code splitNguyễn Thái Ngọc Duy Sat, 19 May 2018 05:28:17 +0000 (07:28 +0200)

commit-slab.h: code split

The struct declaration and implementation macros are moved to
commit-slab-hdr.h and commit-slab-impl.h respectively.

This right now is not needed for current users but if we make a public
commit-slab type, we may want to avoid including the slab
implementation in a header file which gets replicated in every c file
that includes it.

Signed-off-by: Junio C Hamano <gitster@pobox.com>

regex: do not call `regfree()` if compilation failsMartin Ågren Sun, 20 May 2018 10:50:32 +0000 (12:50 +0200)

regex: do not call `regfree()` if compilation fails

It is apparently undefined behavior to call `regfree()` on a regex where
`regcomp()` failed. The language in [1] is a bit muddy, at least to me,
but the clearest hint is this (`preg` is the `regex_t *`):

Upon successful completion, the regcomp() function shall return 0.
Otherwise, it shall return an integer value indicating an error as
described in <regex.h>, and the content of preg is undefined.

Funnily enough, there is also the `regerror()` function which should be
given a pointer to such a "failed" `regex_t` -- the content of which
would supposedly be undefined -- and which may investigate it to come up
with a detailed error message.

In any case, the example in that document shows how `regfree()` is not
called after `regcomp()` fails.

We have quite a few users of this API and most get this right. These
three users do not.

Several implementations can handle this just fine [2] and these code paths
supposedly have not wreaked havoc or we'd have heard about it. (These
are all in code paths where git got bad input and is just about to die
anyway.) But let's just avoid the issue altogether.

[1] http://pubs.opengroup.org/onlinepubs/9699919799/functions/regcomp.html

[2] https://www.redhat.com/archives/libvir-list/2013-September/msg00262.html

Researched-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-byi Martin Ågren <martin.agren@gmail.com>

Signed-off-by: Junio C Hamano <gitster@pobox.com>

config: let `config_store_data_clear()` handle `key`Martin Ågren Sun, 20 May 2018 10:42:35 +0000 (12:42 +0200)

config: let `config_store_data_clear()` handle `key`

Instead of remembering to free `key` in each code path, let
`config_store_data_clear()` handle that.

We still need to free it before replacing it, though. Move that freeing
closer to the replacing to be safe. Note that in that same part of the
code, we can no longer set `key` to the original pointer, but need to
`xstrdup()` it.

Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

config: let `config_store_data_clear()` handle `value_r... Martin Ågren Sun, 20 May 2018 10:42:34 +0000 (12:42 +0200)

config: let `config_store_data_clear()` handle `value_regex`

Instead of duplicating the logic for clearing up `value_regex`, let
`config_store_data_clear()` handle that.

When `regcomp()` fails, the current code does not call `regfree()`. Make
sure we do the same by immediately invalidating `value_regex`. Some
implementations are able to handle such an extra `regfree()`-call [1],
but from the example in [2], we should not do so. (The language itself
in [2] is not super-clear on this.)

[1] https://www.redhat.com/archives/libvir-list/2013-September/msg00262.html

[2] http://pubs.opengroup.org/onlinepubs/9699919799/functions/regcomp.html

Researched-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

config: free resources of `struct config_store_data`Martin Ågren Sun, 20 May 2018 10:42:33 +0000 (12:42 +0200)

config: free resources of `struct config_store_data`

Commit fee8572c6d (config: avoid using the global variable `store`,
2018-04-09) dropped the staticness of a certain struct, instead letting
the users create an instance on the stack and pass around a pointer.

We do not free all the memory that the struct tracks. When the struct
was static, the memory would always be reachable. Now that we keep the
struct on the stack, though, as soon as we return, it goes out of scope
and we leak the memory it points to. In particular, we leak the memory
pointed to by the `parsed` and `seen` fields.

Introduce and use a helper function `config_store_data_clear()` to plug
these leaks. The memory tracked here is config parser events. Once the
users (`git_config_set_multivar_in_file_gently()` and
`git_config_copy_or_rename_section_in_file()` at the moment) are done,
no-one should be holding on to a pointer into this memory.

There are two more members of the struct that are candidates for freeing
in this new function (`key` and `value_regex`). Those are actually
already being taken care of. The next couple of patches will move their
freeing into the function we are adding here.

Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t5300: abstract away SHA-1-specific constantsbrian m. carlson Mon, 21 May 2018 02:01:47 +0000 (02:01 +0000)

t5300: abstract away SHA-1-specific constants

Adjust the test so that it computes variables for object IDs instead of
using hard-coded hashes.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t4208: abstract away SHA-1-specific constantsbrian m. carlson Mon, 21 May 2018 02:01:46 +0000 (02:01 +0000)

t4208: abstract away SHA-1-specific constants

Adjust the test so that it computes variables for object IDs instead of
using hard-coded hashes.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t4045: abstract away SHA-1-specific constantsbrian m. carlson Mon, 21 May 2018 02:01:45 +0000 (02:01 +0000)

t4045: abstract away SHA-1-specific constants

Adjust the test so that it computes variables for blobs instead of using
hard-coded hashes.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t4042: abstract away SHA-1-specific constantsbrian m. carlson Mon, 21 May 2018 02:01:44 +0000 (02:01 +0000)

t4042: abstract away SHA-1-specific constants

Adjust the test so that it computes variables for blobs instead of using
hard-coded hashes.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t4205: sort log output in a hash-independent waybrian m. carlson Mon, 21 May 2018 02:01:43 +0000 (02:01 +0000)

t4205: sort log output in a hash-independent way

This test enumerates log entries and then sorts them. For SHA-1, this
produces results that happen to sort in the order specified in the test,
but for other hash algorithms they sort differently. Ensure we sort the
log entries in a hash-independent way by sorting on the ref name instead
of the object ID.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t/lib-diff-alternative: abstract away SHA-1-specific... brian m. carlson Mon, 21 May 2018 02:01:42 +0000 (02:01 +0000)

t/lib-diff-alternative: abstract away SHA-1-specific constants

Adjust the test code so that it computes variables for blobs instead of
using hard-coded hashes. This makes t4033 and t4050 (the patience and
histogram tests) pass.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t4030: abstract away SHA-1-specific constantsbrian m. carlson Mon, 21 May 2018 02:01:41 +0000 (02:01 +0000)

t4030: abstract away SHA-1-specific constants

Adjust the test so that it computes variables for blobs instead of using
hard-coded hashes.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t4029: abstract away SHA-1-specific constantsbrian m. carlson Mon, 21 May 2018 02:01:40 +0000 (02:01 +0000)

t4029: abstract away SHA-1-specific constants

Adjust the test so that it computes variables for blobs instead of using
hard-coded hashes.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t4029: fix test indentationbrian m. carlson Mon, 21 May 2018 02:01:39 +0000 (02:01 +0000)

t4029: fix test indentation

We typically indent our tests with a single tab, partially so that we
can take advantage of indented heredocs. Make this change and move the
quote marks to be in the typical position for our tests.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t4022: abstract away SHA-1-specific constantsbrian m. carlson Mon, 21 May 2018 02:01:38 +0000 (02:01 +0000)

t4022: abstract away SHA-1-specific constants

Adjust the test so that it computes variables for blobs instead of using
hard-coded hashes.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t4020: abstract away SHA-1-specific constantsbrian m. carlson Mon, 21 May 2018 02:01:37 +0000 (02:01 +0000)

t4020: abstract away SHA-1-specific constants

Adjust the test so that it computes variables for blobs instead of using
hard-coded hashes.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t4014: abstract away SHA-1-specific constantsbrian m. carlson Mon, 21 May 2018 02:01:36 +0000 (02:01 +0000)

t4014: abstract away SHA-1-specific constants

Adjust the test so that it computes values for blobs instead of using
hard-coded hashes.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t4008: abstract away SHA-1-specific constantsbrian m. carlson Mon, 21 May 2018 02:01:35 +0000 (02:01 +0000)

t4008: abstract away SHA-1-specific constants

Adjust the test so that it computes variables for blobs instead of using
hard-coded hashes.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t4007: abstract away SHA-1-specific constantsbrian m. carlson Mon, 21 May 2018 02:01:34 +0000 (02:01 +0000)

t4007: abstract away SHA-1-specific constants

Adjust the test so that it computes variables for blobs and uses the
ZERO_OID variable instead of using hard-coded hashes.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t3905: abstract away SHA-1-specific constantsbrian m. carlson Mon, 21 May 2018 02:01:33 +0000 (02:01 +0000)

t3905: abstract away SHA-1-specific constants

Adjust the test so that it computes variables for blobs instead of using
hard-coded hashes.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t3702: abstract away SHA-1-specific constantsbrian m. carlson Mon, 21 May 2018 02:01:32 +0000 (02:01 +0000)

t3702: abstract away SHA-1-specific constants

Strip out the index lines in the diff before comparing them, as these
will differ between hash algorithms. This leads to a smaller, simpler
change than editing the index line.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

completion: allow to customize the completable command... Nguyễn Thái Ngọc Duy Sun, 20 May 2018 18:40:09 +0000 (20:40 +0200)

completion: allow to customize the completable command list

By default we show porcelain, external commands and a couple others
that are also popular. If you are not happy with this list, you can
now customize it a new config variable.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

completion: add and use --list-cmds=aliasNguyễn Thái Ngọc Duy Sun, 20 May 2018 18:40:08 +0000 (20:40 +0200)

completion: add and use --list-cmds=alias

By providing aliases via --list-cmds=, we could simplify command
collection code in the script. We only issue one git command. Before
this patch that is "git config", after it's "git --list-cmds=". In
"git help" completion case we actually reduce one "git" process (for
getting guides) but that call was added in this series so it does not
really count.

A couple of bash functions are removed because they are not needed
anymore. __git_compute_all_commands() and $__git_all_commands stay
because they are still needed for completing pager.* config and
without "alias" group, the result is still cacheable.

There is a slight (good) change in _git_help() with this patch: before
"git help <tab>" shows external commands (as in _not_ part of git) as
well as part of $__git_all_commands. We have finer control over
command listing now and can exclude that because we can't provide a
man page for external commands anyway.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

completion: add and use --list-cmds=nohelpersNguyễn Thái Ngọc Duy Sun, 20 May 2018 18:40:07 +0000 (20:40 +0200)

completion: add and use --list-cmds=nohelpers

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Move declaration for alias.c to alias.hNguyễn Thái Ngọc Duy Sun, 20 May 2018 18:40:06 +0000 (20:40 +0200)

Move declaration for alias.c to alias.h

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

completion: reduce completable command listNguyễn Thái Ngọc Duy Sun, 20 May 2018 18:40:05 +0000 (20:40 +0200)

completion: reduce completable command list

The following commands are removed from the complete list:

- annotate obsolete, discouraged to use
- filter-branch not often used
- get-tar-commit-id not often used
- imap-send not often used
- interpreter-trailers not for interactive use
- name-rev plumbing, just use git-describe
- p4 too short and probably not often used (*)
- svn same category as p4 (*)
- verify-commit not often used

(*) to be fair, send-email command which is in the same foreignscminterface
group as svn and p4 does get completion, just because it's used by git
and kernel development. So maybe we should include them.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

completion: let git provide the completable command... Nguyễn Thái Ngọc Duy Sun, 20 May 2018 18:40:04 +0000 (20:40 +0200)

completion: let git provide the completable command list

Instead of maintaining a separate list of command classification,
which often could go out of date, let's centralize the information
back in git.

While the function in git-completion.bash implies "list porcelain
commands", that's not exactly what it does. It gets all commands (aka
--list-cmds=main,others) then exclude certain non-porcelain ones. We
could almost recreate this list two lists list-mainporcelain and
others. The non-porcelain-but-included-anyway is added by the third
category list-complete.

Note that the current completion script incorrectly classifies
filter-branch as porcelain and t9902 tests this behavior. We keep it
this way in t9902 because this test does not really care which
particular command is porcelain or plumbing, they're just names.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

command-list.txt: documentation and guide lineNguyễn Thái Ngọc Duy Sun, 20 May 2018 18:40:03 +0000 (20:40 +0200)

command-list.txt: documentation and guide line

This is intended to help anybody who needs to update command-list.txt.
It gives a brief introduction of all attributes a command can take.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

help: use command-list.txt for the source of guidesNguyễn Thái Ngọc Duy Sun, 20 May 2018 18:40:02 +0000 (20:40 +0200)

help: use command-list.txt for the source of guides

The help command currently hard codes the list of guides and their
summary in C. Let's move this list to command-list.txt. This lets us
extract summary lines from Documentation/git*.txt. This also
potentially lets us list guides in git.txt, but I'll leave that for
now.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

help: add "-a --verbose" to list all commands with... Nguyễn Thái Ngọc Duy Sun, 20 May 2018 18:40:01 +0000 (20:40 +0200)

help: add "-a --verbose" to list all commands with synopsis

This lists all recognized commands [1] by category. The group order
follows closely git.txt.

[1] We may actually show commands that are not built (e.g. if you set
NO_PERL you don't have git-instaweb but it's still listed here). I
ignore the problem because on Linux a git package could be split
anyway. The "git-core" package may not contain git-instaweb even if
it's built because it may end up in a separate package. We can't know
anyway.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

git: support --list-cmds=list-<category>Nguyễn Thái Ngọc Duy Sun, 20 May 2018 18:40:00 +0000 (20:40 +0200)

git: support --list-cmds=list-<category>

This allows us to select any group of commands by a category defined
in command-list.txt. This is an internal/hidden option so we don't
have to be picky about the category name or worried about exposing too
much.

This will be used later by git-completion.bash to retrieve certain
command groups.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

completion: implement and use --list-cmds=main,othersNguyễn Thái Ngọc Duy Sun, 20 May 2018 18:39:59 +0000 (20:39 +0200)

completion: implement and use --list-cmds=main,others

This is part of the effort to break down and provide commands by
category in machine-readable form. This could be helpful later on when
completion script switches to use --list-cmds for selecting
completable commands. It would be much easier for the user to choose
to complete _all_ commands instead of the default selection by passing
different values to --list-cmds in git-completino.bash.

While at there, replace "git help -a" in git-completion.bash with
--list-cmds since it's better suited for this task.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

git --list-cmds: collect command list in a string_listNguyễn Thái Ngọc Duy Sun, 20 May 2018 18:39:58 +0000 (20:39 +0200)

git --list-cmds: collect command list in a string_list

Instead of printing the command directly one by one, keep them in a
list and print at the end. This allows more modification before we
print out (e.g. sorting, removing duplicates or even excluding some
items).

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

git.c: convert --list-* to --list-cmds=*Nguyễn Thái Ngọc Duy Sun, 20 May 2018 18:39:57 +0000 (20:39 +0200)

git.c: convert --list-* to --list-cmds=*

Even if these are hidden options, let's make them a bit more generic
since we're introducing more listing types shortly. The code is
structured to allow combining multiple listing types together because
we will soon add more types the 'builtins'.

'parseopt' remains separate because it has separate (SPC) to match
git-completion.bash needs and will not combine with others.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

fmt_with_err: add a comment that truncation is OKJeff King Sat, 19 May 2018 01:58:44 +0000 (18:58 -0700)

fmt_with_err: add a comment that truncation is OK

Functions like die_errno() use fmt_with_err() to combine the
caller-provided format with the strerror() string. We use a
fixed stack buffer because we're already handling an error
and don't have any way to report another one. Our buffer
should generally be big enough to fit this, but if it's not,
truncation is our best option. Let's add a comment to that
effect, so that anybody auditing the code for truncation
bugs knows that this is fine.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

shorten_unambiguous_ref: use xsnprintfJeff King Sat, 19 May 2018 01:58:20 +0000 (18:58 -0700)

shorten_unambiguous_ref: use xsnprintf

We convert the ref_rev_parse_rules array into scanf formats
on the fly, and use snprintf() to write into each string. We
should have enough memory to hold everything because of the
earlier total_len computation. Let's use xsnprintf() to
give runtime confirmation that this is the case, and to make
it easy for people auditing the code to know there's no
truncation bug.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

fsmonitor: use internal argv_array of struct child_processRené Scharfe Sat, 19 May 2018 08:27:46 +0000 (10:27 +0200)

fsmonitor: use internal argv_array of struct child_process

Avoid magic array sizes and indexes by constructing the fsmonitor
command line using the embedded argv_array of the child_process. The
resulting code is shorter and easier to extend.

Getting rid of the snprintf() calls is a bonus -- even though the
buffers were big enough here to avoid truncation -- as it makes auditing
the remaining callers easier.

Inspired-by: Jeff King <peff@peff.net>
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

log_write_email_headers: use strbufsJeff King Sat, 19 May 2018 01:57:44 +0000 (18:57 -0700)

log_write_email_headers: use strbufs

When we write a MIME attachment, we write the mime headers
into fixed-size buffers. These are likely to be big enough
in practice, but technically the input could be arbitrarily
large (e.g., if the caller provided a lot of content in the
extra_headers string), in which case we'd quietly truncate
it and generate bogus output. Let's convert these buffers to
strbufs.

The memory ownership here is a bit funny. The original fixed
buffers were static, and we merely pass out pointers to them
to be used by the caller (and in one case, we even just
stuff our value into the opt->diffopt.stat_sep value).
Ideally we'd actually pass back heap buffers, and the caller
would be responsible for freeing them.

This patch punts on that cleanup for now, and instead just
marks the strbufs as static. That means we keep ownership in
this function, making it not a complete leak. This also
takes us one step closer to fixing it in the long term
(since we can eventually use strbuf_detach() to hand
ownership to the caller, once it's ready).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

http: use strbufs instead of fixed buffersJeff King Sat, 19 May 2018 01:56:37 +0000 (18:56 -0700)

http: use strbufs instead of fixed buffers

We keep the names of incoming packs and objects in fixed
PATH_MAX-size buffers, and snprintf() into them. This is
unlikely to end up with truncated filenames, but it is
possible (especially on systems where PATH_MAX is shorter
than actual paths can be). Let's switch to using strbufs,
which makes the question go away entirely.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

config: die when --blob is used outside a repositoryJeff King Fri, 18 May 2018 22:27:04 +0000 (15:27 -0700)

config: die when --blob is used outside a repository

If you run "config --blob" outside of a repository, then we
eventually try to resolve the blob name and hit a BUG().
Let's catch this earlier and provide a useful message.

Note that we could also catch this much lower in the stack,
in git_config_from_blob_ref(). That might cover other
callsites, too, but it's unclear whether those ones would
actually be bugs or not. So let's leave the low-level
functions to assume the caller knows what it's doing (and
BUG() if it turns out it doesn't).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

get_main_ref_store: BUG() when outside a repositoryJeff King Fri, 18 May 2018 22:25:53 +0000 (15:25 -0700)

get_main_ref_store: BUG() when outside a repository

If we don't have a repository, then we can't initialize the
ref store. Prior to 64a741619d (refs: store the main ref
store inside the repository struct, 2018-04-11), we'd try to
access get_git_dir(), and outside a repository that would
trigger a BUG(). After that commit, though, we directly use
the_repository->git_dir; if it's NULL we'll just segfault.

Let's catch this case and restore the BUG() behavior.
Obviously we don't ever want to hit this code, but a BUG()
is a lot more helpful than a segfault if we do.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t9902-completion: exercise __git_complete_index_file... SZEDER Gábor Fri, 18 May 2018 14:17:51 +0000 (16:17 +0200)

t9902-completion: exercise __git_complete_index_file() directly

The tests added in 2f271cd9cf (t9902-completion: add tests
demonstrating issues with quoted pathnames, 2018-05-08) and in
2ab6eab4fe (completion: improve handling quoted paths in 'git
ls-files's output, 2018-03-28) have a few shortcomings:

- All these tests use the 'test_completion' helper function, thus
they are exercising the whole completion machinery, although they
are only interested in how git-aware path completion, specifically
the __git_complete_index_file() function deals with unusual
characters in pathnames and on the command line.

- These tests can't satisfactorily test the case of pathnames
containing spaces, because 'test_completion' gets the words on the
command line as a single argument and it uses space as word
separator.

- Some of the tests are protected by different FUNNYNAMES_* prereqs
depending on whether they put backslashes and double quotes or
separator characters (FS, GS, RS, US) in pathnames, although a
filesystem not allowing one likely doesn't allow the others
either.

- One of the tests operates on paths containing '|' and '&'
characters without being protected by a FUNNYNAMES prereq, but
some filesystems (notably on Windows) don't allow these characters
in pathnames, either.

Replace these tests with basically equivalent, more focused tests that
call __git_complete_index_file() directly. Since this function only
looks at the current word to be completed, i.e. the $cur variable, we
can easily include pathnames containing spaces in the tests, so use
such pathnames instead of pathnames containing '|' and '&'. Finally,
use only a single FUNNYNAMES prereq for all kinds of special
characters.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

completion: don't return with error from __gitcomp_file... SZEDER Gábor Fri, 18 May 2018 14:17:50 +0000 (16:17 +0200)

completion: don't return with error from __gitcomp_file_direct()

In __gitcomp_file_direct() we tell Bash that it should handle our
possible completion words as filenames with the following piece of
cleverness:

# use a hack to enable file mode in bash < 4
compopt -o filenames +o nospace 2>/dev/null ||
compgen -f /non-existing-dir/ > /dev/null

Unfortunately, this makes this function always return with error
when it is not invoked in real completion, but e.g. in tests of
't9902-completion.sh':

- First the 'compopt' line errors out
- either because in Bash v3.x there is no such command,
- or because in Bash v4.x it complains about "not currently
executing completion function",

- then 'compgen' just silently returns with error because of the
non-existing directory.

Since __gitcomp_file_direct() is now the last command executed in
__git_complete_index_file(), that function returns with error as well,
which prevents it from being invoked in tests directly as is, and
would require extra steps in test to hide its error code.

So let's make sure that __gitcomp_file_direct() doesn't return with
error, because in the tests coming in the following patch we do want
to exercise __git_complete_index_file() directly,

__gitcomp_file() contains the same construct, and thus it, too, always
returns with error. Update that function accordingly as well.

While at it, also remove the space from between the redirection
operator and the filename in both functions.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

merge-recursive: provide pair of `unpack_trees_{start... Elijah Newren Sun, 20 May 2018 10:17:35 +0000 (12:17 +0200)

merge-recursive: provide pair of `unpack_trees_{start,finish}()`

Rename `git_merge_trees()` to `unpack_trees_start()` and extract the
call to `discard_index()` into a new function `unpack_trees_finish()`.
As a result, these are called early resp. late in `merge_trees()`,
making the resource handling clearer. A later commit will expand on
that, teaching `..._finish()` to free more memory. (So rather than
moving the FIXME-comment, just drop it, since it will be addressed soon
enough.)

Also call `..._finish()` when `merge_trees()` returns early.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

merge: setup `opts` later in `checkout_fast_forward()`Martin Ågren Sun, 20 May 2018 10:17:34 +0000 (12:17 +0200)

merge: setup `opts` later in `checkout_fast_forward()`

After we initialize the various fields in `opts` but before we actually
use them, we might return early. Move the initialization further down,
to immediately before we use `opts`.

This limits the scope of `opts` and will help a later commit fix a
memory leak without having to worry about those early returns.

This patch is best viewed using something like this (note the tab!):
--color-moved --anchored=" trees[nr_trees] = parse_tree_indirect"

Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

commit: allow lookup_commit_graft to handle arbitrary... Stefan Beller Thu, 17 May 2018 22:51:54 +0000 (15:51 -0700)

commit: allow lookup_commit_graft to handle arbitrary repositories

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

commit: allow prepare_commit_graft to handle arbitrary... Stefan Beller Thu, 17 May 2018 22:51:53 +0000 (15:51 -0700)

commit: allow prepare_commit_graft to handle arbitrary repositories

Move the global variable 'commit_graft_prepared' into the object
pool and convert the function prepare_commit_graft to work
an arbitrary repositories.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

shallow: migrate shallow information into the object... Stefan Beller Thu, 17 May 2018 22:51:52 +0000 (15:51 -0700)

shallow: migrate shallow information into the object parser

We need to convert the shallow functions all at the same time
as we move the data structures they operate on into the repository.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

path.c: migrate global git_path_* to take a repository... Stefan Beller Thu, 17 May 2018 22:51:51 +0000 (15:51 -0700)

path.c: migrate global git_path_* to take a repository argument

Migrate all git_path_* functions that are defined in path.c to take a
repository argument. Unlike other patches in this series, do not use the
#define trick, as we rewrite the whole function, which is rather small.

This doesn't migrate all the functions, as other builtins have their own
local path functions defined using GIT_PATH_FUNC. So keep that macro
around to serve the other locations.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

cache: convert get_graft_file to handle arbitrary repos... Stefan Beller Thu, 17 May 2018 22:51:50 +0000 (15:51 -0700)

cache: convert get_graft_file to handle arbitrary repositories

This conversion was done without the #define trick used in the earlier
series refactoring to have better repository access, because this function
is easy to review, as all lines are converted and it has only one caller.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

commit: convert read_graft_file to handle arbitrary... Brandon Williams Thu, 17 May 2018 22:51:49 +0000 (15:51 -0700)

commit: convert read_graft_file to handle arbitrary repositories

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

commit: convert register_commit_graft to handle arbitra... Brandon Williams Thu, 17 May 2018 22:51:48 +0000 (15:51 -0700)

commit: convert register_commit_graft to handle arbitrary repositories

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

commit: convert commit_graft_pos() to handle arbitrary... Brandon Williams Thu, 17 May 2018 22:51:47 +0000 (15:51 -0700)

commit: convert commit_graft_pos() to handle arbitrary repositories

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

shallow: add repository argument to is_repository_shallowStefan Beller Thu, 17 May 2018 22:51:46 +0000 (15:51 -0700)

shallow: add repository argument to is_repository_shallow

Add a repository argument to allow callers of is_repository_shallow
to be more specific about which repository to handle. This is a small
mechanical change; it doesn't change the implementation to handle
repositories other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a
repository other than the_repository at compile time.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

shallow: add repository argument to check_shallow_file_... Stefan Beller Thu, 17 May 2018 22:51:45 +0000 (15:51 -0700)

shallow: add repository argument to check_shallow_file_for_update

Add a repository argument to allow callers of check_shallow_file_for_update
to be more specific about which repository to handle. This is a small
mechanical change; it doesn't change the implementation to handle
repositories other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a
repository other than the_repository at compile time.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

shallow: add repository argument to register_shallowStefan Beller Thu, 17 May 2018 22:51:44 +0000 (15:51 -0700)

shallow: add repository argument to register_shallow

Add a repository argument to allow callers of register_shallow
to be more specific about which repository to handle. This is a small
mechanical change; it doesn't change the implementation to handle
repositories other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a
repository other than the_repository at compile time.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

shallow: add repository argument to set_alternate_shall... Stefan Beller Thu, 17 May 2018 22:51:43 +0000 (15:51 -0700)

shallow: add repository argument to set_alternate_shallow_file

Add a repository argument to allow callers of set_alternate_shallow_file
to be more specific about which repository to handle. This is a small
mechanical change; it doesn't change the implementation to handle
repositories other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a
repository other than the_repository at compile time.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

commit: add repository argument to lookup_commit_graftJonathan Nieder Thu, 17 May 2018 22:51:42 +0000 (15:51 -0700)

commit: add repository argument to lookup_commit_graft

Add a repository argument to allow callers of lookup_commit_graft to
be more specific about which repository to handle. This is a small
mechanical change; it doesn't change the implementation to handle
repositories other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a
repository other than the_repository at compile time.

Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

config: a user-provided invalid section is not a BUGJohannes Schindelin Thu, 17 May 2018 21:47:06 +0000 (23:47 +0200)

config: a user-provided invalid section is not a BUG

This was pointed out by Jeff King while the empty-config-section-fix
patch series was cooking, and was not addressed in time for that patch
series to advance to `master`.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

merge-recursive: give notice when submodule commit... Leif Middelschulte Thu, 17 May 2018 18:40:08 +0000 (11:40 -0700)

merge-recursive: give notice when submodule commit gets fast-forwarded

Inform the user about an automatically fast-forwarded submodule. The
silent merge behavior was introduced by commit 68d03e4a6e44 ("Implement
automatic fast-forward merge for submodules", 2010-07-07)).

Signed-off-by: Leif Middelschulte <Leif.Middelschulte@gmail.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>