gitweb.git
t7063: tests for untracked cacheNguyễn Thái Ngọc Duy Sun, 8 Mar 2015 10:12:44 +0000 (17:12 +0700)

t7063: tests for untracked cache

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

update-index: test the system before enabling untracked... Nguyễn Thái Ngọc Duy Sun, 8 Mar 2015 10:12:43 +0000 (17:12 +0700)

update-index: test the system before enabling untracked cache

Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

update-index: manually enable or disable untracked... Nguyễn Thái Ngọc Duy Sun, 8 Mar 2015 10:12:42 +0000 (17:12 +0700)

update-index: manually enable or disable untracked cache

Overall time saving on "git status" is about 40% in the best case
scenario, removing ..collect_untracked() as the most time consuming
function. read and refresh index operations are now at the top (which
should drop when index-helper and/or watchman support is added). More
numbers and analysis below.

webkit.git
==========

169k files. 6k dirs. Lots of test data (i.e. not touched most of the
time)

Base status
-----------

Index version 4 in split index mode and cache-tree populated. No
untracked cache. It shows how time is consumed by "git status". The
same settings are used for other repos below.

18:28:10.199679 builtin/commit.c:1394 performance: 0.000000451 s: cmd_status:setup
18:28:10.474847 read-cache.c:1407 performance: 0.274873831 s: read_index
18:28:10.475295 read-cache.c:1407 performance: 0.000000656 s: read_index
18:28:10.728443 preload-index.c:131 performance: 0.253147487 s: read_index_preload
18:28:10.741422 read-cache.c:1254 performance: 0.012868340 s: refresh_index
18:28:10.752300 wt-status.c:623 performance: 0.010421357 s: wt_status_collect_changes_worktree
18:28:10.762069 wt-status.c:629 performance: 0.009644748 s: wt_status_collect_changes_index
18:28:11.601019 wt-status.c:632 performance: 0.838859547 s: wt_status_collect_untracked
18:28:11.605939 builtin/commit.c:1421 performance: 0.004835004 s: cmd_status:update_index
18:28:11.606580 trace.c:415 performance: 1.407878388 s: git command: 'git' 'status'

Populating status
-----------------

This is after enabling untracked cache and the cache is still empty.
We see a slight increase in .._collect_untracked() and update_index
(because new cache has to be written to $GIT_DIR/index).

18:28:18.915213 builtin/commit.c:1394 performance: 0.000000326 s: cmd_status:setup
18:28:19.197364 read-cache.c:1407 performance: 0.281901416 s: read_index
18:28:19.197754 read-cache.c:1407 performance: 0.000000546 s: read_index
18:28:19.451355 preload-index.c:131 performance: 0.253599607 s: read_index_preload
18:28:19.464400 read-cache.c:1254 performance: 0.012935336 s: refresh_index
18:28:19.475115 wt-status.c:623 performance: 0.010236920 s: wt_status_collect_changes_worktree
18:28:19.486022 wt-status.c:629 performance: 0.010801685 s: wt_status_collect_changes_index
18:28:20.362660 wt-status.c:632 performance: 0.876551366 s: wt_status_collect_untracked
18:28:20.396199 builtin/commit.c:1421 performance: 0.033447969 s: cmd_status:update_index
18:28:20.396939 trace.c:415 performance: 1.482695902 s: git command: 'git' 'status'

Populated status
----------------

After the cache is populated, wt_status_collect_untracked() drops 82%
from 0.838s to 0.144s. Overall time drops 45%. Top offenders are now
read_index() and read_index_preload().

18:28:20.408605 builtin/commit.c:1394 performance: 0.000000457 s: cmd_status:setup
18:28:20.692864 read-cache.c:1407 performance: 0.283980458 s: read_index
18:28:20.693273 read-cache.c:1407 performance: 0.000000661 s: read_index
18:28:20.958814 preload-index.c:131 performance: 0.265540254 s: read_index_preload
18:28:20.972375 read-cache.c:1254 performance: 0.013437429 s: refresh_index
18:28:20.983959 wt-status.c:623 performance: 0.011146646 s: wt_status_collect_changes_worktree
18:28:20.993948 wt-status.c:629 performance: 0.009879094 s: wt_status_collect_changes_index
18:28:21.138125 wt-status.c:632 performance: 0.144084737 s: wt_status_collect_untracked
18:28:21.173678 builtin/commit.c:1421 performance: 0.035463949 s: cmd_status:update_index
18:28:21.174251 trace.c:415 performance: 0.766707355 s: git command: 'git' 'status'

gentoo-x86.git
==============

This repository is a strange one with a balanced, wide and shallow
worktree (about 100k files and 23k dirs) and no .gitignore in
worktree. .._collect_untracked() time drops 88%, total time drops 56%.

Base status
-----------
18:20:40.828642 builtin/commit.c:1394 performance: 0.000000496 s: cmd_status:setup
18:20:41.027233 read-cache.c:1407 performance: 0.198130532 s: read_index
18:20:41.027670 read-cache.c:1407 performance: 0.000000581 s: read_index
18:20:41.171716 preload-index.c:131 performance: 0.144045594 s: read_index_preload
18:20:41.179171 read-cache.c:1254 performance: 0.007320424 s: refresh_index
18:20:41.185785 wt-status.c:623 performance: 0.006144638 s: wt_status_collect_changes_worktree
18:20:41.192701 wt-status.c:629 performance: 0.006780184 s: wt_status_collect_changes_index
18:20:41.991723 wt-status.c:632 performance: 0.798927029 s: wt_status_collect_untracked
18:20:41.994664 builtin/commit.c:1421 performance: 0.002852772 s: cmd_status:update_index
18:20:41.995458 trace.c:415 performance: 1.168427502 s: git command: 'git' 'status'
Populating status
-----------------
18:20:48.968848 builtin/commit.c:1394 performance: 0.000000380 s: cmd_status:setup
18:20:49.172918 read-cache.c:1407 performance: 0.203734214 s: read_index
18:20:49.173341 read-cache.c:1407 performance: 0.000000562 s: read_index
18:20:49.320013 preload-index.c:131 performance: 0.146671391 s: read_index_preload
18:20:49.328039 read-cache.c:1254 performance: 0.007921957 s: refresh_index
18:20:49.334680 wt-status.c:623 performance: 0.006172020 s: wt_status_collect_changes_worktree
18:20:49.342526 wt-status.c:629 performance: 0.007731746 s: wt_status_collect_changes_index
18:20:50.257510 wt-status.c:632 performance: 0.914864222 s: wt_status_collect_untracked
18:20:50.338371 builtin/commit.c:1421 performance: 0.080776477 s: cmd_status:update_index
18:20:50.338900 trace.c:415 performance: 1.371462446 s: git command: 'git' 'status'
Populated status
----------------
18:20:50.351160 builtin/commit.c:1394 performance: 0.000000571 s: cmd_status:setup
18:20:50.577358 read-cache.c:1407 performance: 0.225917338 s: read_index
18:20:50.577794 read-cache.c:1407 performance: 0.000000617 s: read_index
18:20:50.734140 preload-index.c:131 performance: 0.156345564 s: read_index_preload
18:20:50.745717 read-cache.c:1254 performance: 0.011463075 s: refresh_index
18:20:50.755176 wt-status.c:623 performance: 0.008877929 s: wt_status_collect_changes_worktree
18:20:50.763768 wt-status.c:629 performance: 0.008471633 s: wt_status_collect_changes_index
18:20:50.854885 wt-status.c:632 performance: 0.090988721 s: wt_status_collect_untracked
18:20:50.857765 builtin/commit.c:1421 performance: 0.002789097 s: cmd_status:update_index
18:20:50.858411 trace.c:415 performance: 0.508647673 s: git command: 'git' 'status'

linux-2.6
=========

Reference repo. Not too big. .._collect_status() drops 84%. Total time
drops 42%.

Base status
-----------
18:34:09.870122 builtin/commit.c:1394 performance: 0.000000385 s: cmd_status:setup
18:34:09.943218 read-cache.c:1407 performance: 0.072871177 s: read_index
18:34:09.943614 read-cache.c:1407 performance: 0.000000491 s: read_index
18:34:10.004364 preload-index.c:131 performance: 0.060748102 s: read_index_preload
18:34:10.008190 read-cache.c:1254 performance: 0.003714285 s: refresh_index
18:34:10.012087 wt-status.c:623 performance: 0.002775446 s: wt_status_collect_changes_worktree
18:34:10.016054 wt-status.c:629 performance: 0.003862140 s: wt_status_collect_changes_index
18:34:10.214747 wt-status.c:632 performance: 0.198604837 s: wt_status_collect_untracked
18:34:10.216102 builtin/commit.c:1421 performance: 0.001244166 s: cmd_status:update_index
18:34:10.216817 trace.c:415 performance: 0.347670735 s: git command: 'git' 'status'
Populating status
-----------------
18:34:16.595102 builtin/commit.c:1394 performance: 0.000000456 s: cmd_status:setup
18:34:16.666600 read-cache.c:1407 performance: 0.070992413 s: read_index
18:34:16.667012 read-cache.c:1407 performance: 0.000000606 s: read_index
18:34:16.729375 preload-index.c:131 performance: 0.062362492 s: read_index_preload
18:34:16.732565 read-cache.c:1254 performance: 0.003075517 s: refresh_index
18:34:16.736148 wt-status.c:623 performance: 0.002422201 s: wt_status_collect_changes_worktree
18:34:16.739990 wt-status.c:629 performance: 0.003746618 s: wt_status_collect_changes_index
18:34:16.948505 wt-status.c:632 performance: 0.208426710 s: wt_status_collect_untracked
18:34:16.961744 builtin/commit.c:1421 performance: 0.013151887 s: cmd_status:update_index
18:34:16.962233 trace.c:415 performance: 0.368537535 s: git command: 'git' 'status'
Populated status
----------------
18:34:16.970026 builtin/commit.c:1394 performance: 0.000000631 s: cmd_status:setup
18:34:17.046235 read-cache.c:1407 performance: 0.075904673 s: read_index
18:34:17.046644 read-cache.c:1407 performance: 0.000000681 s: read_index
18:34:17.113564 preload-index.c:131 performance: 0.066920253 s: read_index_preload
18:34:17.117281 read-cache.c:1254 performance: 0.003604055 s: refresh_index
18:34:17.121115 wt-status.c:623 performance: 0.002508345 s: wt_status_collect_changes_worktree
18:34:17.125089 wt-status.c:629 performance: 0.003871636 s: wt_status_collect_changes_index
18:34:17.156089 wt-status.c:632 performance: 0.030895703 s: wt_status_collect_untracked
18:34:17.169861 builtin/commit.c:1421 performance: 0.013686404 s: cmd_status:update_index
18:34:17.170391 trace.c:415 performance: 0.201474531 s: git command: 'git' 'status'

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

status: enable untracked cacheNguyễn Thái Ngọc Duy Sun, 8 Mar 2015 10:12:41 +0000 (17:12 +0700)

status: enable untracked cache

update_index_if_able() is moved down so that the updated untracked
cache could be written out.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

untracked-cache: temporarily disable with $GIT_DISABLE_... Nguyễn Thái Ngọc Duy Sun, 8 Mar 2015 10:12:40 +0000 (17:12 +0700)

untracked-cache: temporarily disable with $GIT_DISABLE_UNTRACKED_CACHE

This can be used to double check if results with untracked cache are
correctly, compared to vanilla version. Untracked cache remains in
index, but not used.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

untracked cache: mark index dirty if untracked cache... Nguyễn Thái Ngọc Duy Sun, 8 Mar 2015 10:12:39 +0000 (17:12 +0700)

untracked cache: mark index dirty if untracked cache is updated

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

untracked cache: print stats with $GIT_TRACE_UNTRACKED_... Nguyễn Thái Ngọc Duy Sun, 8 Mar 2015 10:12:38 +0000 (17:12 +0700)

untracked cache: print stats with $GIT_TRACE_UNTRACKED_STATS

This could be used to verify correct behavior in tests

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

untracked cache: avoid racy timestampsNguyễn Thái Ngọc Duy Sun, 8 Mar 2015 10:12:37 +0000 (17:12 +0700)

untracked cache: avoid racy timestamps

When a directory is updated within the same second that its timestamp
is last saved, we cannot realize the directory has been updated by
checking timestamps. Assume the worst (something is update). See
29e4d36 (Racy GIT - 2005-12-20) for more information.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

read-cache.c: split racy stat test to a separate functionNguyễn Thái Ngọc Duy Sun, 8 Mar 2015 10:12:36 +0000 (17:12 +0700)

read-cache.c: split racy stat test to a separate function

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

untracked cache: invalidate at index addition or removalNguyễn Thái Ngọc Duy Sun, 8 Mar 2015 10:12:35 +0000 (17:12 +0700)

untracked cache: invalidate at index addition or removal

Ideally we should implement untracked_cache_remove_from_index() and
untracked_cache_add_to_index() so that they update untracked cache
right away instead of invalidating it and wait for read_directory()
next time to deal with it. But that may need some more work in
unpack-trees.c. So stay simple as the first step.

The new call in add_index_entry_with_check() may look strange because
new calls usually stay close to cache_tree_invalidate_path(). We do it
a bit later than c_t_i_p() in this function because if it's about
replacing the entry with the same name, we don't care (but cache-tree
does).

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

untracked cache: load from UNTR index extensionNguyễn Thái Ngọc Duy Sun, 8 Mar 2015 10:12:34 +0000 (17:12 +0700)

untracked cache: load from UNTR index extension

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

untracked cache: save to an index extensionNguyễn Thái Ngọc Duy Sun, 8 Mar 2015 10:12:33 +0000 (17:12 +0700)

untracked cache: save to an index extension

Helped-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

ewah: add convenient wrapper ewah_serialize_strbuf()Nguyễn Thái Ngọc Duy Sun, 8 Mar 2015 10:12:32 +0000 (17:12 +0700)

ewah: add convenient wrapper ewah_serialize_strbuf()

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

untracked cache: don't open non-existent .gitignoreNguyễn Thái Ngọc Duy Sun, 8 Mar 2015 10:12:31 +0000 (17:12 +0700)

untracked cache: don't open non-existent .gitignore

This cuts down a signficant number of open(.gitignore) because most
directories usually don't have .gitignore files.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

untracked cache: mark what dirs should be recursed... Nguyễn Thái Ngọc Duy Sun, 8 Mar 2015 10:12:30 +0000 (17:12 +0700)

untracked cache: mark what dirs should be recursed/saved

If we redo this thing in a functional style, we would have one struct
untracked_dir as input tree and another as output. The input is used
for verification. The output is a brand new tree, reflecting current
worktree.

But that means recreate a lot of dir nodes even if a lot could be
shared between input and output trees in good cases. So we go with the
messy but efficient way, combining both input and output trees into
one. We need a way to know which node in this combined tree belongs to
the output. This is the purpose of this "recurse" flag.

"valid" bit can't be used for this because it's about data of the node
except the subdirs. When we invalidate a directory, we want to keep
cached data of the subdirs intact even though we don't really know
what subdir still exists (yet). Then we check worktree to see what
actual subdir remains on disk. Those will have 'recurse' bit set
again. If cached data for those are still valid, we may be able to
avoid computing exclude files for them. Those subdirs that are deleted
will have 'recurse' remained clear and their 'valid' bits do not
matter.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

untracked cache: record/validate dir mtime and reuse... Nguyễn Thái Ngọc Duy Sun, 8 Mar 2015 10:12:29 +0000 (17:12 +0700)

untracked cache: record/validate dir mtime and reuse cached output

The main readdir loop in read_directory_recursive() is replaced with a
new one that checks if cached results of a directory is still valid.

If a file is added or removed from the index, the containing directory
is invalidated (but not its subdirs). If directory's mtime is changed,
the same happens. If a .gitignore is updated, the containing directory
and all subdirs are invalidated recursively. If dir_struct#flags or
other conditions change, the cache is ignored.

If a directory is invalidated, we opendir/readdir/closedir and run the
exclude machinery on that directory listing as usual. If untracked
cache is also enabled, we'll update the cache along the way. If a
directory is validated, we simply pull the untracked listing out from
the cache. The cache also records the list of direct subdirs that we
have to recurse in. Fully excluded directories are seen as "untracked
files".

In the best case when no dirs are invalidated, read_directory()
becomes a series of

stat(dir), open(.gitignore), fstat(), read(), close() and optionally
hash_sha1_file()

For comparison, standard read_directory() is a sequence of

opendir(), readdir(), open(.gitignore), fstat(), read(), close(), the
expensive last_exclude_matching() and closedir().

We already try not to open(.gitignore) if we know it does not exist,
so open/fstat/read/close sequence does not apply to every
directory. The sequence could be reduced further, as noted in
prep_exclude() in another patch. So in theory, the entire best-case
read_directory sequence could be reduced to a series of stat() and
nothing else.

This is not a silver bullet approach. When you compile a C file, for
example, the old .o file is removed and a new one with the same name
created, effectively invalidating the containing directory's cache
(but not its subdirectories). If your build process touches every
directory, this cache adds extra overhead for nothing, so it's a good
idea to separate generated files from tracked files.. Editors may use
the same strategy for saving files. And of course you're out of luck
running your repo on an unsupported filesystem and/or operating system.

Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

untracked cache: make a wrapper around {open,read,close... Nguyễn Thái Ngọc Duy Sun, 8 Mar 2015 10:12:28 +0000 (17:12 +0700)

untracked cache: make a wrapper around {open,read,close}dir()

This allows us to feed different info to read_directory_recursive()
based on untracked cache in the next patch.

Helped-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

untracked cache: invalidate dirs recursively if .gitign... Nguyễn Thái Ngọc Duy Sun, 8 Mar 2015 10:12:27 +0000 (17:12 +0700)

untracked cache: invalidate dirs recursively if .gitignore changes

It's easy to see that if an existing .gitignore changes, its SHA-1
would be different and invalidate_gitignore() is called.

If .gitignore is removed, add_excludes() will treat it like an empty
.gitignore, which again should invalidate the cached directory data.

if .gitignore is added, lookup_untracked() already fills initial
.gitignore SHA-1 as "empty file", so again invalidate_gitignore() is
called.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

untracked cache: initial untracked cache validationNguyễn Thái Ngọc Duy Sun, 8 Mar 2015 10:12:26 +0000 (17:12 +0700)

untracked cache: initial untracked cache validation

Make sure the starting conditions and all global exclude files are
good to go. If not, either disable untracked cache completely, or wipe
out the cache and start fresh.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

untracked cache: record .gitignore information and... Nguyễn Thái Ngọc Duy Sun, 8 Mar 2015 10:12:25 +0000 (17:12 +0700)

untracked cache: record .gitignore information and dir hierarchy

The idea is if we can capture all input and (non-rescursive) output of
read_directory_recursive(), and can verify later that all the input is
the same, then the second r_d_r() should produce the same output as in
the first run.

The requirement for this to work is stat info of a directory MUST
change if an entry is added to or removed from that directory (and
should not change often otherwise). If your OS and filesystem do not
meet this requirement, untracked cache is not for you. Most file
systems on *nix should be fine. On Windows, NTFS is fine while FAT may
not be [1] even though FAT on Linux seems to be fine.

The list of input of r_d_r() is in the big comment block in dir.h. In
short, the output of a directory (not counting subdirs) mainly depends
on stat info of the directory in question, all .gitignore leading to
it and the check_only flag when r_d_r() is called recursively. This
patch records all this info (and the output) as r_d_r() runs.

Two hash_sha1_file() are required for $GIT_DIR/info/exclude and
core.excludesfile unless their stat data matches. hash_sha1_file() is
only needed when .gitignore files in the worktree are modified,
otherwise their SHA-1 in index is used (see the previous patch).

We could store stat data for .gitignore files so we don't have to
rehash them if their content is different from index, but I think
.gitignore files are rarely modified, so not worth extra cache data
(and hashing penalty read-cache.c:verify_hdr(), as we will be storing
this as an index extension).

The implication is, if you change .gitignore, you better add it to the
index soon or you lose all the benefit of untracked cache because a
modified .gitignore invalidates all subdirs recursively. This is
especially bad for .gitignore at root.

This cached output is about untracked files only, not ignored files
because the number of tracked files is usually small, so small cache
overhead, while the number of ignored files could go really high
(e.g. *.o files mixing with source code).

[1] "Description of NTFS date and time stamps for files and folders"
http://support.microsoft.com/kb/299648

Helped-by: Torsten Bögershausen <tboegi@web.de>
Helped-by: David Turner <dturner@twopensource.com>
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

dir.c: optionally compute sha-1 of a .gitignore fileNguyễn Thái Ngọc Duy Sun, 8 Mar 2015 10:12:24 +0000 (17:12 +0700)

dir.c: optionally compute sha-1 of a .gitignore file

This is not used anywhere yet. But the goal is to compare quickly if a
.gitignore file has changed when we have the SHA-1 of both old (cached
somewhere) and new (from index or a tree) versions.

Helped-by: Junio C Hamano <gitster@pobox.com>
Helped-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Post 2.3 cycle (batch #9)Junio C Hamano Tue, 10 Mar 2015 20:53:49 +0000 (13:53 -0700)

Post 2.3 cycle (batch #9)

Signed-off-by: Junio C Hamano <gitster@pobox.com>

Merge branch 'mh/expire-updateref-fixes'Junio C Hamano Tue, 10 Mar 2015 20:52:39 +0000 (13:52 -0700)

Merge branch 'mh/expire-updateref-fixes'

Various issues around "reflog expire", e.g. using --updateref when
expiring a reflog for a symbolic reference, have been corrected
and/or made saner.

* mh/expire-updateref-fixes:
reflog_expire(): never update a reference to null_sha1
reflog_expire(): ignore --updateref for symbolic references
reflog: improve and update documentation
struct ref_lock: delete the force_write member
lock_ref_sha1_basic(): do not set force_write for missing references
write_ref_sha1(): move write elision test to callers
write_ref_sha1(): remove check for lock == NULL

Merge branch 'jk/diffcore-rename-duplicate'Junio C Hamano Tue, 10 Mar 2015 20:52:38 +0000 (13:52 -0700)

Merge branch 'jk/diffcore-rename-duplicate'

A corrupt input to "git diff -M" can cause us to segfault.

* jk/diffcore-rename-duplicate:
diffcore-rename: avoid processing duplicate destinations
diffcore-rename: split locate_rename_dst into two functions

Post 2.3 cycle (batch #8)Junio C Hamano Fri, 6 Mar 2015 23:05:39 +0000 (15:05 -0800)

Post 2.3 cycle (batch #8)

Signed-off-by: Junio C Hamano <gitster@pobox.com>

Merge branch 'bw/kwset-use-unsigned'Junio C Hamano Fri, 6 Mar 2015 23:02:33 +0000 (15:02 -0800)

Merge branch 'bw/kwset-use-unsigned'

The borrowed code in kwset API did not follow our usual convention
to use "unsigned char" to store values that range from 0-255.

* bw/kwset-use-unsigned:
kwset: use unsigned char to store values with high-bit set

Merge branch 'ak/t5516-typofix'Junio C Hamano Fri, 6 Mar 2015 23:02:32 +0000 (15:02 -0800)

Merge branch 'ak/t5516-typofix'

* ak/t5516-typofix:
t5516: correct misspelled pushInsteadOf

Merge branch 'ms/submodule-update-config-doc'Junio C Hamano Fri, 6 Mar 2015 23:02:31 +0000 (15:02 -0800)

Merge branch 'ms/submodule-update-config-doc'

The interaction between "git submodule update" and the
submodule.*.update configuration was not clearly documented.

* ms/submodule-update-config-doc:
submodule: improve documentation of update subcommand

Merge branch 'ja/clean-confirm-i18n'Junio C Hamano Fri, 6 Mar 2015 23:02:29 +0000 (15:02 -0800)

Merge branch 'ja/clean-confirm-i18n'

The prompt string "remove?" used when "git clean -i" asks the user
if a path should be removed was localizable, but the code always
expects a substring of "yes" to tell it to go ahead. Always show
[y/N] as part of this prompt to hint that the answer is not (yet)
localized.

* ja/clean-confirm-i18n:
Add hint interactive cleaning

Merge branch 'mk/diff-shortstat-dirstat-fix'Junio C Hamano Fri, 6 Mar 2015 23:02:28 +0000 (15:02 -0800)

Merge branch 'mk/diff-shortstat-dirstat-fix'

"git diff --shortstat --dirstat=changes" showed a dirstat based on
lines that was never asked by the end user in addition to the
dirstat that the user asked for.

* mk/diff-shortstat-dirstat-fix:
diff --shortstat --dirstat: remove duplicate output

Merge branch 'mg/doc-remote-tags-or-not'Junio C Hamano Fri, 6 Mar 2015 23:02:27 +0000 (15:02 -0800)

Merge branch 'mg/doc-remote-tags-or-not'

"git remote add" mentioned "--tags" and "--no-tags" and was not
clear that fetch from the remote in the future will use the default
behaviour when neither is given to override it.

* mg/doc-remote-tags-or-not:
git-remote.txt: describe behavior without --tags and --no-tags

Merge branch 'nd/grep-exclude-standard-help-fix'Junio C Hamano Fri, 6 Mar 2015 23:02:27 +0000 (15:02 -0800)

Merge branch 'nd/grep-exclude-standard-help-fix'

Description given by "grep -h" for its --exclude-standard option
was phrased poorly.

* nd/grep-exclude-standard-help-fix:
grep: correct help string for --exclude-standard

Merge branch 'mr/doc-clean-f-f'Junio C Hamano Fri, 6 Mar 2015 23:02:26 +0000 (15:02 -0800)

Merge branch 'mr/doc-clean-f-f'

Documentation update.

* mr/doc-clean-f-f:
Documentation/git-clean.txt: document that -f may need to be given twice

Merge branch 'ye/http-accept-language'Junio C Hamano Fri, 6 Mar 2015 23:02:24 +0000 (15:02 -0800)

Merge branch 'ye/http-accept-language'

Compilation fix for a recent topic in 'master'.

* ye/http-accept-language:
gettext.c: move get_preferred_languages() from http.c

Sync with 2.3.2Junio C Hamano Fri, 6 Mar 2015 22:59:12 +0000 (14:59 -0800)

Sync with 2.3.2

* maint:
Git 2.3.2

Git 2.3.2 v2.3.2Junio C Hamano Fri, 6 Mar 2015 22:58:14 +0000 (14:58 -0800)

Git 2.3.2

Signed-off-by: Junio C Hamano <gitster@pobox.com>

Merge branch 'rj/no-xopen-source-for-cygwin' into maintJunio C Hamano Fri, 6 Mar 2015 22:57:58 +0000 (14:57 -0800)

Merge branch 'rj/no-xopen-source-for-cygwin' into maint

Code cleanups.

* rj/no-xopen-source-for-cygwin:
git-compat-util.h: remove redundant code

Merge branch 'rs/simple-cleanups' into maintJunio C Hamano Fri, 6 Mar 2015 22:57:57 +0000 (14:57 -0800)

Merge branch 'rs/simple-cleanups' into maint

Code cleanups.

* rs/simple-cleanups:
sha1_name: use strlcpy() to copy strings
pretty: use starts_with() to check for a prefix
for-each-ref: use skip_prefix() to avoid duplicate string comparison
connect: use strcmp() for string comparison

Merge branch 'mm/am-c-doc' into maintJunio C Hamano Fri, 6 Mar 2015 22:57:56 +0000 (14:57 -0800)

Merge branch 'mm/am-c-doc' into maint

The configuration variable 'mailinfo.scissors' was hard to
discover in the documentation.

* mm/am-c-doc:
Documentation/git-am.txt: mention mailinfo.scissors config variable
Documentation/config.txt: document mailinfo.scissors

Merge branch 'ew/svn-maint-fixes' into maintJunio C Hamano Fri, 6 Mar 2015 22:57:55 +0000 (14:57 -0800)

Merge branch 'ew/svn-maint-fixes' into maint

Correct a breakage to git-svn around v2.2 era that triggers
premature closing of FileHandle.

* ew/svn-maint-fixes:
Git::SVN::*: avoid premature FileHandle closure
git-svn: fix localtime=true on non-glibc environments

Merge branch 'km/send-email-getopt-long-workarounds... Junio C Hamano Fri, 6 Mar 2015 22:57:54 +0000 (14:57 -0800)

Merge branch 'km/send-email-getopt-long-workarounds' into maint

Even though we officially haven't dropped Perl 5.8 support, the
Getopt::Long package that came with it does not support "--no-"
prefix to negate a boolean option; manually add support to help
people with older Getopt::Long package.

* km/send-email-getopt-long-workarounds:
git-send-email.perl: support no- prefix with older GetOptions

Sync with maintJunio C Hamano Thu, 5 Mar 2015 21:16:27 +0000 (13:16 -0800)

Sync with maint

* maint:
Prepare for 2.3.2

Prepare for 2.3.2Junio C Hamano Thu, 5 Mar 2015 21:15:53 +0000 (13:15 -0800)

Prepare for 2.3.2

Merge branch 'sb/plug-leak-in-make-cache-entry' into... Junio C Hamano Thu, 5 Mar 2015 21:13:13 +0000 (13:13 -0800)

Merge branch 'sb/plug-leak-in-make-cache-entry' into maint

"update-index --refresh" used to leak when an entry cannot be
refreshed for whatever reason.

* sb/plug-leak-in-make-cache-entry:
read-cache.c: free cache entry when refreshing fails

Merge branch 'jk/fast-import-die-nicely-fix' into maintJunio C Hamano Thu, 5 Mar 2015 21:13:12 +0000 (13:13 -0800)

Merge branch 'jk/fast-import-die-nicely-fix' into maint

"git fast-import" used to crash when it could not close and
conclude the resulting packfile cleanly.

* jk/fast-import-die-nicely-fix:
fast-import: avoid running end_packfile recursively

Merge branch 'es/blame-commit-info-fix' into maintJunio C Hamano Thu, 5 Mar 2015 21:13:12 +0000 (13:13 -0800)

Merge branch 'es/blame-commit-info-fix' into maint

"git blame" died, trying to free an uninitialized piece of memory.

* es/blame-commit-info-fix:
builtin/blame: destroy initialized commit_info only

Merge branch 'ab/merge-file-prefix' into maintJunio C Hamano Thu, 5 Mar 2015 21:13:10 +0000 (13:13 -0800)

Merge branch 'ab/merge-file-prefix' into maint

"git merge-file" did not work correctly in a subdirectory.

* ab/merge-file-prefix:
merge-file: correctly open files when in a subdir

Merge branch 'ps/submodule-sanitize-path-upon-add'... Junio C Hamano Thu, 5 Mar 2015 21:13:09 +0000 (13:13 -0800)

Merge branch 'ps/submodule-sanitize-path-upon-add' into maint

"git submodule add" failed to squash "path/to/././submodule" to
"path/to/submodule".

* ps/submodule-sanitize-path-upon-add:
git-submodule.sh: fix '/././' path normalization

Merge branch 'jk/prune-mtime' into maintJunio C Hamano Thu, 5 Mar 2015 21:13:08 +0000 (13:13 -0800)

Merge branch 'jk/prune-mtime' into maint

In v2.2.0, we broke "git prune" that runs in a repository that
borrows from an alternate object store.

* jk/prune-mtime:
sha1_file: fix iterating loose alternate objects
for_each_loose_file_in_objdir: take an optional strbuf path

Merge branch 'tc/curl-vernum-output-broken-in-7.11... Junio C Hamano Thu, 5 Mar 2015 21:13:07 +0000 (13:13 -0800)

Merge branch 'tc/curl-vernum-output-broken-in-7.11' into maint

Certain older vintages of cURL give irregular output from
"curl-config --vernum", which confused our build system.

* tc/curl-vernum-output-broken-in-7.11:
Makefile: handle broken curl version number in version check

Merge branch 'es/squelch-openssl-warnings-on-macosx... Junio C Hamano Thu, 5 Mar 2015 21:13:06 +0000 (13:13 -0800)

Merge branch 'es/squelch-openssl-warnings-on-macosx' into maint

An earlier workaround to squelch unhelpful deprecation warnings
from the complier on Mac OSX unnecessarily set minimum required
version of the OS, which the user might want to raise (or lower)
for other reasons.

* es/squelch-openssl-warnings-on-macosx:
git-compat-util: do not step on MAC_OS_X_VERSION_MIN_REQUIRED

Merge branch 'jc/conf-var-doc' into maintJunio C Hamano Thu, 5 Mar 2015 21:13:05 +0000 (13:13 -0800)

Merge branch 'jc/conf-var-doc' into maint

Longstanding configuration variable naming rules has been added to
the documentation.

* jc/conf-var-doc:
CodingGuidelines: describe naming rules for configuration variables
config.txt: mark deprecated variables more prominently
config.txt: clarify that add.ignore-errors is deprecated

Merge branch 'av/wincred-with-at-in-username-fix' into... Junio C Hamano Thu, 5 Mar 2015 21:13:04 +0000 (13:13 -0800)

Merge branch 'av/wincred-with-at-in-username-fix' into maint

The credential helper for Windows (in contrib/) used to mishandle
a user name with an at-sign in it.

* av/wincred-with-at-in-username-fix:
wincred: fix get credential if username has "@"

Merge branch 'ch/new-gpg-drops-rfc-1991' into maintJunio C Hamano Thu, 5 Mar 2015 21:13:03 +0000 (13:13 -0800)

Merge branch 'ch/new-gpg-drops-rfc-1991' into maint

Older GnuPG implementations may not correctly import the keyring
material we prepare for the tests to use.

* ch/new-gpg-drops-rfc-1991:
t/lib-gpg: sanity-check that we can actually sign
t/lib-gpg: include separate public keys in keyring.gpg

Merge branch 'jc/remote-set-url-doc' into maintJunio C Hamano Thu, 5 Mar 2015 21:13:03 +0000 (13:13 -0800)

Merge branch 'jc/remote-set-url-doc' into maint

Clarify in the documentation that "remote.<nick>.pushURL" and
"remote.<nick>.URL" are there to name the same repository accessed
via different transports, not two separate repositories.

* jc/remote-set-url-doc:
Documentation/git-remote.txt: stress that set-url is not for triangular

Merge branch 'jk/pack-bitmap' into maintJunio C Hamano Thu, 5 Mar 2015 21:13:02 +0000 (13:13 -0800)

Merge branch 'jk/pack-bitmap' into maint

The pack bitmap support did not build with older versions of GCC.

* jk/pack-bitmap:
ewah: fix building with gcc < 3.4.0

Merge branch 'jk/config-no-ungetc-eof' into maintJunio C Hamano Thu, 5 Mar 2015 21:13:00 +0000 (13:13 -0800)

Merge branch 'jk/config-no-ungetc-eof' into maint

Reading configuration from a blob object, when it ends with a lone
CR, use to confuse the configuration parser.

* jk/config-no-ungetc-eof:
config_buf_ungetc: warn when pushing back a random character
config: do not ungetc EOF

Merge branch 'jk/decimal-width-for-uintmax' into maintJunio C Hamano Thu, 5 Mar 2015 21:12:59 +0000 (13:12 -0800)

Merge branch 'jk/decimal-width-for-uintmax' into maint

We didn't format an integer that wouldn't fit in "int" but in
"uintmax_t" correctly.

* jk/decimal-width-for-uintmax:
decimal_width: avoid integer overflow

Merge branch 'jc/push-cert' into maintJunio C Hamano Thu, 5 Mar 2015 21:12:58 +0000 (13:12 -0800)

Merge branch 'jc/push-cert' into maint

"git push --signed" gave an incorrectly worded error message when
the other side did not support the capability.

* jc/push-cert:
transport-helper: fix typo in error message when --signed is not supported

Merge branch 'mh/deref-symref-over-helper-transport... Junio C Hamano Thu, 5 Mar 2015 21:12:57 +0000 (13:12 -0800)

Merge branch 'mh/deref-symref-over-helper-transport' into maint

"git fetch" over a remote-helper that cannot respond to "list"
command could not fetch from a symbolic reference e.g. HEAD.

* mh/deref-symref-over-helper-transport:
transport-helper: do not request symbolic refs to remote helpers

Merge branch 'ks/rebase-i-abbrev' into maintJunio C Hamano Thu, 5 Mar 2015 21:12:56 +0000 (13:12 -0800)

Merge branch 'ks/rebase-i-abbrev' into maint

The insn sheet "git rebase -i" creates did not fully honor
core.abbrev settings.

* ks/rebase-i-abbrev:
rebase -i: use full object name internally throughout the script

Merge branch 'dp/remove-duplicated-header-inclusion... Junio C Hamano Thu, 5 Mar 2015 21:12:55 +0000 (13:12 -0800)

Merge branch 'dp/remove-duplicated-header-inclusion' into maint

Code clean-up.

* dp/remove-duplicated-header-inclusion:
do not include the same header twice

Merge branch 'sb/hex-object-name-is-at-most-41-bytes... Junio C Hamano Thu, 5 Mar 2015 21:12:54 +0000 (13:12 -0800)

Merge branch 'sb/hex-object-name-is-at-most-41-bytes-long' into maint

Code clean-up.

* sb/hex-object-name-is-at-most-41-bytes-long:
hex.c: reduce memory footprint of sha1_to_hex static buffers

Merge branch 'ak/git-pm-typofix' into maintJunio C Hamano Thu, 5 Mar 2015 21:12:53 +0000 (13:12 -0800)

Merge branch 'ak/git-pm-typofix' into maint

Typofix in comments.

* ak/git-pm-typofix:
Git.pm: two minor typo fixes

Merge branch 'jk/sanity' into maintJunio C Hamano Thu, 5 Mar 2015 21:12:52 +0000 (13:12 -0800)

Merge branch 'jk/sanity' into maint

The tests that wanted to see that file becomes unreadable after
running "chmod a-r file", and the tests that wanted to make sure it
is not run as root, we used "can we write into the / directory?" as
a cheap substitute, but on some platforms that is not a good
heuristics. The tests and their prerequisites have been updated to
check what they really require.

* jk/sanity:
test-lib.sh: set prerequisite SANITY by testing what we really need
tests: correct misuses of POSIXPERM
t/lib-httpd: switch SANITY check for NOT_ROOT

Post 2.3 cycle (batch #7)Junio C Hamano Thu, 5 Mar 2015 20:48:18 +0000 (12:48 -0800)

Post 2.3 cycle (batch #7)

Signed-off-by: Junio C Hamano <gitster@pobox.com>

Merge branch 'ew/svn-fixes'Junio C Hamano Thu, 5 Mar 2015 20:45:45 +0000 (12:45 -0800)

Merge branch 'ew/svn-fixes'

* ew/svn-fixes:
git-svn: lazy load some modules

Merge branch 'ew/svn-maint-fixes'Junio C Hamano Thu, 5 Mar 2015 20:45:45 +0000 (12:45 -0800)

Merge branch 'ew/svn-maint-fixes'

Correct a breakage to git-svn around v2.2 era that triggers
premature closing of FileHandle.

* ew/svn-maint-fixes:
Git::SVN::*: avoid premature FileHandle closure
git-svn: fix localtime=true on non-glibc environments

Merge branch 'tb/connect-ipv6-parse-fix'Junio C Hamano Thu, 5 Mar 2015 20:45:44 +0000 (12:45 -0800)

Merge branch 'tb/connect-ipv6-parse-fix'

We did not parse username followed by literal IPv6 address in SSH
transport URLs, e.g. ssh://user@[2001:db8::1]:22/repo.git
correctly.

* tb/connect-ipv6-parse-fix:
t5500: show user name and host in diag-url
t5601: add more test cases for IPV6
connect.c: allow ssh://user@[2001:db8::1]/repo.git

Merge branch 'jc/diff-test-updates'Junio C Hamano Thu, 5 Mar 2015 20:45:43 +0000 (12:45 -0800)

Merge branch 'jc/diff-test-updates'

Test clean-up.

* jc/diff-test-updates:
test_ln_s_add: refresh stat info of fake symbolic links
t4008: modernise style
t/diff-lib: check exact object names in compare_diff_raw
tests: do not borrow from COPYING and README from the real source
t4010: correct expected object names
t9300: correct expected object names
t4008: correct stale comments

Merge branch 'rs/simple-cleanups'Junio C Hamano Thu, 5 Mar 2015 20:45:42 +0000 (12:45 -0800)

Merge branch 'rs/simple-cleanups'

Code cleanups.

* rs/simple-cleanups:
sha1_name: use strlcpy() to copy strings
pretty: use starts_with() to check for a prefix
for-each-ref: use skip_prefix() to avoid duplicate string comparison
connect: use strcmp() for string comparison

Merge branch 'rj/no-xopen-source-for-cygwin'Junio C Hamano Thu, 5 Mar 2015 20:45:41 +0000 (12:45 -0800)

Merge branch 'rj/no-xopen-source-for-cygwin'

Code cleanups.

* rj/no-xopen-source-for-cygwin:
git-compat-util.h: remove redundant code

Merge branch 'mm/am-c-doc'Junio C Hamano Thu, 5 Mar 2015 20:45:40 +0000 (12:45 -0800)

Merge branch 'mm/am-c-doc'

The configuration variable 'mailinfo.scissors' was hard to
discover in the documentation.

* mm/am-c-doc:
Documentation/git-am.txt: mention mailinfo.scissors config variable
Documentation/config.txt: document mailinfo.scissors

Merge branch 'mh/refs-have-new'Junio C Hamano Thu, 5 Mar 2015 20:45:39 +0000 (12:45 -0800)

Merge branch 'mh/refs-have-new'

Simplify the ref transaction API around how "the ref should be
pointing at this object" is specified.

* mh/refs-have-new:
refs.h: remove duplication in function docstrings
update_ref(): improve documentation
ref_transaction_verify(): new function to check a reference's value
ref_transaction_delete(): check that old_sha1 is not null_sha1
ref_transaction_create(): check that new_sha1 is valid
commit: avoid race when creating orphan commits
commit: add tests of commit races
ref_transaction_delete(): remove "have_old" parameter
ref_transaction_update(): remove "have_old" parameter
struct ref_update: move "have_old" into "flags"
refs.c: change some "flags" to "unsigned int"
refs: remove the gap in the REF_* constant values
refs: move REF_DELETING to refs.c

reflog_expire(): never update a reference to null_sha1Michael Haggerty Tue, 3 Mar 2015 11:43:17 +0000 (12:43 +0100)

reflog_expire(): never update a reference to null_sha1

Currently, if --updateref is specified and the very last reflog entry
is expired or deleted, the reference's value is set to 0{40}. This is
an invalid state of the repository, and breaks, for example, "git
fsck" and "git for-each-ref".

The only place we use --updateref in our own code is when dropping
stash entries. In that code, the very next step is to check if the
reflog has been made empty, and if so, delete the "refs/stash"
reference entirely. Thus that code path ultimately leaves the
repository in a valid state.

But we don't want to the repository in an invalid state even
temporarily, and we don't want to leave an invalid state if other
callers of "git reflog expire|delete --updateref" don't think to do
the extra cleanup step.

So, if "git reflog expire|delete" leaves no more entries in the
reflog, just leave the reference unchanged.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Reviewed-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

reflog_expire(): ignore --updateref for symbolic referencesMichael Haggerty Tue, 3 Mar 2015 11:43:16 +0000 (12:43 +0100)

reflog_expire(): ignore --updateref for symbolic references

If we are expiring reflog entries for a symbolic reference, then how
should --updateref be handled if the newest reflog entry is expired?

Option 1: Update the referred-to reference. (This is what the current
code does.) This doesn't make sense, because the referred-to reference
has its own reflog, which hasn't been rewritten.

Option 2: Update the symbolic reference itself (as in, REF_NODEREF).
This would convert the symbolic reference into a non-symbolic
reference (e.g., detaching HEAD), which is surely not what a user
would expect.

Option 3: Error out. This is plausible, but it would make the
following usage impossible:

git reflog expire ... --updateref --all

Option 4: Ignore --updateref for symbolic references.

We choose to implement option 4.

Note: another problem in this code will be fixed in a moment.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Reviewed-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

reflog: improve and update documentationMichael Haggerty Tue, 3 Mar 2015 11:43:15 +0000 (12:43 +0100)

reflog: improve and update documentation

Revamp the "git reflog" usage documentation in the manpage and the
command help to match the current reality and improve its clarity:

* Add documentation for some options that had been left out.

* Group the subcommands and options more logically and move more
common subcommands/options higher.

* Improve some explanations.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Reviewed-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

struct ref_lock: delete the force_write memberStefan Beller Tue, 3 Mar 2015 11:43:14 +0000 (12:43 +0100)

struct ref_lock: delete the force_write member

Instead, compute the value when it is needed.

Signed-off-by: Stefan Beller <sbeller@google.com>
Edited-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Reviewed-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

lock_ref_sha1_basic(): do not set force_write for missi... Michael Haggerty Mon, 2 Mar 2015 09:29:53 +0000 (10:29 +0100)

lock_ref_sha1_basic(): do not set force_write for missing references

If a reference is missing, its SHA-1 will be null_sha1, which can't
possibly match a new value that ref_transaction_commit() is trying to
update it to. So there is no need to set force_write in this scenario.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Reviewed-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

write_ref_sha1(): move write elision test to callersMichael Haggerty Mon, 2 Mar 2015 09:29:52 +0000 (10:29 +0100)

write_ref_sha1(): move write elision test to callers

write_ref_sha1() previously skipped the write if the reference already
had the desired value, unless lock->force_write was set. Instead,
perform that test at the callers.

Two of the callers (in rename_ref()) unconditionally set force_write
just before calling write_ref_sha1(), so they don't need the extra
check at all. Nor do they need to set force_write anymore.

The last caller, in ref_transaction_commit(), still needs the test.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Reviewed-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

write_ref_sha1(): remove check for lock == NULLMichael Haggerty Mon, 2 Mar 2015 09:29:51 +0000 (10:29 +0100)

write_ref_sha1(): remove check for lock == NULL

None of the callers pass NULL to this function, and there doesn't seem
to be any usefulness to allowing them to do so.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Reviewed-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Post 2.3 cycle (batch #6)Junio C Hamano Tue, 3 Mar 2015 22:39:10 +0000 (14:39 -0800)

Post 2.3 cycle (batch #6)

Signed-off-by: Junio C Hamano <gitster@pobox.com>

Merge branch 'jk/daemon-interpolate'Junio C Hamano Tue, 3 Mar 2015 22:37:05 +0000 (14:37 -0800)

Merge branch 'jk/daemon-interpolate'

The "interpolated-path" option of "git daemon" inserted any string
client declared on the "host=" capability request without checking.
Sanitize and limit %H and %CH to a saner and a valid DNS name.

* jk/daemon-interpolate:
daemon: sanitize incoming virtual hostname
t5570: test git-daemon's --interpolated-path option
git_connect: let user override virtual-host we send to daemon

Merge branch 'rs/daemon-interpolate'Junio C Hamano Tue, 3 Mar 2015 22:37:04 +0000 (14:37 -0800)

Merge branch 'rs/daemon-interpolate'

"git daemon" looked up the hostname even when "%CH" and "%IP"
interpolations are not requested, which was unnecessary.

* rs/daemon-interpolate:
daemon: use callback to build interpolated path
daemon: look up client-supplied hostname lazily

Merge branch 'km/send-email-getopt-long-workarounds'Junio C Hamano Tue, 3 Mar 2015 22:37:03 +0000 (14:37 -0800)

Merge branch 'km/send-email-getopt-long-workarounds'

Even though we officially haven't dropped Perl 5.8 support, the
Getopt::Long package that came with it does not support "--no-"
prefix to negate a boolean option; manually add support to help
people with older Getopt::Long package.

* km/send-email-getopt-long-workarounds:
git-send-email.perl: support no- prefix with older GetOptions

Merge branch 'jc/apply-ws-fix-expands-report'Junio C Hamano Tue, 3 Mar 2015 22:37:02 +0000 (14:37 -0800)

Merge branch 'jc/apply-ws-fix-expands-report'

"git apply --whitespace=fix" fixed whitespace errors in the common
context lines but did so without reporting.

* jc/apply-ws-fix-expands-report:
apply: detect and mark whitespace errors in context lines when fixing

Merge branch 'jc/apply-beyond-symlink'Junio C Hamano Tue, 3 Mar 2015 22:37:01 +0000 (14:37 -0800)

Merge branch 'jc/apply-beyond-symlink'

"git apply" was not very careful about reading from, removing,
updating and creating paths outside the working tree (under
--index/--cached) or the current directory (when used as a
replacement for GNU patch).

* jc/apply-beyond-symlink:
apply: do not touch a file beyond a symbolic link
apply: do not read from beyond a symbolic link
apply: do not read from the filesystem under --index
apply: reject input that touches outside the working area

t5516: correct misspelled pushInsteadOfAnders Kaseorg Sun, 1 Mar 2015 04:18:14 +0000 (23:18 -0500)

t5516: correct misspelled pushInsteadOf

A future breakage to "git push" to make it incorrectly pay attention
to pushInsteadOf when it should not will be left uncaught without
this change.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

submodule: improve documentation of update subcommandMichal Sojka Mon, 2 Mar 2015 22:57:58 +0000 (23:57 +0100)

submodule: improve documentation of update subcommand

The documentation of 'git submodule update' has several problems:

1) It mentions that value 'none' of submodule.$name.update can be
overridden by --checkout, but other combinations of configuration
values and command line options are not mentioned.

2) The documentation of submodule.$name.update is scattered across three
places, which is confusing.

3) The documentation of submodule.$name.update in gitmodules.txt is
incorrect, because the code always uses the value from .git/config
and never from .gitmodules.

4) Documentation of --force was incomplete, because it is only effective
in case of checkout method of update.

Fix all these problems by documenting submodule.*.update in
git-submodule.txt and make everybody else refer to it.

Helped-by: Junio C Hamano <gitster@pobox.com>
Helped-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Michal Sojka <sojkam1@fel.cvut.cz>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

git-remote.txt: describe behavior without --tags and... Michael J Gruber Mon, 2 Mar 2015 13:08:09 +0000 (14:08 +0100)

git-remote.txt: describe behavior without --tags and --no-tags

Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

kwset: use unsigned char to store values with high... Ben Walton Mon, 2 Mar 2015 19:22:31 +0000 (19:22 +0000)

kwset: use unsigned char to store values with high-bit set

Sun Studio on Solaris issues warnings about improper initialization
values being used when defining tolower_trans_tbl[] in ctype.c. The
array wants to store values with high-bit set and treat them as
values between 128 to 255. Unlike the rest of the Git codebase
where we explicitly specify 'unsigned char' for such variables and
arrays, however, kwset code we borrowed from elsewhere uses 'char'
for this and other variables.

Fix the declarations to explicitly use 'unsigned char' where
necessary to bring it in line with the rest of the Git.

Signed-off-by: Ben Walton <bdwalton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Add hint interactive cleaningJean-Noel Avila Sun, 1 Mar 2015 11:58:25 +0000 (12:58 +0100)

Add hint interactive cleaning

For translators, specify that a [y/N] reply is needed.

Also capitalize the first word in the prompt, as all the other
interactive prompts from this command are capitalized.

Signed-off-by: Jean-Noel Avila <jn.avila@free.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

diff --shortstat --dirstat: remove duplicate outputMårten Kongstad Mon, 2 Mar 2015 15:05:39 +0000 (16:05 +0100)

diff --shortstat --dirstat: remove duplicate output

When --shortstat is used in conjunction with --dirstat=changes, git diff will
output the dirstat information twice: first as calculated by the 'lines'
algorithm, then as calculated by the 'changes' algorithm:

$ git diff --dirstat=changes,10 --shortstat v2.2.0..v2.2.1
23 files changed, 453 insertions(+), 54 deletions(-)
33.5% Documentation/RelNotes/
26.2% t/
46.6% Documentation/RelNotes/
16.6% t/

The same duplication happens for --shortstat together with --dirstat=files, but
not for --shortstat together with --dirstat=lines.

Limit output to only include one dirstat part, calculated as specified
by the --dirstat parameter. Also, add test for this.

Signed-off-by: Mårten Kongstad <marten.kongstad@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

diffcore-rename: avoid processing duplicate destinationsJeff King Fri, 27 Feb 2015 01:42:27 +0000 (20:42 -0500)

diffcore-rename: avoid processing duplicate destinations

The rename code cannot handle an input where we have
duplicate destinations (i.e., more than one diff_filepair in
the queue with the same string in its pair->two->path). We
end up allocating only one slot in the rename_dst mapping.
If we fill in the diff_filepair for that slot, when we
re-queue the results, we may queue that filepair multiple
times. When the diff is finally flushed, the filepair is
processed and free()d multiple times, leading to heap
corruption.

This situation should only happen when a tree diff sees
duplicates in one of the trees (see the added test for a
detailed example). Rather than handle it, the sanest thing
is just to turn off rename detection altogether for the
diff.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

diffcore-rename: split locate_rename_dst into two functionsJeff King Fri, 27 Feb 2015 01:39:48 +0000 (20:39 -0500)

diffcore-rename: split locate_rename_dst into two functions

This function manages the mapping of destination pathnames
to filepairs, and it handles both insertion and lookup. This
makes the return value a bit confusing, as we return a newly
created entry (even though no caller cares), and have no
room to indicate to the caller that an entry already
existed.

Instead, let's break this up into two distinct functions,
both backed by a common binary search. The binary search
will use our normal "return the index if we found something,
or negative index minus one to show where it would have
gone" semantics.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

grep: correct help string for --exclude-standardNguyễn Thái Ngọc Duy Fri, 27 Feb 2015 14:01:58 +0000 (21:01 +0700)

grep: correct help string for --exclude-standard

The current help string is about --no-exclude-standard. But "git grep -h"
would show --exclude-standard instead. Flip the string. See 0a93fb8
(grep: teach --untracked and --exclude-standard options - 2011-09-27)
for more info about these options.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

gettext.c: move get_preferred_languages() from http.cJeff King Thu, 26 Feb 2015 03:04:16 +0000 (22:04 -0500)

gettext.c: move get_preferred_languages() from http.c

Calling setlocale(LC_MESSAGES, ...) directly from http.c, without
including <locale.h>, was causing compilation warnings. Move the
helper function to gettext.c that already includes the header and
where locale-related issues are handled.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Merge branch 'svn-maint-fixes' into svn-fixesJunio C Hamano Thu, 26 Feb 2015 22:03:57 +0000 (14:03 -0800)

Merge branch 'svn-maint-fixes' into svn-fixes

* svn-maint-fixes:
Git::SVN::*: avoid premature FileHandle closure
git-svn: fix localtime=true on non-glibc environments

Git::SVN::*: avoid premature FileHandle closureKyle J. McKay Thu, 26 Feb 2015 13:49:34 +0000 (05:49 -0800)

Git::SVN::*: avoid premature FileHandle closure

Since b19138b (git-svn: Make it incrementally faster by minimizing temp
files, v1.6.0), git-svn has been using the Git.pm temp_acquire and
temp_release mechanism to avoid unnecessary temp file churn and provide
a speed boost.

However, that change introduced a call to temp_acquire inside the
Git::SVN::Fetcher::close_file function for an 'svn_hash' temp file.
Because an SVN::Pool is active at the time this function is called, if
the Git::temp_acquire function ends up actually creating a new
FileHandle for the temp file (which it will the first time it's called
with the name 'svn_hash') that FileHandle will end up in the SVN::Pool
and should that pool have SVN::Pool::clear called on it that FileHandle
will be closed out from under Git::temp_acquire.

Since the only call site to Git::temp_acquire with the name 'svn_hash'
is inside the close_file function, if an 'svn_hash' temp file is ever
created its FileHandle is guaranteed to be created in the active
SVN::Pool.

This has not been a problem in the past because the SVN::Pool was not
being cleared. However, since dfa72fdb (git-svn: reload RA every
log-window-size, v2.2.0) the pool has been getting cleared periodically
at which point the FileHandle for the 'svn_hash' temp file gets closed.
Any subsequent calls to Git::temp_acquire for 'svn_hash', however,
succeed without creating/opening a new temporary file since it still has
the now invalid FileHandle in its cache. Callers that then attempt to
use that FileHandle fail with an error.

We avoid this problem by making sure the 'svn_hash' temp file is created
in the same place the 'svn_delta_...' and 'git_blob_...' temp files are
(and then temp_release'd) so that it can be safely used inside the
close_file function without having its FileHandle end up in an SVN::Pool
that gets cleared.

Additionally the Git.pm cat_blob function creates a bidirectional pipe
FileHandle using the IPC::Open2::open2 function. If that handle is
created too late, it also gets caught up in the SVN::Pool and incorrectly
closed by the SVN::Pool::clear call. But this only seems to happen with
more recent versions of Perl and svn.

To avoid this problem we add an explicit call to _open_cat_blob_if_needed
before the first call to SVN::Pool->new_default to make sure the open2
handle does not end up in the SVN::Pool.

Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

git-svn: fix localtime=true on non-glibc environmentsRyuichi Kokubo Wed, 25 Feb 2015 16:04:41 +0000 (01:04 +0900)

git-svn: fix localtime=true on non-glibc environments

git svn uses POSIX::strftime('%s', $sec, $min, ...) to make unix epoch time.
But lowercase %s formatting character is a GNU extention. This causes problem
in git svn fetch --localtime on non-glibc systems, such as msys or cygwin.
Using Time::Local::timelocal($sec, $min, ...) fixes it.

Signed-off-by: Ryuichi Kokubo <ryu1kkb@gmail.com>
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Notes:
lowercase %s format character in strftime is a GNU extension and not widely supported.
POSIX::strftime affected by underlying crt's strftime because POSIX::strftime just calls crt's one.
Time::Local is good function to replace POSIX::strftime because it's a perl core module function.

Document about Time::Local.
http://perldoc.perl.org/Time/Local.html

These are specifications of strftime.

The GNU C Library Reference Manual.
http://www.gnu.org/software/libc/manual/html_node/Formatting-Calendar-Time.html

perl POSIX module's strftime document. It does not have '%s'.
http://perldoc.perl.org/POSIX.html

strftime document of Microsort Windows C Run-Time library.
https://msdn.microsoft.com/en-us/library/fe06s4ak.aspx

The Open Group's old specification does not have '%s' too.
http://pubs.opengroup.org/onlinepubs/007908799/xsh/strftime.html

On my environment, following problems happened.
- msys : git svn fetch does not progress at all with perl.exe consuming CPU.
- cygwin : git svn fetch progresses but time stamp information is dropped.
Every commits have unix epoch timestamp.

I would like to thank git developer and contibutors.
git helps me so much everyday.
Thank you.

Signed-off-by: Junio C Hamano <gitster@pobox.com>