From: Junio C Hamano Date: Fri, 17 Mar 2017 20:50:23 +0000 (-0700) Subject: Merge branch 'cc/split-index-config' X-Git-Tag: v2.13.0-rc0~111 X-Git-Url: https://git.lorimer.id.au/gitweb.git/diff_plain/94c9b5af703eb70adba349cfbfaaa3029849744c?hp=-c Merge branch 'cc/split-index-config' The experimental "split index" feature has gained a few configuration variables to make it easier to use. * cc/split-index-config: (22 commits) Documentation/git-update-index: explain splitIndex.* Documentation/config: add splitIndex.sharedIndexExpire read-cache: use freshen_shared_index() in read_index_from() read-cache: refactor read_index_from() t1700: test shared index file expiration read-cache: unlink old sharedindex files config: add git_config_get_expiry() from gc.c read-cache: touch shared index files when used sha1_file: make check_and_freshen_file() non static Documentation/config: add splitIndex.maxPercentChange t1700: add tests for splitIndex.maxPercentChange read-cache: regenerate shared index if necessary config: add git_config_get_max_percent_split_change() Documentation/git-update-index: talk about core.splitIndex config var Documentation/config: add information for core.splitIndex t1700: add tests for core.splitIndex update-index: warn in case of split-index incoherency read-cache: add and then use tweak_split_index() split-index: add {add,remove}_split_index() functions config: add git_config_get_split_index() ... --- 94c9b5af703eb70adba349cfbfaaa3029849744c diff --combined Documentation/config.txt index 5e5c2ae5fb,58326c8ff2..eccc012672 --- a/Documentation/config.txt +++ b/Documentation/config.txt @@@ -170,9 -170,6 +170,9 @@@ The position of any attributes with res be turned off by prefixing them with `no` or `no-` (e.g., `noreverse`, `no-ul`, etc). + +An empty color string produces no color effect at all. This can be used +to avoid coloring specific elements without disabling color entirely. ++ For git's pre-defined color slots, the attributes are meant to be reset at the beginning of each item in the colored output. So setting `color.decorate.branch` to `black` will paint that branch name in a @@@ -334,6 -331,10 +334,10 @@@ core.trustctime: crawlers and some backup systems). See linkgit:git-update-index[1]. True by default. + core.splitIndex:: + If true, the split-index feature of the index will be used. + See linkgit:git-update-index[1]. False by default. + core.untrackedCache:: Determines what to do about the untracked cache feature of the index. It will be kept, if this variable is unset or set to @@@ -350,19 -351,16 +354,19 @@@ core.checkStat: all fields, including the sub-second part of mtime and ctime. core.quotePath:: - The commands that output paths (e.g. 'ls-files', - 'diff'), when not given the `-z` option, will quote - "unusual" characters in the pathname by enclosing the - pathname in a double-quote pair and with backslashes the - same way strings in C source code are quoted. If this - variable is set to false, the bytes higher than 0x80 are - not quoted but output as verbatim. Note that double - quote, backslash and control characters are always - quoted without `-z` regardless of the setting of this - variable. + Commands that output paths (e.g. 'ls-files', 'diff'), will + quote "unusual" characters in the pathname by enclosing the + pathname in double-quotes and escaping those characters with + backslashes in the same way C escapes control characters (e.g. + `\t` for TAB, `\n` for LF, `\\` for backslash) or bytes with + values larger than 0x80 (e.g. octal `\302\265` for "micro" in + UTF-8). If this variable is set to false, bytes higher than + 0x80 are not considered "unusual" any more. Double-quotes, + backslash and control characters are always escaped regardless + of the setting of this variable. A simple space character is + not considered "unusual". Many commands can output pathnames + completely verbatim using the `-z` option. The default value + is true. core.eol:: Sets the line ending type to use in the working directory for @@@ -523,12 -521,10 +527,12 @@@ core.logAllRefUpdates: "`$GIT_DIR/logs/`", by appending the new and old SHA-1, the date/time and the reason of the update, but only when the file exists. If this configuration - variable is set to true, missing "`$GIT_DIR/logs/`" + variable is set to `true`, missing "`$GIT_DIR/logs/`" file is automatically created for branch heads (i.e. under - refs/heads/), remote refs (i.e. under refs/remotes/), - note refs (i.e. under refs/notes/), and the symbolic ref HEAD. + `refs/heads/`), remote refs (i.e. under `refs/remotes/`), + note refs (i.e. under `refs/notes/`), and the symbolic ref `HEAD`. + If it is set to `always`, then a missing reflog is automatically + created for any ref under `refs/`. + This information can be used to determine what commit was the tip of a branch "2 days ago". @@@ -791,11 -787,10 +795,11 @@@ core.sparseCheckout: linkgit:git-read-tree[1] for more information. core.abbrev:: - Set the length object names are abbreviated to. If unspecified, - many commands abbreviate to 7 hexdigits, which may not be enough - for abbreviated object names to stay unique for sufficiently long - time. + Set the length object names are abbreviated to. If + unspecified or set to "auto", an appropriate value is + computed based on the approximate number of packed objects + in your repository, which hopefully is enough for + abbreviated object names to stay unique for some time. add.ignoreErrors:: add.ignore-errors (deprecated):: @@@ -1405,12 -1400,6 +1409,12 @@@ gc.autoDetach: Make `git gc --auto` return immediately and run in background if the system supports it. Default is true. +gc.logExpiry:: + If the file gc.log exists, then `git gc --auto` won't run + unless that file is more than 'gc.logExpiry' old. Default is + "1.day". See `gc.pruneExpire` for more ways to specify its + value. + gc.packRefs:: Running `git pack-refs` in a repository renders it unclonable by Git versions prior to 1.5.1.2 over dumb @@@ -1424,9 -1413,7 +1428,9 @@@ gc.pruneExpire: Override the grace period with this config variable. The value "now" may be used to disable this grace period and always prune unreachable objects immediately, or "never" may be used to - suppress pruning. + suppress pruning. This feature helps prevent corruption when + 'git gc' runs concurrently with another process writing to the + repository; see the "NOTES" section of linkgit:git-gc[1]. gc.worktreePruneExpire:: When 'git gc' is run, it calls @@@ -1928,10 -1915,7 +1932,10 @@@ http..*: must match exactly between the config key and the URL. . Host/domain name (e.g., `example.com` in `https://example.com/`). - This field must match exactly between the config key and the URL. + This field must match between the config key and the URL. It is + possible to specify a `*` as part of the host name to match all subdomains + at this level. `https://*.example.com/` for example would match + `https://foo.example.com/`, but not `https://foo.bar.example.com/`. . Port number (e.g., `8080` in `http://example.com:8080/`). This field must match exactly between the config key and the URL. @@@ -1966,17 -1950,6 +1970,17 @@@ Environment variable settings always ov matched against are those given directly to Git commands. This means any URLs visited as a result of a redirection do not participate in matching. +ssh.variant:: + Depending on the value of the environment variables `GIT_SSH` or + `GIT_SSH_COMMAND`, or the config setting `core.sshCommand`, Git + auto-detects whether to adjust its command-line parameters for use + with plink or tortoiseplink, as opposed to the default (OpenSSH). ++ +The config variable `ssh.variant` can be set to override this auto-detection; +valid values are `ssh`, `plink`, `putty` or `tortoiseplink`. Any other value +will be treated as normal ssh. This setting can be overridden via the +environment variable `GIT_SSH_VARIANT`. + i18n.commitEncoding:: Character encoding the commit messages are stored in; Git itself does not care per se, but this information is necessary e.g. when @@@ -2064,10 -2037,6 +2068,10 @@@ log.follow: i.e. it cannot be used to follow multiple files and does not work well on non-linear history. +log.graphColors:: + A list of colors, separated by commas, that can be used to draw + history lines in `git log --graph`. + log.showRoot:: If true, the initial commit will be shown as a big creation event. This is equivalent to a diff against an empty tree. @@@ -2353,52 -2322,6 +2357,52 @@@ pretty.: Note that an alias with the same name as a built-in format will be silently ignored. +protocol.allow:: + If set, provide a user defined default policy for all protocols which + don't explicitly have a policy (`protocol..allow`). By default, + if unset, known-safe protocols (http, https, git, ssh, file) have a + default policy of `always`, known-dangerous protocols (ext) have a + default policy of `never`, and all other protocols have a default + policy of `user`. Supported policies: ++ +-- + +* `always` - protocol is always able to be used. + +* `never` - protocol is never able to be used. + +* `user` - protocol is only able to be used when `GIT_PROTOCOL_FROM_USER` is + either unset or has a value of 1. This policy should be used when you want a + protocol to be directly usable by the user but don't want it used by commands which + execute clone/fetch/push commands without user input, e.g. recursive + submodule initialization. + +-- + +protocol..allow:: + Set a policy to be used by protocol `` with clone/fetch/push + commands. See `protocol.allow` above for the available policies. ++ +The protocol names currently used by git are: ++ +-- + - `file`: any local file-based path (including `file://` URLs, + or local paths) + + - `git`: the anonymous git protocol over a direct TCP + connection (or proxy, if configured) + + - `ssh`: git over ssh (including `host:path` syntax, + `ssh://`, etc). + + - `http`: git over http, both "smart http" and "dumb http". + Note that this does _not_ include `https`; if you want to configure + both, you must do so individually. + + - any external helpers are named by their protocol (e.g., use + `hg` to allow the `git-remote-hg` helper) +-- + pull.ff:: By default, Git does not create an extra merge commit when merging a commit that is a descendant of the current commit. Instead, the @@@ -2850,6 -2773,31 +2854,31 @@@ showbranch.default: The default set of branches for linkgit:git-show-branch[1]. See linkgit:git-show-branch[1]. + splitIndex.maxPercentChange:: + When the split index feature is used, this specifies the + percent of entries the split index can contain compared to the + total number of entries in both the split index and the shared + index before a new shared index is written. + The value should be between 0 and 100. If the value is 0 then + a new shared index is always written, if it is 100 a new + shared index is never written. + By default the value is 20, so a new shared index is written + if the number of entries in the split index would be greater + than 20 percent of the total number of entries. + See linkgit:git-update-index[1]. + + splitIndex.sharedIndexExpire:: + When the split index feature is used, shared index files that + were not modified since the time this variable specifies will + be removed when a new shared index file is created. The value + "now" expires all entries immediately, and "never" suppresses + expiration altogether. + The default value is "2.weeks.ago". + Note that a shared index file is considered modified (for the + purpose of expiration) each time a new split-index file is + either created based on it or read from it. + See linkgit:git-update-index[1]. + status.relativePaths:: By default, linkgit:git-status[1] shows paths relative to the current directory. Setting this variable to `false` shows paths @@@ -2972,7 -2920,7 +3001,7 @@@ submodule.alternateLocation: value is set to `superproject` the submodule to be cloned computes its alternates location relative to the superprojects alternate. -submodule.alternateErrorStrategy +submodule.alternateErrorStrategy:: Specifies how to treat errors with the alternates for a submodule as computed via `submodule.alternateLocation`. Possible values are `ignore`, `info`, `die`. Default is `die`. @@@ -3021,11 -2969,6 +3050,11 @@@ is omitted from the advertisements but `refs/namespaces/bar/refs/heads/master` are still advertised as so-called "have" lines. In order to match refs before stripping, add a `^` in front of the ref name. If you combine `!` and `^`, `!` must be specified first. ++ +Even if you hide refs, a client may still be able to steal the target +objects via the techniques described in the "SECURITY" section of the +linkgit:gitnamespaces[7] man page; it's best to keep private data in a +separate repository. transfer.unpackLimit:: When `fetch.unpackLimit` or `receive.unpackLimit` are @@@ -3035,7 -2978,7 +3064,7 @@@ uploadarchive.allowUnreachable:: If true, allow clients to use `git archive --remote` to request any tree, whether reachable from the ref tips or not. See the - discussion in the `SECURITY` section of + discussion in the "SECURITY" section of linkgit:git-upload-archive[1] for more details. Defaults to `false`. @@@ -3049,23 -2992,12 +3078,23 @@@ uploadpack.allowTipSHA1InWant: When `uploadpack.hideRefs` is in effect, allow `upload-pack` to accept a fetch request that asks for an object at the tip of a hidden ref (by default, such a request is rejected). - see also `uploadpack.hideRefs`. + See also `uploadpack.hideRefs`. Even if this is false, a client + may be able to steal objects via the techniques described in the + "SECURITY" section of the linkgit:gitnamespaces[7] man page; it's + best to keep private data in a separate repository. uploadpack.allowReachableSHA1InWant:: Allow `upload-pack` to accept a fetch request that asks for an object that is reachable from any ref tip. However, note that calculating object reachability is computationally expensive. + Defaults to `false`. Even if this is false, a client may be able + to steal objects via the techniques described in the "SECURITY" + section of the linkgit:gitnamespaces[7] man page; it's best to + keep private data in a separate repository. + +uploadpack.allowAnySHA1InWant:: + Allow `upload-pack` to accept a fetch request that asks for any + object at all. Defaults to `false`. uploadpack.keepAlive:: @@@ -3145,39 -3077,17 +3174,39 @@@ user.signingKey: This option is passed unchanged to gpg's --local-user parameter, so you may specify a key using any method that gpg supports. -versionsort.prereleaseSuffix:: - When version sort is used in linkgit:git-tag[1], prerelease - tags (e.g. "1.0-rc1") may appear after the main release - "1.0". By specifying the suffix "-rc" in this variable, - "1.0-rc1" will appear before "1.0". -+ -This variable can be specified multiple times, once per suffix. The -order of suffixes in the config file determines the sorting order -(e.g. if "-pre" appears before "-rc" in the config file then 1.0-preXX -is sorted before 1.0-rcXX). The sorting order between different -suffixes is undefined if they are in multiple config files. +versionsort.prereleaseSuffix (deprecated):: + Deprecated alias for `versionsort.suffix`. Ignored if + `versionsort.suffix` is set. + +versionsort.suffix:: + Even when version sort is used in linkgit:git-tag[1], tagnames + with the same base version but different suffixes are still sorted + lexicographically, resulting e.g. in prerelease tags appearing + after the main release (e.g. "1.0-rc1" after "1.0"). This + variable can be specified to determine the sorting order of tags + with different suffixes. ++ +By specifying a single suffix in this variable, any tagname containing +that suffix will appear before the corresponding main release. E.g. if +the variable is set to "-rc", then all "1.0-rcX" tags will appear before +"1.0". If specified multiple times, once per suffix, then the order of +suffixes in the configuration will determine the sorting order of tagnames +with those suffixes. E.g. if "-pre" appears before "-rc" in the +configuration, then all "1.0-preX" tags will be listed before any +"1.0-rcX" tags. The placement of the main release tag relative to tags +with various suffixes can be determined by specifying the empty suffix +among those other suffixes. E.g. if the suffixes "-rc", "", "-ck" and +"-bfs" appear in the configuration in this order, then all "v4.8-rcX" tags +are listed first, followed by "v4.8", then "v4.8-ckX" and finally +"v4.8-bfsX". ++ +If more than one suffixes match the same tagname, then that tagname will +be sorted according to the suffix which starts at the earliest position in +the tagname. If more than one different matching suffixes start at +that earliest position, then that tagname will be sorted according to the +longest of those suffixes. +The sorting order between different suffixes is undefined if they are +in multiple config files. web.browser:: Specify a web browser that may be used by some commands. diff --combined builtin/gc.c index a2b9e8924e,1e40d45aa2..c2c61a57bb --- a/builtin/gc.c +++ b/builtin/gc.c @@@ -33,8 -33,6 +33,8 @@@ static int aggressive_window = 250 static int gc_auto_threshold = 6700; static int gc_auto_pack_limit = 50; static int detach_auto = 1; +static unsigned long gc_log_expire_time; +static const char *gc_log_expire = "1.day.ago"; static const char *prune_expire = "2.weeks.ago"; static const char *prune_worktrees_expire = "3.months.ago"; @@@ -64,42 -62,13 +64,31 @@@ static void report_pack_garbage(unsigne string_list_append(&pack_garbage, path); } - static void git_config_date_string(const char *key, const char **output) - { - if (git_config_get_string_const(key, output)) - return; - if (strcmp(*output, "now")) { - unsigned long now = approxidate("now"); - if (approxidate(*output) >= now) - git_die_config(key, _("Invalid %s: '%s'"), key, *output); - } - } - static void process_log_file(void) { struct stat st; - if (!fstat(get_lock_file_fd(&log_lock), &st) && st.st_size) + if (fstat(get_lock_file_fd(&log_lock), &st)) { + /* + * Perhaps there was an i/o error or another + * unlikely situation. Try to make a note of + * this in gc.log along with any existing + * messages. + */ + int saved_errno = errno; + fprintf(stderr, _("Failed to fstat %s: %s"), + get_tempfile_path(&log_lock.tempfile), + strerror(saved_errno)); + fflush(stderr); commit_lock_file(&log_lock); - else + errno = saved_errno; + } else if (st.st_size) { + /* There was some error recorded in the lock file */ + commit_lock_file(&log_lock); + } else { + /* No error, clean up any old gc.log */ + unlink(git_path("gc.log")); rollback_lock_file(&log_lock); + } } static void process_log_file_at_exit(void) @@@ -131,10 -100,8 +120,10 @@@ static void gc_config(void git_config_get_int("gc.auto", &gc_auto_threshold); git_config_get_int("gc.autopacklimit", &gc_auto_pack_limit); git_config_get_bool("gc.autodetach", &detach_auto); - git_config_date_string("gc.pruneexpire", &prune_expire); - git_config_date_string("gc.worktreepruneexpire", &prune_worktrees_expire); - git_config_date_string("gc.logexpiry", &gc_log_expire); + git_config_get_expiry("gc.pruneexpire", &prune_expire); + git_config_get_expiry("gc.worktreepruneexpire", &prune_worktrees_expire); ++ git_config_get_expiry("gc.logexpiry", &gc_log_expire); + git_config(git_default_config, NULL); } @@@ -213,11 -180,6 +202,11 @@@ static void add_repack_all_option(void } } +static void add_repack_incremental_option(void) +{ + argv_array_push(&repack, "--no-write-bitmap-index"); +} + static int need_to_gc(void) { /* @@@ -235,9 -197,7 +224,9 @@@ */ if (too_many_packs()) add_repack_all_option(); - else if (!too_many_loose_objects()) + else if (too_many_loose_objects()) + add_repack_incremental_option(); + else return 0; if (run_hook_le(NULL, "pre-auto-gc", NULL)) @@@ -312,34 -272,19 +301,34 @@@ static const char *lock_repo_for_gc(in static int report_last_gc_error(void) { struct strbuf sb = STRBUF_INIT; - int ret; + int ret = 0; + struct stat st; + char *gc_log_path = git_pathdup("gc.log"); + + if (stat(gc_log_path, &st)) { + if (errno == ENOENT) + goto done; + + ret = error_errno(_("Can't stat %s"), gc_log_path); + goto done; + } + + if (st.st_mtime < gc_log_expire_time) + goto done; - ret = strbuf_read_file(&sb, git_path("gc.log"), 0); + ret = strbuf_read_file(&sb, gc_log_path, 0); if (ret > 0) - return error(_("The last gc run reported the following. " + ret = error(_("The last gc run reported the following. " "Please correct the root cause\n" "and remove %s.\n" "Automatic cleanup will not be performed " "until the file is removed.\n\n" "%s"), - git_path("gc.log"), sb.buf); + gc_log_path, sb.buf); strbuf_release(&sb); - return 0; +done: + free(gc_log_path); + return ret; } static int gc_before_repack(void) @@@ -386,10 -331,7 +375,10 @@@ int cmd_gc(int argc, const char **argv argv_array_pushl(&prune_worktrees, "worktree", "prune", "--expire", NULL); argv_array_pushl(&rerere, "rerere", "gc", NULL); + /* default expiry time, overwritten in gc_config */ gc_config(); + if (parse_expiry_date(gc_log_expire, &gc_log_expire_time)) + die(_("Failed to parse gc.logexpiry value %s"), gc_log_expire); if (pack_refs < 0) pack_refs = !is_bare_repository(); @@@ -488,8 -430,5 +477,8 @@@ warning(_("There are too many unreachable loose objects; " "run 'git prune' to remove them.")); + if (!daemonized) + unlink(git_path("gc.log")); + return 0; } diff --combined cache.h index c958269715,279415afbd..d3265cca12 --- a/cache.h +++ b/cache.h @@@ -507,16 -507,14 +507,16 @@@ extern int is_nonbare_repository_dir(st #define READ_GITFILE_ERR_NO_PATH 6 #define READ_GITFILE_ERR_NOT_A_REPO 7 #define READ_GITFILE_ERR_TOO_LARGE 8 +extern void read_gitfile_error_die(int error_code, const char *path, const char *dir); extern const char *read_gitfile_gently(const char *path, int *return_error_code); #define read_gitfile(path) read_gitfile_gently((path), NULL) -extern const char *resolve_gitdir(const char *suspect); +extern const char *resolve_gitdir_gently(const char *suspect, int *return_error_code); +#define resolve_gitdir(path) resolve_gitdir_gently((path), NULL) + extern void set_git_work_tree(const char *tree); #define ALTERNATE_DB_ENVIRONMENT "GIT_ALTERNATE_OBJECT_DIRECTORIES" -extern const char **get_pathspec(const char *prefix, const char **pathspec); extern void setup_work_tree(void); extern const char *setup_git_directory_gently(int *); extern const char *setup_git_directory(void); @@@ -577,26 -575,7 +577,26 @@@ extern int verify_path(const char *path extern int index_dir_exists(struct index_state *istate, const char *name, int namelen); extern void adjust_dirname_case(struct index_state *istate, char *name); extern struct cache_entry *index_file_exists(struct index_state *istate, const char *name, int namelen, int igncase); + +/* + * Searches for an entry defined by name and namelen in the given index. + * If the return value is positive (including 0) it is the position of an + * exact match. If the return value is negative, the negated value minus 1 + * is the position where the entry would be inserted. + * Example: The current index consists of these files and its stages: + * + * b#0, d#0, f#1, f#3 + * + * index_name_pos(&index, "a", 1) -> -1 + * index_name_pos(&index, "b", 1) -> 0 + * index_name_pos(&index, "c", 1) -> -2 + * index_name_pos(&index, "d", 1) -> 1 + * index_name_pos(&index, "e", 1) -> -3 + * index_name_pos(&index, "f", 1) -> -3 + * index_name_pos(&index, "g", 1) -> -5 + */ extern int index_name_pos(const struct index_state *, const char *name, int namelen); + #define ADD_CACHE_OK_TO_ADD 1 /* Ok to add */ #define ADD_CACHE_OK_TO_REPLACE 2 /* Ok to replace file/directory */ #define ADD_CACHE_SKIP_DFCHECK 4 /* Ok to skip DF conflict checks */ @@@ -605,10 -584,7 +605,10 @@@ #define ADD_CACHE_KEEP_CACHE_TREE 32 /* Do not invalidate cache-tree */ extern int add_index_entry(struct index_state *, struct cache_entry *ce, int option); extern void rename_index_entry_at(struct index_state *, int pos, const char *new_name); + +/* Remove entry, return true if there are more entries to go. */ extern int remove_index_entry_at(struct index_state *, int pos); + extern void remove_marked_cache_entries(struct index_state *istate); extern int remove_file_from_index(struct index_state *, const char *path); #define ADD_CACHE_VERBOSE 1 @@@ -616,24 -592,14 +616,24 @@@ #define ADD_CACHE_IGNORE_ERRORS 4 #define ADD_CACHE_IGNORE_REMOVAL 8 #define ADD_CACHE_INTENT 16 +/* + * These two are used to add the contents of the file at path + * to the index, marking the working tree up-to-date by storing + * the cached stat info in the resulting cache entry. A caller + * that has already run lstat(2) on the path can call + * add_to_index(), and all others can call add_file_to_index(); + * the latter will do necessary lstat(2) internally before + * calling the former. + */ extern int add_to_index(struct index_state *, const char *path, struct stat *, int flags); extern int add_file_to_index(struct index_state *, const char *path, int flags); + extern struct cache_entry *make_cache_entry(unsigned int mode, const unsigned char *sha1, const char *path, int stage, unsigned int refresh_options); extern int chmod_index_entry(struct index_state *, struct cache_entry *ce, char flip); extern int ce_same_name(const struct cache_entry *a, const struct cache_entry *b); extern void set_object_name_for_intent_to_add_entry(struct cache_entry *ce); extern int index_name_is_other(const struct index_state *, const char *, int); -extern void *read_blob_data_from_index(struct index_state *, const char *, unsigned long *); +extern void *read_blob_data_from_index(const struct index_state *, const char *, unsigned long *); /* do stat comparison even if CE_VALID is true */ #define CE_MATCH_IGNORE_VALID 01 @@@ -695,6 -661,7 +695,6 @@@ extern int minimum_abbrev, default_abbr extern int ignore_case; extern int assume_unchanged; extern int prefer_symlink_refs; -extern int log_all_ref_updates; extern int warn_ambiguous_refs; extern int warn_on_object_refname_ambiguity; extern const char *apply_default_whitespace; @@@ -703,7 -670,7 +703,7 @@@ extern const char *git_attributes_file extern const char *git_hooks_path; extern int zlib_compression_level; extern int core_compression_level; -extern int core_compression_seen; +extern int pack_compression_level; extern size_t packed_git_window_size; extern size_t packed_git_limit; extern size_t delta_base_cache_limit; @@@ -762,14 -729,6 +762,14 @@@ enum hide_dotfiles_type }; extern enum hide_dotfiles_type hide_dotfiles; +enum log_refs_config { + LOG_REFS_UNSET = -1, + LOG_REFS_NONE = 0, + LOG_REFS_NORMAL, + LOG_REFS_ALWAYS +}; +extern enum log_refs_config log_all_ref_updates; + enum branch_track { BRANCH_TRACK_UNSPECIFIED = -1, BRANCH_TRACK_NEVER = 0, @@@ -1045,6 -1004,9 +1045,6 @@@ static inline int is_empty_tree_oid(con return !hashcmp(oid->hash, EMPTY_TREE_SHA1_BIN); } - -int git_mkstemp(char *path, size_t n, const char *template); - /* set default permissions by passing mode arguments to open(2) */ int git_mkstemps_mode(char *pattern, int suffix_len, int mode); int git_mkstemp_mode(char *pattern, int mode); @@@ -1069,9 -1031,8 +1069,9 @@@ int adjust_shared_perm(const char *path /* * Create the directory containing the named path, using care to be - * somewhat safe against races. Return one of the scld_error values - * to indicate success/failure. + * somewhat safe against races. Return one of the scld_error values to + * indicate success/failure. On error, set errno to describe the + * problem. * * SCLD_VANISHED indicates that one of the ancestor directories of the * path existed at one point during the function call and then @@@ -1095,49 -1056,6 +1095,49 @@@ enum scld_error enum scld_error safe_create_leading_directories(char *path); enum scld_error safe_create_leading_directories_const(const char *path); +/* + * Callback function for raceproof_create_file(). This function is + * expected to do something that makes dirname(path) permanent despite + * the fact that other processes might be cleaning up empty + * directories at the same time. Usually it will create a file named + * path, but alternatively it could create another file in that + * directory, or even chdir() into that directory. The function should + * return 0 if the action was completed successfully. On error, it + * should return a nonzero result and set errno. + * raceproof_create_file() treats two errno values specially: + * + * - ENOENT -- dirname(path) does not exist. In this case, + * raceproof_create_file() tries creating dirname(path) + * (and any parent directories, if necessary) and calls + * the function again. + * + * - EISDIR -- the file already exists and is a directory. In this + * case, raceproof_create_file() removes the directory if + * it is empty (and recursively any empty directories that + * it contains) and calls the function again. + * + * Any other errno causes raceproof_create_file() to fail with the + * callback's return value and errno. + * + * Obviously, this function should be OK with being called again if it + * fails with ENOENT or EISDIR. In other scenarios it will not be + * called again. + */ +typedef int create_file_fn(const char *path, void *cb); + +/* + * Create a file in dirname(path) by calling fn, creating leading + * directories if necessary. Retry a few times in case we are racing + * with another process that is trying to clean up the directory that + * contains path. See the documentation for create_file_fn for more + * details. + * + * Return the value and set the errno that resulted from the most + * recent call of fn. fn is always called at least once, and will be + * called more than once if it returns ENOENT or EISDIR. + */ +int raceproof_create_file(const char *path, create_file_fn fn, void *cb); + int mkdir_in_gitdir(const char *path); extern char *expand_user_path(const char *path); const char *enter_repo(const char *path, int strict); @@@ -1146,13 -1064,9 +1146,13 @@@ static inline int is_absolute_path(cons return is_dir_sep(path[0]) || has_dos_drive_prefix(path); } int is_directory(const char *); +char *strbuf_realpath(struct strbuf *resolved, const char *path, + int die_on_error); const char *real_path(const char *path); const char *real_path_if_valid(const char *path); +char *real_pathdup(const char *path, int die_on_error); const char *absolute_path(const char *path); +char *absolute_pathdup(const char *path); const char *remove_leading_path(const char *in, const char *prefix); const char *relative_path(const char *in, const char *prefix, struct strbuf *sb); int normalize_path_copy_len(char *dst, const char *src, int *prefix_len); @@@ -1211,8 -1125,7 +1211,8 @@@ extern int write_sha1_file(const void * extern int hash_sha1_file_literally(const void *buf, unsigned long len, const char *type, unsigned char *sha1, unsigned flags); extern int pretend_sha1_file(void *, unsigned long, enum object_type, unsigned char *); extern int force_object_loose(const unsigned char *sha1, time_t mtime); -extern int git_open(const char *name); +extern int git_open_cloexec(const char *name, int flags); +#define git_open(name) git_open_cloexec(name, O_RDONLY) extern void *map_sha1_file(const unsigned char *sha1, unsigned long *size); extern int unpack_sha1_header(git_zstream *stream, unsigned char *map, unsigned long mapsize, void *buffer, unsigned long bufsiz); extern int parse_sha1_header(const char *hdr, unsigned long *sizep); @@@ -1226,19 -1139,6 +1226,19 @@@ extern int finalize_object_file(const c extern int has_sha1_pack(const unsigned char *sha1); +/* + * Open the loose object at path, check its sha1, and return the contents, + * type, and size. If the object is a blob, then "contents" may return NULL, + * to allow streaming of large blobs. + * + * Returns 0 on success, negative on error (details may be written to stderr). + */ +int read_loose_object(const char *path, + const unsigned char *expected_sha1, + enum object_type *type, + unsigned long *size, + void **contents); + /* * Return true iff we have an object named sha1, whether local or in * an alternate object database, and whether packed or loose. This @@@ -1270,6 -1170,9 +1270,9 @@@ extern int has_pack_index(const unsigne extern void assert_sha1_type(const unsigned char *sha1, enum object_type expect); + /* Helper to check and "touch" a file */ + extern int check_and_freshen_file(const char *fn, int freshen); + extern const signed char hexval_table[256]; static inline unsigned int hexval(unsigned char c) { @@@ -1360,37 -1263,7 +1363,37 @@@ extern char *oid_to_hex_r(char *out, co extern char *sha1_to_hex(const unsigned char *sha1); /* static buffer result! */ extern char *oid_to_hex(const struct object_id *oid); /* same static buffer as sha1_to_hex */ -extern int interpret_branch_name(const char *str, int len, struct strbuf *); +/* + * This reads short-hand syntax that not only evaluates to a commit + * object name, but also can act as if the end user spelled the name + * of the branch from the command line. + * + * - "@{-N}" finds the name of the Nth previous branch we were on, and + * places the name of the branch in the given buf and returns the + * number of characters parsed if successful. + * + * - "@{upstream}" finds the name of the other ref that + * is configured to merge with (missing defaults + * to the current branch), and places the name of the branch in the + * given buf and returns the number of characters parsed if + * successful. + * + * If the input is not of the accepted format, it returns a negative + * number to signal an error. + * + * If the input was ok but there are not N branch switches in the + * reflog, it returns 0. + * + * If "allowed" is non-zero, it is a treated as a bitfield of allowable + * expansions: local branches ("refs/heads/"), remote branches + * ("refs/remotes/"), or "HEAD". If no "allowed" bits are set, any expansion is + * allowed, even ones to refs outside of those namespaces. + */ +#define INTERPRET_BRANCH_LOCAL (1<<0) +#define INTERPRET_BRANCH_REMOTE (1<<1) +#define INTERPRET_BRANCH_HEAD (1<<2) +extern int interpret_branch_name(const char *str, int len, struct strbuf *, + unsigned allowed); extern int get_oid_mb(const char *str, struct object_id *oid); extern int validate_headref(const char *ref); @@@ -1820,8 -1693,6 +1823,8 @@@ extern int git_default_config(const cha extern int git_config_from_file(config_fn_t fn, const char *, void *); extern int git_config_from_mem(config_fn_t fn, const enum config_origin_type, const char *name, const char *buf, size_t len, void *data); +extern int git_config_from_blob_sha1(config_fn_t fn, const char *name, + const unsigned char *sha1, void *data); extern void git_config_push_parameter(const char *text); extern int git_config_from_parameters(config_fn_t fn, void *data); extern void git_config(config_fn_t fn, void *); @@@ -1890,11 -1761,8 +1893,11 @@@ extern int git_config_include(const cha * * (i.e., what gets handed to a config_fn_t). The caller provides the section; * we return -1 if it does not match, 0 otherwise. The subsection and key - * out-parameters are filled by the function (and subsection is NULL if it is + * out-parameters are filled by the function (and *subsection is NULL if it is * missing). + * + * If the subsection pointer-to-pointer passed in is NULL, returns 0 only if + * there is no subsection at all. */ extern int parse_config_key(const char *var, const char *section, @@@ -1956,6 -1824,11 +1959,11 @@@ extern int git_config_get_bool_or_int(c extern int git_config_get_maybe_bool(const char *key, int *dest); extern int git_config_get_pathname(const char *key, const char **dest); extern int git_config_get_untracked_cache(void); + extern int git_config_get_split_index(void); + extern int git_config_get_max_percent_split_change(void); + + /* This dies if the configured or default date is in the future */ + extern int git_config_get_expiry(const char *key, const char **output); /* * This is a hack for test programs like test-dump-untracked-cache to diff --combined config.c index 0e9e1ebefc,d6c8f8f3ba..bd286b3e36 --- a/config.c +++ b/config.c @@@ -66,8 -66,6 +66,8 @@@ static struct key_value_info *current_c */ static enum config_scope current_parsing_scope; +static int core_compression_seen; +static int pack_compression_seen; static int zlib_compression_seen; /* @@@ -201,105 -199,11 +201,105 @@@ void git_config_push_parameter(const ch strbuf_release(&env); } +static inline int iskeychar(int c) +{ + return isalnum(c) || c == '-'; +} + +/* + * Auxiliary function to sanity-check and split the key into the section + * identifier and variable name. + * + * Returns 0 on success, -1 when there is an invalid character in the key and + * -2 if there is no section name in the key. + * + * store_key - pointer to char* which will hold a copy of the key with + * lowercase section and variable name + * baselen - pointer to int which will hold the length of the + * section + subsection part, can be NULL + */ +static int git_config_parse_key_1(const char *key, char **store_key, int *baselen_, int quiet) +{ + int i, dot, baselen; + const char *last_dot = strrchr(key, '.'); + + /* + * Since "key" actually contains the section name and the real + * key name separated by a dot, we have to know where the dot is. + */ + + if (last_dot == NULL || last_dot == key) { + if (!quiet) + error("key does not contain a section: %s", key); + return -CONFIG_NO_SECTION_OR_NAME; + } + + if (!last_dot[1]) { + if (!quiet) + error("key does not contain variable name: %s", key); + return -CONFIG_NO_SECTION_OR_NAME; + } + + baselen = last_dot - key; + if (baselen_) + *baselen_ = baselen; + + /* + * Validate the key and while at it, lower case it for matching. + */ + if (store_key) + *store_key = xmallocz(strlen(key)); + + dot = 0; + for (i = 0; key[i]; i++) { + unsigned char c = key[i]; + if (c == '.') + dot = 1; + /* Leave the extended basename untouched.. */ + if (!dot || i > baselen) { + if (!iskeychar(c) || + (i == baselen + 1 && !isalpha(c))) { + if (!quiet) + error("invalid key: %s", key); + goto out_free_ret_1; + } + c = tolower(c); + } else if (c == '\n') { + if (!quiet) + error("invalid key (newline): %s", key); + goto out_free_ret_1; + } + if (store_key) + (*store_key)[i] = c; + } + + return 0; + +out_free_ret_1: + if (store_key) { + free(*store_key); + *store_key = NULL; + } + return -CONFIG_INVALID_KEY; +} + +int git_config_parse_key(const char *key, char **store_key, int *baselen) +{ + return git_config_parse_key_1(key, store_key, baselen, 0); +} + +int git_config_key_is_valid(const char *key) +{ + return !git_config_parse_key_1(key, NULL, NULL, 1); +} + int git_config_parse_parameter(const char *text, config_fn_t fn, void *data) { const char *value; + char *canonical_name; struct strbuf **pair; + int ret; pair = strbuf_split_str(text, '=', 2); if (!pair[0]) @@@ -317,15 -221,13 +317,15 @@@ strbuf_list_free(pair); return error("bogus config parameter: %s", text); } - strbuf_tolower(pair[0]); - if (fn(pair[0]->buf, value, data) < 0) { - strbuf_list_free(pair); - return -1; + + if (git_config_parse_key(pair[0]->buf, &canonical_name, NULL)) { + ret = -1; + } else { + ret = (fn(canonical_name, value, data) < 0) ? -1 : 0; + free(canonical_name); } strbuf_list_free(pair); - return 0; + return ret; } int git_config_from_parameters(config_fn_t fn, void *data) @@@ -452,6 -354,11 +452,6 @@@ static char *parse_value(void } } -static inline int iskeychar(int c) -{ - return isalnum(c) || c == '-'; -} - static int get_value(config_fn_t fn, void *data, struct strbuf *name) { int c; @@@ -917,12 -824,7 +917,12 @@@ static int git_default_core_config(cons } if (!strcmp(var, "core.logallrefupdates")) { - log_all_ref_updates = git_config_bool(var, value); + if (value && !strcasecmp(value, "always")) + log_all_ref_updates = LOG_REFS_ALWAYS; + else if (git_config_bool(var, value)) + log_all_ref_updates = LOG_REFS_NORMAL; + else + log_all_ref_updates = LOG_REFS_NONE; return 0; } @@@ -932,16 -834,10 +932,16 @@@ } if (!strcmp(var, "core.abbrev")) { - int abbrev = git_config_int(var, value); - if (abbrev < minimum_abbrev || abbrev > 40) - return -1; - default_abbrev = abbrev; + if (!value) + return config_error_nonbool(var); + if (!strcasecmp(value, "auto")) + default_abbrev = -1; + else { + int abbrev = git_config_int(var, value); + if (abbrev < minimum_abbrev || abbrev > 40) + return error("abbrev length out of range: %d", abbrev); + default_abbrev = abbrev; + } return 0; } @@@ -969,8 -865,6 +969,8 @@@ core_compression_seen = 1; if (!zlib_compression_seen) zlib_compression_level = level; + if (!pack_compression_seen) + pack_compression_level = level; return 0; } @@@ -1231,18 -1125,6 +1231,18 @@@ int git_default_config(const char *var pack_size_limit_cfg = git_config_ulong(var, value); return 0; } + + if (!strcmp(var, "pack.compression")) { + int level = git_config_int(var, value); + if (level == -1) + level = Z_DEFAULT_COMPRESSION; + else if (level < 0 || level > Z_BEST_COMPRESSION) + die(_("bad pack compression level %d"), level); + pack_compression_level = level; + pack_compression_seen = 1; + return 0; + } + /* Add other config variables here and to Documentation/config.txt. */ return 0; } @@@ -1332,10 -1214,10 +1332,10 @@@ int git_config_from_mem(config_fn_t fn return do_config_from(&top, fn, data); } -static int git_config_from_blob_sha1(config_fn_t fn, - const char *name, - const unsigned char *sha1, - void *data) +int git_config_from_blob_sha1(config_fn_t fn, + const char *name, + const unsigned char *sha1, + void *data) { enum object_type type; char *buf; @@@ -1803,6 -1685,19 +1803,19 @@@ int git_config_get_pathname(const char return ret; } + int git_config_get_expiry(const char *key, const char **output) + { + int ret = git_config_get_string_const(key, output); + if (ret) + return ret; + if (strcmp(*output, "now")) { + unsigned long now = approxidate("now"); + if (approxidate(*output) >= now) + git_die_config(key, _("Invalid %s: '%s'"), key, *output); + } + return ret; + } + int git_config_get_untracked_cache(void) { int val = -1; @@@ -1819,14 -1714,39 +1832,39 @@@ if (!strcasecmp(v, "keep")) return -1; - error("unknown core.untrackedCache value '%s'; " - "using 'keep' default value", v); + error(_("unknown core.untrackedCache value '%s'; " + "using 'keep' default value"), v); return -1; } return -1; /* default value */ } + int git_config_get_split_index(void) + { + int val; + + if (!git_config_get_maybe_bool("core.splitindex", &val)) + return val; + + return -1; /* default value */ + } + + int git_config_get_max_percent_split_change(void) + { + int val = -1; + + if (!git_config_get_int("splitindex.maxpercentchange", &val)) { + if (0 <= val && val <= 100) + return val; + + return error(_("splitIndex.maxPercentChange value '%d' " + "should be between 0 and 100"), val); + } + + return -1; /* default value */ + } + NORETURN void git_die_config_linenr(const char *key, const char *filename, int linenr) { @@@ -2080,6 -2000,93 +2118,6 @@@ void git_config_set(const char *key, co git_config_set_multivar(key, value, NULL, 0); } -/* - * Auxiliary function to sanity-check and split the key into the section - * identifier and variable name. - * - * Returns 0 on success, -1 when there is an invalid character in the key and - * -2 if there is no section name in the key. - * - * store_key - pointer to char* which will hold a copy of the key with - * lowercase section and variable name - * baselen - pointer to int which will hold the length of the - * section + subsection part, can be NULL - */ -static int git_config_parse_key_1(const char *key, char **store_key, int *baselen_, int quiet) -{ - int i, dot, baselen; - const char *last_dot = strrchr(key, '.'); - - /* - * Since "key" actually contains the section name and the real - * key name separated by a dot, we have to know where the dot is. - */ - - if (last_dot == NULL || last_dot == key) { - if (!quiet) - error("key does not contain a section: %s", key); - return -CONFIG_NO_SECTION_OR_NAME; - } - - if (!last_dot[1]) { - if (!quiet) - error("key does not contain variable name: %s", key); - return -CONFIG_NO_SECTION_OR_NAME; - } - - baselen = last_dot - key; - if (baselen_) - *baselen_ = baselen; - - /* - * Validate the key and while at it, lower case it for matching. - */ - if (store_key) - *store_key = xmallocz(strlen(key)); - - dot = 0; - for (i = 0; key[i]; i++) { - unsigned char c = key[i]; - if (c == '.') - dot = 1; - /* Leave the extended basename untouched.. */ - if (!dot || i > baselen) { - if (!iskeychar(c) || - (i == baselen + 1 && !isalpha(c))) { - if (!quiet) - error("invalid key: %s", key); - goto out_free_ret_1; - } - c = tolower(c); - } else if (c == '\n') { - if (!quiet) - error("invalid key (newline): %s", key); - goto out_free_ret_1; - } - if (store_key) - (*store_key)[i] = c; - } - - return 0; - -out_free_ret_1: - if (store_key) { - free(*store_key); - *store_key = NULL; - } - return -CONFIG_INVALID_KEY; -} - -int git_config_parse_key(const char *key, char **store_key, int *baselen) -{ - return git_config_parse_key_1(key, store_key, baselen, 0); -} - -int git_config_key_is_valid(const char *key) -{ - return !git_config_parse_key_1(key, NULL, NULL, 1); -} - /* * If value==NULL, unset in (remove from) config, * if value_regex!=NULL, disregard key/value pairs where value does not match. @@@ -2225,12 -2232,7 +2263,12 @@@ int git_config_set_multivar_in_file_gen goto out_free; } - fstat(in_fd, &st); + if (fstat(in_fd, &st) == -1) { + error_errno(_("fstat on %s failed"), config_filename); + ret = CONFIG_INVALID_FILE; + goto out_free; + } + contents_sz = xsize_t(st.st_size); contents = xmmap_gently(NULL, contents_sz, PROT_READ, MAP_PRIVATE, in_fd, 0); @@@ -2432,7 -2434,7 +2470,7 @@@ int git_config_rename_section_in_file(c if (new_name && !section_name_is_ok(new_name)) { ret = error("invalid section name: %s", new_name); - goto out; + goto out_no_rollback; } if (!config_filename) @@@ -2447,13 -2449,10 +2485,13 @@@ if (!(config_file = fopen(config_filename, "rb"))) { /* no config file means nothing to rename, no error */ - goto unlock_and_out; + goto commit_and_out; } - fstat(fileno(config_file), &st); + if (fstat(fileno(config_file), &st) == -1) { + ret = error_errno(_("fstat on %s failed"), config_filename); + goto out; + } if (chmod(get_lock_file_path(lock), st.st_mode & 07777) < 0) { ret = error_errno("chmod on %s failed", @@@ -2509,13 -2508,11 +2547,13 @@@ } } fclose(config_file); -unlock_and_out: +commit_and_out: if (commit_lock_file(lock) < 0) ret = error_errno("could not write config file %s", config_filename); out: + rollback_lock_file(lock); +out_no_rollback: free(filename_buf); return ret; } @@@ -2540,10 -2537,11 +2578,10 @@@ int parse_config_key(const char *var const char **subsection, int *subsection_len, const char **key) { - int section_len = strlen(section); const char *dot; /* Does it start with "section." ? */ - if (!starts_with(var, section) || var[section_len] != '.') + if (!skip_prefix(var, section, &var) || *var != '.') return -1; /* @@@ -2555,16 -2553,12 +2593,16 @@@ *key = dot + 1; /* Did we have a subsection at all? */ - if (dot == var + section_len) { - *subsection = NULL; - *subsection_len = 0; + if (dot == var) { + if (subsection) { + *subsection = NULL; + *subsection_len = 0; + } } else { - *subsection = var + section_len + 1; + if (!subsection) + return -1; + *subsection = var + 1; *subsection_len = dot - *subsection; } diff --combined read-cache.c index 9054369dd0,8cf0673adc..e447751823 --- a/read-cache.c +++ b/read-cache.c @@@ -156,7 -156,14 +156,7 @@@ void fill_stat_cache_info(struct cache_ static int ce_compare_data(const struct cache_entry *ce, struct stat *st) { int match = -1; - static int cloexec = O_CLOEXEC; - int fd = open(ce->name, O_RDONLY | cloexec); - - if ((cloexec & O_CLOEXEC) && fd < 0 && errno == EINVAL) { - /* Try again w/o O_CLOEXEC: the kernel might not support it */ - cloexec &= ~O_CLOEXEC; - fd = open(ce->name, O_RDONLY | cloexec); - } + int fd = git_open_cloexec(ce->name, O_RDONLY); if (fd >= 0) { unsigned char sha1[20]; @@@ -503,6 -510,7 +503,6 @@@ int index_name_pos(const struct index_s return index_name_stage_pos(istate, name, namelen, 0); } -/* Remove entry, return true if there are more entries to go.. */ int remove_index_entry_at(struct index_state *istate, int pos) { struct cache_entry *ce = istate->cache[pos]; @@@ -1558,10 -1566,27 +1558,27 @@@ static void tweak_untracked_cache(struc } } + static void tweak_split_index(struct index_state *istate) + { + switch (git_config_get_split_index()) { + case -1: /* unset: do nothing */ + break; + case 0: /* false */ + remove_split_index(istate); + break; + case 1: /* true */ + add_split_index(istate); + break; + default: /* unknown value: do nothing */ + break; + } + } + static void post_read_index_from(struct index_state *istate) { check_ce_order(istate); tweak_untracked_cache(istate); + tweak_split_index(istate); } /* remember to discard_cache() before reading a different cache! */ @@@ -1657,10 -1682,25 +1674,25 @@@ unmap die("index file corrupt"); } + /* + * Signal that the shared index is used by updating its mtime. + * + * This way, shared index can be removed if they have not been used + * for some time. + */ + static void freshen_shared_index(char *base_sha1_hex, int warn) + { + const char *shared_index = git_path("sharedindex.%s", base_sha1_hex); + if (!check_and_freshen_file(shared_index, 1) && warn) + warning("could not freshen shared index '%s'", shared_index); + } + int read_index_from(struct index_state *istate, const char *path) { struct split_index *split_index; int ret; + char *base_sha1_hex; + const char *base_path; /* istate->initialized covers both .git/index and .git/sharedindex.xxx */ if (istate->initialized) @@@ -1678,15 -1718,16 +1710,16 @@@ discard_index(split_index->base); else split_index->base = xcalloc(1, sizeof(*split_index->base)); - ret = do_read_index(split_index->base, - git_path("sharedindex.%s", - sha1_to_hex(split_index->base_sha1)), 1); + + base_sha1_hex = sha1_to_hex(split_index->base_sha1); + base_path = git_path("sharedindex.%s", base_sha1_hex); + ret = do_read_index(split_index->base, base_path, 1); if (hashcmp(split_index->base_sha1, split_index->base->sha1)) die("broken index, expect %s in %s, got %s", - sha1_to_hex(split_index->base_sha1), - git_path("sharedindex.%s", - sha1_to_hex(split_index->base_sha1)), + base_sha1_hex, base_path, sha1_to_hex(split_index->base->sha1)); + + freshen_shared_index(base_sha1_hex, 0); merge_base_index(istate); post_read_index_from(istate); return ret; @@@ -2169,6 -2210,65 +2202,65 @@@ static int write_split_index(struct ind return ret; } + static const char *shared_index_expire = "2.weeks.ago"; + + static unsigned long get_shared_index_expire_date(void) + { + static unsigned long shared_index_expire_date; + static int shared_index_expire_date_prepared; + + if (!shared_index_expire_date_prepared) { + git_config_get_expiry("splitindex.sharedindexexpire", + &shared_index_expire); + shared_index_expire_date = approxidate(shared_index_expire); + shared_index_expire_date_prepared = 1; + } + + return shared_index_expire_date; + } + + static int should_delete_shared_index(const char *shared_index_path) + { + struct stat st; + unsigned long expiration; + + /* Check timestamp */ + expiration = get_shared_index_expire_date(); + if (!expiration) + return 0; + if (stat(shared_index_path, &st)) + return error_errno(_("could not stat '%s"), shared_index_path); + if (st.st_mtime > expiration) + return 0; + + return 1; + } + + static int clean_shared_index_files(const char *current_hex) + { + struct dirent *de; + DIR *dir = opendir(get_git_dir()); + + if (!dir) + return error_errno(_("unable to open git dir: %s"), get_git_dir()); + + while ((de = readdir(dir)) != NULL) { + const char *sha1_hex; + const char *shared_index_path; + if (!skip_prefix(de->d_name, "sharedindex.", &sha1_hex)) + continue; + if (!strcmp(sha1_hex, current_hex)) + continue; + shared_index_path = git_path("%s", de->d_name); + if (should_delete_shared_index(shared_index_path) > 0 && + unlink(shared_index_path)) + warning_errno(_("unable to unlink: %s"), shared_index_path); + } + closedir(dir); + + return 0; + } + static struct tempfile temporary_sharedindex; static int write_shared_index(struct index_state *istate, @@@ -2190,14 -2290,48 +2282,48 @@@ } ret = rename_tempfile(&temporary_sharedindex, git_path("sharedindex.%s", sha1_to_hex(si->base->sha1))); - if (!ret) + if (!ret) { hashcpy(si->base_sha1, si->base->sha1); + clean_shared_index_files(sha1_to_hex(si->base->sha1)); + } + return ret; } + static const int default_max_percent_split_change = 20; + + static int too_many_not_shared_entries(struct index_state *istate) + { + int i, not_shared = 0; + int max_split = git_config_get_max_percent_split_change(); + + switch (max_split) { + case -1: + /* not or badly configured: use the default value */ + max_split = default_max_percent_split_change; + break; + case 0: + return 1; /* 0% means always write a new shared index */ + case 100: + return 0; /* 100% means never write a new shared index */ + default: + break; /* just use the configured value */ + } + + /* Count not shared entries */ + for (i = 0; i < istate->cache_nr; i++) { + struct cache_entry *ce = istate->cache[i]; + if (!ce->index) + not_shared++; + } + + return (int64_t)istate->cache_nr * max_split < (int64_t)not_shared * 100; + } + int write_locked_index(struct index_state *istate, struct lock_file *lock, unsigned flags) { + int new_shared_index, ret; struct split_index *si = istate->split_index; if (!si || alternate_index_output || @@@ -2212,13 -2346,24 +2338,24 @@@ if ((v & 15) < 6) istate->cache_changed |= SPLIT_INDEX_ORDERED; } - if (istate->cache_changed & SPLIT_INDEX_ORDERED) { - int ret = write_shared_index(istate, lock, flags); + if (too_many_not_shared_entries(istate)) + istate->cache_changed |= SPLIT_INDEX_ORDERED; + + new_shared_index = istate->cache_changed & SPLIT_INDEX_ORDERED; + + if (new_shared_index) { + ret = write_shared_index(istate, lock, flags); if (ret) return ret; } - return write_split_index(istate, lock, flags); + ret = write_split_index(istate, lock, flags); + + /* Freshen the shared index only if the split-index was written */ + if (!ret && !new_shared_index) + freshen_shared_index(sha1_to_hex(si->base_sha1), 1); + + return ret; } /* @@@ -2284,8 -2429,7 +2421,8 @@@ int index_name_is_other(const struct in return 1; } -void *read_blob_data_from_index(struct index_state *istate, const char *path, unsigned long *size) +void *read_blob_data_from_index(const struct index_state *istate, + const char *path, unsigned long *size) { int pos, len; unsigned long sz; diff --combined sha1_file.c index 2ee3c617a6,f5303d955a..f1c83a9248 --- a/sha1_file.c +++ b/sha1_file.c @@@ -28,6 -28,14 +28,6 @@@ #include "mergesort.h" #include "quote.h" -#ifndef O_NOATIME -#if defined(__linux__) && (defined(__i386__) || defined(__PPC__)) -#define O_NOATIME 01000000 -#else -#define O_NOATIME 0 -#endif -#endif - #define SZ_FMT PRIuMAX static inline uintmax_t sz_fmt(size_t s) { return s; } @@@ -129,10 -137,8 +129,10 @@@ enum scld_error safe_create_leading_dir *slash = '\0'; if (!stat(path, &st)) { /* path exists */ - if (!S_ISDIR(st.st_mode)) + if (!S_ISDIR(st.st_mode)) { + errno = ENOTDIR; ret = SCLD_EXISTS; + } } else if (mkdir(path, 0777)) { if (errno == EEXIST && !stat(path, &st) && S_ISDIR(st.st_mode)) @@@ -160,85 -166,13 +160,85 @@@ enum scld_error safe_create_leading_directories_const(const char *path) { + int save_errno; /* path points to cache entries, so xstrdup before messing with it */ char *buf = xstrdup(path); enum scld_error result = safe_create_leading_directories(buf); + + save_errno = errno; free(buf); + errno = save_errno; return result; } +int raceproof_create_file(const char *path, create_file_fn fn, void *cb) +{ + /* + * The number of times we will try to remove empty directories + * in the way of path. This is only 1 because if another + * process is racily creating directories that conflict with + * us, we don't want to fight against them. + */ + int remove_directories_remaining = 1; + + /* + * The number of times that we will try to create the + * directories containing path. We are willing to attempt this + * more than once, because another process could be trying to + * clean up empty directories at the same time as we are + * trying to create them. + */ + int create_directories_remaining = 3; + + /* A scratch copy of path, filled lazily if we need it: */ + struct strbuf path_copy = STRBUF_INIT; + + int ret, save_errno; + + /* Sanity check: */ + assert(*path); + +retry_fn: + ret = fn(path, cb); + save_errno = errno; + if (!ret) + goto out; + + if (errno == EISDIR && remove_directories_remaining-- > 0) { + /* + * A directory is in the way. Maybe it is empty; try + * to remove it: + */ + if (!path_copy.len) + strbuf_addstr(&path_copy, path); + + if (!remove_dir_recursively(&path_copy, REMOVE_DIR_EMPTY_ONLY)) + goto retry_fn; + } else if (errno == ENOENT && create_directories_remaining-- > 0) { + /* + * Maybe the containing directory didn't exist, or + * maybe it was just deleted by a process that is + * racing with us to clean up empty directories. Try + * to create it: + */ + enum scld_error scld_result; + + if (!path_copy.len) + strbuf_addstr(&path_copy, path); + + do { + scld_result = safe_create_leading_directories(path_copy.buf); + if (scld_result == SCLD_OK) + goto retry_fn; + } while (scld_result == SCLD_VANISHED && create_directories_remaining-- > 0); + } + +out: + strbuf_release(&path_copy); + errno = save_errno; + return ret; +} + static void fill_sha1_path(struct strbuf *buf, const unsigned char *sha1) { int i; @@@ -358,7 -292,7 +358,7 @@@ static int link_alt_odb_entry(const cha struct strbuf pathbuf = STRBUF_INIT; if (!is_absolute_path(entry) && relative_base) { - strbuf_addstr(&pathbuf, real_path(relative_base)); + strbuf_realpath(&pathbuf, relative_base, 1); strbuf_addch(&pathbuf, '/'); } strbuf_addstr(&pathbuf, entry); @@@ -667,7 -601,7 +667,7 @@@ static int freshen_file(const char *fn * either does not exist on disk, or has a stale mtime and may be subject to * pruning). */ - static int check_and_freshen_file(const char *fn, int freshen) + int check_and_freshen_file(const char *fn, int freshen) { if (access(fn, F_OK)) return 0; @@@ -1677,81 -1611,66 +1677,81 @@@ int check_sha1_signature(const unsigne return hashcmp(sha1, real_sha1) ? -1 : 0; } -int git_open(const char *name) +int git_open_cloexec(const char *name, int flags) { - static int sha1_file_open_flag = O_NOATIME | O_CLOEXEC; - - for (;;) { - int fd; - - errno = 0; - fd = open(name, O_RDONLY | sha1_file_open_flag); - if (fd >= 0) - return fd; + int fd; + static int o_cloexec = O_CLOEXEC; + fd = open(name, flags | o_cloexec); + if ((o_cloexec & O_CLOEXEC) && fd < 0 && errno == EINVAL) { /* Try again w/o O_CLOEXEC: the kernel might not support it */ - if ((sha1_file_open_flag & O_CLOEXEC) && errno == EINVAL) { - sha1_file_open_flag &= ~O_CLOEXEC; - continue; - } + o_cloexec &= ~O_CLOEXEC; + fd = open(name, flags | o_cloexec); + } - /* Might the failure be due to O_NOATIME? */ - if (errno != ENOENT && (sha1_file_open_flag & O_NOATIME)) { - sha1_file_open_flag &= ~O_NOATIME; - continue; +#if defined(F_GETFL) && defined(F_SETFL) && defined(FD_CLOEXEC) + { + static int fd_cloexec = FD_CLOEXEC; + + if (!o_cloexec && 0 <= fd && fd_cloexec) { + /* Opened w/o O_CLOEXEC? try with fcntl(2) to add it */ + int flags = fcntl(fd, F_GETFL); + if (fcntl(fd, F_SETFL, flags | fd_cloexec)) + fd_cloexec = 0; } - return -1; } +#endif + return fd; } -static int stat_sha1_file(const unsigned char *sha1, struct stat *st) +/* + * Find "sha1" as a loose object in the local repository or in an alternate. + * Returns 0 on success, negative on failure. + * + * The "path" out-parameter will give the path of the object we found (if any). + * Note that it may point to static storage and is only valid until another + * call to sha1_file_name(), etc. + */ +static int stat_sha1_file(const unsigned char *sha1, struct stat *st, + const char **path) { struct alternate_object_database *alt; - if (!lstat(sha1_file_name(sha1), st)) + *path = sha1_file_name(sha1); + if (!lstat(*path, st)) return 0; prepare_alt_odb(); errno = ENOENT; for (alt = alt_odb_list; alt; alt = alt->next) { - const char *path = alt_sha1_path(alt, sha1); - if (!lstat(path, st)) + *path = alt_sha1_path(alt, sha1); + if (!lstat(*path, st)) return 0; } return -1; } -static int open_sha1_file(const unsigned char *sha1) +/* + * Like stat_sha1_file(), but actually open the object and return the + * descriptor. See the caveats on the "path" parameter above. + */ +static int open_sha1_file(const unsigned char *sha1, const char **path) { int fd; struct alternate_object_database *alt; int most_interesting_errno; - fd = git_open(sha1_file_name(sha1)); + *path = sha1_file_name(sha1); + fd = git_open(*path); if (fd >= 0) return fd; most_interesting_errno = errno; prepare_alt_odb(); for (alt = alt_odb_list; alt; alt = alt->next) { - const char *path = alt_sha1_path(alt, sha1); - fd = git_open(path); + *path = alt_sha1_path(alt, sha1); + fd = git_open(*path); if (fd >= 0) return fd; if (most_interesting_errno == ENOENT) @@@ -1761,21 -1680,12 +1761,21 @@@ return -1; } -void *map_sha1_file(const unsigned char *sha1, unsigned long *size) +/* + * Map the loose object at "path" if it is not NULL, or the path found by + * searching for a loose object named "sha1". + */ +static void *map_sha1_file_1(const char *path, + const unsigned char *sha1, + unsigned long *size) { void *map; int fd; - fd = open_sha1_file(sha1); + if (path) + fd = git_open(path); + else + fd = open_sha1_file(sha1, &path); map = NULL; if (fd >= 0) { struct stat st; @@@ -1784,7 -1694,7 +1784,7 @@@ *size = xsize_t(st.st_size); if (!*size) { /* mmap() is forbidden on empty files */ - error("object file %s is empty", sha1_file_name(sha1)); + error("object file %s is empty", path); return NULL; } map = xmmap(NULL, *size, PROT_READ, MAP_PRIVATE, fd, 0); @@@ -1794,11 -1704,6 +1794,11 @@@ return map; } +void *map_sha1_file(const unsigned char *sha1, unsigned long *size) +{ + return map_sha1_file_1(NULL, sha1, size); +} + unsigned long unpack_object_header_buffer(const unsigned char *buf, unsigned long len, enum object_type *type, unsigned long *sizep) { @@@ -2445,10 -2350,11 +2445,10 @@@ static inline void release_delta_base_c void clear_delta_base_cache(void) { - struct hashmap_iter iter; - struct delta_base_cache_entry *entry; - for (entry = hashmap_iter_first(&delta_base_cache, &iter); - entry; - entry = hashmap_iter_next(&iter)) { + struct list_head *lru, *tmp; + list_for_each_safe(lru, tmp, &delta_base_cache_lru) { + struct delta_base_cache_entry *entry = + list_entry(lru, struct delta_base_cache_entry, lru); release_delta_base_cache(entry); } } @@@ -2606,7 -2512,6 +2606,7 @@@ void *unpack_entry(struct packed_git *p while (delta_stack_nr) { void *delta_data; void *base = data; + void *external_base = NULL; unsigned long delta_size, base_size = size; int i; @@@ -2633,7 -2538,6 +2633,7 @@@ p->pack_name); mark_bad_packed_object(p, base_sha1); base = read_object(base_sha1, &type, &base_size); + external_base = base; } } @@@ -2652,7 -2556,6 +2652,7 @@@ "at offset %"PRIuMAX" from %s", (uintmax_t)curpos, p->pack_name); data = NULL; + free(external_base); continue; } @@@ -2672,7 -2575,6 +2672,7 @@@ error("failed to apply delta"); free(delta_data); + free(external_base); } *final_type = type; @@@ -2912,9 -2814,8 +2912,9 @@@ static int sha1_loose_object_info(cons * object even exists. */ if (!oi->typep && !oi->typename && !oi->sizep) { + const char *path; struct stat st; - if (stat_sha1_file(sha1, &st) < 0) + if (stat_sha1_file(sha1, &st, &path) < 0) return -1; if (oi->disk_sizep) *oi->disk_sizep = st.st_size; @@@ -3110,8 -3011,6 +3110,8 @@@ void *read_sha1_file_extended(const uns { void *data; const struct packed_git *p; + const char *path; + struct stat st; const unsigned char *repl = lookup_replace_object_extended(sha1, flag); errno = 0; @@@ -3127,9 -3026,12 +3127,9 @@@ die("replacement %s not found for %s", sha1_to_hex(repl), sha1_to_hex(sha1)); - if (has_loose_object(repl)) { - const char *path = sha1_file_name(sha1); - + if (!stat_sha1_file(repl, &st, &path)) die("loose object %s (stored in %s) is corrupt", sha1_to_hex(repl), path); - } if ((p = has_packed_and_bad(repl)) != NULL) die("packed object %s (stored in %s) is corrupt", @@@ -3899,122 -3801,3 +3899,122 @@@ int for_each_packed_object(each_packed_ } return r ? r : pack_errors; } + +static int check_stream_sha1(git_zstream *stream, + const char *hdr, + unsigned long size, + const char *path, + const unsigned char *expected_sha1) +{ + git_SHA_CTX c; + unsigned char real_sha1[GIT_SHA1_RAWSZ]; + unsigned char buf[4096]; + unsigned long total_read; + int status = Z_OK; + + git_SHA1_Init(&c); + git_SHA1_Update(&c, hdr, stream->total_out); + + /* + * We already read some bytes into hdr, but the ones up to the NUL + * do not count against the object's content size. + */ + total_read = stream->total_out - strlen(hdr) - 1; + + /* + * This size comparison must be "<=" to read the final zlib packets; + * see the comment in unpack_sha1_rest for details. + */ + while (total_read <= size && + (status == Z_OK || status == Z_BUF_ERROR)) { + stream->next_out = buf; + stream->avail_out = sizeof(buf); + if (size - total_read < stream->avail_out) + stream->avail_out = size - total_read; + status = git_inflate(stream, Z_FINISH); + git_SHA1_Update(&c, buf, stream->next_out - buf); + total_read += stream->next_out - buf; + } + git_inflate_end(stream); + + if (status != Z_STREAM_END) { + error("corrupt loose object '%s'", sha1_to_hex(expected_sha1)); + return -1; + } + if (stream->avail_in) { + error("garbage at end of loose object '%s'", + sha1_to_hex(expected_sha1)); + return -1; + } + + git_SHA1_Final(real_sha1, &c); + if (hashcmp(expected_sha1, real_sha1)) { + error("sha1 mismatch for %s (expected %s)", path, + sha1_to_hex(expected_sha1)); + return -1; + } + + return 0; +} + +int read_loose_object(const char *path, + const unsigned char *expected_sha1, + enum object_type *type, + unsigned long *size, + void **contents) +{ + int ret = -1; + int fd = -1; + void *map = NULL; + unsigned long mapsize; + git_zstream stream; + char hdr[32]; + + *contents = NULL; + + map = map_sha1_file_1(path, NULL, &mapsize); + if (!map) { + error_errno("unable to mmap %s", path); + goto out; + } + + if (unpack_sha1_header(&stream, map, mapsize, hdr, sizeof(hdr)) < 0) { + error("unable to unpack header of %s", path); + goto out; + } + + *type = parse_sha1_header(hdr, size); + if (*type < 0) { + error("unable to parse header of %s", path); + git_inflate_end(&stream); + goto out; + } + + if (*type == OBJ_BLOB) { + if (check_stream_sha1(&stream, hdr, *size, path, expected_sha1) < 0) + goto out; + } else { + *contents = unpack_sha1_rest(&stream, hdr, *size, expected_sha1); + if (!*contents) { + error("unable to unpack contents of %s", path); + git_inflate_end(&stream); + goto out; + } + if (check_sha1_signature(expected_sha1, *contents, + *size, typename(*type))) { + error("sha1 mismatch for %s (expected %s)", path, + sha1_to_hex(expected_sha1)); + free(*contents); + goto out; + } + } + + ret = 0; /* everything checks out */ + +out: + if (map) + munmap(map, mapsize); + if (fd >= 0) + close(fd); + return ret; +} diff --combined t/t1700-split-index.sh index 6096f2c630,ea1aeacff0..af3ec0da5a --- a/t/t1700-split-index.sh +++ b/t/t1700-split-index.sh @@@ -8,6 -8,7 +8,7 @@@ test_description='split index mode test sane_unset GIT_TEST_SPLIT_INDEX test_expect_success 'enable split index' ' + git config splitIndex.maxPercentChange 100 && git update-index --split-index && test-dump-split-index .git/index >actual && indexversion=$(test-index-version <.git/index) && @@@ -19,12 -20,12 +20,12 @@@ own=8299b0bcd1ac364e5f1d7768efb62fa2da79a339 base=39d890139ee5356c7ef572216cebcd27aa41f9df fi && - cat >expect <expect <<-EOF && + own $own + base $base + replacements: + deletions: + EOF test_cmp expect actual ' @@@ -32,51 -33,51 +33,51 @@@ test_expect_success 'add one file' : >one && git update-index --add one && git ls-files --stage >ls-files.actual && - cat >ls-files.expect <ls-files.expect <<-EOF && + 100644 $EMPTY_BLOB 0 one + EOF test_cmp ls-files.expect ls-files.actual && test-dump-split-index .git/index | sed "/^own/d" >actual && - cat >expect <expect <<-EOF && + base $base + 100644 $EMPTY_BLOB 0 one + replacements: + deletions: + EOF test_cmp expect actual ' test_expect_success 'disable split index' ' git update-index --no-split-index && git ls-files --stage >ls-files.actual && - cat >ls-files.expect <ls-files.expect <<-EOF && + 100644 $EMPTY_BLOB 0 one + EOF test_cmp ls-files.expect ls-files.actual && BASE=$(test-dump-split-index .git/index | grep "^own" | sed "s/own/base/") && test-dump-split-index .git/index | sed "/^own/d" >actual && - cat >expect <expect <<-EOF && + not a split index + EOF test_cmp expect actual ' test_expect_success 'enable split index again, "one" now belongs to base index"' ' git update-index --split-index && git ls-files --stage >ls-files.actual && - cat >ls-files.expect <ls-files.expect <<-EOF && + 100644 $EMPTY_BLOB 0 one + EOF test_cmp ls-files.expect ls-files.actual && test-dump-split-index .git/index | sed "/^own/d" >actual && - cat >expect <expect <<-EOF && + $BASE + replacements: + deletions: + EOF test_cmp expect actual ' @@@ -84,18 -85,18 +85,18 @@@ test_expect_success 'modify original fi echo modified >one && git update-index one && git ls-files --stage >ls-files.actual && - cat >ls-files.expect <ls-files.expect <<-EOF && + 100644 2e0996000b7e9019eabcad29391bf0f5c7702f0b 0 one + EOF test_cmp ls-files.expect ls-files.actual && test-dump-split-index .git/index | sed "/^own/d" >actual && - q_to_tab >expect <expect <<-EOF && + $BASE + 100644 2e0996000b7e9019eabcad29391bf0f5c7702f0b 0Q + replacements: 0 + deletions: + EOF test_cmp expect actual ' @@@ -103,54 -104,54 +104,54 @@@ test_expect_success 'add another file, : >two && git update-index --add two && git ls-files --stage >ls-files.actual && - cat >ls-files.expect <ls-files.expect <<-EOF && + 100644 2e0996000b7e9019eabcad29391bf0f5c7702f0b 0 one + 100644 $EMPTY_BLOB 0 two + EOF test_cmp ls-files.expect ls-files.actual && test-dump-split-index .git/index | sed "/^own/d" >actual && - q_to_tab >expect <expect <<-EOF && + $BASE + 100644 2e0996000b7e9019eabcad29391bf0f5c7702f0b 0Q + 100644 $EMPTY_BLOB 0 two + replacements: 0 + deletions: + EOF test_cmp expect actual ' test_expect_success 'remove file not in base index' ' git update-index --force-remove two && git ls-files --stage >ls-files.actual && - cat >ls-files.expect <ls-files.expect <<-EOF && + 100644 2e0996000b7e9019eabcad29391bf0f5c7702f0b 0 one + EOF test_cmp ls-files.expect ls-files.actual && test-dump-split-index .git/index | sed "/^own/d" >actual && - q_to_tab >expect <expect <<-EOF && + $BASE + 100644 2e0996000b7e9019eabcad29391bf0f5c7702f0b 0Q + replacements: 0 + deletions: + EOF test_cmp expect actual ' test_expect_success 'remove file in base index' ' git update-index --force-remove one && git ls-files --stage >ls-files.actual && - cat >ls-files.expect <ls-files.expect <<-EOF && + EOF test_cmp ls-files.expect ls-files.actual && test-dump-split-index .git/index | sed "/^own/d" >actual && - cat >expect <expect <<-EOF && + $BASE + replacements: + deletions: 0 + EOF test_cmp expect actual ' @@@ -158,18 -159,18 +159,18 @@@ test_expect_success 'add original file : >one && git update-index --add one && git ls-files --stage >ls-files.actual && - cat >ls-files.expect <ls-files.expect <<-EOF && + 100644 $EMPTY_BLOB 0 one + EOF test_cmp ls-files.expect ls-files.actual && test-dump-split-index .git/index | sed "/^own/d" >actual && - cat >expect <expect <<-EOF && + $BASE + 100644 $EMPTY_BLOB 0 one + replacements: + deletions: 0 + EOF test_cmp expect actual ' @@@ -177,43 -178,180 +178,196 @@@ test_expect_success 'add new file' : >two && git update-index --add two && git ls-files --stage >actual && - cat >expect <expect <<-EOF && + 100644 $EMPTY_BLOB 0 one + 100644 $EMPTY_BLOB 0 two + EOF test_cmp expect actual ' test_expect_success 'unify index, two files remain' ' git update-index --no-split-index && git ls-files --stage >ls-files.actual && - cat >ls-files.expect <ls-files.expect <<-EOF && + 100644 $EMPTY_BLOB 0 one + 100644 $EMPTY_BLOB 0 two + EOF test_cmp ls-files.expect ls-files.actual && test-dump-split-index .git/index | sed "/^own/d" >actual && - cat >expect <expect <<-EOF && + not a split index + EOF test_cmp expect actual ' +test_expect_success 'rev-parse --shared-index-path' ' + test_create_repo split-index && + ( + cd split-index && + git update-index --split-index && + echo .git/sharedindex* >expect && + git rev-parse --shared-index-path >actual && + test_cmp expect actual && + mkdir subdirectory && + cd subdirectory && + echo ../.git/sharedindex* >expect && + git rev-parse --shared-index-path >actual && + test_cmp expect actual + ) +' + + test_expect_success 'set core.splitIndex config variable to true' ' + git config core.splitIndex true && + : >three && + git update-index --add three && + git ls-files --stage >ls-files.actual && + cat >ls-files.expect <<-EOF && + 100644 e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 0 one + 100644 e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 0 three + 100644 e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 0 two + EOF + test_cmp ls-files.expect ls-files.actual && + BASE=$(test-dump-split-index .git/index | grep "^base") && + test-dump-split-index .git/index | sed "/^own/d" >actual && + cat >expect <<-EOF && + $BASE + replacements: + deletions: + EOF + test_cmp expect actual + ' + + test_expect_success 'set core.splitIndex config variable to false' ' + git config core.splitIndex false && + git update-index --force-remove three && + git ls-files --stage >ls-files.actual && + cat >ls-files.expect <<-EOF && + 100644 e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 0 one + 100644 e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 0 two + EOF + test_cmp ls-files.expect ls-files.actual && + test-dump-split-index .git/index | sed "/^own/d" >actual && + cat >expect <<-EOF && + not a split index + EOF + test_cmp expect actual + ' + + test_expect_success 'set core.splitIndex config variable to true' ' + git config core.splitIndex true && + : >three && + git update-index --add three && + BASE=$(test-dump-split-index .git/index | grep "^base") && + test-dump-split-index .git/index | sed "/^own/d" >actual && + cat >expect <<-EOF && + $BASE + replacements: + deletions: + EOF + test_cmp expect actual && + : >four && + git update-index --add four && + test-dump-split-index .git/index | sed "/^own/d" >actual && + cat >expect <<-EOF && + $BASE + 100644 e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 0 four + replacements: + deletions: + EOF + test_cmp expect actual + ' + + test_expect_success 'check behavior with splitIndex.maxPercentChange unset' ' + git config --unset splitIndex.maxPercentChange && + : >five && + git update-index --add five && + BASE=$(test-dump-split-index .git/index | grep "^base") && + test-dump-split-index .git/index | sed "/^own/d" >actual && + cat >expect <<-EOF && + $BASE + replacements: + deletions: + EOF + test_cmp expect actual && + : >six && + git update-index --add six && + test-dump-split-index .git/index | sed "/^own/d" >actual && + cat >expect <<-EOF && + $BASE + 100644 e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 0 six + replacements: + deletions: + EOF + test_cmp expect actual + ' + + test_expect_success 'check splitIndex.maxPercentChange set to 0' ' + git config splitIndex.maxPercentChange 0 && + : >seven && + git update-index --add seven && + BASE=$(test-dump-split-index .git/index | grep "^base") && + test-dump-split-index .git/index | sed "/^own/d" >actual && + cat >expect <<-EOF && + $BASE + replacements: + deletions: + EOF + test_cmp expect actual && + : >eight && + git update-index --add eight && + BASE=$(test-dump-split-index .git/index | grep "^base") && + test-dump-split-index .git/index | sed "/^own/d" >actual && + cat >expect <<-EOF && + $BASE + replacements: + deletions: + EOF + test_cmp expect actual + ' + + test_expect_success 'shared index files expire after 2 weeks by default' ' + : >ten && + git update-index --add ten && + test $(ls .git/sharedindex.* | wc -l) -gt 2 && + just_under_2_weeks_ago=$((5-14*86400)) && + test-chmtime =$just_under_2_weeks_ago .git/sharedindex.* && + : >eleven && + git update-index --add eleven && + test $(ls .git/sharedindex.* | wc -l) -gt 2 && + just_over_2_weeks_ago=$((-1-14*86400)) && + test-chmtime =$just_over_2_weeks_ago .git/sharedindex.* && + : >twelve && + git update-index --add twelve && + test $(ls .git/sharedindex.* | wc -l) -le 2 + ' + + test_expect_success 'check splitIndex.sharedIndexExpire set to 16 days' ' + git config splitIndex.sharedIndexExpire "16.days.ago" && + test-chmtime =$just_over_2_weeks_ago .git/sharedindex.* && + : >thirteen && + git update-index --add thirteen && + test $(ls .git/sharedindex.* | wc -l) -gt 2 && + just_over_16_days_ago=$((-1-16*86400)) && + test-chmtime =$just_over_16_days_ago .git/sharedindex.* && + : >fourteen && + git update-index --add fourteen && + test $(ls .git/sharedindex.* | wc -l) -le 2 + ' + + test_expect_success 'check splitIndex.sharedIndexExpire set to "never" and "now"' ' + git config splitIndex.sharedIndexExpire never && + just_10_years_ago=$((-365*10*86400)) && + test-chmtime =$just_10_years_ago .git/sharedindex.* && + : >fifteen && + git update-index --add fifteen && + test $(ls .git/sharedindex.* | wc -l) -gt 2 && + git config splitIndex.sharedIndexExpire now && + just_1_second_ago=-1 && + test-chmtime =$just_1_second_ago .git/sharedindex.* && + : >sixteen && + git update-index --add sixteen && + test $(ls .git/sharedindex.* | wc -l) -le 2 + ' + test_done