From: Junio C Hamano Date: Tue, 2 Sep 2014 20:24:17 +0000 (-0700) Subject: Merge branch 'ta/config-set' X-Git-Tag: v2.2.0-rc0~178 X-Git-Url: https://git.lorimer.id.au/gitweb.git/diff_plain/56f214e0716dec043b50a7e1e8cc181be2ef7df2?ds=inline;hp=-c Merge branch 'ta/config-set' Add in-core caching layer to let us avoid reading the same configuration files number of times. * ta/config-set: test-config: add tests for the config_set API add `config_set` API for caching config-like files --- 56f214e0716dec043b50a7e1e8cc181be2ef7df2 diff --combined Documentation/technical/api-config.txt index edd5018e15,815c1eecc3..21f280ca6d --- a/Documentation/technical/api-config.txt +++ b/Documentation/technical/api-config.txt @@@ -77,6 -77,86 +77,86 @@@ To read a specific file in git-config f `git_config_from_file`. This takes the same callback and data parameters as `git_config`. + Querying For Specific Variables + ------------------------------- + + For programs wanting to query for specific variables in a non-callback + manner, the config API provides two functions `git_config_get_value` + and `git_config_get_value_multi`. They both read values from an internal + cache generated previously from reading the config files. + + `int git_config_get_value(const char *key, const char **value)`:: + + Finds the highest-priority value for the configuration variable `key`, + stores the pointer to it in `value` and returns 0. When the + configuration variable `key` is not found, returns 1 without touching + `value`. The caller should not free or modify `value`, as it is owned + by the cache. + + `const struct string_list *git_config_get_value_multi(const char *key)`:: + + Finds and returns the value list, sorted in order of increasing priority + for the configuration variable `key`. When the configuration variable + `key` is not found, returns NULL. The caller should not free or modify + the returned pointer, as it is owned by the cache. + + `void git_config_clear(void)`:: + + Resets and invalidates the config cache. + + The config API also provides type specific API functions which do conversion + as well as retrieval for the queried variable, including: + + `int git_config_get_int(const char *key, int *dest)`:: + + Finds and parses the value to an integer for the configuration variable + `key`. Dies on error; otherwise, stores the value of the parsed integer in + `dest` and returns 0. When the configuration variable `key` is not found, + returns 1 without touching `dest`. + + `int git_config_get_ulong(const char *key, unsigned long *dest)`:: + + Similar to `git_config_get_int` but for unsigned longs. + + `int git_config_get_bool(const char *key, int *dest)`:: + + Finds and parses the value into a boolean value, for the configuration + variable `key` respecting keywords like "true" and "false". Integer + values are converted into true/false values (when they are non-zero or + zero, respectively). Other values cause a die(). If parsing is successful, + stores the value of the parsed result in `dest` and returns 0. When the + configuration variable `key` is not found, returns 1 without touching + `dest`. + + `int git_config_get_bool_or_int(const char *key, int *is_bool, int *dest)`:: + + Similar to `git_config_get_bool`, except that integers are copied as-is, + and `is_bool` flag is unset. + + `int git_config_get_maybe_bool(const char *key, int *dest)`:: + + Similar to `git_config_get_bool`, except that it returns -1 on error + rather than dying. + + `int git_config_get_string_const(const char *key, const char **dest)`:: + + Allocates and copies the retrieved string into the `dest` parameter for + the configuration variable `key`; if NULL string is given, prints an + error message and returns -1. When the configuration variable `key` is + not found, returns 1 without touching `dest`. + + `int git_config_get_string(const char *key, char **dest)`:: + + Similar to `git_config_get_string_const`, except that retrieved value + copied into the `dest` parameter is a mutable string. + + `int git_config_get_pathname(const char *key, const char **dest)`:: + + Similar to `git_config_get_string`, but expands `~` or `~user` into + the user's home directory when found at the beginning of the path. + + See test-config.c for usage examples. + Value Parsing Helpers --------------------- @@@ -134,36 -214,69 +214,98 @@@ int read_file_with_include(const char * `git_config` respects includes automatically. The lower-level `git_config_from_file` does not. + Custom Configsets + ----------------- + + A `config_set` can be used to construct an in-memory cache for + config-like files that the caller specifies (i.e., files like `.gitmodules`, + `~/.gitconfig` etc.). For example, + + --------------------------------------- + struct config_set gm_config; + git_configset_init(&gm_config); + int b; + /* we add config files to the config_set */ + git_configset_add_file(&gm_config, ".gitmodules"); + git_configset_add_file(&gm_config, ".gitmodules_alt"); + + if (!git_configset_get_bool(gm_config, "submodule.frotz.ignore", &b)) { + /* hack hack hack */ + } + + /* when we are done with the configset */ + git_configset_clear(&gm_config); + ---------------------------------------- + + Configset API provides functions for the above mentioned work flow, including: + + `void git_configset_init(struct config_set *cs)`:: + + Initializes the config_set `cs`. + + `int git_configset_add_file(struct config_set *cs, const char *filename)`:: + + Parses the file and adds the variable-value pairs to the `config_set`, + dies if there is an error in parsing the file. Returns 0 on success, or + -1 if the file does not exist or is inaccessible. The user has to decide + if he wants to free the incomplete configset or continue using it when + the function returns -1. + + `int git_configset_get_value(struct config_set *cs, const char *key, const char **value)`:: + + Finds the highest-priority value for the configuration variable `key` + and config set `cs`, stores the pointer to it in `value` and returns 0. + When the configuration variable `key` is not found, returns 1 without + touching `value`. The caller should not free or modify `value`, as it + is owned by the cache. + + `const struct string_list *git_configset_get_value_multi(struct config_set *cs, const char *key)`:: + + Finds and returns the value list, sorted in order of increasing priority + for the configuration variable `key` and config set `cs`. When the + configuration variable `key` is not found, returns NULL. The caller + should not free or modify the returned pointer, as it is owned by the cache. + + `void git_configset_clear(struct config_set *cs)`:: + + Clears `config_set` structure, removes all saved variable-value pairs. + + In addition to above functions, the `config_set` API provides type specific + functions in the vein of `git_config_get_int` and family but with an extra + parameter, pointer to struct `config_set`. + They all behave similarly to the `git_config_get*()` family described in + "Querying For Specific Variables" above. + Writing Config Files -------------------- -TODO +Git gives multiple entry points in the Config API to write config values to +files namely `git_config_set_in_file` and `git_config_set`, which write to +a specific config file or to `.git/config` respectively. They both take a +key/value pair as parameter. +In the end they both call `git_config_set_multivar_in_file` which takes four +parameters: + +- the name of the file, as a string, to which key/value pairs will be written. + +- the name of key, as a string. This is in canonical "flat" form: the section, + subsection, and variable segments will be separated by dots, and the section + and variable segments will be all lowercase. + E.g., `core.ignorecase`, `diff.SomeType.textconv`. + +- the value of the variable, as a string. If value is equal to NULL, it will + remove the matching key from the config file. + +- the value regex, as a string. It will disregard key/value pairs where value + does not match. + +- a multi_replace value, as an int. If value is equal to zero, nothing or only + one matching key/value is replaced, else all matching key/values (regardless + how many) are removed, before the new pair is written. + +It returns 0 on success. + +Also, there are functions `git_config_rename_section` and +`git_config_rename_section_in_file` with parameters `old_name` and `new_name` +for renaming or removing sections in the config files. If NULL is passed +through `new_name` parameter, the section will be removed from the config file. diff --combined Makefile index 9f984a9e55,b7462e3398..30cc622f08 --- a/Makefile +++ b/Makefile @@@ -551,6 -551,7 +551,7 @@@ PROGRAMS += $(patsubst %.o,git-%$X,$(PR TEST_PROGRAMS_NEED_X += test-chmtime TEST_PROGRAMS_NEED_X += test-ctype + TEST_PROGRAMS_NEED_X += test-config TEST_PROGRAMS_NEED_X += test-date TEST_PROGRAMS_NEED_X += test-delta TEST_PROGRAMS_NEED_X += test-dump-cache-tree @@@ -1659,11 -1660,7 +1660,11 @@@ endi profile:: profile-clean $(MAKE) PROFILE=GEN all $(MAKE) PROFILE=GEN -j1 test - $(MAKE) PROFILE=GEN -j1 perf + @if test -n "$$GIT_PERF_REPO" || test -d .git; then \ + $(MAKE) PROFILE=GEN -j1 perf; \ + else \ + echo "Skipping profile of perf tests..."; \ + fi $(MAKE) PROFILE=USE all profile-fast: profile-clean diff --combined cache.h index 76be785087,7292aefa2e..4d5b76c76a --- a/cache.h +++ b/cache.h @@@ -585,7 -585,6 +585,7 @@@ extern NORETURN void unable_to_lock_ind extern int hold_lock_file_for_update(struct lock_file *, const char *path, int); extern int hold_lock_file_for_append(struct lock_file *, const char *path, int); extern int commit_lock_file(struct lock_file *); +extern int reopen_lock_file(struct lock_file *); extern void update_index_if_able(struct index_state *, struct lock_file *); extern int hold_locked_index(struct lock_file *, int); @@@ -1062,7 -1061,6 +1062,7 @@@ extern const char *git_author_info(int) extern const char *git_committer_info(int); extern const char *fmt_ident(const char *name, const char *email, const char *date_str, int); extern const char *fmt_name(const char *name, const char *email); +extern const char *ident_default_name(void); extern const char *ident_default_email(void); extern const char *git_editor(void); extern const char *git_pager(int stdout_is_tty); @@@ -1353,6 -1351,38 +1353,38 @@@ extern int parse_config_key(const char const char **subsection, int *subsection_len, const char **key); + struct config_set { + struct hashmap config_hash; + int hash_initialized; + }; + + extern void git_configset_init(struct config_set *cs); + extern int git_configset_add_file(struct config_set *cs, const char *filename); + extern int git_configset_get_value(struct config_set *cs, const char *key, const char **value); + extern const struct string_list *git_configset_get_value_multi(struct config_set *cs, const char *key); + extern void git_configset_clear(struct config_set *cs); + extern int git_configset_get_string_const(struct config_set *cs, const char *key, const char **dest); + extern int git_configset_get_string(struct config_set *cs, const char *key, char **dest); + extern int git_configset_get_int(struct config_set *cs, const char *key, int *dest); + extern int git_configset_get_ulong(struct config_set *cs, const char *key, unsigned long *dest); + extern int git_configset_get_bool(struct config_set *cs, const char *key, int *dest); + extern int git_configset_get_bool_or_int(struct config_set *cs, const char *key, int *is_bool, int *dest); + extern int git_configset_get_maybe_bool(struct config_set *cs, const char *key, int *dest); + extern int git_configset_get_pathname(struct config_set *cs, const char *key, const char **dest); + + extern int git_config_get_value(const char *key, const char **value); + extern const struct string_list *git_config_get_value_multi(const char *key); + extern void git_config_clear(void); + extern void git_config_iter(config_fn_t fn, void *data); + extern int git_config_get_string_const(const char *key, const char **dest); + extern int git_config_get_string(const char *key, char **dest); + extern int git_config_get_int(const char *key, int *dest); + extern int git_config_get_ulong(const char *key, unsigned long *dest); + extern int git_config_get_bool(const char *key, int *dest); + extern int git_config_get_bool_or_int(const char *key, int *is_bool, int *dest); + extern int git_config_get_maybe_bool(const char *key, int *dest); + extern int git_config_get_pathname(const char *key, const char **dest); + extern int committer_ident_sufficiently_given(void); extern int author_ident_sufficiently_given(void); diff --combined config.c index 058505cb8d,d3ad661fca..a191328a9d --- a/config.c +++ b/config.c @@@ -9,6 -9,8 +9,8 @@@ #include "exec_cmd.h" #include "strbuf.h" #include "quote.h" + #include "hashmap.h" + #include "string-list.h" struct config_source { struct config_source *prev; @@@ -33,10 -35,23 +35,23 @@@ long (*do_ftell)(struct config_source *c); }; + struct config_set_element { + struct hashmap_entry ent; + char *key; + struct string_list value_list; + }; + static struct config_source *cf; static int zlib_compression_seen; + /* + * Default config_set that contains key-value pairs from the usual set of config + * config files (i.e repo specific .git/config, user wide ~/.gitconfig, XDG + * config file and the global /etc/gitconfig) + */ + static struct config_set the_config_set; + static int config_file_fgetc(struct config_source *conf) { return fgetc(conf->u.file); @@@ -817,12 -832,14 +832,12 @@@ static int git_default_core_config(cons return git_config_string(&editor_program, var, value); if (!strcmp(var, "core.commentchar")) { - const char *comment; - int ret = git_config_string(&comment, var, value); - if (ret) - return ret; - else if (!strcasecmp(comment, "auto")) + if (!value) + return config_error_nonbool(var); + else if (!strcasecmp(value, "auto")) auto_comment_line_char = 1; - else if (comment[0] && !comment[1]) { - comment_line_char = comment[0]; + else if (value[0] && !value[1]) { + comment_line_char = value[0]; auto_comment_line_char = 0; } else return error("core.commentChar should only be one character"); @@@ -1210,6 -1227,262 +1225,262 @@@ int git_config(config_fn_t fn, void *da return git_config_with_options(fn, data, NULL, 1); } + static struct config_set_element *configset_find_element(struct config_set *cs, const char *key) + { + struct config_set_element k; + struct config_set_element *found_entry; + char *normalized_key; + int ret; + /* + * `key` may come from the user, so normalize it before using it + * for querying entries from the hashmap. + */ + ret = git_config_parse_key(key, &normalized_key, NULL); + + if (ret) + return NULL; + + hashmap_entry_init(&k, strhash(normalized_key)); + k.key = normalized_key; + found_entry = hashmap_get(&cs->config_hash, &k, NULL); + free(normalized_key); + return found_entry; + } + + static int configset_add_value(struct config_set *cs, const char *key, const char *value) + { + struct config_set_element *e; + e = configset_find_element(cs, key); + /* + * Since the keys are being fed by git_config*() callback mechanism, they + * are already normalized. So simply add them without any further munging. + */ + if (!e) { + e = xmalloc(sizeof(*e)); + hashmap_entry_init(e, strhash(key)); + e->key = xstrdup(key); + string_list_init(&e->value_list, 1); + hashmap_add(&cs->config_hash, e); + } + string_list_append_nodup(&e->value_list, value ? xstrdup(value) : NULL); + + return 0; + } + + static int config_set_element_cmp(const struct config_set_element *e1, + const struct config_set_element *e2, const void *unused) + { + return strcmp(e1->key, e2->key); + } + + void git_configset_init(struct config_set *cs) + { + hashmap_init(&cs->config_hash, (hashmap_cmp_fn)config_set_element_cmp, 0); + cs->hash_initialized = 1; + } + + void git_configset_clear(struct config_set *cs) + { + struct config_set_element *entry; + struct hashmap_iter iter; + if (!cs->hash_initialized) + return; + + hashmap_iter_init(&cs->config_hash, &iter); + while ((entry = hashmap_iter_next(&iter))) { + free(entry->key); + string_list_clear(&entry->value_list, 0); + } + hashmap_free(&cs->config_hash, 1); + cs->hash_initialized = 0; + } + + static int config_set_callback(const char *key, const char *value, void *cb) + { + struct config_set *cs = cb; + configset_add_value(cs, key, value); + return 0; + } + + int git_configset_add_file(struct config_set *cs, const char *filename) + { + return git_config_from_file(config_set_callback, filename, cs); + } + + int git_configset_get_value(struct config_set *cs, const char *key, const char **value) + { + const struct string_list *values = NULL; + /* + * Follows "last one wins" semantic, i.e., if there are multiple matches for the + * queried key in the files of the configset, the value returned will be the last + * value in the value list for that key. + */ + values = git_configset_get_value_multi(cs, key); + + if (!values) + return 1; + assert(values->nr > 0); + *value = values->items[values->nr - 1].string; + return 0; + } + + const struct string_list *git_configset_get_value_multi(struct config_set *cs, const char *key) + { + struct config_set_element *e = configset_find_element(cs, key); + return e ? &e->value_list : NULL; + } + + int git_configset_get_string_const(struct config_set *cs, const char *key, const char **dest) + { + const char *value; + if (!git_configset_get_value(cs, key, &value)) + return git_config_string(dest, key, value); + else + return 1; + } + + int git_configset_get_string(struct config_set *cs, const char *key, char **dest) + { + return git_configset_get_string_const(cs, key, (const char **)dest); + } + + int git_configset_get_int(struct config_set *cs, const char *key, int *dest) + { + const char *value; + if (!git_configset_get_value(cs, key, &value)) { + *dest = git_config_int(key, value); + return 0; + } else + return 1; + } + + int git_configset_get_ulong(struct config_set *cs, const char *key, unsigned long *dest) + { + const char *value; + if (!git_configset_get_value(cs, key, &value)) { + *dest = git_config_ulong(key, value); + return 0; + } else + return 1; + } + + int git_configset_get_bool(struct config_set *cs, const char *key, int *dest) + { + const char *value; + if (!git_configset_get_value(cs, key, &value)) { + *dest = git_config_bool(key, value); + return 0; + } else + return 1; + } + + int git_configset_get_bool_or_int(struct config_set *cs, const char *key, + int *is_bool, int *dest) + { + const char *value; + if (!git_configset_get_value(cs, key, &value)) { + *dest = git_config_bool_or_int(key, value, is_bool); + return 0; + } else + return 1; + } + + int git_configset_get_maybe_bool(struct config_set *cs, const char *key, int *dest) + { + const char *value; + if (!git_configset_get_value(cs, key, &value)) { + *dest = git_config_maybe_bool(key, value); + if (*dest == -1) + return -1; + return 0; + } else + return 1; + } + + int git_configset_get_pathname(struct config_set *cs, const char *key, const char **dest) + { + const char *value; + if (!git_configset_get_value(cs, key, &value)) + return git_config_pathname(dest, key, value); + else + return 1; + } + + static void git_config_check_init(void) + { + if (the_config_set.hash_initialized) + return; + git_configset_init(&the_config_set); + git_config(config_set_callback, &the_config_set); + } + + void git_config_clear(void) + { + if (!the_config_set.hash_initialized) + return; + git_configset_clear(&the_config_set); + } + + int git_config_get_value(const char *key, const char **value) + { + git_config_check_init(); + return git_configset_get_value(&the_config_set, key, value); + } + + const struct string_list *git_config_get_value_multi(const char *key) + { + git_config_check_init(); + return git_configset_get_value_multi(&the_config_set, key); + } + + int git_config_get_string_const(const char *key, const char **dest) + { + git_config_check_init(); + return git_configset_get_string_const(&the_config_set, key, dest); + } + + int git_config_get_string(const char *key, char **dest) + { + git_config_check_init(); + return git_config_get_string_const(key, (const char **)dest); + } + + int git_config_get_int(const char *key, int *dest) + { + git_config_check_init(); + return git_configset_get_int(&the_config_set, key, dest); + } + + int git_config_get_ulong(const char *key, unsigned long *dest) + { + git_config_check_init(); + return git_configset_get_ulong(&the_config_set, key, dest); + } + + int git_config_get_bool(const char *key, int *dest) + { + git_config_check_init(); + return git_configset_get_bool(&the_config_set, key, dest); + } + + int git_config_get_bool_or_int(const char *key, int *is_bool, int *dest) + { + git_config_check_init(); + return git_configset_get_bool_or_int(&the_config_set, key, is_bool, dest); + } + + int git_config_get_maybe_bool(const char *key, int *dest) + { + git_config_check_init(); + return git_configset_get_maybe_bool(&the_config_set, key, dest); + } + + int git_config_get_pathname(const char *key, const char **dest) + { + git_config_check_init(); + return git_configset_get_pathname(&the_config_set, key, dest); + } + /* * Find all the stuff for git_config_set() below. */ @@@ -1705,6 -1978,9 +1976,9 @@@ int git_config_set_multivar_in_file(con lock = NULL; ret = 0; + /* Invalidate the config cache */ + git_config_clear(); + out_free: if (lock) rollback_lock_file(lock);