From: Junio C Hamano Date: Mon, 13 May 2019 14:50:31 +0000 (+0900) Subject: Merge branch 'jh/trace2-sid-fix' X-Git-Tag: v2.22.0-rc0~13 X-Git-Url: https://git.lorimer.id.au/gitweb.git/diff_plain/5b2d1c0c6eceb5fd6c9527bc2863179644dce840?ds=inline;hp=-c Merge branch 'jh/trace2-sid-fix' Polishing of the new trace2 facility continues. The system-level configuration can specify site-wide trace2 settings, which can be overridden with per-user configuration and environment variables. * jh/trace2-sid-fix: trace2: fixup access problem on /etc/gitconfig in read_very_early_config trace2: update docs to describe system/global config settings trace2: make SIDs more unique trace2: clarify UTC datetime formatting trace2: report peak memory usage of the process trace2: use system/global config for default trace2 settings config: add read_very_early_config() trace2: find exec-dir before trace2 initialization trace2: add absolute elapsed time to start event trace2: refactor setting process starting time config: initialize opts structure in repo_read_config() --- 5b2d1c0c6eceb5fd6c9527bc2863179644dce840 diff --combined Documentation/technical/api-trace2.txt index d0948ba250,8b6a5e6d4b..9e585b8e79 --- a/Documentation/technical/api-trace2.txt +++ b/Documentation/technical/api-trace2.txt @@@ -22,21 -22,41 +22,41 @@@ Targets are defined using a VTable allo formats in the future. This might be used to define a binary format, for example. + Trace2 is controlled using `trace2.*` config values in the system and + global config files and `GIT_TR2*` environment variables. Trace2 does + not read from repo local or worktree config files or respect `-c` + command line config settings. + == Trace2 Targets Trace2 defines the following set of Trace2 Targets. Format details are given in a later section. - `GIT_TR2` (NORMAL):: + === The Normal Format Target + + The normal format target is a tradition printf format and similar + to GIT_TRACE format. This format is enabled with the `GIT_TR` + environment variable or the `trace2.normalTarget` system or global + config setting. + + For example - a simple printf format like GIT_TRACE. - + ------------ $ export GIT_TR2=~/log.normal $ git version git version 2.20.1.155.g426c96fcdb ------------ - + + + or + + ------------ + $ git config --global trace2.normalTarget ~/log.normal + $ git version + git version 2.20.1.155.g426c96fcdb + ------------ + + yields + ------------ $ cat ~/log.normal 12:28:42.620009 common-main.c:38 version 2.20.1.155.g426c96fcdb @@@ -46,82 -66,82 +66,87 @@@ 12:28:42.621250 trace2/tr2_tgt_normal.c:124 atexit elapsed:0.001265 code:0 ------------ - `GIT_TR2_PERF` (PERF):: + === The Performance Format Target + + The performance format target (PERF) is a column-based format to + replace GIT_TRACE_PERFORMANCE and is suitable for development and + testing, possibly to complement tools like gprof. This format is + enabled with the `GIT_TR2_PERF` environment variable or the + `trace2.perfTarget` system or global config setting. + + For example - a column-based format to replace GIT_TRACE_PERFORMANCE suitable for - development and testing, possibly to complement tools like gprof. - + ------------ $ export GIT_TR2_PERF=~/log.perf $ git version git version 2.20.1.155.g426c96fcdb ------------ - + + + or + + ------------ + $ git config --global trace2.perfTarget ~/log.perf + $ git version + git version 2.20.1.155.g426c96fcdb + ------------ + + yields + ------------ $ cat ~/log.perf 12:28:42.620675 common-main.c:38 | d0 | main | version | | | | | 2.20.1.155.g426c96fcdb - 12:28:42.621001 common-main.c:39 | d0 | main | start | | | | | git version + 12:28:42.621001 common-main.c:39 | d0 | main | start | | 0.001173 | | | git version 12:28:42.621111 git.c:432 | d0 | main | cmd_name | | | | | version (version) 12:28:42.621225 git.c:662 | d0 | main | exit | | 0.001227 | | | code:0 12:28:42.621259 trace2/tr2_tgt_perf.c:211 | d0 | main | atexit | | 0.001265 | | | code:0 ------------ - `GIT_TR2_EVENT` (EVENT):: + === The Event Format Target + + The event format target is a JSON-based format of event data suitable + for telemetry analysis. This format is enabled with the `GIT_TR2_EVENT` + environment variable or the `trace2.eventTarget` system or global config + setting. + + For example - a JSON-based format of event data suitable for telemetry analysis. - + ------------ $ export GIT_TR2_EVENT=~/log.event $ git version git version 2.20.1.155.g426c96fcdb ------------ - + - ------------ - $ cat ~/log.event - {"event":"version","sid":"1547659722619736-11614","thread":"main","time":"2019-01-16 17:28:42.620713","file":"common-main.c","line":38,"evt":"1","exe":"2.20.1.155.g426c96fcdb"} - {"event":"start","sid":"1547659722619736-11614","thread":"main","time":"2019-01-16 17:28:42.621027","file":"common-main.c","line":39,"argv":["git","version"]} - {"event":"cmd_name","sid":"1547659722619736-11614","thread":"main","time":"2019-01-16 17:28:42.621122","file":"git.c","line":432,"name":"version","hierarchy":"version"} - {"event":"exit","sid":"1547659722619736-11614","thread":"main","time":"2019-01-16 17:28:42.621236","file":"git.c","line":662,"t_abs":0.001227,"code":0} - {"event":"atexit","sid":"1547659722619736-11614","thread":"main","time":"2019-01-16 17:28:42.621268","file":"trace2/tr2_tgt_event.c","line":163,"t_abs":0.001265,"code":0} - ------------ - - == Enabling a Target - A Trace2 Target is enabled when the corresponding environment variable - (`GIT_TR2`, `GIT_TR2_PERF`, or `GIT_TR2_EVENT`) is set. The following - values are recognized. + or - `0`:: - `false`:: - - Disables the target. - - `1`:: - `true`:: - - Enables the target and writes stream to `STDERR`. - - `[2-9]`:: + ------------ + $ git config --global trace2.eventTarget ~/log.event + $ git version + git version 2.20.1.155.g426c96fcdb + ------------ - Enables the target and writes to the already opened file descriptor. + yields - ``:: + ------------ + $ cat ~/log.event + {"event":"version","sid":"sid":"20190408T191610.507018Z-H9b68c35f-P000059a8","thread":"main","time":"2019-01-16T17:28:42.620713Z","file":"common-main.c","line":38,"evt":"1","exe":"2.20.1.155.g426c96fcdb"} + {"event":"start","sid":"20190408T191610.507018Z-H9b68c35f-P000059a8","thread":"main","time":"2019-01-16T17:28:42.621027Z","file":"common-main.c","line":39,"t_abs":0.001173,"argv":["git","version"]} + {"event":"cmd_name","sid":"20190408T191610.507018Z-H9b68c35f-P000059a8","thread":"main","time":"2019-01-16T17:28:42.621122Z","file":"git.c","line":432,"name":"version","hierarchy":"version"} + {"event":"exit","sid":"20190408T191610.507018Z-H9b68c35f-P000059a8","thread":"main","time":"2019-01-16T17:28:42.621236Z","file":"git.c","line":662,"t_abs":0.001227,"code":0} + {"event":"atexit","sid":"20190408T191610.507018Z-H9b68c35f-P000059a8","thread":"main","time":"2019-01-16T17:28:42.621268Z","file":"trace2/tr2_tgt_event.c","line":163,"t_abs":0.001265,"code":0} + ------------ - Enables the target, opens and writes to the file in append mode. + === Enabling a Target - If the target already exists and is a directory, the traces will be - written to files (one per process) underneath the given directory. They - will be named according to the last component of the SID (optionally - followed by a counter to avoid filename collisions). + To enable a target, set the corresponding environment variable or + system or global config value to one of the following: - `af_unix:[:]`:: + include::../trace2-target-values.txt[] - Enables the target, opens and writes to a Unix Domain Socket - (on platforms that support them). - + - Socket type can be either `stream` or `dgram`. If the socket type is - omitted, Git will try both. ++If the target already exists and is a directory, the traces will be ++written to files (one per process) underneath the given directory. They ++will be named according to the last component of the SID (optionally ++followed by a counter to avoid filename collisions). + == Trace2 API All public Trace2 functions and macros are defined in `trace2.h` and @@@ -165,17 -185,23 +190,23 @@@ purposes These are concerned with the lifetime of the overall git process. + `void trace2_initialize_clock()`:: + + Initialize the Trace2 start clock and nothing else. This should + be called at the very top of main() to capture the process start + time and reduce startup order dependencies. + `void trace2_initialize()`:: Determines if any Trace2 Targets should be enabled and - initializes the Trace2 facility. This includes starting the - elapsed time clocks and thread local storage (TLS). + initializes the Trace2 facility. This includes setting up the + Trace2 thread local storage (TLS). + This function emits a "version" message containing the version of git and the Trace2 protocol. + This function should be called from `main()` as early as possible in - the life of the process. + the life of the process after essential process initialization. `int trace2_is_enabled()`:: @@@ -242,15 -268,16 +273,16 @@@ significantly affects program performan Emits a "def_param" messages for "important" configuration settings. + - The environment variable `GIT_TR2_CONFIG_PARAMS` can be set to a + The environment variable `GIT_TR2_CONFIG_PARAMS` or the `trace2.configParams` + config value can be set to a list of patterns of important configuration settings, for example: `core.*,remote.*.url`. This function will iterate over all config settings and emit a "def_param" message for each match. `void trace2_cmd_set_config(const char *key, const char *value)`:: - Emits a "def_param" message for a specific configuration - setting IFF it matches the `GIT_TR2_CONFIG_PARAMS` pattern. + Emits a "def_param" message for a new or updated key/value + pair IF `key` is considered important. + This is used to hook into `git_config_set()` and catch any configuration changes and update a value previously reported by @@@ -417,9 -444,6 +449,6 @@@ recursive tree walk === NORMAL Format - NORMAL format is enabled when the `GIT_TR2` environment variable is - set. - Events are written as lines of the form: ------------ @@@ -436,8 -460,8 +465,8 @@@ Note that this may contain embedded LF or CRLF characters that are not escaped, so the event may spill across multiple lines. - If `GIT_TR2_BRIEF` is true, the `time`, `filename`, and `line` fields - are omitted. + If `GIT_TR2_BRIEF` or `trace2.normalBrief` is true, the `time`, `filename`, + and `line` fields are omitted. This target is intended to be more of a summary (like GIT_TRACE) and less detailed than the other targets. It ignores thread, region, and @@@ -445,9 -469,6 +474,6 @@@ data messages, for example === PERF Format - PERF format is enabled when the `GIT_TR2_PERF` environment variable - is set. - Events are written as lines of the form: ------------ @@@ -507,8 -528,8 +533,8 @@@ This field is in anticipation of in-pro 15:33:33.532712 wt-status.c:2331 | d0 | main | region_leave | r1 | 0.127568 | 0.001504 | status | label:print ------------ - If `GIT_TR2_PERF_BRIEF` is true, the `time`, `file`, and `line` - fields are omitted. + If `GIT_TR2_PERF_BRIEF` or `trace2.perfBrief` is true, the `time`, `file`, + and `line` fields are omitted. ------------ d0 | main | region_leave | r1 | 0.011717 | 0.009122 | index | label:preload @@@ -519,9 -540,6 +545,6 @@@ during development and is quite noisy === EVENT Format - EVENT format is enabled when the `GIT_TR2_EVENT` environment - variable is set. - Each event is a JSON-object containing multiple key/value pairs written as a single line and followed by a LF. @@@ -539,11 -557,11 +562,11 @@@ The following key/value pairs are commo ------------ { "event":"version", - "sid":"1547659722619736-11614", + "sid":"20190408T191827.272759Z-H9b68c35f-P00003510", "thread":"main", - "time":"2019-01-16 17:28:42.620713", + "time":"2019-04-08T19:18:27.282761Z", "file":"common-main.c", - "line":38, + "line":42, ... } ------------ @@@ -575,9 -593,9 +598,9 @@@ `"repo":`:: when present, is the integer repo-id as described previously. - If `GIT_TR2_EVENT_BRIEF` is true, the `file` and `line` fields are omitted - from all events and the `time` field is only present on the "start" and - "atexit" events. + If `GIT_TR2_EVENT_BRIEF` or `trace2.eventBrief` is true, the `file` + and `line` fields are omitted from all events and the `time` field is + only present on the "start" and "atexit" events. ==== Event-Specific Key/Value Pairs @@@ -600,6 -618,7 +623,7 @@@ { "event":"start", ... + "t_abs":0.001227, # elapsed time in seconds "argv":["git","version"] } ------------ @@@ -887,7 -906,7 +911,7 @@@ visited The `category` field may be used in a future enhancement to do category-based filtering. + - The `GIT_TR2_EVENT_NESTING` environment variable can be used to + `GIT_TR2_EVENT_NESTING` or `trace2.eventNesting` can be used to filter deeply nested regions and data events. It defaults to "2". `"region_leave"`:: @@@ -1117,7 -1136,7 +1141,7 @@@ $ git statu $ cat ~/log.perf d0 | main | version | | | | | 2.20.1.160.g5676107ecd.dirty - d0 | main | start | | | | | git status + d0 | main | start | | 0.001173 | | | git status d0 | main | def_repo | r1 | | | | worktree:/Users/jeffhost/work/gfw d0 | main | cmd_name | | | | | status (status) ... @@@ -1162,7 -1181,7 +1186,7 @@@ $ git statu ... $ cat ~/log.perf d0 | main | version | | | | | 2.20.1.162.gb4ccea44db.dirty - d0 | main | start | | | | | git status + d0 | main | start | | 0.001173 | | | git status d0 | main | def_repo | r1 | | | | worktree:/Users/jeffhost/work/gfw d0 | main | cmd_name | | | | | status (status) ... @@@ -1218,7 -1237,7 +1242,7 @@@ $ git statu ... $ cat ~/log.perf d0 | main | version | | | | | 2.20.1.156.gf9916ae094.dirty - d0 | main | start | | | | | git status + d0 | main | start | | 0.001173 | | | git status d0 | main | def_repo | r1 | | | | worktree:/Users/jeffhost/work/gfw d0 | main | cmd_name | | | | | status (status) d0 | main | region_enter | r1 | 0.001791 | | index | label:do_read_index .git/index diff --combined Makefile index 5143cab9bd,9ddfa3dfe7..af247878c5 --- a/Makefile +++ b/Makefile @@@ -592,7 -592,6 +592,7 @@@ FUZZ_PROGRAMS LIB_OBJS = PROGRAM_OBJS = PROGRAMS = +EXCLUDED_PROGRAMS = SCRIPT_PERL = SCRIPT_PYTHON = SCRIPT_SH = @@@ -614,8 -613,10 +614,8 @@@ SCRIPT_SH += git-merge-one-file.s SCRIPT_SH += git-merge-resolve.sh SCRIPT_SH += git-mergetool.sh SCRIPT_SH += git-quiltimport.sh -SCRIPT_SH += git-legacy-rebase.sh -SCRIPT_SH += git-remote-testgit.sh +SCRIPT_SH += git-legacy-stash.sh SCRIPT_SH += git-request-pull.sh -SCRIPT_SH += git-stash.sh SCRIPT_SH += git-submodule.sh SCRIPT_SH += git-web--browse.sh @@@ -637,11 -638,17 +637,11 @@@ SCRIPT_PERL += git-svn.per SCRIPT_PYTHON += git-p4.py -NO_INSTALL += git-remote-testgit - # Generated files for scripts SCRIPT_SH_GEN = $(patsubst %.sh,%,$(SCRIPT_SH)) SCRIPT_PERL_GEN = $(patsubst %.perl,%,$(SCRIPT_PERL)) SCRIPT_PYTHON_GEN = $(patsubst %.py,%,$(SCRIPT_PYTHON)) -SCRIPT_SH_INS = $(filter-out $(NO_INSTALL),$(SCRIPT_SH_GEN)) -SCRIPT_PERL_INS = $(filter-out $(NO_INSTALL),$(SCRIPT_PERL_GEN)) -SCRIPT_PYTHON_INS = $(filter-out $(NO_INSTALL),$(SCRIPT_PYTHON_GEN)) - # Individual rules to allow e.g. # "make -C ../.. SCRIPT_PERL=contrib/foo/bar.perl build-perl-script" # from subdirectories like contrib/*/ @@@ -651,11 -658,11 +651,11 @@@ build-sh-script: $(SCRIPT_SH_GEN build-python-script: $(SCRIPT_PYTHON_GEN) .PHONY: install-perl-script install-sh-script install-python-script -install-sh-script: $(SCRIPT_SH_INS) +install-sh-script: $(SCRIPT_SH_GEN) $(INSTALL) $^ '$(DESTDIR_SQ)$(gitexec_instdir_SQ)' -install-perl-script: $(SCRIPT_PERL_INS) +install-perl-script: $(SCRIPT_PERL_GEN) $(INSTALL) $^ '$(DESTDIR_SQ)$(gitexec_instdir_SQ)' -install-python-script: $(SCRIPT_PYTHON_INS) +install-python-script: $(SCRIPT_PYTHON_GEN) $(INSTALL) $^ '$(DESTDIR_SQ)$(gitexec_instdir_SQ)' .PHONY: clean-perl-script clean-sh-script clean-python-script @@@ -666,9 -673,9 +666,9 @@@ clean-perl-script clean-python-script: $(RM) $(SCRIPT_PYTHON_GEN) -SCRIPTS = $(SCRIPT_SH_INS) \ - $(SCRIPT_PERL_INS) \ - $(SCRIPT_PYTHON_INS) \ +SCRIPTS = $(SCRIPT_SH_GEN) \ + $(SCRIPT_PERL_GEN) \ + $(SCRIPT_PYTHON_GEN) \ git-instaweb ETAGS_TARGET = TAGS @@@ -738,7 -745,6 +738,7 @@@ TEST_BUILTINS_OBJS += test-repository. TEST_BUILTINS_OBJS += test-revision-walking.o TEST_BUILTINS_OBJS += test-run-command.o TEST_BUILTINS_OBJS += test-scrap-cache-tree.o +TEST_BUILTINS_OBJS += test-serve-v2.o TEST_BUILTINS_OBJS += test-sha1.o TEST_BUILTINS_OBJS += test-sha1-array.o TEST_BUILTINS_OBJS += test-sha256.o @@@ -999,6 -1005,7 +999,7 @@@ LIB_OBJS += trace2/tr2_cfg. LIB_OBJS += trace2/tr2_cmd_name.o LIB_OBJS += trace2/tr2_dst.o LIB_OBJS += trace2/tr2_sid.o + LIB_OBJS += trace2/tr2_sysenv.o LIB_OBJS += trace2/tr2_tbuf.o LIB_OBJS += trace2/tr2_tgt_event.o LIB_OBJS += trace2/tr2_tgt_normal.o @@@ -1120,11 -1127,11 +1121,11 @@@ BUILTIN_OBJS += builtin/rev-parse. BUILTIN_OBJS += builtin/revert.o BUILTIN_OBJS += builtin/rm.o BUILTIN_OBJS += builtin/send-pack.o -BUILTIN_OBJS += builtin/serve.o BUILTIN_OBJS += builtin/shortlog.o BUILTIN_OBJS += builtin/show-branch.o BUILTIN_OBJS += builtin/show-index.o BUILTIN_OBJS += builtin/show-ref.o +BUILTIN_OBJS += builtin/stash.o BUILTIN_OBJS += builtin/stripspace.o BUILTIN_OBJS += builtin/submodule--helper.o BUILTIN_OBJS += builtin/symbolic-ref.o @@@ -1192,7 -1199,6 +1193,7 @@@ BASIC_CFLAGS += -fsanitize=$(SANITIZE) BASIC_CFLAGS += -fno-omit-frame-pointer ifneq ($(filter undefined,$(SANITIZERS)),) BASIC_CFLAGS += -DNO_UNALIGNED_LOADS +BASIC_CFLAGS += -DSHA1DC_FORCE_ALIGNED_ACCESS endif ifneq ($(filter leak,$(SANITIZERS)),) BASIC_CFLAGS += -DSUPPRESS_ANNOTATED_LEAKS @@@ -1336,7 -1342,6 +1337,7 @@@ ifdef NO_CUR REMOTE_CURL_PRIMARY = REMOTE_CURL_ALIASES = REMOTE_CURL_NAMES = + EXCLUDED_PROGRAMS += git-http-fetch git-http-push else ifdef CURLDIR # Try "-Wl,-rpath=$(CURLDIR)/$(lib)" in such a case. @@@ -1361,11 -1366,7 +1362,11 @@@ endi ifeq "$(curl_check)" "070908" ifndef NO_EXPAT PROGRAM_OBJS += http-push.o + else + EXCLUDED_PROGRAMS += git-http-push endif + else + EXCLUDED_PROGRAMS += git-http-push endif curl_check := $(shell (echo 072200; $(CURL_CONFIG) --vernum | sed -e '/^70[BC]/s/^/0/') 2>/dev/null | sort -r | sed -ne 2p) ifeq "$(curl_check)" "072200" @@@ -1613,7 -1614,6 +1614,7 @@@ ifdef NO_INET_PTO endif ifdef NO_UNIX_SOCKETS BASIC_CFLAGS += -DNO_UNIX_SOCKETS + EXCLUDED_PROGRAMS += git-credential-cache git-credential-cache--daemon else LIB_OBJS += unix-socket.o PROGRAM_OBJS += credential-cache.o @@@ -2133,9 -2133,7 +2134,9 @@@ $(BUILT_INS): git$ command-list.h: generate-cmdlist.sh command-list.txt command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Documentation/config/*.txt - $(QUIET_GEN)$(SHELL_PATH) ./generate-cmdlist.sh command-list.txt >$@+ && mv $@+ $@ + $(QUIET_GEN)$(SHELL_PATH) ./generate-cmdlist.sh \ + $(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \ + command-list.txt >$@+ && mv $@+ $@ SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\ $(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\ @@@ -2468,14 -2466,6 +2469,14 @@@ $(VCSSVN_LIB): $(VCSSVN_OBJS export DEFAULT_EDITOR DEFAULT_PAGER +Documentation/GIT-EXCLUDED-PROGRAMS: FORCE + @EXCLUDED='EXCLUDED_PROGRAMS := $(EXCLUDED_PROGRAMS)'; \ + if test x"$$EXCLUDED" != \ + x"`cat Documentation/GIT-EXCLUDED-PROGRAMS 2>/dev/null`" ; then \ + echo >&2 " * new documentation flags"; \ + echo "$$EXCLUDED" >Documentation/GIT-EXCLUDED-PROGRAMS; \ + fi + .PHONY: doc man man-perl html info pdf doc: man-perl $(MAKE) -C Documentation all @@@ -2714,6 -2704,7 +2715,6 @@@ endi test_bindir_programs := $(patsubst %,bin-wrappers/%,$(BINDIR_PROGRAMS_NEED_X) $(BINDIR_PROGRAMS_NO_X) $(TEST_PROGRAMS_NEED_X)) all:: $(TEST_PROGRAMS) $(test_bindir_programs) -all:: $(NO_INSTALL) bin-wrappers/%: wrap-for-bin.sh @mkdir -p bin-wrappers @@@ -3000,7 -2991,7 +3001,7 @@@ rpm: artifacts-tar:: $(ALL_PROGRAMS) $(SCRIPT_LIB) $(BUILT_INS) $(OTHER_PROGRAMS) \ GIT-BUILD-OPTIONS $(TEST_PROGRAMS) $(test_bindir_programs) \ - $(NO_INSTALL) $(MOFILES) + $(MOFILES) $(QUIET_SUBDIR0)templates $(QUIET_SUBDIR1) \ SHELL_PATH='$(SHELL_PATH_SQ)' PERL_PATH='$(PERL_PATH_SQ)' test -n "$(ARTIFACTS_DIRECTORY)" @@@ -3049,7 -3040,7 +3050,7 @@@ clean: profile-clean coverage-clean coc $(RM) $(OBJECTS) $(RM) $(LIB_FILE) $(XDIFF_LIB) $(VCSSVN_LIB) $(RM) $(ALL_PROGRAMS) $(SCRIPT_LIB) $(BUILT_INS) git$X - $(RM) $(TEST_PROGRAMS) $(NO_INSTALL) + $(RM) $(TEST_PROGRAMS) $(RM) $(FUZZ_PROGRAMS) $(RM) -r bin-wrappers $(dep_dirs) $(RM) -r po/build/ @@@ -3058,7 -3049,6 +3059,7 @@@ $(RM) $(GIT_TARNAME).tar.gz git-core_$(GIT_VERSION)-*.tar.gz $(RM) $(htmldocs).tar.gz $(manpages).tar.gz $(MAKE) -C Documentation/ clean + $(RM) Documentation/GIT-EXCLUDED-PROGRAMS ifndef NO_PERL $(MAKE) -C gitweb clean $(RM) -r perl/build/ @@@ -3088,13 -3078,13 +3089,13 @@@ ALL_COMMANDS += git-gui git-citoo .PHONY: check-docs check-docs:: $(MAKE) -C Documentation lint-docs - @(for v in $(ALL_COMMANDS); \ + @(for v in $(patsubst %$X,%,$(ALL_COMMANDS)); \ do \ case "$$v" in \ git-merge-octopus | git-merge-ours | git-merge-recursive | \ git-merge-resolve | git-merge-subtree | \ git-fsck-objects | git-init-db | \ - git-remote-* | git-stage | \ + git-remote-* | git-stage | git-legacy-* | \ git-?*--?* ) continue ;; \ esac ; \ test -f "Documentation/$$v.txt" || \ @@@ -3109,16 -3099,15 +3110,16 @@@ ( \ sed -e '1,/^### command list/d' \ -e '/^#/d' \ + -e '/guide$$/d' \ -e 's/[ ].*//' \ -e 's/^/listed /' command-list.txt; \ $(MAKE) -C Documentation print-man1 | \ grep '\.txt$$' | \ - sed -e 's|Documentation/|documented |' \ + sed -e 's|^|documented |' \ -e 's/\.txt//'; \ ) | while read how cmd; \ do \ - case " $(ALL_COMMANDS) " in \ + case " $(patsubst %$X,%,$(ALL_COMMANDS) $(EXCLUDED_PROGRAMS)) " in \ *" $$cmd "*) ;; \ *) echo "removed but $$how: $$cmd" ;; \ esac; \ diff --combined config.c index c2846df3f1,3f5ad9dc3c..296a6d9cc4 --- a/config.c +++ b/config.c @@@ -242,7 -242,7 +242,7 @@@ again } ret = !wildmatch(pattern.buf + prefix, text.buf + prefix, - icase ? WM_CASEFOLD : 0); + WM_PATHNAME | (icase ? WM_CASEFOLD : 0)); if (!ret && !already_tried_absolute) { /* @@@ -1676,7 -1676,9 +1676,9 @@@ static int do_git_config_sequence(cons repo_config = NULL; current_parsing_scope = CONFIG_SCOPE_SYSTEM; - if (git_config_system() && !access_or_die(git_etc_gitconfig(), R_OK, 0)) + if (git_config_system() && !access_or_die(git_etc_gitconfig(), R_OK, + opts->system_gently ? + ACCESS_EACCES_OK : 0)) ret += git_config_from_file(fn, git_etc_gitconfig(), data); @@@ -1688,14 -1690,15 +1690,15 @@@ ret += git_config_from_file(fn, user_config, data); current_parsing_scope = CONFIG_SCOPE_REPO; - if (repo_config && !access_or_die(repo_config, R_OK, 0)) + if (!opts->ignore_repo && repo_config && + !access_or_die(repo_config, R_OK, 0)) ret += git_config_from_file(fn, repo_config, data); /* * Note: this should have a new scope, CONFIG_SCOPE_WORKTREE. * But let's not complicate things before it's actually needed. */ - if (repository_format_worktree_config) { + if (!opts->ignore_worktree && repository_format_worktree_config) { char *path = git_pathdup("config.worktree"); if (!access_or_die(path, R_OK, 0)) ret += git_config_from_file(fn, path, data); @@@ -1703,7 -1706,7 +1706,7 @@@ } current_parsing_scope = CONFIG_SCOPE_CMDLINE; - if (git_config_from_parameters(fn, data) < 0) + if (!opts->ignore_cmdline && git_config_from_parameters(fn, data) < 0) die(_("unable to parse command-line config")); current_parsing_scope = CONFIG_SCOPE_UNKNOWN; @@@ -1794,6 -1797,23 +1797,23 @@@ void read_early_config(config_fn_t cb, strbuf_release(&gitdir); } + /* + * Read config but only enumerate system and global settings. + * Omit any repo-local, worktree-local, or command-line settings. + */ + void read_very_early_config(config_fn_t cb, void *data) + { + struct config_options opts = { 0 }; + + opts.respect_includes = 1; + opts.ignore_repo = 1; + opts.ignore_worktree = 1; + opts.ignore_cmdline = 1; + opts.system_gently = 1; + + config_with_options(cb, data, NULL, &opts); + } + static struct config_set_element *configset_find_element(struct config_set *cs, const char *key) { struct config_set_element k; @@@ -2011,7 -2031,7 +2031,7 @@@ int git_configset_get_pathname(struct c /* Functions use to read configuration from a repository */ static void repo_read_config(struct repository *repo) { - struct config_options opts; + struct config_options opts = { 0 }; opts.respect_includes = 1; opts.commondir = repo->commondir; diff --combined t/t0210-trace2-normal.sh index 819430658b,8d17e1e6f1..71194a3623 --- a/t/t0210-trace2-normal.sh +++ b/t/t0210-trace2-normal.sh @@@ -3,6 -3,11 +3,11 @@@ test_description='test trace2 facility (normal target)' . ./test-lib.sh + # Turn off any inherited trace2 settings for this test. + sane_unset GIT_TR2 GIT_TR2_PERF GIT_TR2_EVENT + sane_unset GIT_TR2_BRIEF + sane_unset GIT_TR2_CONFIG_PARAMS + # Add t/helper directory to PATH so that we can use a relative # path to run nested instances of test-tool.exe (see 004child). # This helps with HEREDOC comparisons later. @@@ -15,11 -20,6 +20,6 @@@ PATH="$TTDIR:$PATH" && export PAT # Warning: So you may see extra lines in artifact files when # Warning: interactively debugging. - # Turn off any inherited trace2 settings for this test. - unset GIT_TR2 GIT_TR2_PERF GIT_TR2_EVENT - unset GIT_TR2_BRIEF - unset GIT_TR2_CONFIG_PARAMS - V=$(git version | sed -e 's/^git version //') && export V # There are multiple trace2 targets: normal, perf, and event. @@@ -80,21 -80,6 +80,21 @@@ test_expect_success 'normal stream, ret test_cmp expect actual ' +test_expect_success 'automatic filename' ' + test_when_finished "rm -r traces actual expect" && + mkdir traces && + GIT_TR2="$(pwd)/traces" test-tool trace2 001return 0 && + perl "$TEST_DIRECTORY/t0210/scrub_normal.perl" <"$(ls traces/*)" >actual && + cat >expect <<-EOF && + version $V + start _EXE_ trace2 001return 0 + cmd_name trace2 (trace2) + exit elapsed:_TIME_ code:0 + atexit elapsed:_TIME_ code:0 + EOF + test_cmp expect actual +' + # Verb 002exit # # Explicit exit(code) from within cmd_ propagates . @@@ -147,4 -132,43 +147,43 @@@ test_expect_success 'normal stream, err test_cmp expect actual ' + sane_unset GIT_TR2_BRIEF + + # Now test without environment variables and get all Trace2 settings + # from the global config. + + test_expect_success 'using global config, normal stream, return code 0' ' + test_when_finished "rm trace.normal actual expect" && + test_config_global trace2.normalBrief 1 && + test_config_global trace2.normalTarget "$(pwd)/trace.normal" && + test-tool trace2 001return 0 && + perl "$TEST_DIRECTORY/t0210/scrub_normal.perl" actual && + cat >expect <<-EOF && + version $V + start _EXE_ trace2 001return 0 + cmd_name trace2 (trace2) + exit elapsed:_TIME_ code:0 + atexit elapsed:_TIME_ code:0 + EOF + test_cmp expect actual + ' + + test_expect_success 'using global config with include' ' + test_when_finished "rm trace.normal actual expect real.gitconfig" && + test_config_global trace2.normalBrief 1 && + test_config_global trace2.normalTarget "$(pwd)/trace.normal" && + mv "$(pwd)/.gitconfig" "$(pwd)/real.gitconfig" && + test_config_global include.path "$(pwd)/real.gitconfig" && + test-tool trace2 001return 0 && + perl "$TEST_DIRECTORY/t0210/scrub_normal.perl" actual && + cat >expect <<-EOF && + version $V + start _EXE_ trace2 001return 0 + cmd_name trace2 (trace2) + exit elapsed:_TIME_ code:0 + atexit elapsed:_TIME_ code:0 + EOF + test_cmp expect actual + ' + test_done diff --combined trace2.c index 8bbad56887,6baa65cdf9..eb759f3266 --- a/trace2.c +++ b/trace2.c @@@ -10,6 -10,7 +10,7 @@@ #include "trace2/tr2_cmd_name.h" #include "trace2/tr2_dst.h" #include "trace2/tr2_sid.h" + #include "trace2/tr2_sysenv.h" #include "trace2/tr2_tgt.h" #include "trace2/tr2_tls.h" @@@ -120,6 -121,7 +121,7 @@@ static void tr2main_atexit_handler(void tr2_sid_release(); tr2_cmd_name_release(); tr2_cfg_free_patterns(); + tr2_sysenv_release(); trace2_enabled = 0; } @@@ -142,6 -144,11 +144,11 @@@ static void tr2main_signal_handler(int raise(signo); } + void trace2_initialize_clock(void) + { + tr2tls_start_process_clock(); + } + void trace2_initialize_fl(const char *file, int line) { struct tr2_tgt *tgt_j; @@@ -150,6 -157,8 +157,8 @@@ if (trace2_enabled) return; + tr2_sysenv_load(); + if (!tr2_tgt_want_builtins()) return; trace2_enabled = 1; @@@ -177,13 -186,19 +186,19 @@@ void trace2_cmd_start_fl(const char *fi { struct tr2_tgt *tgt_j; int j; + uint64_t us_now; + uint64_t us_elapsed_absolute; if (!trace2_enabled) return; + us_now = getnanotime() / 1000; + us_elapsed_absolute = tr2tls_absolute_elapsed(us_now); + for_each_wanted_builtin (j, tgt_j) if (tgt_j->pfn_start_fl) - tgt_j->pfn_start_fl(file, line, argv); + tgt_j->pfn_start_fl(file, line, us_elapsed_absolute, + argv); } int trace2_cmd_exit_fl(const char *file, int line, int code) @@@ -198,6 -213,8 +213,8 @@@ if (!trace2_enabled) return code; + trace2_collect_process_info(TRACE2_PROCESS_INFO_EXIT); + tr2main_exit_code = code; us_now = getnanotime() / 1000; @@@ -428,7 -445,7 +445,7 @@@ void trace2_thread_start_fl(const char us_now = getnanotime() / 1000; us_elapsed_absolute = tr2tls_absolute_elapsed(us_now); - tr2tls_create_self(thread_name); + tr2tls_create_self(thread_name, us_now); for_each_wanted_builtin (j, tgt_j) if (tgt_j->pfn_thread_start_fl) @@@ -548,14 -565,10 +565,14 @@@ void trace2_region_enter_printf_va_fl(c } void trace2_region_enter_fl(const char *file, int line, const char *category, - const char *label, const struct repository *repo) + const char *label, const struct repository *repo, ...) { + va_list ap; + va_start(ap, repo); trace2_region_enter_printf_va_fl(file, line, category, label, repo, - NULL, NULL); + NULL, ap); + va_end(ap); + } void trace2_region_enter_printf_fl(const char *file, int line, @@@ -625,13 -638,10 +642,13 @@@ void trace2_region_leave_printf_va_fl(c } void trace2_region_leave_fl(const char *file, int line, const char *category, - const char *label, const struct repository *repo) + const char *label, const struct repository *repo, ...) { + va_list ap; + va_start(ap, repo); trace2_region_leave_printf_va_fl(file, line, category, label, repo, - NULL, NULL); + NULL, ap); + va_end(ap); } void trace2_region_leave_printf_fl(const char *file, int line, diff --combined trace2.h index b330a54a89,888531eb08..f189ef5984 --- a/trace2.h +++ b/trace2.h @@@ -19,9 -19,27 +19,27 @@@ struct json_writer * [] trace2_printf* -- legacy trace[1] messages. */ + /* + * Initialize the TRACE2 clock and do nothing else, in particular + * no mallocs, no system inspection, and no environment inspection. + * + * This should be called at the very top of main() to capture the + * process start time. This is intended to reduce chicken-n-egg + * bootstrap pressure. + * + * It is safe to call this more than once. This allows capturing + * absolute startup costs on Windows which uses a little trickery + * to do setup work before common-main.c:main() is called. + * + * The main trace2_initialize_fl() may be called a little later + * after more infrastructure is established. + */ + void trace2_initialize_clock(void); + /* * Initialize TRACE2 tracing facility if any of the builtin TRACE2 - * targets are enabled in the environment. Emits a 'version' event. + * targets are enabled in the system config or the environment. + * Emits a 'version' event. * * Cleanup/Termination is handled automatically by a registered * atexit() routine. @@@ -108,10 -126,11 +126,11 @@@ void trace2_cmd_alias_fl(const char *fi * Emit one or more 'def_param' events for "interesting" configuration * settings. * - * The environment variable "GIT_TR2_CONFIG_PARAMS" can be set to a - * list of patterns considered important. For example: - * - * GIT_TR2_CONFIG_PARAMS="core.*,remote.*.url" + * Use the TR2_SYSENV_CFG_PARAM setting to register a comma-separated + * list of patterns configured important. For example: + * git config --system trace2.configParams 'core.*,remote.*.url' + * or: + * GIT_TR2_CONFIG_PARAMS=core.*,remote.*.url" * * Note: this routine does a read-only iteration on the config data * (using read_early_config()), so it must not be called until enough @@@ -238,7 -257,7 +257,7 @@@ void trace2_def_repo_fl(const char *fil * on this thread. */ void trace2_region_enter_fl(const char *file, int line, const char *category, - const char *label, const struct repository *repo); + const char *label, const struct repository *repo, ...); #define trace2_region_enter(category, label, repo) \ trace2_region_enter_fl(__FILE__, __LINE__, (category), (label), (repo)) @@@ -278,7 -297,7 +297,7 @@@ void trace2_region_enter_printf(const c * in this nesting level. */ void trace2_region_leave_fl(const char *file, int line, const char *category, - const char *label, const struct repository *repo); + const char *label, const struct repository *repo, ...); #define trace2_region_leave(category, label, repo) \ trace2_region_leave_fl(__FILE__, __LINE__, (category), (label), (repo)) @@@ -372,13 -391,19 +391,19 @@@ void trace2_printf(const char *fmt, ... * Optional platform-specific code to dump information about the * current and any parent process(es). This is intended to allow * post-processors to know who spawned this git instance and anything - * else the platform may be able to tell us about the current process. + * else that the platform may be able to tell us about the current process. */ + + enum trace2_process_info_reason { + TRACE2_PROCESS_INFO_STARTUP, + TRACE2_PROCESS_INFO_EXIT, + }; + #if defined(GIT_WINDOWS_NATIVE) - void trace2_collect_process_info(void); + void trace2_collect_process_info(enum trace2_process_info_reason reason); #else - #define trace2_collect_process_info() \ - do { \ + #define trace2_collect_process_info(reason) \ + do { \ } while (0) #endif diff --combined trace2/tr2_dst.c index c3d82ca6a4,7d96f33420..5dda0ca1cd --- a/trace2/tr2_dst.c +++ b/trace2/tr2_dst.c @@@ -1,29 -1,20 +1,19 @@@ #include "cache.h" #include "trace2/tr2_dst.h" +#include "trace2/tr2_sid.h" - - /* - * If a Trace2 target cannot be opened for writing, we should issue a - * warning to stderr, but this is very annoying if the target is a pipe - * or socket and beyond the user's control -- especially since every - * git command (and sub-command) will print the message. So we silently - * eat these warnings and just discard the trace data. - * - * Enable the following environment variable to see these warnings. - */ - #define TR2_ENVVAR_DST_DEBUG "GIT_TR2_DST_DEBUG" + #include "trace2/tr2_sysenv.h" /* - * If a Trace2 target cannot be opened for writing, we should issue a - * warning to stderr, but this is very annoying if the target is a pipe - * or socket and beyond the user's control -- especially since every - * git command (and sub-command) will print the message. So we silently - * eat these warnings and just discard the trace data. + * How many attempts we will make at creating an automatically-named trace file. */ +#define MAX_AUTO_ATTEMPTS 10 + static int tr2_dst_want_warning(void) { static int tr2env_dst_debug = -1; if (tr2env_dst_debug == -1) { - const char *env_value = getenv(TR2_ENVVAR_DST_DEBUG); + const char *env_value = tr2_sysenv_get(TR2_SYSENV_DST_DEBUG); if (!env_value || !*env_value) tr2env_dst_debug = 0; else @@@ -42,62 -33,15 +32,65 @@@ void tr2_dst_trace_disable(struct tr2_d dst->need_close = 0; } +static int tr2_dst_try_auto_path(struct tr2_dst *dst, const char *tgt_prefix) +{ + int fd; + const char *last_slash, *sid = tr2_sid_get(); + struct strbuf path = STRBUF_INIT; + size_t base_path_len; + unsigned attempt_count; + + last_slash = strrchr(sid, '/'); + if (last_slash) + sid = last_slash + 1; + + strbuf_addstr(&path, tgt_prefix); + if (!is_dir_sep(path.buf[path.len - 1])) + strbuf_addch(&path, '/'); + strbuf_addstr(&path, sid); + base_path_len = path.len; + + for (attempt_count = 0; attempt_count < MAX_AUTO_ATTEMPTS; attempt_count++) { + if (attempt_count > 0) { + strbuf_setlen(&path, base_path_len); + strbuf_addf(&path, ".%d", attempt_count); + } + + fd = open(path.buf, O_WRONLY | O_CREAT | O_EXCL, 0666); + if (fd != -1) + break; + } + + if (fd == -1) { + if (tr2_dst_want_warning()) + warning("trace2: could not open '%.*s' for '%s' tracing: %s", + (int) base_path_len, path.buf, - dst->env_var_name, strerror(errno)); ++ tr2_sysenv_display_name(dst->sysenv_var), ++ strerror(errno)); + + tr2_dst_trace_disable(dst); + strbuf_release(&path); + return 0; + } + + strbuf_release(&path); + + dst->fd = fd; + dst->need_close = 1; + dst->initialized = 1; + + return dst->fd; +} + static int tr2_dst_try_path(struct tr2_dst *dst, const char *tgt_value) { int fd = open(tgt_value, O_WRONLY | O_APPEND | O_CREAT, 0666); if (fd == -1) { if (tr2_dst_want_warning()) warning("trace2: could not open '%s' for '%s' tracing: %s", - tgt_value, dst->env_var_name, strerror(errno)); + tgt_value, + tr2_sysenv_display_name(dst->sysenv_var), + strerror(errno)); tr2_dst_trace_disable(dst); return 0; @@@ -171,7 -115,8 +164,8 @@@ static int tr2_dst_try_unix_domain_sock if (!path || !*path) { if (tr2_dst_want_warning()) warning("trace2: invalid AF_UNIX value '%s' for '%s' tracing", - tgt_value, dst->env_var_name); + tgt_value, + tr2_sysenv_display_name(dst->sysenv_var)); tr2_dst_trace_disable(dst); return 0; @@@ -181,7 -126,7 +175,7 @@@ strlen(path) >= sizeof(((struct sockaddr_un *)0)->sun_path)) { if (tr2_dst_want_warning()) warning("trace2: invalid AF_UNIX path '%s' for '%s' tracing", - path, dst->env_var_name); + path, tr2_sysenv_display_name(dst->sysenv_var)); tr2_dst_trace_disable(dst); return 0; @@@ -203,7 -148,8 +197,8 @@@ error: if (tr2_dst_want_warning()) warning("trace2: could not connect to socket '%s' for '%s' tracing: %s", - path, dst->env_var_name, strerror(e)); + path, tr2_sysenv_display_name(dst->sysenv_var), + strerror(e)); tr2_dst_trace_disable(dst); return 0; @@@ -223,7 -169,7 +218,7 @@@ static void tr2_dst_malformed_warning(s struct strbuf buf = STRBUF_INIT; strbuf_addf(&buf, "trace2: unknown value for '%s': '%s'", - dst->env_var_name, tgt_value); + tr2_sysenv_display_name(dst->sysenv_var), tgt_value); warning("%s", buf.buf); strbuf_release(&buf); @@@ -239,7 -185,7 +234,7 @@@ int tr2_dst_get_trace_fd(struct tr2_ds dst->initialized = 1; - tgt_value = getenv(dst->env_var_name); + tgt_value = tr2_sysenv_get(dst->sysenv_var); if (!tgt_value || !strcmp(tgt_value, "") || !strcmp(tgt_value, "0") || !strcasecmp(tgt_value, "false")) { @@@ -257,12 -203,8 +252,12 @@@ return dst->fd; } - if (is_absolute_path(tgt_value)) - return tr2_dst_try_path(dst, tgt_value); + if (is_absolute_path(tgt_value)) { + if (is_directory(tgt_value)) + return tr2_dst_try_auto_path(dst, tgt_value); + else + return tr2_dst_try_path(dst, tgt_value); + } #ifndef NO_UNIX_SOCKETS if (starts_with(tgt_value, PREFIX_AF_UNIX)) @@@ -305,7 -247,8 +300,8 @@@ void tr2_dst_write_line(struct tr2_dst return; if (tr2_dst_want_warning()) - warning("unable to write trace to '%s': %s", dst->env_var_name, + warning("unable to write trace to '%s': %s", + tr2_sysenv_display_name(dst->sysenv_var), strerror(errno)); tr2_dst_trace_disable(dst); } diff --combined trace2/tr2_tgt_event.c index 1cf4f62441,2c97cf54be..c2852d1bd2 --- a/trace2/tr2_tgt_event.c +++ b/trace2/tr2_tgt_event.c @@@ -6,10 -6,11 +6,11 @@@ #include "trace2/tr2_dst.h" #include "trace2/tr2_tbuf.h" #include "trace2/tr2_sid.h" + #include "trace2/tr2_sysenv.h" #include "trace2/tr2_tgt.h" #include "trace2/tr2_tls.h" - static struct tr2_dst tr2dst_event = { "GIT_TR2_EVENT", 0, 0, 0 }; + static struct tr2_dst tr2dst_event = { TR2_SYSENV_EVENT, 0, 0, 0 }; /* * The version number of the JSON data generated by the EVENT target @@@ -28,37 -29,36 +29,36 @@@ * are primarily intended for the performance target during debugging. * * Some of the outer-most messages, however, may be of interest to the - * event target. Set this environment variable to a larger integer for - * more detail in the event target. + * event target. Use the TR2_SYSENV_EVENT_NESTING setting to increase + * region details in the event target. */ - #define TR2_ENVVAR_EVENT_NESTING "GIT_TR2_EVENT_NESTING" - static int tr2env_event_nesting_wanted = 2; + static int tr2env_event_max_nesting_levels = 2; /* - * Set this environment variable to true to omit the