t / perf / READMEon commit sha1_file: restore OBJECT_INFO_QUICK functionality (2b7750c)
   1Git performance tests
   2=====================
   3
   4This directory holds performance testing scripts for git tools.  The
   5first part of this document describes the various ways in which you
   6can run them.
   7
   8When fixing the tools or adding enhancements, you are strongly
   9encouraged to add tests in this directory to cover what you are
  10trying to fix or enhance.  The later part of this short document
  11describes how your test scripts should be organized.
  12
  13
  14Running Tests
  15-------------
  16
  17The easiest way to run tests is to say "make".  This runs all
  18the tests on the current git repository.
  19
  20    === Running 2 tests in this tree ===
  21    [...]
  22    Test                                     this tree
  23    ---------------------------------------------------------
  24    0001.1: rev-list --all                   0.54(0.51+0.02)
  25    0001.2: rev-list --all --objects         6.14(5.99+0.11)
  26    7810.1: grep worktree, cheap regex       0.16(0.16+0.35)
  27    7810.2: grep worktree, expensive regex   7.90(29.75+0.37)
  28    7810.3: grep --cached, cheap regex       3.07(3.02+0.25)
  29    7810.4: grep --cached, expensive regex   9.39(30.57+0.24)
  30
  31You can compare multiple repositories and even git revisions with the
  32'run' script:
  33
  34    $ ./run . origin/next /path/to/git-tree p0001-rev-list.sh
  35
  36where . stands for the current git tree.  The full invocation is
  37
  38    ./run [<revision|directory>...] [--] [<test-script>...]
  39
  40A '.' argument is implied if you do not pass any other
  41revisions/directories.
  42
  43You can also manually test this or another git build tree, and then
  44call the aggregation script to summarize the results:
  45
  46    $ ./p0001-rev-list.sh
  47    [...]
  48    $ GIT_BUILD_DIR=/path/to/other/git ./p0001-rev-list.sh
  49    [...]
  50    $ ./aggregate.perl . /path/to/other/git ./p0001-rev-list.sh
  51
  52aggregate.perl has the same invocation as 'run', it just does not run
  53anything beforehand.
  54
  55You can set the following variables (also in your config.mak):
  56
  57    GIT_PERF_REPEAT_COUNT
  58        Number of times a test should be repeated for best-of-N
  59        measurements.  Defaults to 3.
  60
  61    GIT_PERF_MAKE_OPTS
  62        Options to use when automatically building a git tree for
  63        performance testing. E.g., -j6 would be useful. Passed
  64        directly to make as "make $GIT_PERF_MAKE_OPTS".
  65
  66    GIT_PERF_MAKE_COMMAND
  67        An arbitrary command that'll be run in place of the make
  68        command, if set the GIT_PERF_MAKE_OPTS variable is
  69        ignored. Useful in cases where source tree changes might
  70        require issuing a different make command to different
  71        revisions.
  72
  73        This can be (ab)used to monkeypatch or otherwise change the
  74        tree about to be built. Note that the build directory can be
  75        re-used for subsequent runs so the make command might get
  76        executed multiple times on the same tree, but don't count on
  77        any of that, that's an implementation detail that might change
  78        in the future.
  79
  80    GIT_PERF_REPO
  81    GIT_PERF_LARGE_REPO
  82        Repositories to copy for the performance tests.  The normal
  83        repo should be at least git.git size.  The large repo should
  84        probably be about linux.git size for optimal results.
  85        Both default to the git.git you are running from.
  86
  87You can also pass the options taken by ordinary git tests; the most
  88useful one is:
  89
  90--root=<directory>::
  91        Create "trash" directories used to store all temporary data during
  92        testing under <directory>, instead of the t/ directory.
  93        Using this option with a RAM-based filesystem (such as tmpfs)
  94        can massively speed up the test suite.
  95
  96
  97Naming Tests
  98------------
  99
 100The performance test files are named as:
 101
 102        pNNNN-commandname-details.sh
 103
 104where N is a decimal digit.  The same conventions for choosing NNNN as
 105for normal tests apply.
 106
 107
 108Writing Tests
 109-------------
 110
 111The perf script starts much like a normal test script, except it
 112sources perf-lib.sh:
 113
 114        #!/bin/sh
 115        #
 116        # Copyright (c) 2005 Junio C Hamano
 117        #
 118
 119        test_description='xxx performance test'
 120        . ./perf-lib.sh
 121
 122After that you will want to use some of the following:
 123
 124        test_perf_fresh_repo    # sets up an empty repository
 125        test_perf_default_repo  # sets up a "normal" repository
 126        test_perf_large_repo    # sets up a "large" repository
 127
 128        test_perf_default_repo sub  # ditto, in a subdir "sub"
 129
 130        test_checkout_worktree  # if you need the worktree too
 131
 132At least one of the first two is required!
 133
 134You can use test_expect_success as usual. In both test_expect_success
 135and in test_perf, running "git" points to the version that is being
 136perf-tested. The $MODERN_GIT variable points to the git wrapper for the
 137currently checked-out version (i.e., the one that matches the t/perf
 138scripts you are running).  This is useful if your setup uses commands
 139that only work with newer versions of git than what you might want to
 140test (but obviously your new commands must still create a state that can
 141be used by the older version of git you are testing).
 142
 143For actual performance tests, use
 144
 145        test_perf 'descriptive string' '
 146                command1 &&
 147                command2
 148        '
 149
 150test_perf spawns a subshell, for lack of better options.  This means
 151that
 152
 153* you _must_ export all variables that you need in the subshell
 154
 155* you _must_ flag all variables that you want to persist from the
 156  subshell with 'test_export':
 157
 158        test_perf 'descriptive string' '
 159                foo=$(git rev-parse HEAD) &&
 160                test_export foo
 161        '
 162
 163  The so-exported variables are automatically marked for export in the
 164  shell executing the perf test.  For your convenience, test_export is
 165  the same as export in the main shell.
 166
 167  This feature relies on a bit of magic using 'set' and 'source'.
 168  While we have tried to make sure that it can cope with embedded
 169  whitespace and other special characters, it will not work with
 170  multi-line data.