1Git performance tests 2===================== 3 4This directory holds performance testing scripts for git tools. The 5first part of this document describes the various ways in which you 6can run them. 7 8When fixing the tools or adding enhancements, you are strongly 9encouraged to add tests in this directory to cover what you are 10trying to fix or enhance. The later part of this short document 11describes how your test scripts should be organized. 12 13 14Running Tests 15------------- 16 17The easiest way to run tests is to say "make". This runs all 18the tests on the current git repository. 19 20 === Running 2 tests in this tree === 21 [...] 22 Test this tree 23 --------------------------------------------------------- 24 0001.1: rev-list --all 0.54(0.51+0.02) 25 0001.2: rev-list --all --objects 6.14(5.99+0.11) 26 7810.1: grep worktree, cheap regex 0.16(0.16+0.35) 27 7810.2: grep worktree, expensive regex 7.90(29.75+0.37) 28 7810.3: grep --cached, cheap regex 3.07(3.02+0.25) 29 7810.4: grep --cached, expensive regex 9.39(30.57+0.24) 30 31You can compare multiple repositories and even git revisions with the 32'run' script: 33 34 $ ./run . origin/next /path/to/git-tree p0001-rev-list.sh 35 36where . stands for the current git tree. The full invocation is 37 38 ./run [<revision|directory>...] [--] [<test-script>...] 39 40A '.' argument is implied if you do not pass any other 41revisions/directories. 42 43You can also manually test this or another git build tree, and then 44call the aggregation script to summarize the results: 45 46 $ ./p0001-rev-list.sh 47 [...] 48 $ GIT_BUILD_DIR=/path/to/other/git ./p0001-rev-list.sh 49 [...] 50 $ ./aggregate.perl . /path/to/other/git ./p0001-rev-list.sh 51 52aggregate.perl has the same invocation as 'run', it just does not run 53anything beforehand. 54 55You can set the following variables (also in your config.mak): 56 57 GIT_PERF_REPEAT_COUNT 58 Number of times a test should be repeated for best-of-N 59 measurements. Defaults to 3. 60 61 GIT_PERF_MAKE_OPTS 62 Options to use when automatically building a git tree for 63 performance testing. E.g., -j6 would be useful. 64 65 GIT_PERF_REPO 66 GIT_PERF_LARGE_REPO 67 Repositories to copy for the performance tests. The normal 68 repo should be at least git.git size. The large repo should 69 probably be about linux.git size for optimal results. 70 Both default to the git.git you are running from. 71 72You can also pass the options taken by ordinary git tests; the most 73useful one is: 74 75--root=<directory>:: 76 Create "trash" directories used to store all temporary data during 77 testing under <directory>, instead of the t/ directory. 78 Using this option with a RAM-based filesystem (such as tmpfs) 79 can massively speed up the test suite. 80 81 82Naming Tests 83------------ 84 85The performance test files are named as: 86 87 pNNNN-commandname-details.sh 88 89where N is a decimal digit. The same conventions for choosing NNNN as 90for normal tests apply. 91 92 93Writing Tests 94------------- 95 96The perf script starts much like a normal test script, except it 97sources perf-lib.sh: 98 99 #!/bin/sh 100 # 101 # Copyright (c) 2005 Junio C Hamano 102 # 103 104 test_description='xxx performance test' 105 . ./perf-lib.sh 106 107After that you will want to use some of the following: 108 109 test_perf_fresh_repo # sets up an empty repository 110 test_perf_default_repo # sets up a "normal" repository 111 test_perf_large_repo # sets up a "large" repository 112 113 test_perf_default_repo sub # ditto, in a subdir "sub" 114 115 test_checkout_worktree # if you need the worktree too 116 117At least one of the first two is required! 118 119You can use test_expect_success as usual. In both test_expect_success 120and in test_perf, running "git" points to the version that is being 121perf-tested. The $MODERN_GIT variable points to the git wrapper for the 122currently checked-out version (i.e., the one that matches the t/perf 123scripts you are running). This is useful if your setup uses commands 124that only work with newer versions of git than what you might want to 125test (but obviously your new commands must still create a state that can 126be used by the older version of git you are testing). 127 128For actual performance tests, use 129 130 test_perf 'descriptive string' ' 131 command1 && 132 command2 133 ' 134 135test_perf spawns a subshell, for lack of better options. This means 136that 137 138* you _must_ export all variables that you need in the subshell 139 140* you _must_ flag all variables that you want to persist from the 141 subshell with 'test_export': 142 143 test_perf 'descriptive string' ' 144 foo=$(git rev-parse HEAD) && 145 test_export foo 146 ' 147 148 The so-exported variables are automatically marked for export in the 149 shell executing the perf test. For your convenience, test_export is 150 the same as export in the main shell. 151 152 This feature relies on a bit of magic using 'set' and 'source'. 153 While we have tried to make sure that it can cope with embedded 154 whitespace and other special characters, it will not work with 155 multi-line data.