1git-filter-branch(1) 2==================== 3 4NAME 5---- 6git-filter-branch - Rewrite branches 7 8SYNOPSIS 9-------- 10[verse] 11'git filter-branch' [--env-filter <command>] [--tree-filter <command>] 12 [--index-filter <command>] [--parent-filter <command>] 13 [--msg-filter <command>] [--commit-filter <command>] 14 [--tag-name-filter <command>] [--subdirectory-filter <directory>] 15 [--prune-empty] 16 [--original <namespace>] [-d <directory>] [-f | --force] 17 [--] [<rev-list options>...] 18 19DESCRIPTION 20----------- 21Lets you rewrite git revision history by rewriting the branches mentioned 22in the <rev-list options>, applying custom filters on each revision. 23Those filters can modify each tree (e.g. removing a file or running 24a perl rewrite on all files) or information about each commit. 25Otherwise, all information (including original commit times or merge 26information) will be preserved. 27 28The command will only rewrite the _positive_ refs mentioned in the 29command line (e.g. if you pass 'a..b', only 'b' will be rewritten). 30If you specify no filters, the commits will be recommitted without any 31changes, which would normally have no effect. Nevertheless, this may be 32useful in the future for compensating for some git bugs or such, 33therefore such a usage is permitted. 34 35*NOTE*: This command honors `.git/info/grafts` and `.git/refs/replace/`. 36If you have any grafts or replacement refs defined, running this command 37will make them permanent. 38 39*WARNING*! The rewritten history will have different object names for all 40the objects and will not converge with the original branch. You will not 41be able to easily push and distribute the rewritten branch on top of the 42original branch. Please do not use this command if you do not know the 43full implications, and avoid using it anyway, if a simple single commit 44would suffice to fix your problem. (See the "RECOVERING FROM UPSTREAM 45REBASE" section in linkgit:git-rebase[1] for further information about 46rewriting published history.) 47 48Always verify that the rewritten version is correct: The original refs, 49if different from the rewritten ones, will be stored in the namespace 50'refs/original/'. 51 52Note that since this operation is very I/O expensive, it might 53be a good idea to redirect the temporary directory off-disk with the 54'-d' option, e.g. on tmpfs. Reportedly the speedup is very noticeable. 55 56 57Filters 58~~~~~~~ 59 60The filters are applied in the order as listed below. The <command> 61argument is always evaluated in the shell context using the 'eval' command 62(with the notable exception of the commit filter, for technical reasons). 63Prior to that, the $GIT_COMMIT environment variable will be set to contain 64the id of the commit being rewritten. Also, GIT_AUTHOR_NAME, 65GIT_AUTHOR_EMAIL, GIT_AUTHOR_DATE, GIT_COMMITTER_NAME, GIT_COMMITTER_EMAIL, 66and GIT_COMMITTER_DATE are set according to the current commit. The values 67of these variables after the filters have run, are used for the new commit. 68If any evaluation of <command> returns a non-zero exit status, the whole 69operation will be aborted. 70 71A 'map' function is available that takes an "original sha1 id" argument 72and outputs a "rewritten sha1 id" if the commit has been already 73rewritten, and "original sha1 id" otherwise; the 'map' function can 74return several ids on separate lines if your commit filter emitted 75multiple commits. 76 77 78OPTIONS 79------- 80 81--env-filter <command>:: 82 This filter may be used if you only need to modify the environment 83 in which the commit will be performed. Specifically, you might 84 want to rewrite the author/committer name/email/time environment 85 variables (see linkgit:git-commit-tree[1] for details). Do not forget 86 to re-export the variables. 87 88--tree-filter <command>:: 89 This is the filter for rewriting the tree and its contents. 90 The argument is evaluated in shell with the working 91 directory set to the root of the checked out tree. The new tree 92 is then used as-is (new files are auto-added, disappeared files 93 are auto-removed - neither .gitignore files nor any other ignore 94 rules *HAVE ANY EFFECT*!). 95 96--index-filter <command>:: 97 This is the filter for rewriting the index. It is similar to the 98 tree filter but does not check out the tree, which makes it much 99 faster. Frequently used with `git rm --cached 100 --ignore-unmatch ...`, see EXAMPLES below. For hairy 101 cases, see linkgit:git-update-index[1]. 102 103--parent-filter <command>:: 104 This is the filter for rewriting the commit's parent list. 105 It will receive the parent string on stdin and shall output 106 the new parent string on stdout. The parent string is in 107 the format described in linkgit:git-commit-tree[1]: empty for 108 the initial commit, "-p parent" for a normal commit and 109 "-p parent1 -p parent2 -p parent3 ..." for a merge commit. 110 111--msg-filter <command>:: 112 This is the filter for rewriting the commit messages. 113 The argument is evaluated in the shell with the original 114 commit message on standard input; its standard output is 115 used as the new commit message. 116 117--commit-filter <command>:: 118 This is the filter for performing the commit. 119 If this filter is specified, it will be called instead of the 120 'git commit-tree' command, with arguments of the form 121 "<TREE_ID> [(-p <PARENT_COMMIT_ID>)...]" and the log message on 122 stdin. The commit id is expected on stdout. 123+ 124As a special extension, the commit filter may emit multiple 125commit ids; in that case, the rewritten children of the original commit will 126have all of them as parents. 127+ 128You can use the 'map' convenience function in this filter, and other 129convenience functions, too. For example, calling 'skip_commit "$@"' 130will leave out the current commit (but not its changes! If you want 131that, use 'git rebase' instead). 132+ 133You can also use the `git_commit_non_empty_tree "$@"` instead of 134`git commit-tree "$@"` if you don't wish to keep commits with a single parent 135and that makes no change to the tree. 136 137--tag-name-filter <command>:: 138 This is the filter for rewriting tag names. When passed, 139 it will be called for every tag ref that points to a rewritten 140 object (or to a tag object which points to a rewritten object). 141 The original tag name is passed via standard input, and the new 142 tag name is expected on standard output. 143+ 144The original tags are not deleted, but can be overwritten; 145use "--tag-name-filter cat" to simply update the tags. In this 146case, be very careful and make sure you have the old tags 147backed up in case the conversion has run afoul. 148+ 149Nearly proper rewriting of tag objects is supported. If the tag has 150a message attached, a new tag object will be created with the same message, 151author, and timestamp. If the tag has a signature attached, the 152signature will be stripped. It is by definition impossible to preserve 153signatures. The reason this is "nearly" proper, is because ideally if 154the tag did not change (points to the same object, has the same name, etc.) 155it should retain any signature. That is not the case, signatures will always 156be removed, buyer beware. There is also no support for changing the 157author or timestamp (or the tag message for that matter). Tags which point 158to other tags will be rewritten to point to the underlying commit. 159 160--subdirectory-filter <directory>:: 161 Only look at the history which touches the given subdirectory. 162 The result will contain that directory (and only that) as its 163 project root. Implies <<Remap_to_ancestor>>. 164 165--prune-empty:: 166 Some kind of filters will generate empty commits, that left the tree 167 untouched. This switch allow git-filter-branch to ignore such 168 commits. Though, this switch only applies for commits that have one 169 and only one parent, it will hence keep merges points. Also, this 170 option is not compatible with the use of '--commit-filter'. Though you 171 just need to use the function 'git_commit_non_empty_tree "$@"' instead 172 of the `git commit-tree "$@"` idiom in your commit filter to make that 173 happen. 174 175--original <namespace>:: 176 Use this option to set the namespace where the original commits 177 will be stored. The default value is 'refs/original'. 178 179-d <directory>:: 180 Use this option to set the path to the temporary directory used for 181 rewriting. When applying a tree filter, the command needs to 182 temporarily check out the tree to some directory, which may consume 183 considerable space in case of large projects. By default it 184 does this in the '.git-rewrite/' directory but you can override 185 that choice by this parameter. 186 187-f:: 188--force:: 189 'git filter-branch' refuses to start with an existing temporary 190 directory or when there are already refs starting with 191 'refs/original/', unless forced. 192 193<rev-list options>...:: 194 Arguments for 'git rev-list'. All positive refs included by 195 these options are rewritten. You may also specify options 196 such as '--all', but you must use '--' to separate them from 197 the 'git filter-branch' options. Implies <<Remap_to_ancestor>>. 198 199 200[[Remap_to_ancestor]] 201Remap to ancestor 202~~~~~~~~~~~~~~~~~ 203 204By using linkgit:rev-list[1] arguments, e.g., path limiters, you can limit the 205set of revisions which get rewritten. However, positive refs on the command 206line are distinguished: we don't let them be excluded by such limiters. For 207this purpose, they are instead rewritten to point at the nearest ancestor that 208was not excluded. 209 210 211Examples 212-------- 213 214Suppose you want to remove a file (containing confidential information 215or copyright violation) from all commits: 216 217------------------------------------------------------- 218git filter-branch --tree-filter 'rm filename' HEAD 219------------------------------------------------------- 220 221However, if the file is absent from the tree of some commit, 222a simple `rm filename` will fail for that tree and commit. 223Thus you may instead want to use `rm -f filename` as the script. 224 225Using `--index-filter` with 'git rm' yields a significantly faster 226version. Like with using `rm filename`, `git rm --cached filename` 227will fail if the file is absent from the tree of a commit. If you 228want to "completely forget" a file, it does not matter when it entered 229history, so we also add `--ignore-unmatch`: 230 231-------------------------------------------------------------------------- 232git filter-branch --index-filter 'git rm --cached --ignore-unmatch filename' HEAD 233-------------------------------------------------------------------------- 234 235Now, you will get the rewritten history saved in HEAD. 236 237To rewrite the repository to look as if `foodir/` had been its project 238root, and discard all other history: 239 240------------------------------------------------------- 241git filter-branch --subdirectory-filter foodir -- --all 242------------------------------------------------------- 243 244Thus you can, e.g., turn a library subdirectory into a repository of 245its own. Note the `--` that separates 'filter-branch' options from 246revision options, and the `--all` to rewrite all branches and tags. 247 248To set a commit (which typically is at the tip of another 249history) to be the parent of the current initial commit, in 250order to paste the other history behind the current history: 251 252------------------------------------------------------------------- 253git filter-branch --parent-filter 'sed "s/^\$/-p <graft-id>/"' HEAD 254------------------------------------------------------------------- 255 256(if the parent string is empty - which happens when we are dealing with 257the initial commit - add graftcommit as a parent). Note that this assumes 258history with a single root (that is, no merge without common ancestors 259happened). If this is not the case, use: 260 261-------------------------------------------------------------------------- 262git filter-branch --parent-filter \ 263 'test $GIT_COMMIT = <commit-id> && echo "-p <graft-id>" || cat' HEAD 264-------------------------------------------------------------------------- 265 266or even simpler: 267 268----------------------------------------------- 269echo "$commit-id $graft-id" >> .git/info/grafts 270git filter-branch $graft-id..HEAD 271----------------------------------------------- 272 273To remove commits authored by "Darl McBribe" from the history: 274 275------------------------------------------------------------------------------ 276git filter-branch --commit-filter ' 277 if [ "$GIT_AUTHOR_NAME" = "Darl McBribe" ]; 278 then 279 skip_commit "$@"; 280 else 281 git commit-tree "$@"; 282 fi' HEAD 283------------------------------------------------------------------------------ 284 285The function 'skip_commit' is defined as follows: 286 287-------------------------- 288skip_commit() 289{ 290 shift; 291 while [ -n "$1" ]; 292 do 293 shift; 294 map "$1"; 295 shift; 296 done; 297} 298-------------------------- 299 300The shift magic first throws away the tree id and then the -p 301parameters. Note that this handles merges properly! In case Darl 302committed a merge between P1 and P2, it will be propagated properly 303and all children of the merge will become merge commits with P1,P2 304as their parents instead of the merge commit. 305 306You can rewrite the commit log messages using `--msg-filter`. For 307example, 'git svn-id' strings in a repository created by 'git svn' can 308be removed this way: 309 310------------------------------------------------------- 311git filter-branch --msg-filter ' 312 sed -e "/^git-svn-id:/d" 313' 314------------------------------------------------------- 315 316To restrict rewriting to only part of the history, specify a revision 317range in addition to the new branch name. The new branch name will 318point to the top-most revision that a 'git rev-list' of this range 319will print. 320 321If you need to add 'Acked-by' lines to, say, the last 10 commits (none 322of which is a merge), use this command: 323 324-------------------------------------------------------- 325git filter-branch --msg-filter ' 326 cat && 327 echo "Acked-by: Bugs Bunny <bunny@bugzilla.org>" 328' HEAD~10..HEAD 329-------------------------------------------------------- 330 331*NOTE* the changes introduced by the commits, and which are not reverted 332by subsequent commits, will still be in the rewritten branch. If you want 333to throw out _changes_ together with the commits, you should use the 334interactive mode of 'git rebase'. 335 336 337Consider this history: 338 339------------------ 340 D--E--F--G--H 341 / / 342A--B-----C 343------------------ 344 345To rewrite only commits D,E,F,G,H, but leave A, B and C alone, use: 346 347-------------------------------- 348git filter-branch ... C..H 349-------------------------------- 350 351To rewrite commits E,F,G,H, use one of these: 352 353---------------------------------------- 354git filter-branch ... C..H --not D 355git filter-branch ... D..H --not C 356---------------------------------------- 357 358To move the whole tree into a subdirectory, or remove it from there: 359 360--------------------------------------------------------------- 361git filter-branch --index-filter \ 362 'git ls-files -s | sed "s-\t\"*-&newsubdir/-" | 363 GIT_INDEX_FILE=$GIT_INDEX_FILE.new \ 364 git update-index --index-info && 365 mv "$GIT_INDEX_FILE.new" "$GIT_INDEX_FILE"' HEAD 366--------------------------------------------------------------- 367 368 369 370Checklist for Shrinking a Repository 371------------------------------------ 372 373git-filter-branch is often used to get rid of a subset of files, 374usually with some combination of `--index-filter` and 375`--subdirectory-filter`. People expect the resulting repository to 376be smaller than the original, but you need a few more steps to 377actually make it smaller, because git tries hard not to lose your 378objects until you tell it to. First make sure that: 379 380* You really removed all variants of a filename, if a blob was moved 381 over its lifetime. `git log --name-only --follow --all -- filename` 382 can help you find renames. 383 384* You really filtered all refs: use `--tag-name-filter cat -- --all` 385 when calling git-filter-branch. 386 387Then there are two ways to get a smaller repository. A safer way is 388to clone, that keeps your original intact. 389 390* Clone it with `git clone file:///path/to/repo`. The clone 391 will not have the removed objects. See linkgit:git-clone[1]. (Note 392 that cloning with a plain path just hardlinks everything!) 393 394If you really don't want to clone it, for whatever reasons, check the 395following points instead (in this order). This is a very destructive 396approach, so *make a backup* or go back to cloning it. You have been 397warned. 398 399* Remove the original refs backed up by git-filter-branch: say `git 400 for-each-ref --format="%(refname)" refs/original/ | xargs -n 1 git 401 update-ref -d`. 402 403* Expire all reflogs with `git reflog expire --expire=now --all`. 404 405* Garbage collect all unreferenced objects with `git gc --prune=now` 406 (or if your git-gc is not new enough to support arguments to 407 `--prune`, use `git repack -ad; git prune` instead). 408 409GIT 410--- 411Part of the linkgit:git[1] suite