Documentation / git-pack-objects.txton commit wt-status.h: drop stdio.h include (7a06fb0)
   1git-pack-objects(1)
   2===================
   3
   4NAME
   5----
   6git-pack-objects - Create a packed archive of objects
   7
   8
   9SYNOPSIS
  10--------
  11[verse]
  12'git pack-objects' [-q | --progress | --all-progress] [--all-progress-implied]
  13        [--no-reuse-delta] [--delta-base-offset] [--non-empty]
  14        [--local] [--incremental] [--window=<n>] [--depth=<n>]
  15        [--revs [--unpacked | --all]] [--keep-pack=<pack-name>]
  16        [--stdout [--filter=<filter-spec>] | base-name]
  17        [--shallow] [--keep-true-parents] [--sparse] < object-list
  18
  19
  20DESCRIPTION
  21-----------
  22Reads list of objects from the standard input, and writes either one or
  23more packed archives with the specified base-name to disk, or a packed
  24archive to the standard output.
  25
  26A packed archive is an efficient way to transfer a set of objects
  27between two repositories as well as an access efficient archival
  28format.  In a packed archive, an object is either stored as a
  29compressed whole or as a difference from some other object.
  30The latter is often called a delta.
  31
  32The packed archive format (.pack) is designed to be self-contained
  33so that it can be unpacked without any further information. Therefore,
  34each object that a delta depends upon must be present within the pack.
  35
  36A pack index file (.idx) is generated for fast, random access to the
  37objects in the pack. Placing both the index file (.idx) and the packed
  38archive (.pack) in the pack/ subdirectory of $GIT_OBJECT_DIRECTORY (or
  39any of the directories on $GIT_ALTERNATE_OBJECT_DIRECTORIES)
  40enables Git to read from the pack archive.
  41
  42The 'git unpack-objects' command can read the packed archive and
  43expand the objects contained in the pack into "one-file
  44one-object" format; this is typically done by the smart-pull
  45commands when a pack is created on-the-fly for efficient network
  46transport by their peers.
  47
  48
  49OPTIONS
  50-------
  51base-name::
  52        Write into pairs of files (.pack and .idx), using
  53        <base-name> to determine the name of the created file.
  54        When this option is used, the two files in a pair are written in
  55        <base-name>-<SHA-1>.{pack,idx} files.  <SHA-1> is a hash
  56        based on the pack content and is written to the standard
  57        output of the command.
  58
  59--stdout::
  60        Write the pack contents (what would have been written to
  61        .pack file) out to the standard output.
  62
  63--revs::
  64        Read the revision arguments from the standard input, instead of
  65        individual object names.  The revision arguments are processed
  66        the same way as 'git rev-list' with the `--objects` flag
  67        uses its `commit` arguments to build the list of objects it
  68        outputs.  The objects on the resulting list are packed.
  69        Besides revisions, `--not` or `--shallow <SHA-1>` lines are
  70        also accepted.
  71
  72--unpacked::
  73        This implies `--revs`.  When processing the list of
  74        revision arguments read from the standard input, limit
  75        the objects packed to those that are not already packed.
  76
  77--all::
  78        This implies `--revs`.  In addition to the list of
  79        revision arguments read from the standard input, pretend
  80        as if all refs under `refs/` are specified to be
  81        included.
  82
  83--include-tag::
  84        Include unasked-for annotated tags if the object they
  85        reference was included in the resulting packfile.  This
  86        can be useful to send new tags to native Git clients.
  87
  88--window=<n>::
  89--depth=<n>::
  90        These two options affect how the objects contained in
  91        the pack are stored using delta compression.  The
  92        objects are first internally sorted by type, size and
  93        optionally names and compared against the other objects
  94        within --window to see if using delta compression saves
  95        space.  --depth limits the maximum delta depth; making
  96        it too deep affects the performance on the unpacker
  97        side, because delta data needs to be applied that many
  98        times to get to the necessary object.
  99+
 100The default value for --window is 10 and --depth is 50. The maximum
 101depth is 4095.
 102
 103--window-memory=<n>::
 104        This option provides an additional limit on top of `--window`;
 105        the window size will dynamically scale down so as to not take
 106        up more than '<n>' bytes in memory.  This is useful in
 107        repositories with a mix of large and small objects to not run
 108        out of memory with a large window, but still be able to take
 109        advantage of the large window for the smaller objects.  The
 110        size can be suffixed with "k", "m", or "g".
 111        `--window-memory=0` makes memory usage unlimited.  The default
 112        is taken from the `pack.windowMemory` configuration variable.
 113
 114--max-pack-size=<n>::
 115        In unusual scenarios, you may not be able to create files
 116        larger than a certain size on your filesystem, and this option
 117        can be used to tell the command to split the output packfile
 118        into multiple independent packfiles, each not larger than the
 119        given size. The size can be suffixed with
 120        "k", "m", or "g". The minimum size allowed is limited to 1 MiB.
 121        This option
 122        prevents the creation of a bitmap index.
 123        The default is unlimited, unless the config variable
 124        `pack.packSizeLimit` is set.
 125
 126--honor-pack-keep::
 127        This flag causes an object already in a local pack that
 128        has a .keep file to be ignored, even if it would have
 129        otherwise been packed.
 130
 131--keep-pack=<pack-name>::
 132        This flag causes an object already in the given pack to be
 133        ignored, even if it would have otherwise been
 134        packed. `<pack-name>` is the the pack file name without
 135        leading directory (e.g. `pack-123.pack`). The option could be
 136        specified multiple times to keep multiple packs.
 137
 138--incremental::
 139        This flag causes an object already in a pack to be ignored
 140        even if it would have otherwise been packed.
 141
 142--local::
 143        This flag causes an object that is borrowed from an alternate
 144        object store to be ignored even if it would have otherwise been
 145        packed.
 146
 147--non-empty::
 148        Only create a packed archive if it would contain at
 149        least one object.
 150
 151--progress::
 152        Progress status is reported on the standard error stream
 153        by default when it is attached to a terminal, unless -q
 154        is specified. This flag forces progress status even if
 155        the standard error stream is not directed to a terminal.
 156
 157--all-progress::
 158        When --stdout is specified then progress report is
 159        displayed during the object count and compression phases
 160        but inhibited during the write-out phase. The reason is
 161        that in some cases the output stream is directly linked
 162        to another command which may wish to display progress
 163        status of its own as it processes incoming pack data.
 164        This flag is like --progress except that it forces progress
 165        report for the write-out phase as well even if --stdout is
 166        used.
 167
 168--all-progress-implied::
 169        This is used to imply --all-progress whenever progress display
 170        is activated.  Unlike --all-progress this flag doesn't actually
 171        force any progress display by itself.
 172
 173-q::
 174        This flag makes the command not to report its progress
 175        on the standard error stream.
 176
 177--no-reuse-delta::
 178        When creating a packed archive in a repository that
 179        has existing packs, the command reuses existing deltas.
 180        This sometimes results in a slightly suboptimal pack.
 181        This flag tells the command not to reuse existing deltas
 182        but compute them from scratch.
 183
 184--no-reuse-object::
 185        This flag tells the command not to reuse existing object data at all,
 186        including non deltified object, forcing recompression of everything.
 187        This implies --no-reuse-delta. Useful only in the obscure case where
 188        wholesale enforcement of a different compression level on the
 189        packed data is desired.
 190
 191--compression=<n>::
 192        Specifies compression level for newly-compressed data in the
 193        generated pack.  If not specified,  pack compression level is
 194        determined first by pack.compression,  then by core.compression,
 195        and defaults to -1,  the zlib default,  if neither is set.
 196        Add --no-reuse-object if you want to force a uniform compression
 197        level on all data no matter the source.
 198
 199--sparse::
 200        Use the "sparse" algorithm to determine which objects to include in
 201        the pack, when combined with the "--revs" option. This algorithm
 202        only walks trees that appear in paths that introduce new objects.
 203        This can have significant performance benefits when computing
 204        a pack to send a small change. However, it is possible that extra
 205        objects are added to the pack-file if the included commits contain
 206        certain types of direct renames.
 207
 208--thin::
 209        Create a "thin" pack by omitting the common objects between a
 210        sender and a receiver in order to reduce network transfer. This
 211        option only makes sense in conjunction with --stdout.
 212+
 213Note: A thin pack violates the packed archive format by omitting
 214required objects and is thus unusable by Git without making it
 215self-contained. Use `git index-pack --fix-thin`
 216(see linkgit:git-index-pack[1]) to restore the self-contained property.
 217
 218--shallow::
 219        Optimize a pack that will be provided to a client with a shallow
 220        repository.  This option, combined with --thin, can result in a
 221        smaller pack at the cost of speed.
 222
 223--delta-base-offset::
 224        A packed archive can express the base object of a delta as
 225        either a 20-byte object name or as an offset in the
 226        stream, but ancient versions of Git don't understand the
 227        latter.  By default, 'git pack-objects' only uses the
 228        former format for better compatibility.  This option
 229        allows the command to use the latter format for
 230        compactness.  Depending on the average delta chain
 231        length, this option typically shrinks the resulting
 232        packfile by 3-5 per-cent.
 233+
 234Note: Porcelain commands such as `git gc` (see linkgit:git-gc[1]),
 235`git repack` (see linkgit:git-repack[1]) pass this option by default
 236in modern Git when they put objects in your repository into pack files.
 237So does `git bundle` (see linkgit:git-bundle[1]) when it creates a bundle.
 238
 239--threads=<n>::
 240        Specifies the number of threads to spawn when searching for best
 241        delta matches.  This requires that pack-objects be compiled with
 242        pthreads otherwise this option is ignored with a warning.
 243        This is meant to reduce packing time on multiprocessor machines.
 244        The required amount of memory for the delta search window is
 245        however multiplied by the number of threads.
 246        Specifying 0 will cause Git to auto-detect the number of CPU's
 247        and set the number of threads accordingly.
 248
 249--index-version=<version>[,<offset>]::
 250        This is intended to be used by the test suite only. It allows
 251        to force the version for the generated pack index, and to force
 252        64-bit index entries on objects located above the given offset.
 253
 254--keep-true-parents::
 255        With this option, parents that are hidden by grafts are packed
 256        nevertheless.
 257
 258--filter=<filter-spec>::
 259        Requires `--stdout`.  Omits certain objects (usually blobs) from
 260        the resulting packfile.  See linkgit:git-rev-list[1] for valid
 261        `<filter-spec>` forms.
 262
 263--no-filter::
 264        Turns off any previous `--filter=` argument.
 265
 266--missing=<missing-action>::
 267        A debug option to help with future "partial clone" development.
 268        This option specifies how missing objects are handled.
 269+
 270The form '--missing=error' requests that pack-objects stop with an error if
 271a missing object is encountered.  This is the default action.
 272+
 273The form '--missing=allow-any' will allow object traversal to continue
 274if a missing object is encountered.  Missing objects will silently be
 275omitted from the results.
 276+
 277The form '--missing=allow-promisor' is like 'allow-any', but will only
 278allow object traversal to continue for EXPECTED promisor missing objects.
 279Unexpected missing object will raise an error.
 280
 281--exclude-promisor-objects::
 282        Omit objects that are known to be in the promisor remote.  (This
 283        option has the purpose of operating only on locally created objects,
 284        so that when we repack, we still maintain a distinction between
 285        locally created objects [without .promisor] and objects from the
 286        promisor remote [with .promisor].)  This is used with partial clone.
 287
 288--keep-unreachable::
 289        Objects unreachable from the refs in packs named with
 290        --unpacked= option are added to the resulting pack, in
 291        addition to the reachable objects that are not in packs marked
 292        with *.keep files. This implies `--revs`.
 293
 294--pack-loose-unreachable::
 295        Pack unreachable loose objects (and their loose counterparts
 296        removed). This implies `--revs`.
 297
 298--unpack-unreachable::
 299        Keep unreachable objects in loose form. This implies `--revs`.
 300
 301--delta-islands::
 302        Restrict delta matches based on "islands". See DELTA ISLANDS
 303        below.
 304
 305
 306DELTA ISLANDS
 307-------------
 308
 309When possible, `pack-objects` tries to reuse existing on-disk deltas to
 310avoid having to search for new ones on the fly. This is an important
 311optimization for serving fetches, because it means the server can avoid
 312inflating most objects at all and just send the bytes directly from
 313disk. This optimization can't work when an object is stored as a delta
 314against a base which the receiver does not have (and which we are not
 315already sending). In that case the server "breaks" the delta and has to
 316find a new one, which has a high CPU cost. Therefore it's important for
 317performance that the set of objects in on-disk delta relationships match
 318what a client would fetch.
 319
 320In a normal repository, this tends to work automatically. The objects
 321are mostly reachable from the branches and tags, and that's what clients
 322fetch. Any deltas we find on the server are likely to be between objects
 323the client has or will have.
 324
 325But in some repository setups, you may have several related but separate
 326groups of ref tips, with clients tending to fetch those groups
 327independently. For example, imagine that you are hosting several "forks"
 328of a repository in a single shared object store, and letting clients
 329view them as separate repositories through `GIT_NAMESPACE` or separate
 330repos using the alternates mechanism. A naive repack may find that the
 331optimal delta for an object is against a base that is only found in
 332another fork. But when a client fetches, they will not have the base
 333object, and we'll have to find a new delta on the fly.
 334
 335A similar situation may exist if you have many refs outside of
 336`refs/heads/` and `refs/tags/` that point to related objects (e.g.,
 337`refs/pull` or `refs/changes` used by some hosting providers). By
 338default, clients fetch only heads and tags, and deltas against objects
 339found only in those other groups cannot be sent as-is.
 340
 341Delta islands solve this problem by allowing you to group your refs into
 342distinct "islands". Pack-objects computes which objects are reachable
 343from which islands, and refuses to make a delta from an object `A`
 344against a base which is not present in all of `A`'s islands. This
 345results in slightly larger packs (because we miss some delta
 346opportunities), but guarantees that a fetch of one island will not have
 347to recompute deltas on the fly due to crossing island boundaries.
 348
 349When repacking with delta islands the delta window tends to get
 350clogged with candidates that are forbidden by the config. Repacking
 351with a big --window helps (and doesn't take as long as it otherwise
 352might because we can reject some object pairs based on islands before
 353doing any computation on the content).
 354
 355Islands are configured via the `pack.island` option, which can be
 356specified multiple times. Each value is a left-anchored regular
 357expressions matching refnames. For example:
 358
 359-------------------------------------------
 360[pack]
 361island = refs/heads/
 362island = refs/tags/
 363-------------------------------------------
 364
 365puts heads and tags into an island (whose name is the empty string; see
 366below for more on naming). Any refs which do not match those regular
 367expressions (e.g., `refs/pull/123`) is not in any island. Any object
 368which is reachable only from `refs/pull/` (but not heads or tags) is
 369therefore not a candidate to be used as a base for `refs/heads/`.
 370
 371Refs are grouped into islands based on their "names", and two regexes
 372that produce the same name are considered to be in the same
 373island. The names are computed from the regexes by concatenating any
 374capture groups from the regex, with a '-' dash in between. (And if
 375there are no capture groups, then the name is the empty string, as in
 376the above example.) This allows you to create arbitrary numbers of
 377islands. Only up to 14 such capture groups are supported though.
 378
 379For example, imagine you store the refs for each fork in
 380`refs/virtual/ID`, where `ID` is a numeric identifier. You might then
 381configure:
 382
 383-------------------------------------------
 384[pack]
 385island = refs/virtual/([0-9]+)/heads/
 386island = refs/virtual/([0-9]+)/tags/
 387island = refs/virtual/([0-9]+)/(pull)/
 388-------------------------------------------
 389
 390That puts the heads and tags for each fork in their own island (named
 391"1234" or similar), and the pull refs for each go into their own
 392"1234-pull".
 393
 394Note that we pick a single island for each regex to go into, using "last
 395one wins" ordering (which allows repo-specific config to take precedence
 396over user-wide config, and so forth).
 397
 398SEE ALSO
 399--------
 400linkgit:git-rev-list[1]
 401linkgit:git-repack[1]
 402linkgit:git-prune-packed[1]
 403
 404GIT
 405---
 406Part of the linkgit:git[1] suite