Documentation / technical / http-protocol.txton commit finish_tmp_packfile():use strbuf for pathname construction (5889271)
   1HTTP transfer protocols
   2=======================
   3
   4Git supports two HTTP based transfer protocols.  A "dumb" protocol
   5which requires only a standard HTTP server on the server end of the
   6connection, and a "smart" protocol which requires a Git aware CGI
   7(or server module).  This document describes both protocols.
   8
   9As a design feature smart clients can automatically upgrade "dumb"
  10protocol URLs to smart URLs.  This permits all users to have the
  11same published URL, and the peers automatically select the most
  12efficient transport available to them.
  13
  14
  15URL Format
  16----------
  17
  18URLs for Git repositories accessed by HTTP use the standard HTTP
  19URL syntax documented by RFC 1738, so they are of the form:
  20
  21  http://<host>:<port>/<path>?<searchpart>
  22
  23Within this documentation the placeholder `$GIT_URL` will stand for
  24the http:// repository URL entered by the end-user.
  25
  26Servers SHOULD handle all requests to locations matching `$GIT_URL`, as
  27both the "smart" and "dumb" HTTP protocols used by Git operate
  28by appending additional path components onto the end of the user
  29supplied `$GIT_URL` string.
  30
  31An example of a dumb client requesting for a loose object:
  32
  33  $GIT_URL:     http://example.com:8080/git/repo.git
  34  URL request:  http://example.com:8080/git/repo.git/objects/d0/49f6c27a2244e12041955e262a404c7faba355
  35
  36An example of a smart request to a catch-all gateway:
  37
  38  $GIT_URL:     http://example.com/daemon.cgi?svc=git&q=
  39  URL request:  http://example.com/daemon.cgi?svc=git&q=/info/refs&service=git-receive-pack
  40
  41An example of a request to a submodule:
  42
  43  $GIT_URL:     http://example.com/git/repo.git/path/submodule.git
  44  URL request:  http://example.com/git/repo.git/path/submodule.git/info/refs
  45
  46Clients MUST strip a trailing `/`, if present, from the user supplied
  47`$GIT_URL` string to prevent empty path tokens (`//`) from appearing
  48in any URL sent to a server.  Compatible clients MUST expand
  49`$GIT_URL/info/refs` as `foo/info/refs` and not `foo//info/refs`.
  50
  51
  52Authentication
  53--------------
  54
  55Standard HTTP authentication is used if authentication is required
  56to access a repository, and MAY be configured and enforced by the
  57HTTP server software.
  58
  59Because Git repositories are accessed by standard path components
  60server administrators MAY use directory based permissions within
  61their HTTP server to control repository access.
  62
  63Clients SHOULD support Basic authentication as described by RFC 2616.
  64Servers SHOULD support Basic authentication by relying upon the
  65HTTP server placed in front of the Git server software.
  66
  67Servers SHOULD NOT require HTTP cookies for the purposes of
  68authentication or access control.
  69
  70Clients and servers MAY support other common forms of HTTP based
  71authentication, such as Digest authentication.
  72
  73
  74SSL
  75---
  76
  77Clients and servers SHOULD support SSL, particularly to protect
  78passwords when relying on Basic HTTP authentication.
  79
  80
  81Session State
  82-------------
  83
  84The Git over HTTP protocol (much like HTTP itself) is stateless
  85from the perspective of the HTTP server side.  All state MUST be
  86retained and managed by the client process.  This permits simple
  87round-robin load-balancing on the server side, without needing to
  88worry about state management.
  89
  90Clients MUST NOT require state management on the server side in
  91order to function correctly.
  92
  93Servers MUST NOT require HTTP cookies in order to function correctly.
  94Clients MAY store and forward HTTP cookies during request processing
  95as described by RFC 2616 (HTTP/1.1).  Servers SHOULD ignore any
  96cookies sent by a client.
  97
  98
  99General Request Processing
 100--------------------------
 101
 102Except where noted, all standard HTTP behavior SHOULD be assumed
 103by both client and server.  This includes (but is not necessarily
 104limited to):
 105
 106If there is no repository at `$GIT_URL`, or the resource pointed to by a
 107location matching `$GIT_URL` does not exist, the server MUST NOT respond
 108with `200 OK` response.  A server SHOULD respond with
 109`404 Not Found`, `410 Gone`, or any other suitable HTTP status code
 110which does not imply the resource exists as requested.
 111
 112If there is a repository at `$GIT_URL`, but access is not currently
 113permitted, the server MUST respond with the `403 Forbidden` HTTP
 114status code.
 115
 116Servers SHOULD support both HTTP 1.0 and HTTP 1.1.
 117Servers SHOULD support chunked encoding for both request and response
 118bodies.
 119
 120Clients SHOULD support both HTTP 1.0 and HTTP 1.1.
 121Clients SHOULD support chunked encoding for both request and response
 122bodies.
 123
 124Servers MAY return ETag and/or Last-Modified headers.
 125
 126Clients MAY revalidate cached entities by including If-Modified-Since
 127and/or If-None-Match request headers.
 128
 129Servers MAY return `304 Not Modified` if the relevant headers appear
 130in the request and the entity has not changed.  Clients MUST treat
 131`304 Not Modified` identical to `200 OK` by reusing the cached entity.
 132
 133Clients MAY reuse a cached entity without revalidation if the
 134Cache-Control and/or Expires header permits caching.  Clients and
 135servers MUST follow RFC 2616 for cache controls.
 136
 137
 138Discovering References
 139----------------------
 140
 141All HTTP clients MUST begin either a fetch or a push exchange by
 142discovering the references available on the remote repository.
 143
 144Dumb Clients
 145~~~~~~~~~~~~
 146
 147HTTP clients that only support the "dumb" protocol MUST discover
 148references by making a request for the special info/refs file of
 149the repository.
 150
 151Dumb HTTP clients MUST make a `GET` request to `$GIT_URL/info/refs`,
 152without any search/query parameters.
 153
 154   C: GET $GIT_URL/info/refs HTTP/1.0
 155
 156   S: 200 OK
 157   S:
 158   S: 95dcfa3633004da0049d3d0fa03f80589cbcaf31  refs/heads/maint
 159   S: d049f6c27a2244e12041955e262a404c7faba355  refs/heads/master
 160   S: 2cb58b79488a98d2721cea644875a8dd0026b115  refs/tags/v1.0
 161   S: a3c2e2402b99163d1d59756e5f207ae21cccba4c  refs/tags/v1.0^{}
 162
 163The Content-Type of the returned info/refs entity SHOULD be
 164`text/plain; charset=utf-8`, but MAY be any content type.
 165Clients MUST NOT attempt to validate the returned Content-Type.
 166Dumb servers MUST NOT return a return type starting with
 167`application/x-git-`.
 168
 169Cache-Control headers MAY be returned to disable caching of the
 170returned entity.
 171
 172When examining the response clients SHOULD only examine the HTTP
 173status code.  Valid responses are `200 OK`, or `304 Not Modified`.
 174
 175The returned content is a UNIX formatted text file describing
 176each ref and its known value.  The file SHOULD be sorted by name
 177according to the C locale ordering.  The file SHOULD NOT include
 178the default ref named `HEAD`.
 179
 180  info_refs   =  *( ref_record )
 181  ref_record  =  any_ref / peeled_ref
 182
 183  any_ref     =  obj-id HTAB refname LF
 184  peeled_ref  =  obj-id HTAB refname LF
 185                 obj-id HTAB refname "^{}" LF
 186
 187Smart Clients
 188~~~~~~~~~~~~~
 189
 190HTTP clients that support the "smart" protocol (or both the
 191"smart" and "dumb" protocols) MUST discover references by making
 192a parameterized request for the info/refs file of the repository.
 193
 194The request MUST contain exactly one query parameter,
 195`service=$servicename`, where `$servicename` MUST be the service
 196name the client wishes to contact to complete the operation.
 197The request MUST NOT contain additional query parameters.
 198
 199   C: GET $GIT_URL/info/refs?service=git-upload-pack HTTP/1.0
 200
 201dumb server reply:
 202
 203   S: 200 OK
 204   S:
 205   S: 95dcfa3633004da0049d3d0fa03f80589cbcaf31  refs/heads/maint
 206   S: d049f6c27a2244e12041955e262a404c7faba355  refs/heads/master
 207   S: 2cb58b79488a98d2721cea644875a8dd0026b115  refs/tags/v1.0
 208   S: a3c2e2402b99163d1d59756e5f207ae21cccba4c  refs/tags/v1.0^{}
 209
 210smart server reply:
 211
 212   S: 200 OK
 213   S: Content-Type: application/x-git-upload-pack-advertisement
 214   S: Cache-Control: no-cache
 215   S:
 216   S: 001e# service=git-upload-pack\n
 217   S: 004895dcfa3633004da0049d3d0fa03f80589cbcaf31 refs/heads/maint\0multi_ack\n
 218   S: 0042d049f6c27a2244e12041955e262a404c7faba355 refs/heads/master\n
 219   S: 003c2cb58b79488a98d2721cea644875a8dd0026b115 refs/tags/v1.0\n
 220   S: 003fa3c2e2402b99163d1d59756e5f207ae21cccba4c refs/tags/v1.0^{}\n
 221
 222Dumb Server Response
 223^^^^^^^^^^^^^^^^^^^^
 224Dumb servers MUST respond with the dumb server reply format.
 225
 226See the prior section under dumb clients for a more detailed
 227description of the dumb server response.
 228
 229Smart Server Response
 230^^^^^^^^^^^^^^^^^^^^^
 231If the server does not recognize the requested service name, or the
 232requested service name has been disabled by the server administrator,
 233the server MUST respond with the `403 Forbidden` HTTP status code.
 234
 235Otherwise, smart servers MUST respond with the smart server reply
 236format for the requested service name.
 237
 238Cache-Control headers SHOULD be used to disable caching of the
 239returned entity.
 240
 241The Content-Type MUST be `application/x-$servicename-advertisement`.
 242Clients SHOULD fall back to the dumb protocol if another content
 243type is returned.  When falling back to the dumb protocol clients
 244SHOULD NOT make an additional request to `$GIT_URL/info/refs`, but
 245instead SHOULD use the response already in hand.  Clients MUST NOT
 246continue if they do not support the dumb protocol.
 247
 248Clients MUST validate the status code is either `200 OK` or
 249`304 Not Modified`.
 250
 251Clients MUST validate the first five bytes of the response entity
 252matches the regex `^[0-9a-f]{4}#`.  If this test fails, clients
 253MUST NOT continue.
 254
 255Clients MUST parse the entire response as a sequence of pkt-line
 256records.
 257
 258Clients MUST verify the first pkt-line is `# service=$servicename`.
 259Servers MUST set $servicename to be the request parameter value.
 260Servers SHOULD include an LF at the end of this line.
 261Clients MUST ignore an LF at the end of the line.
 262
 263Servers MUST terminate the response with the magic `0000` end
 264pkt-line marker.
 265
 266The returned response is a pkt-line stream describing each ref and
 267its known value.  The stream SHOULD be sorted by name according to
 268the C locale ordering.  The stream SHOULD include the default ref
 269named `HEAD` as the first ref.  The stream MUST include capability
 270declarations behind a NUL on the first ref.
 271
 272  smart_reply     =  PKT-LINE("# service=$servicename" LF)
 273                     ref_list
 274                     "0000"
 275  ref_list        =  empty_list / non_empty_list
 276
 277  empty_list      =  PKT-LINE(zero-id SP "capabilities^{}" NUL cap-list LF)
 278
 279  non_empty_list  =  PKT-LINE(obj-id SP name NUL cap_list LF)
 280                     *ref_record
 281
 282  cap-list        =  capability *(SP capability)
 283  capability      =  1*(LC_ALPHA / DIGIT / "-" / "_")
 284  LC_ALPHA        =  %x61-7A
 285
 286  ref_record      =  any_ref / peeled_ref
 287  any_ref         =  PKT-LINE(obj-id SP name LF)
 288  peeled_ref      =  PKT-LINE(obj-id SP name LF)
 289                     PKT-LINE(obj-id SP name "^{}" LF
 290
 291
 292Smart Service git-upload-pack
 293------------------------------
 294This service reads from the repository pointed to by `$GIT_URL`.
 295
 296Clients MUST first perform ref discovery with
 297`$GIT_URL/info/refs?service=git-upload-pack`.
 298
 299   C: POST $GIT_URL/git-upload-pack HTTP/1.0
 300   C: Content-Type: application/x-git-upload-pack-request
 301   C:
 302   C: 0032want 0a53e9ddeaddad63ad106860237bbf53411d11a7\n
 303   C: 0032have 441b40d833fdfa93eb2908e52742248faf0ee993\n
 304   C: 0000
 305
 306   S: 200 OK
 307   S: Content-Type: application/x-git-upload-pack-result
 308   S: Cache-Control: no-cache
 309   S:
 310   S: ....ACK %s, continue
 311   S: ....NAK
 312
 313Clients MUST NOT reuse or revalidate a cached response.
 314Servers MUST include sufficient Cache-Control headers
 315to prevent caching of the response.
 316
 317Servers SHOULD support all capabilities defined here.
 318
 319Clients MUST send at least one "want" command in the request body.
 320Clients MUST NOT reference an id in a "want" command which did not
 321appear in the response obtained through ref discovery unless the
 322server advertises capability `allow-tip-sha1-in-want`.
 323
 324  compute_request   =  want_list
 325                       have_list
 326                       request_end
 327  request_end       =  "0000" / "done"
 328
 329  want_list         =  PKT-LINE(want NUL cap_list LF)
 330                       *(want_pkt)
 331  want_pkt          =  PKT-LINE(want LF)
 332  want              =  "want" SP id
 333  cap_list          =  *(SP capability) SP
 334
 335  have_list         =  *PKT-LINE("have" SP id LF)
 336
 337TODO: Document this further.
 338
 339The Negotiation Algorithm
 340~~~~~~~~~~~~~~~~~~~~~~~~~
 341The computation to select the minimal pack proceeds as follows
 342(C = client, S = server):
 343
 344'init step:'
 345
 346C: Use ref discovery to obtain the advertised refs.
 347
 348C: Place any object seen into set `advertised`.
 349
 350C: Build an empty set, `common`, to hold the objects that are later
 351   determined to be on both ends.
 352
 353C: Build a set, `want`, of the objects from `advertised` the client
 354   wants to fetch, based on what it saw during ref discovery.
 355
 356C: Start a queue, `c_pending`, ordered by commit time (popping newest
 357   first).  Add all client refs.  When a commit is popped from
 358   the queue its parents SHOULD be automatically inserted back.
 359   Commits MUST only enter the queue once.
 360
 361'one compute step:'
 362
 363C: Send one `$GIT_URL/git-upload-pack` request:
 364
 365   C: 0032want <want #1>...............................
 366   C: 0032want <want #2>...............................
 367   ....
 368   C: 0032have <common #1>.............................
 369   C: 0032have <common #2>.............................
 370   ....
 371   C: 0032have <have #1>...............................
 372   C: 0032have <have #2>...............................
 373   ....
 374   C: 0000
 375
 376The stream is organized into "commands", with each command
 377appearing by itself in a pkt-line.  Within a command line
 378the text leading up to the first space is the command name,
 379and the remainder of the line to the first LF is the value.
 380Command lines are terminated with an LF as the last byte of
 381the pkt-line value.
 382
 383Commands MUST appear in the following order, if they appear
 384at all in the request stream:
 385
 386* "want"
 387* "have"
 388
 389The stream is terminated by a pkt-line flush (`0000`).
 390
 391A single "want" or "have" command MUST have one hex formatted
 392SHA-1 as its value.  Multiple SHA-1s MUST be sent by sending
 393multiple commands.
 394
 395The `have` list is created by popping the first 32 commits
 396from `c_pending`.  Less can be supplied if `c_pending` empties.
 397
 398If the client has sent 256 "have" commits and has not yet
 399received one of those back from `s_common`, or the client has
 400emptied `c_pending` it SHOULD include a "done" command to let
 401the server know it won't proceed:
 402
 403   C: 0009done
 404
 405S: Parse the git-upload-pack request:
 406
 407Verify all objects in `want` are directly reachable from refs.
 408
 409The server MAY walk backwards through history or through
 410the reflog to permit slightly stale requests.
 411
 412If no "want" objects are received, send an error:
 413TODO: Define error if no "want" lines are requested.
 414
 415If any "want" object is not reachable, send an error:
 416TODO: Define error if an invalid "want" is requested.
 417
 418Create an empty list, `s_common`.
 419
 420If "have" was sent:
 421
 422Loop through the objects in the order supplied by the client.
 423
 424For each object, if the server has the object reachable from
 425a ref, add it to `s_common`.  If a commit is added to `s_common`,
 426do not add any ancestors, even if they also appear in `have`.
 427
 428S: Send the git-upload-pack response:
 429
 430If the server has found a closed set of objects to pack or the
 431request ends with "done", it replies with the pack.
 432TODO: Document the pack based response
 433
 434   S: PACK...
 435
 436The returned stream is the side-band-64k protocol supported
 437by the git-upload-pack service, and the pack is embedded into
 438stream 1.  Progress messages from the server side MAY appear
 439in stream 2.
 440
 441Here a "closed set of objects" is defined to have at least
 442one path from every "want" to at least one "common" object.
 443
 444If the server needs more information, it replies with a
 445status continue response:
 446TODO: Document the non-pack response
 447
 448C: Parse the upload-pack response:
 449   TODO: Document parsing response
 450
 451'Do another compute step.'
 452
 453
 454Smart Service git-receive-pack
 455------------------------------
 456This service reads from the repository pointed to by `$GIT_URL`.
 457
 458Clients MUST first perform ref discovery with
 459`$GIT_URL/info/refs?service=git-receive-pack`.
 460
 461   C: POST $GIT_URL/git-receive-pack HTTP/1.0
 462   C: Content-Type: application/x-git-receive-pack-request
 463   C:
 464   C: ....0a53e9ddeaddad63ad106860237bbf53411d11a7 441b40d833fdfa93eb2908e52742248faf0ee993 refs/heads/maint\0 report-status
 465   C: 0000
 466   C: PACK....
 467
 468   S: 200 OK
 469   S: Content-Type: application/x-git-receive-pack-result
 470   S: Cache-Control: no-cache
 471   S:
 472   S: ....
 473
 474Clients MUST NOT reuse or revalidate a cached response.
 475Servers MUST include sufficient Cache-Control headers
 476to prevent caching of the response.
 477
 478Servers SHOULD support all capabilities defined here.
 479
 480Clients MUST send at least one command in the request body.
 481Within the command portion of the request body clients SHOULD send
 482the id obtained through ref discovery as old_id.
 483
 484  update_request  =  command_list
 485                     "PACK" <binary data>
 486
 487  command_list    =  PKT-LINE(command NUL cap_list LF)
 488                     *(command_pkt)
 489  command_pkt     =  PKT-LINE(command LF)
 490  cap_list        =  *(SP capability) SP
 491
 492  command         =  create / delete / update
 493  create          =  zero-id SP new_id SP name
 494  delete          =  old_id SP zero-id SP name
 495  update          =  old_id SP new_id SP name
 496
 497TODO: Document this further.
 498
 499
 500References
 501----------
 502
 503link:http://www.ietf.org/rfc/rfc1738.txt[RFC 1738: Uniform Resource Locators (URL)]
 504link:http://www.ietf.org/rfc/rfc2616.txt[RFC 2616: Hypertext Transfer Protocol -- HTTP/1.1]
 505link:technical/pack-protocol.html
 506link:technical/protocol-capabilities.html