diff options
Diffstat (limited to 'Documentation/technical')
| -rw-r--r-- | Documentation/technical/api-builtin.txt | 6 | ||||
| -rw-r--r-- | Documentation/technical/api-diff.txt | 10 | ||||
| -rw-r--r-- | Documentation/technical/api-directory-listing.txt | 25 | ||||
| -rw-r--r-- | Documentation/technical/api-parse-options.txt | 24 | ||||
| -rw-r--r-- | Documentation/technical/api-ref-iteration.txt | 4 | ||||
| -rw-r--r-- | Documentation/technical/api-remote.txt | 20 | ||||
| -rw-r--r-- | Documentation/technical/api-revision-walking.txt | 2 | ||||
| -rw-r--r-- | Documentation/technical/api-setup.txt | 38 | ||||
| -rw-r--r-- | Documentation/technical/api-strbuf.txt | 5 | ||||
| -rw-r--r-- | Documentation/technical/http-protocol.txt | 503 | ||||
| -rw-r--r-- | Documentation/technical/index-format.txt | 2 | ||||
| -rw-r--r-- | Documentation/technical/pack-heuristics.txt | 10 | ||||
| -rw-r--r-- | Documentation/technical/pack-protocol.txt | 10 | ||||
| -rw-r--r-- | Documentation/technical/protocol-capabilities.txt | 69 | ||||
| -rw-r--r-- | Documentation/technical/racy-git.txt | 8 |
15 files changed, 679 insertions, 57 deletions
diff --git a/Documentation/technical/api-builtin.txt b/Documentation/technical/api-builtin.txt index 4a4228b896..e3d6e7a79a 100644 --- a/Documentation/technical/api-builtin.txt +++ b/Documentation/technical/api-builtin.txt @@ -14,8 +14,8 @@ Git: . Add the external declaration for the function to `builtin.h`. -. Add the command to `commands[]` table in `handle_internal_command()`, - defined in `git.c`. The entry should look like: +. Add the command to the `commands[]` table defined in `git.c`. + The entry should look like: { "foo", cmd_foo, <options> }, + @@ -39,7 +39,7 @@ where options is the bitwise-or of: on bare repositories. This only makes sense when `RUN_SETUP` is also set. -. Add `builtin-foo.o` to `BUILTIN_OBJS` in `Makefile`. +. Add `builtin/foo.o` to `BUILTIN_OBJS` in `Makefile`. Additionally, if `foo` is a new command, there are 3 more things to do: diff --git a/Documentation/technical/api-diff.txt b/Documentation/technical/api-diff.txt index 2d2ebc04b7..8b001de0db 100644 --- a/Documentation/technical/api-diff.txt +++ b/Documentation/technical/api-diff.txt @@ -28,7 +28,8 @@ Calling sequence * Call `diff_setup_done()`; this inspects the options set up so far for internal consistency and make necessary tweaking to it (e.g. if - textual patch output was asked, recursive behaviour is turned on). + textual patch output was asked, recursive behaviour is turned on); + the callback set_default in diff_options can be used to tweak this more. * As you find different pairs of files, call `diff_change()` to feed modified files, `diff_addremove()` to feed created or deleted files, @@ -115,6 +116,13 @@ Notable members are: operation, but some do not have anything to do with the diffcore library. +`touched_flags`:: + Records whether a flag has been changed due to user request + (rather than just set/unset by default). + +`set_default`:: + Callback which allows tweaking the options in diff_setup_done(). + BINARY, TEXT;; Affects the way how a file that is seemingly binary is treated. diff --git a/Documentation/technical/api-directory-listing.txt b/Documentation/technical/api-directory-listing.txt index 1f349b28ae..7f8e78d916 100644 --- a/Documentation/technical/api-directory-listing.txt +++ b/Documentation/technical/api-directory-listing.txt @@ -22,12 +22,23 @@ The notable options are: `flags`:: - A bit-field of options: + A bit-field of options (the `*IGNORED*` flags are mutually exclusive): `DIR_SHOW_IGNORED`::: - The traversal is for finding just ignored files, not unignored - files. + Return just ignored files in `entries[]`, not untracked files. + +`DIR_SHOW_IGNORED_TOO`::: + + Similar to `DIR_SHOW_IGNORED`, but return ignored files in `ignored[]` + in addition to untracked files in `entries[]`. + +`DIR_COLLECT_IGNORED`::: + + Special mode for git-add. Return ignored files in `ignored[]` and + untracked files in `entries[]`. Only returns ignored files that match + pathspec exactly (no wildcards). Does not recurse into ignored + directories. `DIR_SHOW_OTHER_DIRECTORIES`::: @@ -57,6 +68,14 @@ The result of the enumeration is left in these fields: Internal use; keeps track of allocation of `entries[]` array. +`ignored[]`:: + + An array of `struct dir_entry`, used for ignored paths with the + `DIR_SHOW_IGNORED_TOO` and `DIR_COLLECT_IGNORED` flags. + +`ignored_nr`:: + + The number of members in `ignored[]` array. Calling sequence ---------------- diff --git a/Documentation/technical/api-parse-options.txt b/Documentation/technical/api-parse-options.txt index 32ddc1cf13..be50cf4de3 100644 --- a/Documentation/technical/api-parse-options.txt +++ b/Documentation/technical/api-parse-options.txt @@ -29,9 +29,9 @@ that allow to change the behavior of a command. The parse-options API allows: -* 'sticked' and 'separate form' of options with arguments. - `-oArg` is sticked, `-o Arg` is separate form. - `--option=Arg` is sticked, `--option Arg` is separate form. +* 'stuck' and 'separate form' of options with arguments. + `-oArg` is stuck, `-o Arg` is separate form. + `--option=Arg` is stuck, `--option Arg` is separate form. * Long options may be 'abbreviated', as long as the abbreviation is unambiguous. @@ -41,6 +41,8 @@ The parse-options API allows: * Boolean long options can be 'negated' (or 'unset') by prepending `no-`, e.g. `--no-abbrev` instead of `--abbrev`. Conversely, options that begin with `no-` can be 'negated' by removing it. + Other long options can be unset (e.g., set string to NULL, set + integer to 0) by prepending `no-`. * Options and non-option arguments can clearly be separated using the `--` option, e.g. `-a -b --option -- --this-is-a-file` indicates that @@ -174,6 +176,10 @@ There are some macros to easily define options: Introduce an option with date argument, see `approxidate()`. The timestamp is put into `int_var`. +`OPT_EXPIRY_DATE(short, long, &int_var, description)`:: + Introduce an option with expiry date argument, see `parse_expiry_date()`. + The timestamp is put into `int_var`. + `OPT_CALLBACK(short, long, &var, arg_str, description, func_ptr)`:: Introduce an option with argument. The argument will be fed into the function given by `func_ptr` @@ -269,10 +275,10 @@ Examples -------- See `test-parse-options.c` and -`builtin-add.c`, -`builtin-clone.c`, -`builtin-commit.c`, -`builtin-fetch.c`, -`builtin-fsck.c`, -`builtin-rm.c` +`builtin/add.c`, +`builtin/clone.c`, +`builtin/commit.c`, +`builtin/fetch.c`, +`builtin/fsck.c`, +`builtin/rm.c` for real-world examples. diff --git a/Documentation/technical/api-ref-iteration.txt b/Documentation/technical/api-ref-iteration.txt index aa1c50f181..02adfd45d3 100644 --- a/Documentation/technical/api-ref-iteration.txt +++ b/Documentation/technical/api-ref-iteration.txt @@ -50,10 +50,10 @@ submodules object database. You can do this by a code-snippet like this: const char *path = "path/to/submodule" - if (!add_submodule_odb(path)) + if (add_submodule_odb(path)) die("Error submodule '%s' not populated.", path); -`add_submodule_odb()` will return an non-zero value on success. If you +`add_submodule_odb()` will return zero on success. If you do not do this you will get an error for each ref that it does not point to a valid object. diff --git a/Documentation/technical/api-remote.txt b/Documentation/technical/api-remote.txt index 4be87768f6..5d245aa9d1 100644 --- a/Documentation/technical/api-remote.txt +++ b/Documentation/technical/api-remote.txt @@ -58,16 +58,16 @@ default remote, given the current branch and configuration. struct refspec -------------- -A struct refspec holds the parsed interpretation of a refspec. If it -will force updates (starts with a '+'), force is true. If it is a -pattern (sides end with '*') pattern is true. src and dest are the two -sides (if a pattern, only the part outside of the wildcards); if there -is only one side, it is src, and dst is NULL; if sides exist but are -empty (i.e., the refspec either starts or ends with ':'), the -corresponding side is "". - -This parsing can be done to an array of strings to give an array of -struct refpsecs with parse_ref_spec(). +A struct refspec holds the parsed interpretation of a refspec. If it +will force updates (starts with a '+'), force is true. If it is a +pattern (sides end with '*') pattern is true. src and dest are the +two sides (including '*' characters if present); if there is only one +side, it is src, and dst is NULL; if sides exist but are empty (i.e., +the refspec either starts or ends with ':'), the corresponding side is +"". + +An array of strings can be parsed into an array of struct refspecs +using parse_fetch_refspec() or parse_push_refspec(). remote_find_tracking(), given a remote and a struct refspec with either src or dst filled out, will fill out the other such that the diff --git a/Documentation/technical/api-revision-walking.txt b/Documentation/technical/api-revision-walking.txt index b7d0d9a8a7..55b878ade8 100644 --- a/Documentation/technical/api-revision-walking.txt +++ b/Documentation/technical/api-revision-walking.txt @@ -59,7 +59,7 @@ function. `reset_revision_walk`:: Reset the flags used by the revision walking api. You can use - this to do multiple sequencial revision walks. + this to do multiple sequential revision walks. Data structures --------------- diff --git a/Documentation/technical/api-setup.txt b/Documentation/technical/api-setup.txt index 4f63a04d7d..540e455689 100644 --- a/Documentation/technical/api-setup.txt +++ b/Documentation/technical/api-setup.txt @@ -8,6 +8,42 @@ Talk about * is_inside_git_dir() * is_inside_work_tree() * setup_work_tree() -* get_pathspec() (Dscho) + +Pathspec +-------- + +See glossary-context.txt for the syntax of pathspec. In memory, a +pathspec set is represented by "struct pathspec" and is prepared by +parse_pathspec(). This function takes several arguments: + +- magic_mask specifies what features that are NOT supported by the + following code. If a user attempts to use such a feature, + parse_pathspec() can reject it early. + +- flags specifies other things that the caller wants parse_pathspec to + perform. + +- prefix and args come from cmd_* functions + +get_pathspec() is obsolete and should never be used in new code. + +parse_pathspec() helps catch unsupported features and reject them +politely. At a lower level, different pathspec-related functions may +not support the same set of features. Such pathspec-sensitive +functions are guarded with GUARD_PATHSPEC(), which will die in an +unfriendly way when an unsupported feature is requested. + +The command designers are supposed to make sure that GUARD_PATHSPEC() +never dies. They have to make sure all unsupported features are caught +by parse_pathspec(), not by GUARD_PATHSPEC. grepping GUARD_PATHSPEC() +should give the designers all pathspec-sensitive codepaths and what +features they support. + +A similar process is applied when a new pathspec magic is added. The +designer lifts the GUARD_PATHSPEC restriction in the functions that +support the new magic. At the same time (s)he has to make sure this +new feature will be caught at parse_pathspec() in commands that cannot +handle the new magic in some cases. grepping parse_pathspec() should +help. diff --git a/Documentation/technical/api-strbuf.txt b/Documentation/technical/api-strbuf.txt index 2c59cb2259..3350d97dda 100644 --- a/Documentation/technical/api-strbuf.txt +++ b/Documentation/technical/api-strbuf.txt @@ -230,6 +230,11 @@ which can be used by the programmer of the callback as she sees fit. destination. This is useful for literal data to be fed to either strbuf_expand or to the *printf family of functions. +`strbuf_humanise_bytes`:: + + Append the given byte size as a human-readable string (i.e. 12.23 KiB, + 3.50 MiB). + `strbuf_addf`:: Add a formatted string to the buffer. diff --git a/Documentation/technical/http-protocol.txt b/Documentation/technical/http-protocol.txt new file mode 100644 index 0000000000..d21d77d1de --- /dev/null +++ b/Documentation/technical/http-protocol.txt @@ -0,0 +1,503 @@ +HTTP transfer protocols +======================= + +Git supports two HTTP based transfer protocols. A "dumb" protocol +which requires only a standard HTTP server on the server end of the +connection, and a "smart" protocol which requires a Git aware CGI +(or server module). This document describes both protocols. + +As a design feature smart clients can automatically upgrade "dumb" +protocol URLs to smart URLs. This permits all users to have the +same published URL, and the peers automatically select the most +efficient transport available to them. + + +URL Format +---------- + +URLs for Git repositories accessed by HTTP use the standard HTTP +URL syntax documented by RFC 1738, so they are of the form: + + http://<host>:<port>/<path>?<searchpart> + +Within this documentation the placeholder $GIT_URL will stand for +the http:// repository URL entered by the end-user. + +Servers SHOULD handle all requests to locations matching $GIT_URL, as +both the "smart" and "dumb" HTTP protocols used by Git operate +by appending additional path components onto the end of the user +supplied $GIT_URL string. + +An example of a dumb client requesting for a loose object: + + $GIT_URL: http://example.com:8080/git/repo.git + URL request: http://example.com:8080/git/repo.git/objects/d0/49f6c27a2244e12041955e262a404c7faba355 + +An example of a smart request to a catch-all gateway: + + $GIT_URL: http://example.com/daemon.cgi?svc=git&q= + URL request: http://example.com/daemon.cgi?svc=git&q=/info/refs&service=git-receive-pack + +An example of a request to a submodule: + + $GIT_URL: http://example.com/git/repo.git/path/submodule.git + URL request: http://example.com/git/repo.git/path/submodule.git/info/refs + +Clients MUST strip a trailing '/', if present, from the user supplied +$GIT_URL string to prevent empty path tokens ('//') from appearing +in any URL sent to a server. Compatible clients MUST expand +'$GIT_URL/info/refs' as 'foo/info/refs' and not 'foo//info/refs'. + + +Authentication +-------------- + +Standard HTTP authentication is used if authentication is required +to access a repository, and MAY be configured and enforced by the +HTTP server software. + +Because Git repositories are accessed by standard path components +server administrators MAY use directory based permissions within +their HTTP server to control repository access. + +Clients SHOULD support Basic authentication as described by RFC 2616. +Servers SHOULD support Basic authentication by relying upon the +HTTP server placed in front of the Git server software. + +Servers SHOULD NOT require HTTP cookies for the purposes of +authentication or access control. + +Clients and servers MAY support other common forms of HTTP based +authentication, such as Digest authentication. + + +SSL +--- + +Clients and servers SHOULD support SSL, particularly to protect +passwords when relying on Basic HTTP authentication. + + +Session State +------------- + +The Git over HTTP protocol (much like HTTP itself) is stateless +from the perspective of the HTTP server side. All state MUST be +retained and managed by the client process. This permits simple +round-robin load-balancing on the server side, without needing to +worry about state management. + +Clients MUST NOT require state management on the server side in +order to function correctly. + +Servers MUST NOT require HTTP cookies in order to function correctly. +Clients MAY store and forward HTTP cookies during request processing +as described by RFC 2616 (HTTP/1.1). Servers SHOULD ignore any +cookies sent by a client. + + +General Request Processing +-------------------------- + +Except where noted, all standard HTTP behavior SHOULD be assumed +by both client and server. This includes (but is not necessarily +limited to): + +If there is no repository at $GIT_URL, or the resource pointed to by a +location matching $GIT_URL does not exist, the server MUST NOT respond +with '200 OK' response. A server SHOULD respond with +'404 Not Found', '410 Gone', or any other suitable HTTP status code +which does not imply the resource exists as requested. + +If there is a repository at $GIT_URL, but access is not currently +permitted, the server MUST respond with the '403 Forbidden' HTTP +status code. + +Servers SHOULD support both HTTP 1.0 and HTTP 1.1. +Servers SHOULD support chunked encoding for both request and response +bodies. + +Clients SHOULD support both HTTP 1.0 and HTTP 1.1. +Clients SHOULD support chunked encoding for both request and response +bodies. + +Servers MAY return ETag and/or Last-Modified headers. + +Clients MAY revalidate cached entities by including If-Modified-Since +and/or If-None-Match request headers. + +Servers MAY return '304 Not Modified' if the relevant headers appear +in the request and the entity has not changed. Clients MUST treat +'304 Not Modified' identical to '200 OK' by reusing the cached entity. + +Clients MAY reuse a cached entity without revalidation if the +Cache-Control and/or Expires header permits caching. Clients and +servers MUST follow RFC 2616 for cache controls. + + +Discovering References +---------------------- + +All HTTP clients MUST begin either a fetch or a push exchange by +discovering the references available on the remote repository. + +Dumb Clients +~~~~~~~~~~~~ + +HTTP clients that only support the "dumb" protocol MUST discover +references by making a request for the special info/refs file of +the repository. + +Dumb HTTP clients MUST make a GET request to $GIT_URL/info/refs, +without any search/query parameters. + + C: GET $GIT_URL/info/refs HTTP/1.0 + + S: 200 OK + S: + S: 95dcfa3633004da0049d3d0fa03f80589cbcaf31 refs/heads/maint + S: d049f6c27a2244e12041955e262a404c7faba355 refs/heads/master + S: 2cb58b79488a98d2721cea644875a8dd0026b115 refs/tags/v1.0 + S: a3c2e2402b99163d1d59756e5f207ae21cccba4c refs/tags/v1.0^{} + +The Content-Type of the returned info/refs entity SHOULD be +"text/plain; charset=utf-8", but MAY be any content type. +Clients MUST NOT attempt to validate the returned Content-Type. +Dumb servers MUST NOT return a return type starting with +"application/x-git-". + +Cache-Control headers MAY be returned to disable caching of the +returned entity. + +When examining the response clients SHOULD only examine the HTTP +status code. Valid responses are '200 OK', or '304 Not Modified'. + +The returned content is a UNIX formatted text file describing +each ref and its known value. The file SHOULD be sorted by name +according to the C locale ordering. The file SHOULD NOT include +the default ref named 'HEAD'. + + info_refs = *( ref_record ) + ref_record = any_ref / peeled_ref + + any_ref = obj-id HTAB refname LF + peeled_ref = obj-id HTAB refname LF + obj-id HTAB refname "^{}" LF + +Smart Clients +~~~~~~~~~~~~~ + +HTTP clients that support the "smart" protocol (or both the +"smart" and "dumb" protocols) MUST discover references by making +a parameterized request for the info/refs file of the repository. + +The request MUST contain exactly one query parameter, +'service=$servicename', where $servicename MUST be the service +name the client wishes to contact to complete the operation. +The request MUST NOT contain additional query parameters. + + C: GET $GIT_URL/info/refs?service=git-upload-pack HTTP/1.0 + + dumb server reply: + S: 200 OK + S: + S: 95dcfa3633004da0049d3d0fa03f80589cbcaf31 refs/heads/maint + S: d049f6c27a2244e12041955e262a404c7faba355 refs/heads/master + S: 2cb58b79488a98d2721cea644875a8dd0026b115 refs/tags/v1.0 + S: a3c2e2402b99163d1d59756e5f207ae21cccba4c refs/tags/v1.0^{} + + smart server reply: + S: 200 OK + S: Content-Type: application/x-git-upload-pack-advertisement + S: Cache-Control: no-cache + S: + S: 001e# service=git-upload-pack\n + S: 004895dcfa3633004da0049d3d0fa03f80589cbcaf31 refs/heads/maint\0multi_ack\n + S: 0042d049f6c27a2244e12041955e262a404c7faba355 refs/heads/master\n + S: 003c2cb58b79488a98d2721cea644875a8dd0026b115 refs/tags/v1.0\n + S: 003fa3c2e2402b99163d1d59756e5f207ae21cccba4c refs/tags/v1.0^{}\n + +Dumb Server Response +^^^^^^^^^^^^^^^^^^^^ +Dumb servers MUST respond with the dumb server reply format. + +See the prior section under dumb clients for a more detailed +description of the dumb server response. + +Smart Server Response +^^^^^^^^^^^^^^^^^^^^^ +If the server does not recognize the requested service name, or the +requested service name has been disabled by the server administrator, +the server MUST respond with the '403 Forbidden' HTTP status code. + +Otherwise, smart servers MUST respond with the smart server reply +format for the requested service name. + +Cache-Control headers SHOULD be used to disable caching of the +returned entity. + +The Content-Type MUST be 'application/x-$servicename-advertisement'. +Clients SHOULD fall back to the dumb protocol if another content +type is returned. When falling back to the dumb protocol clients +SHOULD NOT make an additional request to $GIT_URL/info/refs, but +instead SHOULD use the response already in hand. Clients MUST NOT +continue if they do not support the dumb protocol. + +Clients MUST validate the status code is either '200 OK' or +'304 Not Modified'. + +Clients MUST validate the first five bytes of the response entity +matches the regex "^[0-9a-f]{4}#". If this test fails, clients +MUST NOT continue. + +Clients MUST parse the entire response as a sequence of pkt-line +records. + +Clients MUST verify the first pkt-line is "# service=$servicename". +Servers MUST set $servicename to be the request parameter value. +Servers SHOULD include an LF at the end of this line. +Clients MUST ignore an LF at the end of the line. + +Servers MUST terminate the response with the magic "0000" end +pkt-line marker. + +The returned response is a pkt-line stream describing each ref and +its known value. The stream SHOULD be sorted by name according to +the C locale ordering. The stream SHOULD include the default ref +named 'HEAD' as the first ref. The stream MUST include capability +declarations behind a NUL on the first ref. + + smart_reply = PKT-LINE("# service=$servicename" LF) + ref_list + "0000" + ref_list = empty_list / non_empty_list + + empty_list = PKT-LINE(zero-id SP "capabilities^{}" NUL cap-list LF) + + non_empty_list = PKT-LINE(obj-id SP name NUL cap_list LF) + *ref_record + + cap-list = capability *(SP capability) + capability = 1*(LC_ALPHA / DIGIT / "-" / "_") + LC_ALPHA = %x61-7A + + ref_record = any_ref / peeled_ref + any_ref = PKT-LINE(obj-id SP name LF) + peeled_ref = PKT-LINE(obj-id SP name LF) + PKT-LINE(obj-id SP name "^{}" LF + +Smart Service git-upload-pack +------------------------------ +This service reads from the repository pointed to by $GIT_URL. + +Clients MUST first perform ref discovery with +'$GIT_URL/info/refs?service=git-upload-pack'. + + C: POST $GIT_URL/git-upload-pack HTTP/1.0 + C: Content-Type: application/x-git-upload-pack-request + C: + C: 0032want 0a53e9ddeaddad63ad106860237bbf53411d11a7\n + C: 0032have 441b40d833fdfa93eb2908e52742248faf0ee993\n + C: 0000 + + S: 200 OK + S: Content-Type: application/x-git-upload-pack-result + S: Cache-Control: no-cache + S: + S: ....ACK %s, continue + S: ....NAK + +Clients MUST NOT reuse or revalidate a cached response. +Servers MUST include sufficient Cache-Control headers +to prevent caching of the response. + +Servers SHOULD support all capabilities defined here. + +Clients MUST send at least one 'want' command in the request body. +Clients MUST NOT reference an id in a 'want' command which did not +appear in the response obtained through ref discovery unless the +server advertises capability "allow-tip-sha1-in-want". + + compute_request = want_list + have_list + request_end + request_end = "0000" / "done" + + want_list = PKT-LINE(want NUL cap_list LF) + *(want_pkt) + want_pkt = PKT-LINE(want LF) + want = "want" SP id + cap_list = *(SP capability) SP + + have_list = *PKT-LINE("have" SP id LF) + +TODO: Document this further. +TODO: Don't use uppercase for variable names below. + +The Negotiation Algorithm +~~~~~~~~~~~~~~~~~~~~~~~~~ +The computation to select the minimal pack proceeds as follows +(c = client, s = server): + + init step: + (c) Use ref discovery to obtain the advertised refs. + (c) Place any object seen into set ADVERTISED. + + (c) Build an empty set, COMMON, to hold the objects that are later + determined to be on both ends. + (c) Build a set, WANT, of the objects from ADVERTISED the client + wants to fetch, based on what it saw during ref discovery. + + (c) Start a queue, C_PENDING, ordered by commit time (popping newest + first). Add all client refs. When a commit is popped from + the queue its parents SHOULD be automatically inserted back. + Commits MUST only enter the queue once. + + one compute step: + (c) Send one $GIT_URL/git-upload-pack request: + + C: 0032want <WANT #1>............................... + C: 0032want <WANT #2>............................... + .... + C: 0032have <COMMON #1>............................. + C: 0032have <COMMON #2>............................. + .... + C: 0032have <HAVE #1>............................... + C: 0032have <HAVE #2>............................... + .... + C: 0000 + + The stream is organized into "commands", with each command + appearing by itself in a pkt-line. Within a command line + the text leading up to the first space is the command name, + and the remainder of the line to the first LF is the value. + Command lines are terminated with an LF as the last byte of + the pkt-line value. + + Commands MUST appear in the following order, if they appear + at all in the request stream: + + * want + * have + + The stream is terminated by a pkt-line flush ("0000"). + + A single "want" or "have" command MUST have one hex formatted + SHA-1 as its value. Multiple SHA-1s MUST be sent by sending + multiple commands. + + The HAVE list is created by popping the first 32 commits + from C_PENDING. Less can be supplied if C_PENDING empties. + + If the client has sent 256 HAVE commits and has not yet + received one of those back from S_COMMON, or the client has + emptied C_PENDING it SHOULD include a "done" command to let + the server know it won't proceed: + + C: 0009done + + (s) Parse the git-upload-pack request: + + Verify all objects in WANT are directly reachable from refs. + + The server MAY walk backwards through history or through + the reflog to permit slightly stale requests. + + If no WANT objects are received, send an error: + +TODO: Define error if no want lines are requested. + + If any WANT object is not reachable, send an error: + +TODO: Define error if an invalid want is requested. + + Create an empty list, S_COMMON. + + If 'have' was sent: + + Loop through the objects in the order supplied by the client. + For each object, if the server has the object reachable from + a ref, add it to S_COMMON. If a commit is added to S_COMMON, + do not add any ancestors, even if they also appear in HAVE. + + (s) Send the git-upload-pack response: + + If the server has found a closed set of objects to pack or the + request ends with "done", it replies with the pack. + +TODO: Document the pack based response + S: PACK... + + The returned stream is the side-band-64k protocol supported + by the git-upload-pack service, and the pack is embedded into + stream 1. Progress messages from the server side MAY appear + in stream 2. + + Here a "closed set of objects" is defined to have at least + one path from every WANT to at least one COMMON object. + + If the server needs more information, it replies with a + status continue response: + +TODO: Document the non-pack response + + (c) Parse the upload-pack response: + +TODO: Document parsing response + + Do another compute step. + + +Smart Service git-receive-pack +------------------------------ +This service reads from the repository pointed to by $GIT_URL. + +Clients MUST first perform ref discovery with +'$GIT_URL/info/refs?service=git-receive-pack'. + + C: POST $GIT_URL/git-receive-pack HTTP/1.0 + C: Content-Type: application/x-git-receive-pack-request + C: + C: ....0a53e9ddeaddad63ad106860237bbf53411d11a7 441b40d833fdfa93eb2908e52742248faf0ee993 refs/heads/maint\0 report-status + C: 0000 + C: PACK.... + + S: 200 OK + S: Content-Type: application/x-git-receive-pack-result + S: Cache-Control: no-cache + S: + S: .... + +Clients MUST NOT reuse or revalidate a cached response. +Servers MUST include sufficient Cache-Control headers +to prevent caching of the response. + +Servers SHOULD support all capabilities defined here. + +Clients MUST send at least one command in the request body. +Within the command portion of the request body clients SHOULD send +the id obtained through ref discovery as old_id. + + update_request = command_list + "PACK" <binary data> + + command_list = PKT-LINE(command NUL cap_list LF) + *(command_pkt) + command_pkt = PKT-LINE(command LF) + cap_list = *(SP capability) SP + + command = create / delete / update + create = zero-id SP new_id SP name + delete = old_id SP zero-id SP name + update = old_id SP new_id SP name + +TODO: Document this further. + + +References +---------- + +link:http://www.ietf.org/rfc/rfc1738.txt[RFC 1738: Uniform Resource Locators (URL)] +link:http://www.ietf.org/rfc/rfc2616.txt[RFC 2616: Hypertext Transfer Protocol -- HTTP/1.1] +link:technical/pack-protocol.html +link:technical/protocol-capabilities.html diff --git a/Documentation/technical/index-format.txt b/Documentation/technical/index-format.txt index 0810251f5a..f352a9b22e 100644 --- a/Documentation/technical/index-format.txt +++ b/Documentation/technical/index-format.txt @@ -175,7 +175,7 @@ Git index format A conflict is represented in the index as a set of higher stage entries. When a conflict is resolved (e.g. with "git add path"), these higher - stage entries will be removed and a stage-0 entry with proper resoluton + stage entries will be removed and a stage-0 entry with proper resolution is added. When these higher stage entries are removed, they are saved in the diff --git a/Documentation/technical/pack-heuristics.txt b/Documentation/technical/pack-heuristics.txt index 8b7ae1c140..95a07db6e8 100644 --- a/Documentation/technical/pack-heuristics.txt +++ b/Documentation/technical/pack-heuristics.txt @@ -1,5 +1,5 @@ - Concerning Git's Packing Heuristics - =================================== +Concerning Git's Packing Heuristics +=================================== Oh, here's a really stupid question: @@ -366,12 +366,6 @@ been detailed! <linus> Yes, we always write out most recent first -For the other record: - - <pasky> njs`: http://pastebin.com/547965 - -The 'net never forgets, so that should be good until the end of time. - <njs`> And, yeah, I got the part about deeper-in-history stuff having worse IO characteristics, one sort of doesn't care. diff --git a/Documentation/technical/pack-protocol.txt b/Documentation/technical/pack-protocol.txt index f1a51edf47..c73b62f5e1 100644 --- a/Documentation/technical/pack-protocol.txt +++ b/Documentation/technical/pack-protocol.txt @@ -161,6 +161,7 @@ MUST peel the ref if it's an annotated tag. ---- advertised-refs = (no-refs / list-of-refs) + *shallow flush-pkt no-refs = PKT-LINE(zero-id SP "capabilities^{}" @@ -174,6 +175,8 @@ MUST peel the ref if it's an annotated tag. other-tip = obj-id SP refname LF other-peeled = obj-id SP refname "^{}" LF + shallow = PKT-LINE("shallow" SP obj-id) + capability-list = capability *(SP capability) capability = 1*(LC_ALPHA / DIGIT / "-" / "_") LC_ALPHA = %x61-7A @@ -228,8 +231,7 @@ obtained through ref discovery. The client MUST write all obj-ids which it only has shallow copies of (meaning that it does not have the parents of a commit) as 'shallow' lines so that the server is aware of the limitations of -the client's history. Clients MUST NOT mention an obj-id which -it does not know exists on the server. +the client's history. The client now sends the maximum commit history depth it wants for this transaction, which is the number of commits it wants from the @@ -462,7 +464,9 @@ contain all the objects that the server will need to complete the new references. ---- - update-request = command-list [pack-file] + update-request = *shallow command-list [pack-file] + + shallow = PKT-LINE("shallow" SP obj-id) command-list = PKT-LINE(command NUL capability-list LF) *PKT-LINE(command LF) diff --git a/Documentation/technical/protocol-capabilities.txt b/Documentation/technical/protocol-capabilities.txt index b15517fa06..e3e792476e 100644 --- a/Documentation/technical/protocol-capabilities.txt +++ b/Documentation/technical/protocol-capabilities.txt @@ -18,11 +18,12 @@ was sent. Server MUST NOT ignore capabilities that client requested and server advertised. As a consequence of these rules, server MUST NOT advertise capabilities it does not understand. -The 'report-status' and 'delete-refs' capabilities are sent and +The 'report-status', 'delete-refs', and 'quiet' capabilities are sent and recognized by the receive-pack (push to server) process. -The 'ofs-delta' capability is sent and recognized by both upload-pack -and receive-pack protocols. +The 'ofs-delta' and 'side-band-64k' capabilities are sent and recognized +by both upload-pack and receive-pack protocols. The 'agent' capability +may optionally be sent in both protocols. All other capabilities are only recognized by the upload-pack (fetch from server) process. @@ -71,14 +72,29 @@ interleaved with S-R-Q. thin-pack --------- -This capability means that the server can send a 'thin' pack, a pack -which does not contain base objects; if those base objects are available -on client side. Client requests 'thin-pack' capability when it -understands how to "thicken" it by adding required delta bases making -it self-contained. +A thin pack is one with deltas which reference base objects not +contained within the pack (but are known to exist at the receiving +end). This can reduce the network traffic significantly, but it +requires the receiving end to know how to "thicken" these packs by +adding the missing bases to the pack. -Client MUST NOT request 'thin-pack' capability if it cannot turn a thin -pack into a self-contained pack. +The upload-pack server advertises 'thin-pack' when it can generate +and send a thin pack. A client requests the 'thin-pack' capability +when it understands how to "thicken" it, notifying the server that +it can receive such a pack. A client MUST NOT request the +'thin-pack' capability if it cannot turn a thin pack into a +self-contained pack. + +Receive-pack, on the other hand, is assumed by default to be able to +handle thin packs, but can ask the client not to use the feature by +advertising the 'no-thin' capability. A client MUST NOT send a thin +pack if the server advertises the 'no-thin' capability. + +The reasons for this asymmetry are historical. The receive-pack +program did not exist until after the invention of thin packs, so +historically the reference implementation of receive-pack always +understood thin packs. Adding 'no-thin' later allowed receive-pack +to disable the feature in a backwards-compatible manner. side-band, side-band-64k @@ -123,6 +139,20 @@ Server can send, and client understand PACKv2 with delta referring to its base by position in pack rather than by an obj-id. That is, they can send/read OBJ_OFS_DELTA (aka type 6) in a packfile. +agent +----- + +The server may optionally send a capability of the form `agent=X` to +notify the client that the server is running version `X`. The client may +optionally return its own agent string by responding with an `agent=Y` +capability (but it MUST NOT do so if the server did not mention the +agent capability). The `X` and `Y` strings may contain any printable +ASCII characters except space (i.e., the byte range 32 < x < 127), and +are typically of the form "package/version" (e.g., "git/1.8.3.1"). The +agent strings are purely informative for statistics and debugging +purposes, and MUST NOT be used to programatically assume the presence +or absence of particular features. + shallow ------- @@ -168,7 +198,7 @@ of whether or not there are tags available. report-status ------------- -The upload-pack process can receive a 'report-status' capability, +The receive-pack process can receive a 'report-status' capability, which tells it that the client wants a report of what happened after a packfile upload and reference update. If the pushing client requests this capability, after unpacking and updating references the server @@ -185,3 +215,20 @@ it is capable of accepting a zero-id value as the target value of a reference update. It is not sent back by the client, it simply informs the client that it can be sent zero-id values to delete references. + +quiet +----- + +If the receive-pack server advertises the 'quiet' capability, it is +capable of silencing human-readable progress output which otherwise may +be shown when processing the received pack. A send-pack client should +respond with the 'quiet' capability to suppress server-side progress +reporting if the local progress reporting is also being suppressed +(e.g., via `push -q`, or if stderr does not go to a tty). + +allow-tip-sha1-in-want +---------------------- + +If the upload-pack server advertises this capability, fetch-pack may +send "want" lines with SHA-1s that exist at the server but are not +advertised by upload-pack. diff --git a/Documentation/technical/racy-git.txt b/Documentation/technical/racy-git.txt index 6dc82ca5a8..242a044db9 100644 --- a/Documentation/technical/racy-git.txt +++ b/Documentation/technical/racy-git.txt @@ -46,7 +46,7 @@ because in-core timestamps can have finer granularity than on-disk timestamps, resulting in meaningless changes when an inode is evicted from the inode cache. See commit 8ce13b0 of git://git.kernel.org/pub/scm/linux/kernel/git/tglx/history.git -([PATCH] Sync in core time granuality with filesystems, +([PATCH] Sync in core time granularity with filesystems, 2005-01-04). Racy Git @@ -135,9 +135,9 @@ them, and give the same timestamp to the index file: $ git ls-files | git update-index --stdin $ touch -r .datestamp .git/index -This will make all index entries racily clean. The linux-2.6 -project, for example, there are over 20,000 files in the working -tree. On my Athlon 64 X2 3800+, after the above: +This will make all index entries racily clean. The linux project, for +example, there are over 20,000 files in the working tree. On my +Athlon 64 X2 3800+, after the above: $ /usr/bin/time git diff-files 1.68user 0.54system 0:02.22elapsed 100%CPU (0avgtext+0avgdata 0maxresident)k |