diff options
Diffstat (limited to 'Documentation/technical')
26 files changed, 794 insertions, 693 deletions
diff --git a/Documentation/technical/api-argv-array.txt b/Documentation/technical/api-argv-array.txt index 1a797812fb..cfc063018c 100644 --- a/Documentation/technical/api-argv-array.txt +++ b/Documentation/technical/api-argv-array.txt @@ -46,6 +46,9 @@ Functions Format a string and push it onto the end of the array. This is a convenience wrapper combining `strbuf_addf` and `argv_array_push`. +`argv_array_pushv`:: + Push a null-terminated array of strings onto the end of the array. + `argv_array_pop`:: Remove the final element from the array. If there are no elements in the array, do nothing. @@ -53,3 +56,10 @@ Functions `argv_array_clear`:: Free all memory associated with the array and return it to the initial, empty state. + +`argv_array_detach`:: + Disconnect the `argv` member from the `argv_array` struct and + return it. The caller is responsible for freeing the memory used + by the array, and by the strings it references. After detaching, + the `argv_array` is in a reinitialized state and can be pushed + into again. diff --git a/Documentation/technical/api-config.txt b/Documentation/technical/api-config.txt index 0d8b99b368..20741f345e 100644 --- a/Documentation/technical/api-config.txt +++ b/Documentation/technical/api-config.txt @@ -63,13 +63,6 @@ parse for configuration, rather than looking in the usual files. Regular Specify whether include directives should be followed in parsed files. Regular `git_config` defaults to `1`. -There is a special version of `git_config` called `git_config_early`. -This version takes an additional parameter to specify the repository -config, instead of having it looked up via `git_path`. This is useful -early in a Git program before the repository has been found. Unless -you're working with early setup code, you probably don't want to use -this. - Reading Specific Files ---------------------- diff --git a/Documentation/technical/api-credentials.txt b/Documentation/technical/api-credentials.txt index c1b42a40d3..75368f26ca 100644 --- a/Documentation/technical/api-credentials.txt +++ b/Documentation/technical/api-credentials.txt @@ -243,12 +243,15 @@ appended to its command line, which is one of: The details of the credential will be provided on the helper's stdin stream. The exact format is the same as the input/output format of the `git credential` plumbing command (see the section `INPUT/OUTPUT -FORMAT` in linkgit:git-credential[7] for a detailed specification). +FORMAT` in linkgit:git-credential[1] for a detailed specification). For a `get` operation, the helper should produce a list of attributes on stdout in the same format. A helper is free to produce a subset, or even no values at all if it has nothing useful to provide. Any provided -attributes will overwrite those already known about by Git. +attributes will overwrite those already known about by Git. If a helper +outputs a `quit` attribute with a value of `true` or `1`, no further +helpers will be consulted, nor will the user be prompted (if no +credential has been provided, the operation will then fail). For a `store` or `erase` operation, the helper's output is ignored. If it fails to perform the requested operation, it may complain to @@ -265,4 +268,4 @@ See also linkgit:gitcredentials[7] -linkgit:git-config[5] (See configuration variables `credential.*`) +linkgit:git-config[1] (See configuration variables `credential.*`) diff --git a/Documentation/technical/api-error-handling.txt b/Documentation/technical/api-error-handling.txt new file mode 100644 index 0000000000..ceeedd485c --- /dev/null +++ b/Documentation/technical/api-error-handling.txt @@ -0,0 +1,75 @@ +Error reporting in git +====================== + +`die`, `usage`, `error`, and `warning` report errors of various +kinds. + +- `die` is for fatal application errors. It prints a message to + the user and exits with status 128. + +- `usage` is for errors in command line usage. After printing its + message, it exits with status 129. (See also `usage_with_options` + in the link:api-parse-options.html[parse-options API].) + +- `error` is for non-fatal library errors. It prints a message + to the user and returns -1 for convenience in signaling the error + to the caller. + +- `warning` is for reporting situations that probably should not + occur but which the user (and Git) can continue to work around + without running into too many problems. Like `error`, it + returns -1 after reporting the situation to the caller. + +Customizable error handlers +--------------------------- + +The default behavior of `die` and `error` is to write a message to +stderr and then exit or return as appropriate. This behavior can be +overridden using `set_die_routine` and `set_error_routine`. For +example, "git daemon" uses set_die_routine to write the reason `die` +was called to syslog before exiting. + +Library errors +-------------- + +Functions return a negative integer on error. Details beyond that +vary from function to function: + +- Some functions return -1 for all errors. Others return a more + specific value depending on how the caller might want to react + to the error. + +- Some functions report the error to stderr with `error`, + while others leave that for the caller to do. + +- errno is not meaningful on return from most functions (except + for thin wrappers for system calls). + +Check the function's API documentation to be sure. + +Caller-handled errors +--------------------- + +An increasing number of functions take a parameter 'struct strbuf *err'. +On error, such functions append a message about what went wrong to the +'err' strbuf. The message is meant to be complete enough to be passed +to `die` or `error` as-is. For example: + + if (ref_transaction_commit(transaction, &err)) + die("%s", err.buf); + +The 'err' parameter will be untouched if no error occurred, so multiple +function calls can be chained: + + t = ref_transaction_begin(&err); + if (!t || + ref_transaction_update(t, "HEAD", ..., &err) || + ret_transaction_commit(t, &err)) + die("%s", err.buf); + +The 'err' parameter must be a pointer to a valid strbuf. To silence +a message, pass a strbuf that is explicitly ignored: + + if (thing_that_can_fail_in_an_ignorable_way(..., &err)) + /* This failure is okay. */ + strbuf_reset(&err); diff --git a/Documentation/technical/api-gitattributes.txt b/Documentation/technical/api-gitattributes.txt index 2602668677..e7cbb7c13a 100644 --- a/Documentation/technical/api-gitattributes.txt +++ b/Documentation/technical/api-gitattributes.txt @@ -16,10 +16,15 @@ Data Structure of no interest to the calling programs. The name of the attribute can be retrieved by calling `git_attr_name()`. -`struct git_attr_check`:: +`struct attr_check_item`:: - This structure represents a set of attributes to check in a call - to `git_check_attr()` function, and receives the results. + This structure represents one attribute and its value. + +`struct attr_check`:: + + This structure represents a collection of `attr_check_item`. + It is passed to `git_check_attr()` function, specifying the + attributes to check, and receives their values. Attribute Values @@ -27,7 +32,7 @@ Attribute Values An attribute for a path can be in one of four states: Set, Unset, Unspecified or set to a string, and `.value` member of `struct -git_attr_check` records it. There are three macros to check these: +attr_check_item` records it. There are three macros to check these: `ATTR_TRUE()`:: @@ -48,49 +53,51 @@ value of the attribute for the path. Querying Specific Attributes ---------------------------- -* Prepare an array of `struct git_attr_check` to define the list of - attributes you would want to check. To populate this array, you would - need to define necessary attributes by calling `git_attr()` function. +* Prepare `struct attr_check` using attr_check_initl() + function, enumerating the names of attributes whose values you are + interested in, terminated with a NULL pointer. Alternatively, an + empty `struct attr_check` can be prepared by calling + `attr_check_alloc()` function and then attributes you want to + ask about can be added to it with `attr_check_append()` + function. * Call `git_check_attr()` to check the attributes for the path. -* Inspect `git_attr_check` structure to see how each of the attribute in - the array is defined for the path. +* Inspect `attr_check` structure to see how each of the + attribute in the array is defined for the path. Example ------- -To see how attributes "crlf" and "indent" are set for different paths. +To see how attributes "crlf" and "ident" are set for different paths. -. Prepare an array of `struct git_attr_check` with two elements (because - we are checking two attributes). Initialize their `attr` member with - pointers to `struct git_attr` obtained by calling `git_attr()`: +. Prepare a `struct attr_check` with two elements (because + we are checking two attributes): ------------ -static struct git_attr_check check[2]; +static struct attr_check *check; static void setup_check(void) { - if (check[0].attr) + if (check) return; /* already done */ - check[0].attr = git_attr("crlf"); - check[1].attr = git_attr("ident"); + check = attr_check_initl("crlf", "ident", NULL); } ------------ -. Call `git_check_attr()` with the prepared array of `struct git_attr_check`: +. Call `git_check_attr()` with the prepared `struct attr_check`: ------------ const char *path; setup_check(); - git_check_attr(path, ARRAY_SIZE(check), check); + git_check_attr(path, check); ------------ -. Act on `.value` member of the result, left in `check[]`: +. Act on `.value` member of the result, left in `check->items[]`: ------------ - const char *value = check[0].value; + const char *value = check->items[0].value; if (ATTR_TRUE(value)) { The attribute is Set, by listing only the name of the @@ -109,20 +116,39 @@ static void setup_check(void) } ------------ +To see how attributes in argv[] are set for different paths, only +the first step in the above would be different. + +------------ +static struct attr_check *check; +static void setup_check(const char **argv) +{ + check = attr_check_alloc(); + while (*argv) { + struct git_attr *attr = git_attr(*argv); + attr_check_append(check, attr); + argv++; + } +} +------------ + Querying All Attributes ----------------------- To get the values of all attributes associated with a file: -* Call `git_all_attrs()`, which returns an array of `git_attr_check` - structures. +* Prepare an empty `attr_check` structure by calling + `attr_check_alloc()`. + +* Call `git_all_attrs()`, which populates the `attr_check` + with the attributes attached to the path. -* Iterate over the `git_attr_check` array to examine the attribute - names and values. The name of the attribute described by a - `git_attr_check` object can be retrieved via - `git_attr_name(check[i].attr)`. (Please note that no items will be - returned for unset attributes, so `ATTR_UNSET()` will return false - for all returned `git_array_check` objects.) +* Iterate over the `attr_check.items[]` array to examine + the attribute names and values. The name of the attribute + described by a `attr_check.items[]` object can be retrieved via + `git_attr_name(check->items[i].attr)`. (Please note that no items + will be returned for unset attributes, so `ATTR_UNSET()` will return + false for all returned `attr_check.items[]` objects.) -* Free the `git_array_check` array. +* Free the `attr_check` struct by calling `attr_check_free()`. diff --git a/Documentation/technical/api-hashmap.txt b/Documentation/technical/api-hashmap.txt index ad7a5bddd2..ccc634bbd7 100644 --- a/Documentation/technical/api-hashmap.txt +++ b/Documentation/technical/api-hashmap.txt @@ -21,6 +21,9 @@ that the hashmap is initialized. It may also be useful for statistical purposes `cmpfn` stores the comparison function specified in `hashmap_init()`. In advanced scenarios, it may be useful to change this, e.g. to switch between case-sensitive and case-insensitive lookup. ++ +When `disallow_rehash` is set, automatic rehashes are prevented during inserts +and deletes. `struct hashmap_entry`:: @@ -57,6 +60,7 @@ Functions `unsigned int strihash(const char *buf)`:: `unsigned int memhash(const void *buf, size_t len)`:: `unsigned int memihash(const void *buf, size_t len)`:: +`unsigned int memihash_cont(unsigned int hash_seed, const void *buf, size_t len)`:: Ready-to-use hash functions for strings, using the FNV-1 algorithm (see http://www.isthe.com/chongo/tech/comp/fnv). @@ -65,6 +69,9 @@ Functions `memihash` operate on arbitrary-length memory. + `strihash` and `memihash` are case insensitive versions. ++ +`memihash_cont` is a variant of `memihash` that allows a computation to be +continued with another chunk of data. `unsigned int sha1hash(const unsigned char *sha1)`:: @@ -104,6 +111,11 @@ If `free_entries` is true, each hashmap_entry in the map is freed as well `entry` points to the entry to initialize. + `hash` is the hash code of the entry. ++ +The hashmap_entry structure does not hold references to external resources, +and it is safe to just discard it once you are done with it (i.e. if +your structure was allocated with xmalloc(), you can just free(3) it, +and if it is on stack, you can just let it go out of scope). `void *hashmap_get(const struct hashmap *map, const void *key, const void *keydata)`:: @@ -179,11 +191,28 @@ passed to `hashmap_cmp_fn` to decide whether the entry matches the key. + Returns the removed entry, or NULL if not found. +`void hashmap_disallow_rehash(struct hashmap *map, unsigned value)`:: + + Disallow/allow automatic rehashing of the hashmap during inserts + and deletes. ++ +This is useful if the caller knows that the hashmap will be accessed +by multiple threads. ++ +The caller is still responsible for any necessary locking; this simply +prevents unexpected rehashing. The caller is also responsible for properly +sizing the initial hashmap to ensure good performance. ++ +A call to allow rehashing does not force a rehash; that might happen +with the next insert or delete. + `void hashmap_iter_init(struct hashmap *map, struct hashmap_iter *iter)`:: `void *hashmap_iter_next(struct hashmap_iter *iter)`:: `void *hashmap_iter_first(struct hashmap *map, struct hashmap_iter *iter)`:: - Used to iterate over all entries of a hashmap. + Used to iterate over all entries of a hashmap. Note that it is + not safe to add or remove entries to the hashmap while + iterating. + `hashmap_iter_init` initializes a `hashmap_iter` structure. + diff --git a/Documentation/technical/api-in-core-index.txt b/Documentation/technical/api-in-core-index.txt deleted file mode 100644 index adbdbf5d75..0000000000 --- a/Documentation/technical/api-in-core-index.txt +++ /dev/null @@ -1,21 +0,0 @@ -in-core index API -================= - -Talk about <read-cache.c> and <cache-tree.c>, things like: - -* cache -> the_index macros -* read_index() -* write_index() -* ie_match_stat() and ie_modified(); how they are different and when to - use which. -* index_name_pos() -* remove_index_entry_at() -* remove_file_from_index() -* add_file_to_index() -* add_index_entry() -* refresh_index() -* discard_index() -* cache_tree_invalidate_path() -* cache_tree_update() - -(JC, Linus) diff --git a/Documentation/technical/api-lockfile.txt b/Documentation/technical/api-lockfile.txt deleted file mode 100644 index 93b5f23e4c..0000000000 --- a/Documentation/technical/api-lockfile.txt +++ /dev/null @@ -1,220 +0,0 @@ -lockfile API -============ - -The lockfile API serves two purposes: - -* Mutual exclusion and atomic file updates. When we want to change a - file, we create a lockfile `<filename>.lock`, write the new file - contents into it, and then rename the lockfile to its final - destination `<filename>`. We create the `<filename>.lock` file with - `O_CREAT|O_EXCL` so that we can notice and fail if somebody else has - already locked the file, then atomically rename the lockfile to its - final destination to commit the changes and unlock the file. - -* Automatic cruft removal. If the program exits after we lock a file - but before the changes have been committed, we want to make sure - that we remove the lockfile. This is done by remembering the - lockfiles we have created in a linked list and setting up an - `atexit(3)` handler and a signal handler that clean up the - lockfiles. This mechanism ensures that outstanding lockfiles are - cleaned up if the program exits (including when `die()` is called) - or if the program dies on a signal. - -Please note that lockfiles only block other writers. Readers do not -block, but they are guaranteed to see either the old contents of the -file or the new contents of the file (assuming that the filesystem -implements `rename(2)` atomically). - - -Calling sequence ----------------- - -The caller: - -* Allocates a `struct lock_file` either as a static variable or on the - heap, initialized to zeros. Once you use the structure to call the - `hold_lock_file_*` family of functions, it belongs to the lockfile - subsystem and its storage must remain valid throughout the life of - the program (i.e. you cannot use an on-stack variable to hold this - structure). - -* Attempts to create a lockfile by passing that variable and the path - of the final destination (e.g. `$GIT_DIR/index`) to - `hold_lock_file_for_update` or `hold_lock_file_for_append`. - -* Writes new content for the destination file by either: - - * writing to the file descriptor returned by the `hold_lock_file_*` - functions (also available via `lock->fd`). - - * calling `fdopen_lock_file` to get a `FILE` pointer for the open - file and writing to the file using stdio. - -When finished writing, the caller can: - -* Close the file descriptor and rename the lockfile to its final - destination by calling `commit_lock_file` or `commit_lock_file_to`. - -* Close the file descriptor and remove the lockfile by calling - `rollback_lock_file`. - -* Close the file descriptor without removing or renaming the lockfile - by calling `close_lock_file`, and later call `commit_lock_file`, - `commit_lock_file_to`, `rollback_lock_file`, or `reopen_lock_file`. - -Even after the lockfile is committed or rolled back, the `lock_file` -object must not be freed or altered by the caller. However, it may be -reused; just pass it to another call of `hold_lock_file_for_update` or -`hold_lock_file_for_append`. - -If the program exits before you have called one of `commit_lock_file`, -`commit_lock_file_to`, `rollback_lock_file`, or `close_lock_file`, an -`atexit(3)` handler will close and remove the lockfile, rolling back -any uncommitted changes. - -If you need to close the file descriptor you obtained from a -`hold_lock_file_*` function yourself, do so by calling -`close_lock_file`. You should never call `close(2)` or `fclose(3)` -yourself! Otherwise the `struct lock_file` structure would still think -that the file descriptor needs to be closed, and a commit or rollback -would result in duplicate calls to `close(2)`. Worse yet, if you close -and then later open another file descriptor for a completely different -purpose, then a commit or rollback might close that unrelated file -descriptor. - - -Error handling --------------- - -The `hold_lock_file_*` functions return a file descriptor on success -or -1 on failure (unless `LOCK_DIE_ON_ERROR` is used; see below). On -errors, `errno` describes the reason for failure. Errors can be -reported by passing `errno` to one of the following helper functions: - -unable_to_lock_message:: - - Append an appropriate error message to a `strbuf`. - -unable_to_lock_error:: - - Emit an appropriate error message using `error()`. - -unable_to_lock_die:: - - Emit an appropriate error message and `die()`. - -Similarly, `commit_lock_file`, `commit_lock_file_to`, and -`close_lock_file` return 0 on success. On failure they set `errno` -appropriately, do their best to roll back the lockfile, and return -1. - - -Flags ------ - -The following flags can be passed to `hold_lock_file_for_update` or -`hold_lock_file_for_append`: - -LOCK_NO_DEREF:: - - Usually symbolic links in the destination path are resolved - and the lockfile is created by adding ".lock" to the resolved - path. If `LOCK_NO_DEREF` is set, then the lockfile is created - by adding ".lock" to the path argument itself. This option is - used, for example, when locking a symbolic reference, which - for backwards-compatibility reasons can be a symbolic link - containing the name of the referred-to-reference. - -LOCK_DIE_ON_ERROR:: - - If a lock is already taken for the file, `die()` with an error - message. If this option is not specified, trying to lock a - file that is already locked returns -1 to the caller. - - -The functions -------------- - -hold_lock_file_for_update:: - - Take a pointer to `struct lock_file`, the path of the file to - be locked (e.g. `$GIT_DIR/index`) and a flags argument (see - above). Attempt to create a lockfile for the destination and - return the file descriptor for writing to the file. - -hold_lock_file_for_append:: - - Like `hold_lock_file_for_update`, but before returning copy - the existing contents of the file (if any) to the lockfile and - position its write pointer at the end of the file. - -fdopen_lock_file:: - - Associate a stdio stream with the lockfile. Return NULL - (*without* rolling back the lockfile) on error. The stream is - closed automatically when `close_lock_file` is called or when - the file is committed or rolled back. - -get_locked_file_path:: - - Return the path of the file that is locked by the specified - lock_file object. The caller must free the memory. - -commit_lock_file:: - - Take a pointer to the `struct lock_file` initialized with an - earlier call to `hold_lock_file_for_update` or - `hold_lock_file_for_append`, close the file descriptor, and - rename the lockfile to its final destination. Return 0 upon - success. On failure, roll back the lock file and return -1, - with `errno` set to the value from the failing call to - `close(2)` or `rename(2)`. It is a bug to call - `commit_lock_file` for a `lock_file` object that is not - currently locked. - -commit_lock_file_to:: - - Like `commit_lock_file()`, except that it takes an explicit - `path` argument to which the lockfile should be renamed. The - `path` must be on the same filesystem as the lock file. - -rollback_lock_file:: - - Take a pointer to the `struct lock_file` initialized with an - earlier call to `hold_lock_file_for_update` or - `hold_lock_file_for_append`, close the file descriptor and - remove the lockfile. It is a NOOP to call - `rollback_lock_file()` for a `lock_file` object that has - already been committed or rolled back. - -close_lock_file:: - - Take a pointer to the `struct lock_file` initialized with an - earlier call to `hold_lock_file_for_update` or - `hold_lock_file_for_append`. Close the file descriptor (and - the file pointer if it has been opened using - `fdopen_lock_file`). Return 0 upon success. On failure to - `close(2)`, return a negative value and roll back the lock - file. Usually `commit_lock_file`, `commit_lock_file_to`, or - `rollback_lock_file` should eventually be called if - `close_lock_file` succeeds. - -reopen_lock_file:: - - Re-open a lockfile that has been closed (using - `close_lock_file`) but not yet committed or rolled back. This - can be used to implement a sequence of operations like the - following: - - * Lock file. - - * Write new contents to lockfile, then `close_lock_file` to - cause the contents to be written to disk. - - * Pass the name of the lockfile to another program to allow it - (and nobody else) to inspect the contents you wrote, while - still holding the lock yourself. - - * `reopen_lock_file` to reopen the lockfile. Make further - updates to the contents. - - * `commit_lock_file` to make the final version permanent. diff --git a/Documentation/technical/api-parse-options.txt b/Documentation/technical/api-parse-options.txt index 1f2db31312..36768b479e 100644 --- a/Documentation/technical/api-parse-options.txt +++ b/Documentation/technical/api-parse-options.txt @@ -144,8 +144,12 @@ There are some macros to easily define options: `OPT_COUNTUP(short, long, &int_var, description)`:: Introduce a count-up option. - `int_var` is incremented on each use of `--option`, and - reset to zero with `--no-option`. + Each use of `--option` increments `int_var`, starting from zero + (even if initially negative), and `--no-option` resets it to + zero. To determine if `--option` or `--no-option` was encountered at + all, initialize `int_var` to a negative value, and if it is still + negative after parse_options(), then neither `--option` nor + `--no-option` was seen. `OPT_BIT(short, long, &int_var, description, mask)`:: Introduce a boolean option. @@ -164,10 +168,21 @@ There are some macros to easily define options: Introduce an option with string argument. The string argument is put into `str_var`. +`OPT_STRING_LIST(short, long, &struct string_list, arg_str, description)`:: + Introduce an option with string argument. + The string argument is stored as an element in `string_list`. + Use of `--no-option` will clear the list of preceding values. + `OPT_INTEGER(short, long, &int_var, description)`:: Introduce an option with integer argument. The integer is put into `int_var`. +`OPT_MAGNITUDE(short, long, &unsigned_long_var, description)`:: + Introduce an option with a size argument. The argument must be a + non-negative integer and may include a suffix of 'k', 'm' or 'g' to + scale the provided value by 1024, 1024^2 or 1024^3 respectively. + The scaled value is put into `unsigned_long_var`. + `OPT_DATE(short, long, &int_var, description)`:: Introduce an option with date argument, see `approxidate()`. The timestamp is put into `int_var`. @@ -212,6 +227,26 @@ There are some macros to easily define options: Use it to hide deprecated options that are still to be recognized and ignored silently. +`OPT_PASSTHRU(short, long, &char_var, arg_str, description, flags)`:: + Introduce an option that will be reconstructed into a char* string, + which must be initialized to NULL. This is useful when you need to + pass the command-line option to another command. Any previous value + will be overwritten, so this should only be used for options where + the last one specified on the command line wins. + +`OPT_PASSTHRU_ARGV(short, long, &argv_array_var, arg_str, description, flags)`:: + Introduce an option where all instances of it on the command-line will + be reconstructed into an argv_array. This is useful when you need to + pass the command-line option, which can be specified multiple times, + to another command. + +`OPT_CMDMODE(short, long, &int_var, description, enum_val)`:: + Define an "operation mode" option, only one of which in the same + group of "operating mode" options that share the same `int_var` + can be given by the user. `enum_val` is set to `int_var` when the + option is used, but an error is reported if other "operating mode" + option has already set its value to the same `int_var`. + The last element of the array must be `OPT_END()`. diff --git a/Documentation/technical/api-ref-iteration.txt b/Documentation/technical/api-ref-iteration.txt index 02adfd45d3..37379d8337 100644 --- a/Documentation/technical/api-ref-iteration.txt +++ b/Documentation/technical/api-ref-iteration.txt @@ -6,7 +6,7 @@ Iteration of refs is done by using an iterate function which will call a callback function for every ref. The callback function has this signature: - int handle_one_ref(const char *refname, const unsigned char *sha1, + int handle_one_ref(const char *refname, const struct object_id *oid, int flags, void *cb_data); There are different kinds of iterate functions which all take a diff --git a/Documentation/technical/api-remote.txt b/Documentation/technical/api-remote.txt index 5d245aa9d1..f10941b2e8 100644 --- a/Documentation/technical/api-remote.txt +++ b/Documentation/technical/api-remote.txt @@ -51,6 +51,10 @@ struct remote The proxy to use for curl (http, https, ftp, etc.) URLs. +`http_proxy_authmethod`:: + + The method used for authenticating against `http_proxy`. + struct remotes can be found by name with remote_get(), and iterated through with for_each_remote(). remote_get(NULL) will return the default remote, given the current branch and configuration. @@ -97,10 +101,6 @@ It contains: The name of the remote listed in the configuration. -`remote`:: - - The struct remote for that remote. - `merge_name`:: An array of the "merge" lines in the configuration. diff --git a/Documentation/technical/api-run-command.txt b/Documentation/technical/api-run-command.txt index a9fdb45b93..8bf3e37f53 100644 --- a/Documentation/technical/api-run-command.txt +++ b/Documentation/technical/api-run-command.txt @@ -46,6 +46,13 @@ Functions The argument dir corresponds the member .dir. The argument env corresponds to the member .env. +`child_process_clear`:: + + Release the memory associated with the struct child_process. + Most users of the run-command API don't need to call this + function explicitly because `start_command` invokes it on + failure and `finish_command` calls it automatically already. + The functions above do the following: . If a system call failed, errno is set and -1 is returned. A diagnostic diff --git a/Documentation/technical/api-setup.txt b/Documentation/technical/api-setup.txt index 540e455689..eb1fa9853e 100644 --- a/Documentation/technical/api-setup.txt +++ b/Documentation/technical/api-setup.txt @@ -27,8 +27,6 @@ parse_pathspec(). This function takes several arguments: - prefix and args come from cmd_* functions -get_pathspec() is obsolete and should never be used in new code. - parse_pathspec() helps catch unsupported features and reject them politely. At a lower level, different pathspec-related functions may not support the same set of features. Such pathspec-sensitive diff --git a/Documentation/technical/api-sha1-array.txt b/Documentation/technical/api-sha1-array.txt index 3e75497a37..dcc52943a5 100644 --- a/Documentation/technical/api-sha1-array.txt +++ b/Documentation/technical/api-sha1-array.txt @@ -38,16 +38,20 @@ Functions `sha1_array_for_each_unique`:: Efficiently iterate over each unique element of the list, executing the callback function for each one. If the array is - not sorted, this function has the side effect of sorting it. + not sorted, this function has the side effect of sorting it. If + the callback returns a non-zero value, the iteration ends + immediately and the callback's return is propagated; otherwise, + 0 is returned. Examples -------- ----------------------------------------- -void print_callback(const unsigned char sha1[20], +int print_callback(const unsigned char sha1[20], void *data) { printf("%s\n", sha1_to_hex(sha1)); + return 0; /* always continue */ } void some_func(void) diff --git a/Documentation/technical/api-strbuf.txt b/Documentation/technical/api-strbuf.txt deleted file mode 100644 index cca6543234..0000000000 --- a/Documentation/technical/api-strbuf.txt +++ /dev/null @@ -1,351 +0,0 @@ -strbuf API -========== - -strbuf's are meant to be used with all the usual C string and memory -APIs. Given that the length of the buffer is known, it's often better to -use the mem* functions than a str* one (memchr vs. strchr e.g.). -Though, one has to be careful about the fact that str* functions often -stop on NULs and that strbufs may have embedded NULs. - -A strbuf is NUL terminated for convenience, but no function in the -strbuf API actually relies on the string being free of NULs. - -strbufs have some invariants that are very important to keep in mind: - -. The `buf` member is never NULL, so it can be used in any usual C -string operations safely. strbuf's _have_ to be initialized either by -`strbuf_init()` or by `= STRBUF_INIT` before the invariants, though. -+ -Do *not* assume anything on what `buf` really is (e.g. if it is -allocated memory or not), use `strbuf_detach()` to unwrap a memory -buffer from its strbuf shell in a safe way. That is the sole supported -way. This will give you a malloced buffer that you can later `free()`. -+ -However, it is totally safe to modify anything in the string pointed by -the `buf` member, between the indices `0` and `len-1` (inclusive). - -. The `buf` member is a byte array that has at least `len + 1` bytes - allocated. The extra byte is used to store a `'\0'`, allowing the - `buf` member to be a valid C-string. Every strbuf function ensure this - invariant is preserved. -+ -NOTE: It is OK to "play" with the buffer directly if you work it this - way: -+ ----- -strbuf_grow(sb, SOME_SIZE); <1> -strbuf_setlen(sb, sb->len + SOME_OTHER_SIZE); ----- -<1> Here, the memory array starting at `sb->buf`, and of length -`strbuf_avail(sb)` is all yours, and you can be sure that -`strbuf_avail(sb)` is at least `SOME_SIZE`. -+ -NOTE: `SOME_OTHER_SIZE` must be smaller or equal to `strbuf_avail(sb)`. -+ -Doing so is safe, though if it has to be done in many places, adding the -missing API to the strbuf module is the way to go. -+ -WARNING: Do _not_ assume that the area that is yours is of size `alloc -- 1` even if it's true in the current implementation. Alloc is somehow a -"private" member that should not be messed with. Use `strbuf_avail()` -instead. - -Data structures ---------------- - -* `struct strbuf` - -This is the string buffer structure. The `len` member can be used to -determine the current length of the string, and `buf` member provides -access to the string itself. - -Functions ---------- - -* Life cycle - -`strbuf_init`:: - - Initialize the structure. The second parameter can be zero or a bigger - number to allocate memory, in case you want to prevent further reallocs. - -`strbuf_release`:: - - Release a string buffer and the memory it used. You should not use the - string buffer after using this function, unless you initialize it again. - -`strbuf_detach`:: - - Detach the string from the strbuf and returns it; you now own the - storage the string occupies and it is your responsibility from then on - to release it with `free(3)` when you are done with it. - -`strbuf_attach`:: - - Attach a string to a buffer. You should specify the string to attach, - the current length of the string and the amount of allocated memory. - The amount must be larger than the string length, because the string you - pass is supposed to be a NUL-terminated string. This string _must_ be - malloc()ed, and after attaching, the pointer cannot be relied upon - anymore, and neither be free()d directly. - -`strbuf_swap`:: - - Swap the contents of two string buffers. - -* Related to the size of the buffer - -`strbuf_avail`:: - - Determine the amount of allocated but unused memory. - -`strbuf_grow`:: - - Ensure that at least this amount of unused memory is available after - `len`. This is used when you know a typical size for what you will add - and want to avoid repetitive automatic resizing of the underlying buffer. - This is never a needed operation, but can be critical for performance in - some cases. - -`strbuf_setlen`:: - - Set the length of the buffer to a given value. This function does *not* - allocate new memory, so you should not perform a `strbuf_setlen()` to a - length that is larger than `len + strbuf_avail()`. `strbuf_setlen()` is - just meant as a 'please fix invariants from this strbuf I just messed - with'. - -`strbuf_reset`:: - - Empty the buffer by setting the size of it to zero. - -* Related to the contents of the buffer - -`strbuf_trim`:: - - Strip whitespace from the beginning and end of a string. - Equivalent to performing `strbuf_rtrim()` followed by `strbuf_ltrim()`. - -`strbuf_rtrim`:: - - Strip whitespace from the end of a string. - -`strbuf_ltrim`:: - - Strip whitespace from the beginning of a string. - -`strbuf_reencode`:: - - Replace the contents of the strbuf with a reencoded form. Returns -1 - on error, 0 on success. - -`strbuf_tolower`:: - - Lowercase each character in the buffer using `tolower`. - -`strbuf_cmp`:: - - Compare two buffers. Returns an integer less than, equal to, or greater - than zero if the first buffer is found, respectively, to be less than, - to match, or be greater than the second buffer. - -* Adding data to the buffer - -NOTE: All of the functions in this section will grow the buffer as necessary. -If they fail for some reason other than memory shortage and the buffer hadn't -been allocated before (i.e. the `struct strbuf` was set to `STRBUF_INIT`), -then they will free() it. - -`strbuf_addch`:: - - Add a single character to the buffer. - -`strbuf_addchars`:: - - Add a character the specified number of times to the buffer. - -`strbuf_insert`:: - - Insert data to the given position of the buffer. The remaining contents - will be shifted, not overwritten. - -`strbuf_remove`:: - - Remove given amount of data from a given position of the buffer. - -`strbuf_splice`:: - - Remove the bytes between `pos..pos+len` and replace it with the given - data. - -`strbuf_add_commented_lines`:: - - Add a NUL-terminated string to the buffer. Each line will be prepended - by a comment character and a blank. - -`strbuf_add`:: - - Add data of given length to the buffer. - -`strbuf_addstr`:: - -Add a NUL-terminated string to the buffer. -+ -NOTE: This function will *always* be implemented as an inline or a macro -that expands to: -+ ----- -strbuf_add(..., s, strlen(s)); ----- -+ -Meaning that this is efficient to write things like: -+ ----- -strbuf_addstr(sb, "immediate string"); ----- - -`strbuf_addbuf`:: - - Copy the contents of another buffer at the end of the current one. - -`strbuf_adddup`:: - - Copy part of the buffer from a given position till a given length to the - end of the buffer. - -`strbuf_expand`:: - - This function can be used to expand a format string containing - placeholders. To that end, it parses the string and calls the specified - function for every percent sign found. -+ -The callback function is given a pointer to the character after the `%` -and a pointer to the struct strbuf. It is expected to add the expanded -version of the placeholder to the strbuf, e.g. to add a newline -character if the letter `n` appears after a `%`. The function returns -the length of the placeholder recognized and `strbuf_expand()` skips -over it. -+ -The format `%%` is automatically expanded to a single `%` as a quoting -mechanism; callers do not need to handle the `%` placeholder themselves, -and the callback function will not be invoked for this placeholder. -+ -All other characters (non-percent and not skipped ones) are copied -verbatim to the strbuf. If the callback returned zero, meaning that the -placeholder is unknown, then the percent sign is copied, too. -+ -In order to facilitate caching and to make it possible to give -parameters to the callback, `strbuf_expand()` passes a context pointer, -which can be used by the programmer of the callback as she sees fit. - -`strbuf_expand_dict_cb`:: - - Used as callback for `strbuf_expand()`, expects an array of - struct strbuf_expand_dict_entry as context, i.e. pairs of - placeholder and replacement string. The array needs to be - terminated by an entry with placeholder set to NULL. - -`strbuf_addbuf_percentquote`:: - - Append the contents of one strbuf to another, quoting any - percent signs ("%") into double-percents ("%%") in the - destination. This is useful for literal data to be fed to either - strbuf_expand or to the *printf family of functions. - -`strbuf_humanise_bytes`:: - - Append the given byte size as a human-readable string (i.e. 12.23 KiB, - 3.50 MiB). - -`strbuf_addf`:: - - Add a formatted string to the buffer. - -`strbuf_commented_addf`:: - - Add a formatted string prepended by a comment character and a - blank to the buffer. - -`strbuf_fread`:: - - Read a given size of data from a FILE* pointer to the buffer. -+ -NOTE: The buffer is rewound if the read fails. If -1 is returned, -`errno` must be consulted, like you would do for `read(3)`. -`strbuf_read()`, `strbuf_read_file()` and `strbuf_getline()` has the -same behaviour as well. - -`strbuf_read`:: - - Read the contents of a given file descriptor. The third argument can be - used to give a hint about the file size, to avoid reallocs. - -`strbuf_read_file`:: - - Read the contents of a file, specified by its path. The third argument - can be used to give a hint about the file size, to avoid reallocs. - -`strbuf_readlink`:: - - Read the target of a symbolic link, specified by its path. The third - argument can be used to give a hint about the size, to avoid reallocs. - -`strbuf_getline`:: - - Read a line from a FILE *, overwriting the existing contents - of the strbuf. The second argument specifies the line - terminator character, typically `'\n'`. - Reading stops after the terminator or at EOF. The terminator - is removed from the buffer before returning. Returns 0 unless - there was nothing left before EOF, in which case it returns `EOF`. - -`strbuf_getwholeline`:: - - Like `strbuf_getline`, but keeps the trailing terminator (if - any) in the buffer. - -`strbuf_getwholeline_fd`:: - - Like `strbuf_getwholeline`, but operates on a file descriptor. - It reads one character at a time, so it is very slow. Do not - use it unless you need the correct position in the file - descriptor. - -`strbuf_getcwd`:: - - Set the buffer to the path of the current working directory. - -`strbuf_add_absolute_path` - - Add a path to a buffer, converting a relative path to an - absolute one in the process. Symbolic links are not - resolved. - -`stripspace`:: - - Strip whitespace from a buffer. The second parameter controls if - comments are considered contents to be removed or not. - -`strbuf_split_buf`:: -`strbuf_split_str`:: -`strbuf_split_max`:: -`strbuf_split`:: - - Split a string or strbuf into a list of strbufs at a specified - terminator character. The returned substrings include the - terminator characters. Some of these functions take a `max` - parameter, which, if positive, limits the output to that - number of substrings. - -`strbuf_list_free`:: - - Free a list of strbufs (for example, the return values of the - `strbuf_split()` functions). - -`launch_editor`:: - - Launch the user preferred editor to edit a file and fill the buffer - with the file's contents upon the user completing their editing. The - third argument can be used to set the environment which the editor is - run in. If the buffer is NULL the editor is launched as usual but the - file's contents are not read into the buffer upon completion. diff --git a/Documentation/technical/api-string-list.txt b/Documentation/technical/api-string-list.txt index d51a6579c8..c08402b12e 100644 --- a/Documentation/technical/api-string-list.txt +++ b/Documentation/technical/api-string-list.txt @@ -29,7 +29,7 @@ member (you need this if you add things later) and you should set the `unsorted_string_list_has_string` and get it from the list using `string_list_lookup` for sorted lists. -. Can sort an unsorted list using `sort_string_list`. +. Can sort an unsorted list using `string_list_sort`. . Can remove duplicate items from a sorted list using `string_list_remove_duplicates`. @@ -146,7 +146,7 @@ write `string_list_insert(...)->util = ...;`. ownership of a malloc()ed string to a `string_list` that has `strdup_string` set. -`sort_string_list`:: +`string_list_sort`:: Sort the list's entries by string value in `strcmp()` order. diff --git a/Documentation/technical/api-submodule-config.txt b/Documentation/technical/api-submodule-config.txt new file mode 100644 index 0000000000..3dce003fda --- /dev/null +++ b/Documentation/technical/api-submodule-config.txt @@ -0,0 +1,66 @@ +submodule config cache API +========================== + +The submodule config cache API allows to read submodule +configurations/information from specified revisions. Internally +information is lazily read into a cache that is used to avoid +unnecessary parsing of the same .gitmodule files. Lookups can be done by +submodule path or name. + +Usage +----- + +To initialize the cache with configurations from the worktree the caller +typically first calls `gitmodules_config()` to read values from the +worktree .gitmodules and then to overlay the local git config values +`parse_submodule_config_option()` from the config parsing +infrastructure. + +The caller can look up information about submodules by using the +`submodule_from_path()` or `submodule_from_name()` functions. They return +a `struct submodule` which contains the values. The API automatically +initializes and allocates the needed infrastructure on-demand. If the +caller does only want to lookup values from revisions the initialization +can be skipped. + +If the internal cache might grow too big or when the caller is done with +the API, all internally cached values can be freed with submodule_free(). + +Data Structures +--------------- + +`struct submodule`:: + + This structure is used to return the information about one + submodule for a certain revision. It is returned by the lookup + functions. + +Functions +--------- + +`void submodule_free()`:: + + Use these to free the internally cached values. + +`int parse_submodule_config_option(const char *var, const char *value)`:: + + Can be passed to the config parsing infrastructure to parse + local (worktree) submodule configurations. + +`const struct submodule *submodule_from_path(const unsigned char *treeish_name, const char *path)`:: + + Given a tree-ish in the superproject and a path, return the + submodule that is bound at the path in the named tree. + +`const struct submodule *submodule_from_name(const unsigned char *treeish_name, const char *name)`:: + + The same as above but lookup by name. + +Whenever a submodule configuration is parsed in `parse_submodule_config_option` +via e.g. `gitmodules_config()`, it will overwrite the null_sha1 entry. +So in the normal case, when HEAD:.gitmodules is parsed first and then overlayed +with the repository configuration, the null_sha1 entry contains the local +configuration of a submodule (e.g. consolidated values from local git +configuration and the .gitmodules file in the worktree). + +For an example usage see test-submodule-config.c. diff --git a/Documentation/technical/api-trace.txt b/Documentation/technical/api-trace.txt index 097a651d96..fadb5979c4 100644 --- a/Documentation/technical/api-trace.txt +++ b/Documentation/technical/api-trace.txt @@ -28,7 +28,7 @@ static struct trace_key trace_foo = TRACE_KEY_INIT(FOO); static void trace_print_foo(const char *message) { - trace_print_key(&trace_foo, message); + trace_printf_key(&trace_foo, "%s", message); } ------------ + @@ -95,3 +95,46 @@ for (;;) { } trace_performance(t, "frotz"); ------------ + +Bugs & Caveats +-------------- + +GIT_TRACE_* environment variables can be used to tell Git to show +trace output to its standard error stream. Git can often spawn a pager +internally to run its subcommand and send its standard output and +standard error to it. + +Because GIT_TRACE_PERFORMANCE trace is generated only at the very end +of the program with atexit(), which happens after the pager exits, it +would not work well if you send its log to the standard error output +and let Git spawn the pager at the same time. + +As a work around, you can for example use '--no-pager', or set +GIT_TRACE_PERFORMANCE to another file descriptor which is redirected +to stderr, or set GIT_TRACE_PERFORMANCE to a file specified by its +absolute path. + +For example instead of the following command which by default may not +print any performance information: + +------------ +GIT_TRACE_PERFORMANCE=2 git log -1 +------------ + +you may want to use: + +------------ +GIT_TRACE_PERFORMANCE=2 git --no-pager log -1 +------------ + +or: + +------------ +GIT_TRACE_PERFORMANCE=3 3>&2 git log -1 +------------ + +or: + +------------ +GIT_TRACE_PERFORMANCE=/path/to/log/file git log -1 +------------ diff --git a/Documentation/technical/http-protocol.txt b/Documentation/technical/http-protocol.txt index 229f845dfa..1c561bdd92 100644 --- a/Documentation/technical/http-protocol.txt +++ b/Documentation/technical/http-protocol.txt @@ -319,7 +319,8 @@ Servers SHOULD support all capabilities defined here. Clients MUST send at least one "want" command in the request body. Clients MUST NOT reference an id in a "want" command which did not appear in the response obtained through ref discovery unless the -server advertises capability `allow-tip-sha1-in-want`. +server advertises capability `allow-tip-sha1-in-want` or +`allow-reachable-sha1-in-want`. compute_request = want_list have_list diff --git a/Documentation/technical/index-format.txt b/Documentation/technical/index-format.txt index 1250b5ca8b..ade0b0c445 100644 --- a/Documentation/technical/index-format.txt +++ b/Documentation/technical/index-format.txt @@ -170,7 +170,7 @@ Git index format The entries are written out in the top-down, depth-first order. The first entry represents the root level of the repository, followed by the - first subtree---let's call this A---of the root level (with its name + first subtree--let's call this A--of the root level (with its name relative to the root level), followed by the first subtree of A (with its name relative to A), ... @@ -207,7 +207,7 @@ Git index format in a separate file. This extension records the changes to be made on top of that to produce the final index. - The signature for this extension is { 'l', 'i, 'n', 'k' }. + The signature for this extension is { 'l', 'i', 'n', 'k' }. The extension consists of: @@ -233,3 +233,65 @@ Git index format The remaining index entries after replaced ones will be added to the final index. These added entries are also sorted by entry name then stage. + +== Untracked cache + + Untracked cache saves the untracked file list and necessary data to + verify the cache. The signature for this extension is { 'U', 'N', + 'T', 'R' }. + + The extension starts with + + - A sequence of NUL-terminated strings, preceded by the size of the + sequence in variable width encoding. Each string describes the + environment where the cache can be used. + + - Stat data of $GIT_DIR/info/exclude. See "Index entry" section from + ctime field until "file size". + + - Stat data of core.excludesfile + + - 32-bit dir_flags (see struct dir_struct) + + - 160-bit SHA-1 of $GIT_DIR/info/exclude. Null SHA-1 means the file + does not exist. + + - 160-bit SHA-1 of core.excludesfile. Null SHA-1 means the file does + not exist. + + - NUL-terminated string of per-dir exclude file name. This usually + is ".gitignore". + + - The number of following directory blocks, variable width + encoding. If this number is zero, the extension ends here with a + following NUL. + + - A number of directory blocks in depth-first-search order, each + consists of + + - The number of untracked entries, variable width encoding. + + - The number of sub-directory blocks, variable width encoding. + + - The directory name terminated by NUL. + + - A number of untracked file/dir names terminated by NUL. + +The remaining data of each directory block is grouped by type: + + - An ewah bitmap, the n-th bit marks whether the n-th directory has + valid untracked cache entries. + + - An ewah bitmap, the n-th bit records "check-only" bit of + read_directory_recursive() for the n-th directory. + + - An ewah bitmap, the n-th bit indicates whether SHA-1 and stat data + is valid for the n-th directory and exists in the next data. + + - An array of stat data. The n-th data corresponds with the n-th + "one" bit in the previous ewah bitmap. + + - An array of SHA-1. The n-th SHA-1 corresponds with the n-th "one" bit + in the previous ewah bitmap. + + - One NUL. diff --git a/Documentation/technical/pack-protocol.txt b/Documentation/technical/pack-protocol.txt index 462e20645f..c59ac9936a 100644 --- a/Documentation/technical/pack-protocol.txt +++ b/Documentation/technical/pack-protocol.txt @@ -1,11 +1,11 @@ Packfile transfer protocols =========================== -Git supports transferring data in packfiles over the ssh://, git:// and +Git supports transferring data in packfiles over the ssh://, git://, http:// and file:// transports. There exist two sets of protocols, one for pushing data from a client to a server and another for fetching data from a -server to a client. All three transports (ssh, git, file) use the same -protocol to transfer data. +server to a client. The three transports (ssh, git, file) use the same +protocol to transfer data. http is documented in http-protocol.txt. The processes invoked in the canonical Git implementation are 'upload-pack' on the server side and 'fetch-pack' on the client side for fetching data; @@ -14,6 +14,14 @@ data. The protocol functions to have a server tell a client what is currently on the server, then for the two to negotiate the smallest amount of data to send in order to fully update one or the other. +pkt-line Format +--------------- + +The descriptions below build on the pkt-line format described in +protocol-common.txt. When the grammar indicate `PKT-LINE(...)`, unless +otherwise noted the usual pkt-line LF rules apply: the sender SHOULD +include a LF, but the receiver MUST NOT complain if it is not present. + Transports ---------- There are three transports over which the packfile protocol is @@ -143,9 +151,6 @@ with the object name that each reference currently points to. 003fe92df48743b7bc7d26bcaabfddde0a1e20cae47c refs/tags/v1.0^{} 0000 -Server SHOULD terminate each non-flush line using LF ("\n") terminator; -client MUST NOT complain if there is no terminator. - The returned response is a pkt-line stream describing each ref and its current value. The stream MUST be sorted by name according to the C locale ordering. @@ -165,15 +170,15 @@ MUST peel the ref if it's an annotated tag. flush-pkt no-refs = PKT-LINE(zero-id SP "capabilities^{}" - NUL capability-list LF) + NUL capability-list) list-of-refs = first-ref *other-ref first-ref = PKT-LINE(obj-id SP refname - NUL capability-list LF) + NUL capability-list) other-ref = PKT-LINE(other-tip / other-peeled) - other-tip = obj-id SP refname LF - other-peeled = obj-id SP refname "^{}" LF + other-tip = obj-id SP refname + other-peeled = obj-id SP refname "^{}" shallow = PKT-LINE("shallow" SP obj-id) @@ -214,10 +219,12 @@ out of what the server said it could do with the first 'want' line. shallow-line = PKT-LINE("shallow" SP obj-id) - depth-request = PKT-LINE("deepen" SP depth) + depth-request = PKT-LINE("deepen" SP depth) / + PKT-LINE("deepen-since" SP timestamp) / + PKT-LINE("deepen-not" SP ref) - first-want = PKT-LINE("want" SP obj-id SP capability-list LF) - additional-want = PKT-LINE("want" SP obj-id LF) + first-want = PKT-LINE("want" SP obj-id SP capability-list) + additional-want = PKT-LINE("want" SP obj-id) depth = 1*DIGIT ---- @@ -284,7 +291,7 @@ so that there is always a block of 32 "in-flight on the wire" at a time. compute-end have-list = *have-line - have-line = PKT-LINE("have" SP obj-id LF) + have-line = PKT-LINE("have" SP obj-id) compute-end = flush-pkt / PKT-LINE("done") ---- @@ -302,7 +309,7 @@ In multi_ack mode: ready to make a packfile, it will blindly ACK all 'have' obj-ids back to the client. - * the server will then send a 'NACK' and then wait for another response + * the server will then send a 'NAK' and then wait for another response from the client - either a 'done' or another list of 'have' lines. In multi_ack_detailed mode: @@ -348,10 +355,10 @@ Then the server will start sending its packfile data. ---- server-response = *ack_multi ack / nak - ack_multi = PKT-LINE("ACK" SP obj-id ack_status LF) + ack_multi = PKT-LINE("ACK" SP obj-id ack_status) ack_status = "continue" / "common" / "ready" - ack = PKT-LINE("ACK SP obj-id LF) - nak = PKT-LINE("NAK" LF) + ack = PKT-LINE("ACK" SP obj-id) + nak = PKT-LINE("NAK") ---- A simple clone may look like this (with no 'have' lines): @@ -449,7 +456,8 @@ The reference discovery phase is done nearly the same way as it is in the fetching protocol. Each reference obj-id and name on the server is sent in packet-line format to the client, followed by a flush-pkt. The only real difference is that the capability listing is different - the only -possible values are 'report-status', 'delete-refs' and 'ofs-delta'. +possible values are 'report-status', 'delete-refs', 'ofs-delta' and +'push-options'. Reference Update Request and Packfile Transfer ---------------------------------------------- @@ -460,17 +468,18 @@ that it wants to update, it sends a line listing the obj-id currently on the server, the obj-id the client would like to update it to and the name of the reference. -This list is followed by a flush-pkt and then the packfile that should -contain all the objects that the server will need to complete the new -references. +This list is followed by a flush-pkt. Then the push options are transmitted +one per packet followed by another flush-pkt. After that the packfile that +should contain all the objects that the server will need to complete the new +references will be sent. ---- - update-request = *shallow ( command-list | push-cert ) [pack-file] + update-request = *shallow ( command-list | push-cert ) [packfile] - shallow = PKT-LINE("shallow" SP obj-id LF) + shallow = PKT-LINE("shallow" SP obj-id) - command-list = PKT-LINE(command NUL capability-list LF) - *PKT-LINE(command LF) + command-list = PKT-LINE(command NUL capability-list) + *PKT-LINE(command) flush-pkt command = create / delete / update @@ -491,7 +500,7 @@ references. *PKT-LINE(gpg-signature-lines LF) PKT-LINE("push-cert-end" LF) - pack-file = "PACK" 28*(OCTET) + packfile = "PACK" 28*(OCTET) ---- If the receiving end does not support delete-refs, the sending end MUST @@ -502,11 +511,11 @@ MUST NOT send a push-cert command. When a push-cert command is sent, command-list MUST NOT be sent; the commands recorded in the push certificate is used instead. -The pack-file MUST NOT be sent if the only command used is 'delete'. +The packfile MUST NOT be sent if the only command used is 'delete'. -A pack-file MUST be sent if either create or update command is used, +A packfile MUST be sent if either create or update command is used, even if the server already has all the necessary objects. In this -case the client MUST send an empty pack-file. The only time this +case the client MUST send an empty packfile. The only time this is likely to happen is if the client is creating a new branch or a tag that points to an existing obj-id. @@ -521,7 +530,8 @@ Push Certificate A push certificate begins with a set of header lines. After the header and an empty line, the protocol commands follow, one per -line. +line. Note that the trailing LF in push-cert PKT-LINEs is _not_ +optional; it must be present. Currently, the following header fields are defined: @@ -560,12 +570,12 @@ update was successful, or 'ng [refname] [error]' if the update was not. 1*(command-status) flush-pkt - unpack-status = PKT-LINE("unpack" SP unpack-result LF) + unpack-status = PKT-LINE("unpack" SP unpack-result) unpack-result = "ok" / error-msg command-status = command-ok / command-fail - command-ok = PKT-LINE("ok" SP refname LF) - command-fail = PKT-LINE("ng" SP refname SP error-msg LF) + command-ok = PKT-LINE("ok" SP refname) + command-fail = PKT-LINE("ng" SP refname SP error-msg) error-msg = 1*(OCTECT) ; where not "ok" ---- diff --git a/Documentation/technical/protocol-capabilities.txt b/Documentation/technical/protocol-capabilities.txt index 6d5424c1bd..26dcc6f502 100644 --- a/Documentation/technical/protocol-capabilities.txt +++ b/Documentation/technical/protocol-capabilities.txt @@ -18,8 +18,9 @@ was sent. Server MUST NOT ignore capabilities that client requested and server advertised. As a consequence of these rules, server MUST NOT advertise capabilities it does not understand. -The 'report-status', 'delete-refs', 'quiet', and 'push-cert' capabilities -are sent and recognized by the receive-pack (push to server) process. +The 'atomic', 'report-status', 'delete-refs', 'quiet', and 'push-cert' +capabilities are sent and recognized by the receive-pack (push to server) +process. The 'ofs-delta' and 'side-band-64k' capabilities are sent and recognized by both upload-pack and receive-pack protocols. The 'agent' capability @@ -178,6 +179,31 @@ This capability adds "deepen", "shallow" and "unshallow" commands to the fetch-pack/upload-pack protocol so clients can request shallow clones. +deepen-since +------------ + +This capability adds "deepen-since" command to fetch-pack/upload-pack +protocol so the client can request shallow clones that are cut at a +specific time, instead of depth. Internally it's equivalent of doing +"rev-list --max-age=<timestamp>" on the server side. "deepen-since" +cannot be used with "deepen". + +deepen-not +---------- + +This capability adds "deepen-not" command to fetch-pack/upload-pack +protocol so the client can request shallow clones that are cut at a +specific revision, instead of depth. Internally it's equivalent of +doing "rev-list --not <rev>" on the server side. "deepen-not" +cannot be used with "deepen", but can be used with "deepen-since". + +deepen-relative +--------------- + +If this capability is requested by the client, the semantics of +"deepen" command is changed. The "depth" argument is the depth from +the current shallow boundary, instead of the depth from remote refs. + no-progress ----------- @@ -244,6 +270,23 @@ respond with the 'quiet' capability to suppress server-side progress reporting if the local progress reporting is also being suppressed (e.g., via `push -q`, or if stderr does not go to a tty). +atomic +------ + +If the server sends the 'atomic' capability it is capable of accepting +atomic pushes. If the pushing client requests this capability, the server +will update the refs in one atomic transaction. Either all refs are +updated or none. + +push-options +------------ + +If the server sends the 'push-options' capability it is able to accept +push options after the update commands have been sent, but before the +packfile is streamed. If the pushing client requests this capability, +the server will pass the options to the pre- and post- receive hooks +that process this push request. + allow-tip-sha1-in-want ---------------------- @@ -251,6 +294,13 @@ If the upload-pack server advertises this capability, fetch-pack may send "want" lines with SHA-1s that exist at the server but are not advertised by upload-pack. +allow-reachable-sha1-in-want +---------------------------- + +If the upload-pack server advertises this capability, fetch-pack may +send "want" lines with SHA-1s that exist at the server but are not +advertised by upload-pack. + push-cert=<nonce> ----------------- diff --git a/Documentation/technical/protocol-common.txt b/Documentation/technical/protocol-common.txt index 889985f707..ecedb34bba 100644 --- a/Documentation/technical/protocol-common.txt +++ b/Documentation/technical/protocol-common.txt @@ -62,11 +62,14 @@ A pkt-line MAY contain binary data, so implementors MUST ensure pkt-line parsing/formatting routines are 8-bit clean. A non-binary line SHOULD BE terminated by an LF, which if present -MUST be included in the total length. - -The maximum length of a pkt-line's data component is 65520 bytes. -Implementations MUST NOT send pkt-line whose length exceeds 65524 -(65520 bytes of payload + 4 bytes of length data). +MUST be included in the total length. Receivers MUST treat pkt-lines +with non-binary data the same whether or not they contain the trailing +LF (stripping the LF if present, and not complaining when it is +missing). + +The maximum length of a pkt-line's data component is 65516 bytes. +Implementations MUST NOT send pkt-line whose length exceeds 65520 +(65516 bytes of payload + 4 bytes of length data). Implementations SHOULD NOT send an empty pkt-line ("0004"). diff --git a/Documentation/technical/racy-git.txt b/Documentation/technical/racy-git.txt index 242a044db9..4a8be4d144 100644 --- a/Documentation/technical/racy-git.txt +++ b/Documentation/technical/racy-git.txt @@ -41,13 +41,17 @@ With a `USE_STDEV` compile-time option, `st_dev` is also compared, but this is not enabled by default because this member is not stable on network filesystems. With `USE_NSEC` compile-time option, `st_mtim.tv_nsec` and `st_ctim.tv_nsec` -members are also compared, but this is not enabled by default +members are also compared. On Linux, this is not enabled by default because in-core timestamps can have finer granularity than on-disk timestamps, resulting in meaningless changes when an inode is evicted from the inode cache. See commit 8ce13b0 of git://git.kernel.org/pub/scm/linux/kernel/git/tglx/history.git ([PATCH] Sync in core time granularity with filesystems, -2005-01-04). +2005-01-04). This patch is included in kernel 2.6.11 and newer, but +only fixes the issue for file systems with exactly 1 ns or 1 s +resolution. Other file systems are still broken in current Linux +kernels (e.g. CEPH, CIFS, NTFS, UDF), see +https://lkml.org/lkml/2015/6/9/714 Racy Git -------- diff --git a/Documentation/technical/repository-version.txt b/Documentation/technical/repository-version.txt new file mode 100644 index 0000000000..00ad37986e --- /dev/null +++ b/Documentation/technical/repository-version.txt @@ -0,0 +1,88 @@ +Git Repository Format Versions +============================== + +Every git repository is marked with a numeric version in the +`core.repositoryformatversion` key of its `config` file. This version +specifies the rules for operating on the on-disk repository data. An +implementation of git which does not understand a particular version +advertised by an on-disk repository MUST NOT operate on that repository; +doing so risks not only producing wrong results, but actually losing +data. + +Because of this rule, version bumps should be kept to an absolute +minimum. Instead, we generally prefer these strategies: + + - bumping format version numbers of individual data files (e.g., + index, packfiles, etc). This restricts the incompatibilities only to + those files. + + - introducing new data that gracefully degrades when used by older + clients (e.g., pack bitmap files are ignored by older clients, which + simply do not take advantage of the optimization they provide). + +A whole-repository format version bump should only be part of a change +that cannot be independently versioned. For instance, if one were to +change the reachability rules for objects, or the rules for locking +refs, that would require a bump of the repository format version. + +Note that this applies only to accessing the repository's disk contents +directly. An older client which understands only format `0` may still +connect via `git://` to a repository using format `1`, as long as the +server process understands format `1`. + +The preferred strategy for rolling out a version bump (whether whole +repository or for a single file) is to teach git to read the new format, +and allow writing the new format with a config switch or command line +option (for experimentation or for those who do not care about backwards +compatibility with older gits). Then after a long period to allow the +reading capability to become common, we may switch to writing the new +format by default. + +The currently defined format versions are: + +Version `0` +----------- + +This is the format defined by the initial version of git, including but +not limited to the format of the repository directory, the repository +configuration file, and the object and ref storage. Specifying the +complete behavior of git is beyond the scope of this document. + +Version `1` +----------- + +This format is identical to version `0`, with the following exceptions: + + 1. When reading the `core.repositoryformatversion` variable, a git + implementation which supports version 1 MUST also read any + configuration keys found in the `extensions` section of the + configuration file. + + 2. If a version-1 repository specifies any `extensions.*` keys that + the running git has not implemented, the operation MUST NOT + proceed. Similarly, if the value of any known key is not understood + by the implementation, the operation MUST NOT proceed. + +Note that if no extensions are specified in the config file, then +`core.repositoryformatversion` SHOULD be set to `0` (setting it to `1` +provides no benefit, and makes the repository incompatible with older +implementations of git). + +This document will serve as the master list for extensions. Any +implementation wishing to define a new extension should make a note of +it here, in order to claim the name. + +The defined extensions are: + +`noop` +~~~~~~ + +This extension does not change git's behavior at all. It is useful only +for testing format-1 compatibility. + +`preciousObjects` +~~~~~~~~~~~~~~~~~ + +When the config key `extensions.preciousObjects` is set to `true`, +objects in the repository MUST NOT be deleted (e.g., by `git-prune` or +`git repack -d`). diff --git a/Documentation/technical/signature-format.txt b/Documentation/technical/signature-format.txt new file mode 100644 index 0000000000..2c9406a56a --- /dev/null +++ b/Documentation/technical/signature-format.txt @@ -0,0 +1,186 @@ +Git signature format +==================== + +== Overview + +Git uses cryptographic signatures in various places, currently objects (tags, +commits, mergetags) and transactions (pushes). In every case, the command which +is about to create an object or transaction determines a payload from that, +calls gpg to obtain a detached signature for the payload (`gpg -bsa`) and +embeds the signature into the object or transaction. + +Signatures always begin with `-----BEGIN PGP SIGNATURE-----` +and end with `-----END PGP SIGNATURE-----`, unless gpg is told to +produce RFC1991 signatures which use `MESSAGE` instead of `SIGNATURE`. + +The signed payload and the way the signature is embedded depends +on the type of the object resp. transaction. + +== Tag signatures + +- created by: `git tag -s` +- payload: annotated tag object +- embedding: append the signature to the unsigned tag object +- example: tag `signedtag` with subject `signed tag` + +---- +object 04b871796dc0420f8e7561a895b52484b701d51a +type commit +tag signedtag +tagger C O Mitter <committer@example.com> 1465981006 +0000 + +signed tag + +signed tag message body +-----BEGIN PGP SIGNATURE----- +Version: GnuPG v1 + +iQEcBAABAgAGBQJXYRhOAAoJEGEJLoW3InGJklkIAIcnhL7RwEb/+QeX9enkXhxn +rxfdqrvWd1K80sl2TOt8Bg/NYwrUBw/RWJ+sg/hhHp4WtvE1HDGHlkEz3y11Lkuh +8tSxS3qKTxXUGozyPGuE90sJfExhZlW4knIQ1wt/yWqM+33E9pN4hzPqLwyrdods +q8FWEqPPUbSJXoMbRPw04S5jrLtZSsUWbRYjmJCHzlhSfFWW4eFd37uquIaLUBS0 +rkC3Jrx7420jkIpgFcTI2s60uhSQLzgcCwdA2ukSYIRnjg/zDkj8+3h/GaROJ72x +lZyI6HWixKJkWw8lE9aAOD9TmTW9sFJwcVAzmAuFX2kUreDUKMZduGcoRYGpD7E= +=jpXa +-----END PGP SIGNATURE----- +---- + +- verify with: `git verify-tag [-v]` or `git tag -v` + +---- +gpg: Signature made Wed Jun 15 10:56:46 2016 CEST using RSA key ID B7227189 +gpg: Good signature from "Eris Discordia <discord@example.net>" +gpg: WARNING: This key is not certified with a trusted signature! +gpg: There is no indication that the signature belongs to the owner. +Primary key fingerprint: D4BE 2231 1AD3 131E 5EDA 29A4 6109 2E85 B722 7189 +object 04b871796dc0420f8e7561a895b52484b701d51a +type commit +tag signedtag +tagger C O Mitter <committer@example.com> 1465981006 +0000 + +signed tag + +signed tag message body +---- + +== Commit signatures + +- created by: `git commit -S` +- payload: commit object +- embedding: header entry `gpgsig` + (content is preceded by a space) +- example: commit with subject `signed commit` + +---- +tree eebfed94e75e7760540d1485c740902590a00332 +parent 04b871796dc0420f8e7561a895b52484b701d51a +author A U Thor <author@example.com> 1465981137 +0000 +committer C O Mitter <committer@example.com> 1465981137 +0000 +gpgsig -----BEGIN PGP SIGNATURE----- + Version: GnuPG v1 + + iQEcBAABAgAGBQJXYRjRAAoJEGEJLoW3InGJ3IwIAIY4SA6GxY3BjL60YyvsJPh/ + HRCJwH+w7wt3Yc/9/bW2F+gF72kdHOOs2jfv+OZhq0q4OAN6fvVSczISY/82LpS7 + DVdMQj2/YcHDT4xrDNBnXnviDO9G7am/9OE77kEbXrp7QPxvhjkicHNwy2rEflAA + zn075rtEERDHr8nRYiDh8eVrefSO7D+bdQ7gv+7GsYMsd2auJWi1dHOSfTr9HIF4 + HJhWXT9d2f8W+diRYXGh4X0wYiGg6na/soXc+vdtDYBzIxanRqjg8jCAeo1eOTk1 + EdTwhcTZlI0x5pvJ3H0+4hA2jtldVtmPM4OTB0cTrEWBad7XV6YgiyuII73Ve3I= + =jKHM + -----END PGP SIGNATURE----- + +signed commit + +signed commit message body +---- + +- verify with: `git verify-commit [-v]` (or `git show --show-signature`) + +---- +gpg: Signature made Wed Jun 15 10:58:57 2016 CEST using RSA key ID B7227189 +gpg: Good signature from "Eris Discordia <discord@example.net>" +gpg: WARNING: This key is not certified with a trusted signature! +gpg: There is no indication that the signature belongs to the owner. +Primary key fingerprint: D4BE 2231 1AD3 131E 5EDA 29A4 6109 2E85 B722 7189 +tree eebfed94e75e7760540d1485c740902590a00332 +parent 04b871796dc0420f8e7561a895b52484b701d51a +author A U Thor <author@example.com> 1465981137 +0000 +committer C O Mitter <committer@example.com> 1465981137 +0000 + +signed commit + +signed commit message body +---- + +== Mergetag signatures + +- created by: `git merge` on signed tag +- payload/embedding: the whole signed tag object is embedded into + the (merge) commit object as header entry `mergetag` +- example: merge of the signed tag `signedtag` as above + +---- +tree c7b1cff039a93f3600a1d18b82d26688668c7dea +parent c33429be94b5f2d3ee9b0adad223f877f174b05d +parent 04b871796dc0420f8e7561a895b52484b701d51a +author A U Thor <author@example.com> 1465982009 +0000 +committer C O Mitter <committer@example.com> 1465982009 +0000 +mergetag object 04b871796dc0420f8e7561a895b52484b701d51a + type commit + tag signedtag + tagger C O Mitter <committer@example.com> 1465981006 +0000 + + signed tag + + signed tag message body + -----BEGIN PGP SIGNATURE----- + Version: GnuPG v1 + + iQEcBAABAgAGBQJXYRhOAAoJEGEJLoW3InGJklkIAIcnhL7RwEb/+QeX9enkXhxn + rxfdqrvWd1K80sl2TOt8Bg/NYwrUBw/RWJ+sg/hhHp4WtvE1HDGHlkEz3y11Lkuh + 8tSxS3qKTxXUGozyPGuE90sJfExhZlW4knIQ1wt/yWqM+33E9pN4hzPqLwyrdods + q8FWEqPPUbSJXoMbRPw04S5jrLtZSsUWbRYjmJCHzlhSfFWW4eFd37uquIaLUBS0 + rkC3Jrx7420jkIpgFcTI2s60uhSQLzgcCwdA2ukSYIRnjg/zDkj8+3h/GaROJ72x + lZyI6HWixKJkWw8lE9aAOD9TmTW9sFJwcVAzmAuFX2kUreDUKMZduGcoRYGpD7E= + =jpXa + -----END PGP SIGNATURE----- + +Merge tag 'signedtag' into downstream + +signed tag + +signed tag message body + +# gpg: Signature made Wed Jun 15 08:56:46 2016 UTC using RSA key ID B7227189 +# gpg: Good signature from "Eris Discordia <discord@example.net>" +# gpg: WARNING: This key is not certified with a trusted signature! +# gpg: There is no indication that the signature belongs to the owner. +# Primary key fingerprint: D4BE 2231 1AD3 131E 5EDA 29A4 6109 2E85 B722 7189 +---- + +- verify with: verification is embedded in merge commit message by default, + alternatively with `git show --show-signature`: + +---- +commit 9863f0c76ff78712b6800e199a46aa56afbcbd49 +merged tag 'signedtag' +gpg: Signature made Wed Jun 15 10:56:46 2016 CEST using RSA key ID B7227189 +gpg: Good signature from "Eris Discordia <discord@example.net>" +gpg: WARNING: This key is not certified with a trusted signature! +gpg: There is no indication that the signature belongs to the owner. +Primary key fingerprint: D4BE 2231 1AD3 131E 5EDA 29A4 6109 2E85 B722 7189 +Merge: c33429b 04b8717 +Author: A U Thor <author@example.com> +Date: Wed Jun 15 09:13:29 2016 +0000 + + Merge tag 'signedtag' into downstream + + signed tag + + signed tag message body + + # gpg: Signature made Wed Jun 15 08:56:46 2016 UTC using RSA key ID B7227189 + # gpg: Good signature from "Eris Discordia <discord@example.net>" + # gpg: WARNING: This key is not certified with a trusted signature! + # gpg: There is no indication that the signature belongs to the owner. + # Primary key fingerprint: D4BE 2231 1AD3 131E 5EDA 29A4 6109 2E85 B722 7189 +---- |