diff options
Diffstat (limited to 'Documentation/technical')
| -rw-r--r-- | Documentation/technical/api-argv-array.txt | 51 | ||||
| -rw-r--r-- | Documentation/technical/api-builtin.txt | 2 | ||||
| -rw-r--r-- | Documentation/technical/api-config.txt | 140 | ||||
| -rw-r--r-- | Documentation/technical/api-credentials.txt | 301 | ||||
| -rw-r--r-- | Documentation/technical/api-gitattributes.txt | 61 | ||||
| -rw-r--r-- | Documentation/technical/api-parse-options.txt | 58 | ||||
| -rw-r--r-- | Documentation/technical/api-ref-iteration.txt | 81 | ||||
| -rw-r--r-- | Documentation/technical/api-revision-walking.txt | 5 | ||||
| -rw-r--r-- | Documentation/technical/api-sha1-array.txt | 79 | ||||
| -rw-r--r-- | Documentation/technical/api-strbuf.txt | 18 | ||||
| -rw-r--r-- | Documentation/technical/api-string-list.txt | 14 | ||||
| -rw-r--r-- | Documentation/technical/index-format.txt | 18 | ||||
| -rw-r--r-- | Documentation/technical/pack-protocol.txt | 114 | ||||
| -rw-r--r-- | Documentation/technical/protocol-common.txt | 2 |
14 files changed, 863 insertions, 81 deletions
diff --git a/Documentation/technical/api-argv-array.txt b/Documentation/technical/api-argv-array.txt new file mode 100644 index 0000000000..1b7d8f140c --- /dev/null +++ b/Documentation/technical/api-argv-array.txt @@ -0,0 +1,51 @@ +argv-array API +============== + +The argv-array API allows one to dynamically build and store +NULL-terminated lists. An argv-array maintains the invariant that the +`argv` member always points to a non-NULL array, and that the array is +always NULL-terminated at the element pointed to by `argv[argc]`. This +makes the result suitable for passing to functions expecting to receive +argv from main(), or the link:api-run-command.html[run-command API]. + +The link:api-string-list.html[string-list API] is similar, but cannot be +used for these purposes; instead of storing a straight string pointer, +it contains an item structure with a `util` field that is not compatible +with the traditional argv interface. + +Each `argv_array` manages its own memory. Any strings pushed into the +array are duplicated, and all memory is freed by argv_array_clear(). + +Data Structures +--------------- + +`struct argv_array`:: + + A single array. This should be initialized by assignment from + `ARGV_ARRAY_INIT`, or by calling `argv_array_init`. The `argv` + member contains the actual array; the `argc` member contains the + number of elements in the array, not including the terminating + NULL. + +Functions +--------- + +`argv_array_init`:: + Initialize an array. This is no different than assigning from + `ARGV_ARRAY_INIT`. + +`argv_array_push`:: + Push a copy of a string onto the end of the array. + +`argv_array_pushl`:: + Push a list of strings onto the end of the array. The arguments + should be a list of `const char *` strings, terminated by a NULL + argument. + +`argv_array_pushf`:: + Format a string and push it onto the end of the array. This is a + convenience wrapper combining `strbuf_addf` and `argv_array_push`. + +`argv_array_clear`:: + Free all memory associated with the array and return it to the + initial, empty state. diff --git a/Documentation/technical/api-builtin.txt b/Documentation/technical/api-builtin.txt index 5cb2b0590a..b0cafe87be 100644 --- a/Documentation/technical/api-builtin.txt +++ b/Documentation/technical/api-builtin.txt @@ -49,6 +49,8 @@ Additionally, if `foo` is a new command, there are 3 more things to do: . Add an entry for `git-foo` to `command-list.txt`. +. Add an entry for `/git-foo` to `.gitignore`. + How a built-in is called ------------------------ diff --git a/Documentation/technical/api-config.txt b/Documentation/technical/api-config.txt new file mode 100644 index 0000000000..edf8dfb99b --- /dev/null +++ b/Documentation/technical/api-config.txt @@ -0,0 +1,140 @@ +config API +========== + +The config API gives callers a way to access git configuration files +(and files which have the same syntax). See linkgit:git-config[1] for a +discussion of the config file syntax. + +General Usage +------------- + +Config files are parsed linearly, and each variable found is passed to a +caller-provided callback function. The callback function is responsible +for any actions to be taken on the config option, and is free to ignore +some options. It is not uncommon for the configuration to be parsed +several times during the run of a git program, with different callbacks +picking out different variables useful to themselves. + +A config callback function takes three parameters: + +- the name of the parsed variable. This is in canonical "flat" form: the + section, subsection, and variable segments will be separated by dots, + and the section and variable segments will be all lowercase. E.g., + `core.ignorecase`, `diff.SomeType.textconv`. + +- the value of the found variable, as a string. If the variable had no + value specified, the value will be NULL (typically this means it + should be interpreted as boolean true). + +- a void pointer passed in by the caller of the config API; this can + contain callback-specific data + +A config callback should return 0 for success, or -1 if the variable +could not be parsed properly. + +Basic Config Querying +--------------------- + +Most programs will simply want to look up variables in all config files +that git knows about, using the normal precedence rules. To do this, +call `git_config` with a callback function and void data pointer. + +`git_config` will read all config sources in order of increasing +priority. Thus a callback should typically overwrite previously-seen +entries with new ones (e.g., if both the user-wide `~/.gitconfig` and +repo-specific `.git/config` contain `color.ui`, the config machinery +will first feed the user-wide one to the callback, and then the +repo-specific one; by overwriting, the higher-priority repo-specific +value is left at the end). + +The `git_config_with_options` function lets the caller examine config +while adjusting some of the default behavior of `git_config`. It should +almost never be used by "regular" git code that is looking up +configuration variables. It is intended for advanced callers like +`git-config`, which are intentionally tweaking the normal config-lookup +process. It takes two extra parameters: + +`filename`:: +If this parameter is non-NULL, it specifies the name of a file to +parse for configuration, rather than looking in the usual files. Regular +`git_config` defaults to `NULL`. + +`respect_includes`:: +Specify whether include directives should be followed in parsed files. +Regular `git_config` defaults to `1`. + +There is a special version of `git_config` called `git_config_early`. +This version takes an additional parameter to specify the repository +config, instead of having it looked up via `git_path`. This is useful +early in a git program before the repository has been found. Unless +you're working with early setup code, you probably don't want to use +this. + +Reading Specific Files +---------------------- + +To read a specific file in git-config format, use +`git_config_from_file`. This takes the same callback and data parameters +as `git_config`. + +Value Parsing Helpers +--------------------- + +To aid in parsing string values, the config API provides callbacks with +a number of helper functions, including: + +`git_config_int`:: +Parse the string to an integer, including unit factors. Dies on error; +otherwise, returns the parsed result. + +`git_config_ulong`:: +Identical to `git_config_int`, but for unsigned longs. + +`git_config_bool`:: +Parse a string into a boolean value, respecting keywords like "true" and +"false". Integer values are converted into true/false values (when they +are non-zero or zero, respectively). Other values cause a die(). If +parsing is successful, the return value is the result. + +`git_config_bool_or_int`:: +Same as `git_config_bool`, except that integers are returned as-is, and +an `is_bool` flag is unset. + +`git_config_maybe_bool`:: +Same as `git_config_bool`, except that it returns -1 on error rather +than dying. + +`git_config_string`:: +Allocates and copies the value string into the `dest` parameter; if no +string is given, prints an error message and returns -1. + +`git_config_pathname`:: +Similar to `git_config_string`, but expands `~` or `~user` into the +user's home directory when found at the beginning of the path. + +Include Directives +------------------ + +By default, the config parser does not respect include directives. +However, a caller can use the special `git_config_include` wrapper +callback to support them. To do so, you simply wrap your "real" callback +function and data pointer in a `struct config_include_data`, and pass +the wrapper to the regular config-reading functions. For example: + +------------------------------------------- +int read_file_with_include(const char *file, config_fn_t fn, void *data) +{ + struct config_include_data inc = CONFIG_INCLUDE_INIT; + inc.fn = fn; + inc.data = data; + return git_config_from_file(git_config_include, file, &inc); +} +------------------------------------------- + +`git_config` respects includes automatically. The lower-level +`git_config_from_file` does not. + +Writing Config Files +-------------------- + +TODO diff --git a/Documentation/technical/api-credentials.txt b/Documentation/technical/api-credentials.txt new file mode 100644 index 0000000000..adb6f0c896 --- /dev/null +++ b/Documentation/technical/api-credentials.txt @@ -0,0 +1,301 @@ +credentials API +=============== + +The credentials API provides an abstracted way of gathering username and +password credentials from the user (even though credentials in the wider +world can take many forms, in this document the word "credential" always +refers to a username and password pair). + +This document describes two interfaces: the C API that the credential +subsystem provides to the rest of git, and the protocol that git uses to +communicate with system-specific "credential helpers". If you are +writing git code that wants to look up or prompt for credentials, see +the section "C API" below. If you want to write your own helper, see +the section on "Credential Helpers" below. + +Typical setup +------------- + +------------ ++-----------------------+ +| git code (C) |--- to server requiring ---> +| | authentication +|.......................| +| C credential API |--- prompt ---> User ++-----------------------+ + ^ | + | pipe | + | v ++-----------------------+ +| git credential helper | ++-----------------------+ +------------ + +The git code (typically a remote-helper) will call the C API to obtain +credential data like a login/password pair (credential_fill). The +API will itself call a remote helper (e.g. "git credential-cache" or +"git credential-store") that may retrieve credential data from a +store. If the credential helper cannot find the information, the C API +will prompt the user. Then, the caller of the API takes care of +contacting the server, and does the actual authentication. + +C API +----- + +The credential C API is meant to be called by git code which needs to +acquire or store a credential. It is centered around an object +representing a single credential and provides three basic operations: +fill (acquire credentials by calling helpers and/or prompting the user), +approve (mark a credential as successfully used so that it can be stored +for later use), and reject (mark a credential as unsuccessful so that it +can be erased from any persistent storage). + +Data Structures +~~~~~~~~~~~~~~~ + +`struct credential`:: + + This struct represents a single username/password combination + along with any associated context. All string fields should be + heap-allocated (or NULL if they are not known or not applicable). + The meaning of the individual context fields is the same as + their counterparts in the helper protocol; see the section below + for a description of each field. ++ +The `helpers` member of the struct is a `string_list` of helpers. Each +string specifies an external helper which will be run, in order, to +either acquire or store credentials. See the section on credential +helpers below. This list is filled-in by the API functions +according to the corresponding configuration variables before +consulting helpers, so there usually is no need for a caller to +modify the helpers field at all. ++ +This struct should always be initialized with `CREDENTIAL_INIT` or +`credential_init`. + + +Functions +~~~~~~~~~ + +`credential_init`:: + + Initialize a credential structure, setting all fields to empty. + +`credential_clear`:: + + Free any resources associated with the credential structure, + returning it to a pristine initialized state. + +`credential_fill`:: + + Instruct the credential subsystem to fill the username and + password fields of the passed credential struct by first + consulting helpers, then asking the user. After this function + returns, the username and password fields of the credential are + guaranteed to be non-NULL. If an error occurs, the function will + die(). + +`credential_reject`:: + + Inform the credential subsystem that the provided credentials + have been rejected. This will cause the credential subsystem to + notify any helpers of the rejection (which allows them, for + example, to purge the invalid credentials from storage). It + will also free() the username and password fields of the + credential and set them to NULL (readying the credential for + another call to `credential_fill`). Any errors from helpers are + ignored. + +`credential_approve`:: + + Inform the credential subsystem that the provided credentials + were successfully used for authentication. This will cause the + credential subsystem to notify any helpers of the approval, so + that they may store the result to be used again. Any errors + from helpers are ignored. + +`credential_from_url`:: + + Parse a URL into broken-down credential fields. + +Example +~~~~~~~ + +The example below shows how the functions of the credential API could be +used to login to a fictitious "foo" service on a remote host: + +----------------------------------------------------------------------- +int foo_login(struct foo_connection *f) +{ + int status; + /* + * Create a credential with some context; we don't yet know the + * username or password. + */ + + struct credential c = CREDENTIAL_INIT; + c.protocol = xstrdup("foo"); + c.host = xstrdup(f->hostname); + + /* + * Fill in the username and password fields by contacting + * helpers and/or asking the user. The function will die if it + * fails. + */ + credential_fill(&c); + + /* + * Otherwise, we have a username and password. Try to use it. + */ + status = send_foo_login(f, c.username, c.password); + switch (status) { + case FOO_OK: + /* It worked. Store the credential for later use. */ + credential_accept(&c); + break; + case FOO_BAD_LOGIN: + /* Erase the credential from storage so we don't try it + * again. */ + credential_reject(&c); + break; + default: + /* + * Some other error occured. We don't know if the + * credential is good or bad, so report nothing to the + * credential subsystem. + */ + } + + /* Free any associated resources. */ + credential_clear(&c); + + return status; +} +----------------------------------------------------------------------- + + +Credential Helpers +------------------ + +Credential helpers are programs executed by git to fetch or save +credentials from and to long-term storage (where "long-term" is simply +longer than a single git process; e.g., credentials may be stored +in-memory for a few minutes, or indefinitely on disk). + +Each helper is specified by a single string in the configuration +variable `credential.helper` (and others, see linkgit:git-config[1]). +The string is transformed by git into a command to be executed using +these rules: + + 1. If the helper string begins with "!", it is considered a shell + snippet, and everything after the "!" becomes the command. + + 2. Otherwise, if the helper string begins with an absolute path, the + verbatim helper string becomes the command. + + 3. Otherwise, the string "git credential-" is prepended to the helper + string, and the result becomes the command. + +The resulting command then has an "operation" argument appended to it +(see below for details), and the result is executed by the shell. + +Here are some example specifications: + +---------------------------------------------------- +# run "git credential-foo" +foo + +# same as above, but pass an argument to the helper +foo --bar=baz + +# the arguments are parsed by the shell, so use shell +# quoting if necessary +foo --bar="whitespace arg" + +# you can also use an absolute path, which will not use the git wrapper +/path/to/my/helper --with-arguments + +# or you can specify your own shell snippet +!f() { echo "password=`cat $HOME/.secret`"; }; f +---------------------------------------------------- + +Generally speaking, rule (3) above is the simplest for users to specify. +Authors of credential helpers should make an effort to assist their +users by naming their program "git-credential-$NAME", and putting it in +the $PATH or $GIT_EXEC_PATH during installation, which will allow a user +to enable it with `git config credential.helper $NAME`. + +When a helper is executed, it will have one "operation" argument +appended to its command line, which is one of: + +`get`:: + + Return a matching credential, if any exists. + +`store`:: + + Store the credential, if applicable to the helper. + +`erase`:: + + Remove a matching credential, if any, from the helper's storage. + +The details of the credential will be provided on the helper's stdin +stream. The credential is split into a set of named attributes. +Attributes are provided to the helper, one per line. Each attribute is +specified by a key-value pair, separated by an `=` (equals) sign, +followed by a newline. The key may contain any bytes except `=`, +newline, or NUL. The value may contain any bytes except newline or NUL. +In both cases, all bytes are treated as-is (i.e., there is no quoting, +and one cannot transmit a value with newline or NUL in it). The list of +attributes is terminated by a blank line or end-of-file. + +Git will send the following attributes (but may not send all of +them for a given credential; for example, a `host` attribute makes no +sense when dealing with a non-network protocol): + +`protocol`:: + + The protocol over which the credential will be used (e.g., + `https`). + +`host`:: + + The remote hostname for a network credential. + +`path`:: + + The path with which the credential will be used. E.g., for + accessing a remote https repository, this will be the + repository's path on the server. + +`username`:: + + The credential's username, if we already have one (e.g., from a + URL, from the user, or from a previously run helper). + +`password`:: + + The credential's password, if we are asking it to be stored. + +For a `get` operation, the helper should produce a list of attributes +on stdout in the same format. A helper is free to produce a subset, or +even no values at all if it has nothing useful to provide. Any provided +attributes will overwrite those already known about by git. + +For a `store` or `erase` operation, the helper's output is ignored. +If it fails to perform the requested operation, it may complain to +stderr to inform the user. If it does not support the requested +operation (e.g., a read-only store), it should silently ignore the +request. + +If a helper receives any other operation, it should silently ignore the +request. This leaves room for future operations to be added (older +helpers will just ignore the new requests). + +See also +-------- + +linkgit:gitcredentials[7] + +linkgit:git-config[5] (See configuration variables `credential.*`) diff --git a/Documentation/technical/api-gitattributes.txt b/Documentation/technical/api-gitattributes.txt index 9d97eaa9de..ce363b6305 100644 --- a/Documentation/technical/api-gitattributes.txt +++ b/Documentation/technical/api-gitattributes.txt @@ -11,27 +11,15 @@ Data Structure `struct git_attr`:: An attribute is an opaque object that is identified by its name. - Pass the name and its length to `git_attr()` function to obtain - the object of this type. The internal representation of this - structure is of no interest to the calling programs. + Pass the name to `git_attr()` function to obtain the object of + this type. The internal representation of this structure is + of no interest to the calling programs. The name of the + attribute can be retrieved by calling `git_attr_name()`. `struct git_attr_check`:: This structure represents a set of attributes to check in a call - to `git_checkattr()` function, and receives the results. - - -Calling Sequence ----------------- - -* Prepare an array of `struct git_attr_check` to define the list of - attributes you would want to check. To populate this array, you would - need to define necessary attributes by calling `git_attr()` function. - -* Call git_checkattr() to check the attributes for the path. - -* Inspect `git_attr_check` structure to see how each of the attribute in - the array is defined for the path. + to `git_check_attr()` function, and receives the results. Attribute Values @@ -57,6 +45,19 @@ If none of the above returns true, `.value` member points at a string value of the attribute for the path. +Querying Specific Attributes +---------------------------- + +* Prepare an array of `struct git_attr_check` to define the list of + attributes you would want to check. To populate this array, you would + need to define necessary attributes by calling `git_attr()` function. + +* Call `git_check_attr()` to check the attributes for the path. + +* Inspect `git_attr_check` structure to see how each of the attribute in + the array is defined for the path. + + Example ------- @@ -72,18 +73,18 @@ static void setup_check(void) { if (check[0].attr) return; /* already done */ - check[0].attr = git_attr("crlf", 4); - check[1].attr = git_attr("ident", 5); + check[0].attr = git_attr("crlf"); + check[1].attr = git_attr("ident"); } ------------ -. Call `git_checkattr()` with the prepared array of `struct git_attr_check`: +. Call `git_check_attr()` with the prepared array of `struct git_attr_check`: ------------ const char *path; setup_check(); - git_checkattr(path, ARRAY_SIZE(check), check); + git_check_attr(path, ARRAY_SIZE(check), check); ------------ . Act on `.value` member of the result, left in `check[]`: @@ -108,4 +109,20 @@ static void setup_check(void) } ------------ -(JC) + +Querying All Attributes +----------------------- + +To get the values of all attributes associated with a file: + +* Call `git_all_attrs()`, which returns an array of `git_attr_check` + structures. + +* Iterate over the `git_attr_check` array to examine the attribute + names and values. The name of the attribute described by a + `git_attr_check` object can be retrieved via + `git_attr_name(check[i].attr)`. (Please note that no items will be + returned for unset attributes, so `ATTR_UNSET()` will return false + for all returned `git_array_check` objects.) + +* Free the `git_array_check` array. diff --git a/Documentation/technical/api-parse-options.txt b/Documentation/technical/api-parse-options.txt index f6a4a361bd..3062389404 100644 --- a/Documentation/technical/api-parse-options.txt +++ b/Documentation/technical/api-parse-options.txt @@ -21,7 +21,7 @@ that allow to change the behavior of a command. * There are basically two forms of options: 'Short options' consist of one dash (`-`) and one alphanumeric character. - 'Long options' begin with two dashes (`\--`) and some + 'Long options' begin with two dashes (`--`) and some alphanumeric characters. * Options are case-sensitive. @@ -31,7 +31,7 @@ The parse-options API allows: * 'sticked' and 'separate form' of options with arguments. `-oArg` is sticked, `-o Arg` is separate form. - `\--option=Arg` is sticked, `\--option Arg` is separate form. + `--option=Arg` is sticked, `--option Arg` is separate form. * Long options may be 'abbreviated', as long as the abbreviation is unambiguous. @@ -39,11 +39,12 @@ The parse-options API allows: * Short options may be bundled, e.g. `-a -b` can be specified as `-ab`. * Boolean long options can be 'negated' (or 'unset') by prepending - `no-`, e.g. `\--no-abbrev` instead of `\--abbrev`. + `no-`, e.g. `--no-abbrev` instead of `--abbrev`. Conversely, + options that begin with `no-` can be 'negated' by removing it. -* Options and non-option arguments can clearly be separated using the `\--` - option, e.g. `-a -b \--option \-- \--this-is-a-file` indicates that - `\--this-is-a-file` must not be processed as an option. +* Options and non-option arguments can clearly be separated using the `--` + option, e.g. `-a -b --option -- --this-is-a-file` indicates that + `--this-is-a-file` must not be processed as an option. Steps to parse options ---------------------- @@ -75,7 +76,7 @@ before the full parser, which in turn shows the full help message. Flags are the bitwise-or of: `PARSE_OPT_KEEP_DASHDASH`:: - Keep the `\--` that usually separates options from + Keep the `--` that usually separates options from non-option arguments. `PARSE_OPT_STOP_AT_NON_OPTION`:: @@ -113,31 +114,36 @@ say `static struct option builtin_add_options[]`. There are some macros to easily define options: `OPT__ABBREV(&int_var)`:: - Add `\--abbrev[=<n>]`. + Add `--abbrev[=<n>]`. `OPT__COLOR(&int_var, description)`:: - Add `\--color[=<when>]` and `--no-color`. + Add `--color[=<when>]` and `--no-color`. `OPT__DRY_RUN(&int_var, description)`:: - Add `-n, \--dry-run`. + Add `-n, --dry-run`. `OPT__FORCE(&int_var, description)`:: - Add `-f, \--force`. + Add `-f, --force`. `OPT__QUIET(&int_var, description)`:: - Add `-q, \--quiet`. + Add `-q, --quiet`. `OPT__VERBOSE(&int_var, description)`:: - Add `-v, \--verbose`. + Add `-v, --verbose`. `OPT_GROUP(description)`:: Start an option group. `description` is a short string that describes the group or an empty string. Start the description with an upper-case letter. -`OPT_BOOLEAN(short, long, &int_var, description)`:: - Introduce a boolean option. - `int_var` is incremented on each use. +`OPT_BOOL(short, long, &int_var, description)`:: + Introduce a boolean option. `int_var` is set to one with + `--option` and set to zero with `--no-option`. + +`OPT_COUNTUP(short, long, &int_var, description)`:: + Introduce a count-up option. + `int_var` is incremented on each use of `--option`, and + reset to zero with `--no-option`. `OPT_BIT(short, long, &int_var, description, mask)`:: Introduce a boolean option. @@ -148,8 +154,9 @@ There are some macros to easily define options: If used, `int_var` is bitwise-anded with the inverted `mask`. `OPT_SET_INT(short, long, &int_var, description, integer)`:: - Introduce a boolean option. - If used, set `int_var` to `integer`. + Introduce an integer option. + `int_var` is set to `integer` with `--option`, and + reset to zero with `--no-option`. `OPT_SET_PTR(short, long, &ptr_var, description, ptr)`:: Introduce a boolean option. @@ -198,16 +205,21 @@ There are some macros to easily define options: "auto", set `int_var` to 1 if stdout is a tty or a pager, 0 otherwise. +`OPT_NOOP_NOARG(short, long)`:: + Introduce an option that has no effect and takes no arguments. + Use it to hide deprecated options that are still to be recognized + and ignored silently. + The last element of the array must be `OPT_END()`. If not stated otherwise, interpret the arguments as follows: * `short` is a character for the short option - (e.g. `{apostrophe}e{apostrophe}` for `-e`, use `0` to omit), + (e.g. `'e'` for `-e`, use `0` to omit), * `long` is a string for the long option - (e.g. `"example"` for `\--example`, use `NULL` to omit), + (e.g. `"example"` for `--example`, use `NULL` to omit), * `int_var` is an integer variable, @@ -231,10 +243,10 @@ The function must be defined in this form: The callback mechanism is as follows: * Inside `func`, the only interesting member of the structure - given by `opt` is the void pointer `opt\->value`. - `\*opt\->value` will be the value that is saved into `var`, if you + given by `opt` is the void pointer `opt->value`. + `*opt->value` will be the value that is saved into `var`, if you use `OPT_CALLBACK()`. - For example, do `*(unsigned long *)opt\->value = 42;` to get 42 + For example, do `*(unsigned long *)opt->value = 42;` to get 42 into an `unsigned long` variable. * Return value `0` indicates success and non-zero return diff --git a/Documentation/technical/api-ref-iteration.txt b/Documentation/technical/api-ref-iteration.txt new file mode 100644 index 0000000000..dbbea95db7 --- /dev/null +++ b/Documentation/technical/api-ref-iteration.txt @@ -0,0 +1,81 @@ +ref iteration API +================= + + +Iteration of refs is done by using an iterate function which will call a +callback function for every ref. The callback function has this +signature: + + int handle_one_ref(const char *refname, const unsigned char *sha1, + int flags, void *cb_data); + +There are different kinds of iterate functions which all take a +callback of this type. The callback is then called for each found ref +until the callback returns nonzero. The returned value is then also +returned by the iterate function. + +Iteration functions +------------------- + +* `head_ref()` just iterates the head ref. + +* `for_each_ref()` iterates all refs. + +* `for_each_ref_in()` iterates all refs which have a defined prefix and + strips that prefix from the passed variable refname. + +* `for_each_tag_ref()`, `for_each_branch_ref()`, `for_each_remote_ref()`, + `for_each_replace_ref()` iterate refs from the respective area. + +* `for_each_glob_ref()` iterates all refs that match the specified glob + pattern. + +* `for_each_glob_ref_in()` the previous and `for_each_ref_in()` combined. + +* `head_ref_submodule()`, `for_each_ref_submodule()`, + `for_each_ref_in_submodule()`, `for_each_tag_ref_submodule()`, + `for_each_branch_ref_submodule()`, `for_each_remote_ref_submodule()` + do the same as the functions descibed above but for a specified + submodule. + +* `for_each_rawref()` can be used to learn about broken ref and symref. + +* `for_each_reflog()` iterates each reflog file. + +Submodules +---------- + +If you want to iterate the refs of a submodule you first need to add the +submodules object database. You can do this by a code-snippet like +this: + + const char *path = "path/to/submodule" + if (!add_submodule_odb(path)) + die("Error submodule '%s' not populated.", path); + +`add_submodule_odb()` will return an non-zero value on success. If you +do not do this you will get an error for each ref that it does not point +to a valid object. + +Note: As a side-effect of this you can not safely assume that all +objects you lookup are available in superproject. All submodule objects +will be available the same way as the superprojects objects. + +Example: +-------- + +---- +static int handle_remote_ref(const char *refname, + const unsigned char *sha1, int flags, void *cb_data) +{ + struct strbuf *output = cb_data; + strbuf_addf(output, "%s\n", refname); + return 0; +} + +... + + struct strbuf output = STRBUF_INIT; + for_each_remote_ref(handle_remote_ref, &output); + printf("%s", output.buf); +---- diff --git a/Documentation/technical/api-revision-walking.txt b/Documentation/technical/api-revision-walking.txt index 996da0503a..b7d0d9a8a7 100644 --- a/Documentation/technical/api-revision-walking.txt +++ b/Documentation/technical/api-revision-walking.txt @@ -56,6 +56,11 @@ function. returning a `struct commit *` each time you call it. The end of the revision list is indicated by returning a NULL pointer. +`reset_revision_walk`:: + + Reset the flags used by the revision walking api. You can use + this to do multiple sequencial revision walks. + Data structures --------------- diff --git a/Documentation/technical/api-sha1-array.txt b/Documentation/technical/api-sha1-array.txt new file mode 100644 index 0000000000..4a4bae8109 --- /dev/null +++ b/Documentation/technical/api-sha1-array.txt @@ -0,0 +1,79 @@ +sha1-array API +============== + +The sha1-array API provides storage and manipulation of sets of SHA1 +identifiers. The emphasis is on storage and processing efficiency, +making them suitable for large lists. Note that the ordering of items is +not preserved over some operations. + +Data Structures +--------------- + +`struct sha1_array`:: + + A single array of SHA1 hashes. This should be initialized by + assignment from `SHA1_ARRAY_INIT`. The `sha1` member contains + the actual data. The `nr` member contains the number of items in + the set. The `alloc` and `sorted` members are used internally, + and should not be needed by API callers. + +Functions +--------- + +`sha1_array_append`:: + Add an item to the set. The sha1 will be placed at the end of + the array (but note that some operations below may lose this + ordering). + +`sha1_array_sort`:: + Sort the elements in the array. + +`sha1_array_lookup`:: + Perform a binary search of the array for a specific sha1. + If found, returns the offset (in number of elements) of the + sha1. If not found, returns a negative integer. If the array is + not sorted, this function has the side effect of sorting it. + +`sha1_array_clear`:: + Free all memory associated with the array and return it to the + initial, empty state. + +`sha1_array_for_each_unique`:: + Efficiently iterate over each unique element of the list, + executing the callback function for each one. If the array is + not sorted, this function has the side effect of sorting it. + +Examples +-------- + +----------------------------------------- +void print_callback(const unsigned char sha1[20], + void *data) +{ + printf("%s\n", sha1_to_hex(sha1)); +} + +void some_func(void) +{ + struct sha1_array hashes = SHA1_ARRAY_INIT; + unsigned char sha1[20]; + + /* Read objects into our set */ + while (read_object_from_stdin(sha1)) + sha1_array_append(&hashes, sha1); + + /* Check if some objects are in our set */ + while (read_object_from_stdin(sha1)) { + if (sha1_array_lookup(&hashes, sha1) >= 0) + printf("it's in there!\n"); + + /* + * Print the unique set of objects. We could also have + * avoided adding duplicate objects in the first place, + * but we would end up re-sorting the array repeatedly. + * Instead, this will sort once and then skip duplicates + * in linear time. + */ + sha1_array_for_each_unique(&hashes, print_callback, NULL); +} +----------------------------------------- diff --git a/Documentation/technical/api-strbuf.txt b/Documentation/technical/api-strbuf.txt index afe2759951..95a8bf3846 100644 --- a/Documentation/technical/api-strbuf.txt +++ b/Documentation/technical/api-strbuf.txt @@ -255,8 +255,24 @@ same behaviour as well. `strbuf_getline`:: - Read a line from a FILE* pointer. The second argument specifies the line + Read a line from a FILE *, overwriting the existing contents + of the strbuf. The second argument specifies the line terminator character, typically `'\n'`. + Reading stops after the terminator or at EOF. The terminator + is removed from the buffer before returning. Returns 0 unless + there was nothing left before EOF, in which case it returns `EOF`. + +`strbuf_getwholeline`:: + + Like `strbuf_getline`, but keeps the trailing terminator (if + any) in the buffer. + +`strbuf_getwholeline_fd`:: + + Like `strbuf_getwholeline`, but operates on a file descriptor. + It reads one character at a time, so it is very slow. Do not + use it unless you need the correct position in the file + descriptor. `stripspace`:: diff --git a/Documentation/technical/api-string-list.txt b/Documentation/technical/api-string-list.txt index 3f575bdcff..5a0c14fceb 100644 --- a/Documentation/technical/api-string-list.txt +++ b/Documentation/technical/api-string-list.txt @@ -29,6 +29,9 @@ member (you need this if you add things later) and you should set the . Can sort an unsorted list using `sort_string_list`. +. Can remove individual items of an unsorted list using + `unsorted_string_list_delete_item`. + . Finally it should free the list using `string_list_clear`. Example: @@ -80,7 +83,9 @@ Functions Insert a new element to the string_list. The returned pointer can be handy if you want to write something to the `util` pointer of the - string_list_item containing the just added string. + string_list_item containing the just added string. If the given + string already exists the insertion will be skipped and the + pointer to the existing item returned. + Since this function uses xrealloc() (which die()s if it fails) if the list needs to grow, it is safe not to check the pointer. I.e. you may @@ -112,6 +117,13 @@ write `string_list_insert(...)->util = ...;`. The above two functions need to look through all items, as opposed to their counterpart for sorted lists, which performs a binary search. +`unsorted_string_list_delete_item`:: + + Remove an item from a string_list. The `string` pointer of the items + will be freed in case the `strdup_strings` member of the string_list + is set. The third parameter controls if the `util` pointer of the + items should be freed or not. + Data structures --------------- diff --git a/Documentation/technical/index-format.txt b/Documentation/technical/index-format.txt index 7b233ca196..9d25b30178 100644 --- a/Documentation/technical/index-format.txt +++ b/Documentation/technical/index-format.txt @@ -113,9 +113,22 @@ GIT index format are encoded in 7-bit ASCII and the encoding cannot contain a NUL byte (iow, this is a UNIX pathname). + (Version 4) In version 4, the entry path name is prefix-compressed + relative to the path name for the previous entry (the very first + entry is encoded as if the path name for the previous entry is an + empty string). At the beginning of an entry, an integer N in the + variable width encoding (the same encoding as the offset is encoded + for OFS_DELTA pack entries; see pack-format.txt) is stored, followed + by a NUL-terminated string S. Removing N bytes from the end of the + path name for the previous entry, and replacing it with the string S + yields the path name for this entry. + 1-8 nul bytes as necessary to pad the entry to a multiple of eight bytes while keeping the name NUL-terminated. + (Version 4) In version 4, the padding after the pathname does not + exist. + == Extensions === Cached tree @@ -147,8 +160,9 @@ GIT index format - 160-bit object name for the object that would result from writing this span of index as a tree. - An entry can be in an invalidated state and is represented by having -1 - in the entry_count field. + An entry can be in an invalidated state and is represented by having + -1 in the entry_count field. In this case, there is no object name + and the next entry starts immediately after the newline. The entries are written out in the top-down, depth-first order. The first entry represents the root level of the repository, followed by the diff --git a/Documentation/technical/pack-protocol.txt b/Documentation/technical/pack-protocol.txt index 369f91d3b9..49cdc571cd 100644 --- a/Documentation/technical/pack-protocol.txt +++ b/Documentation/technical/pack-protocol.txt @@ -60,6 +60,13 @@ process on the server side over the Git protocol is this: "0039git-upload-pack /schacon/gitbook.git\0host=example.com\0" | nc -v example.com 9418 +If the server refuses the request for some reasons, it could abort +gracefully with an error message. + +---- + error-line = PKT-LINE("ERR" SP explanation-text) +---- + SSH Transport ------------- @@ -179,34 +186,36 @@ and descriptions. Packfile Negotiation -------------------- -After reference and capabilities discovery, the client can decide -to terminate the connection by sending a flush-pkt, telling the -server it can now gracefully terminate (as happens with the ls-remote -command) or it can enter the negotiation phase, where the client and -server determine what the minimal packfile necessary for transport is. - -Once the client has the initial list of references that the server -has, as well as the list of capabilities, it will begin telling the -server what objects it wants and what objects it has, so the server -can make a packfile that only contains the objects that the client needs. -The client will also send a list of the capabilities it wants to be in -effect, out of what the server said it could do with the first 'want' line. +After reference and capabilities discovery, the client can decide to +terminate the connection by sending a flush-pkt, telling the server it can +now gracefully terminate, and disconnect, when it does not need any pack +data. This can happen with the ls-remote command, and also can happen when +the client already is up-to-date. + +Otherwise, it enters the negotiation phase, where the client and +server determine what the minimal packfile necessary for transport is, +by telling the server what objects it wants, its shallow objects +(if any), and the maximum commit depth it wants (if any). The client +will also send a list of the capabilities it wants to be in effect, +out of what the server said it could do with the first 'want' line. ---- upload-request = want-list - have-list - compute-end + *shallow-line + *1depth-request + flush-pkt want-list = first-want *additional-want - flush-pkt + + shallow-line = PKT_LINE("shallow" SP obj-id) + + depth-request = PKT_LINE("deepen" SP depth) first-want = PKT-LINE("want" SP obj-id SP capability-list LF) additional-want = PKT-LINE("want" SP obj-id LF) - have-list = *have-line - have-line = PKT-LINE("have" SP obj-id LF) - compute-end = flush-pkt / PKT-LINE("done") + depth = 1*DIGIT ---- Clients MUST send all the obj-ids it wants from the reference @@ -215,21 +224,64 @@ discovery phase as 'want' lines. Clients MUST send at least one obj-id in a 'want' command which did not appear in the response obtained through ref discovery. -If client is requesting a shallow clone, it will now send a 'deepen' -line with the depth it is requesting. +The client MUST write all obj-ids which it only has shallow copies +of (meaning that it does not have the parents of a commit) as +'shallow' lines so that the server is aware of the limitations of +the client's history. Clients MUST NOT mention an obj-id which +it does not know exists on the server. + +The client now sends the maximum commit history depth it wants for +this transaction, which is the number of commits it wants from the +tip of the history, if any, as a 'deepen' line. A depth of 0 is the +same as not making a depth request. The client does not want to receive +any commits beyond this depth, nor objects needed only to complete +those commits. Commits whose parents are not received as a result are +defined as shallow and marked as such in the server. This information +is sent back to the client in the next step. + +Once all the 'want's and 'shallow's (and optional 'deepen') are +transferred, clients MUST send a flush-pkt, to tell the server side +that it is done sending the list. + +Otherwise, if the client sent a positive depth request, the server +will determine which commits will and will not be shallow and +send this information to the client. If the client did not request +a positive depth, this step is skipped. -Once all the "want"s (and optional 'deepen') are transferred, -clients MUST send a flush-pkt. If the client has all the references -on the server, client flushes and disconnects. +---- + shallow-update = *shallow-line + *unshallow-line + flush-pkt + + shallow-line = PKT-LINE("shallow" SP obj-id) + + unshallow-line = PKT-LINE("unshallow" SP obj-id) +---- -TODO: shallow/unshallow response and document the deepen command in the ABNF. +If the client has requested a positive depth, the server will compute +the set of commits which are no deeper than the desired depth, starting +at the client's wants. The server writes 'shallow' lines for each +commit whose parents will not be sent as a result. The server writes +an 'unshallow' line for each commit which the client has indicated is +shallow, but is no longer shallow at the currently requested depth +(that is, its parents will now be sent). The server MUST NOT mark +as unshallow anything which the client has not indicated was shallow. Now the client will send a list of the obj-ids it has using 'have' -lines. In multi_ack mode, the canonical implementation will send up -to 32 of these at a time, then will send a flush-pkt. The canonical -implementation will skip ahead and send the next 32 immediately, -so that there is always a block of 32 "in-flight on the wire" at a -time. +lines, so the server can make a packfile that only contains the objects +that the client needs. In multi_ack mode, the canonical implementation +will send up to 32 of these at a time, then will send a flush-pkt. The +canonical implementation will skip ahead and send the next 32 immediately, +so that there is always a block of 32 "in-flight on the wire" at a time. + +---- + upload-haves = have-list + compute-end + + have-list = *have-line + have-line = PKT-LINE("have" SP obj-id LF) + compute-end = flush-pkt / PKT-LINE("done") +---- If the server reads 'have' lines, it then will respond by ACKing any of the obj-ids the client said it had that the server also has. The @@ -299,7 +351,7 @@ Then the server will start sending its packfile data. A simple clone may look like this (with no 'have' lines): ---- - C: 0054want 74730d410fcb6603ace96f1dc55ea6196122532d\0multi_ack \ + C: 0054want 74730d410fcb6603ace96f1dc55ea6196122532d multi_ack \ side-band-64k ofs-delta\n C: 0032want 7d1665144a3a975c05f1f43902ddaf084e784dbe\n C: 0032want 5a3f6be755bbb7deae50065988cbfa1ffa9ab68a\n @@ -315,7 +367,7 @@ A simple clone may look like this (with no 'have' lines): An incremental update (fetch) response might look like this: ---- - C: 0054want 74730d410fcb6603ace96f1dc55ea6196122532d\0multi_ack \ + C: 0054want 74730d410fcb6603ace96f1dc55ea6196122532d multi_ack \ side-band-64k ofs-delta\n C: 0032want 7d1665144a3a975c05f1f43902ddaf084e784dbe\n C: 0032want 5a3f6be755bbb7deae50065988cbfa1ffa9ab68a\n diff --git a/Documentation/technical/protocol-common.txt b/Documentation/technical/protocol-common.txt index d30a1b9510..fb7ff084f8 100644 --- a/Documentation/technical/protocol-common.txt +++ b/Documentation/technical/protocol-common.txt @@ -36,7 +36,7 @@ More specifically, they: . They cannot have ASCII control characters (i.e. bytes whose values are lower than \040, or \177 `DEL`), space, tilde `~`, - caret `{caret}`, colon `:`, question-mark `?`, asterisk `*`, + caret `^`, colon `:`, question-mark `?`, asterisk `*`, or open bracket `[` anywhere. . They cannot end with a slash `/` nor a dot `.`. |