diff options
Diffstat (limited to 'Documentation')
-rw-r--r-- | Documentation/Makefile | 2 | ||||
-rw-r--r-- | Documentation/RelNotes/2.19.0.txt | 31 | ||||
-rw-r--r-- | Documentation/SubmittingPatches | 4 | ||||
-rw-r--r-- | Documentation/config.txt | 32 | ||||
-rwxr-xr-x | Documentation/doc-diff | 39 | ||||
-rw-r--r-- | Documentation/fetch-options.txt | 15 | ||||
-rw-r--r-- | Documentation/git-branch.txt | 9 | ||||
-rw-r--r-- | Documentation/git-format-patch.txt | 29 | ||||
-rw-r--r-- | Documentation/git-interpret-trailers.txt | 10 | ||||
-rw-r--r-- | Documentation/git-multi-pack-index.txt | 56 | ||||
-rw-r--r-- | Documentation/git-pack-objects.txt | 97 | ||||
-rw-r--r-- | Documentation/git-push.txt | 57 | ||||
-rw-r--r-- | Documentation/git-repack.txt | 5 | ||||
-rw-r--r-- | Documentation/git-rerere.txt | 6 | ||||
-rw-r--r-- | Documentation/git-worktree.txt | 12 | ||||
-rw-r--r-- | Documentation/git.txt | 4 | ||||
-rw-r--r-- | Documentation/gitrevisions.txt | 7 | ||||
-rw-r--r-- | Documentation/pull-fetch-param.txt | 39 | ||||
-rw-r--r-- | Documentation/technical/multi-pack-index.txt | 109 | ||||
-rw-r--r-- | Documentation/technical/pack-format.txt | 77 | ||||
-rw-r--r-- | Documentation/technical/rerere.txt | 186 |
21 files changed, 770 insertions, 56 deletions
diff --git a/Documentation/Makefile b/Documentation/Makefile index a42dcfc745..95f6a321f2 100644 --- a/Documentation/Makefile +++ b/Documentation/Makefile @@ -344,7 +344,7 @@ $(OBSOLETE_HTML): %.html : %.txto asciidoc.conf mv $@+ $@ manpage-base-url.xsl: manpage-base-url.xsl.in - sed "s|@@MAN_BASE_URL@@|$(MAN_BASE_URL)|" $< > $@ + $(QUIET_GEN)sed "s|@@MAN_BASE_URL@@|$(MAN_BASE_URL)|" $< > $@ %.1 %.5 %.7 : %.xml manpage-base-url.xsl $(QUIET_XMLTO)$(RM) $@ && \ diff --git a/Documentation/RelNotes/2.19.0.txt b/Documentation/RelNotes/2.19.0.txt index 6a331587e4..a06ccf6e2a 100644 --- a/Documentation/RelNotes/2.19.0.txt +++ b/Documentation/RelNotes/2.19.0.txt @@ -126,9 +126,6 @@ Performance, Internal Implementation, Development Support etc. * Build and test procedure for netrc credential helper (in contrib/) has been updated. - * The conversion to pass "the_repository" and then "a_repository" - throughout the object access API continues. - * Remove unused function definitions and declarations from ewah bitmap subsystem. @@ -267,6 +264,11 @@ Performance, Internal Implementation, Development Support etc. config.txt file but are described in another file that is included by it, which has been corrected. + * The test linter code has learned that the end of here-doc mark + "EOF" can be quoted in a double-quote pair, not just in a + single-quote pair. + + Fixes since v2.18 ----------------- @@ -311,11 +313,6 @@ Fixes since v2.18 to the submodule was changed in the range of commits in the superproject, sometimes showing "(null)". This has been corrected. - * "git submodule" did not correctly adjust core.worktree setting that - indicates whether/where a submodule repository has its associated - working tree across various state transitions, which has been - corrected. - * Bugfix for "rebase -i" corner case regression. (merge a9279c6785 pw/rebase-i-keep-reword-after-conflict later to maint). @@ -556,6 +553,23 @@ Fixes since v2.18 which has been fixed. (merge 9bf5d4c4e2 nd/config-core-checkstat-doc later to maint). + * Recent "git rebase -i" update started to write bogusly formatted + author-script, with a matching broken reading code. These are + fixed. + + * Recent addition of "directory rename" heuristics to the + merge-recursive backend makes the command susceptible to false + positives and false negatives. In the context of "git am -3", + which does not know about surrounding unmodified paths and thus + cannot inform the merge machinery about the full trees involved, + this risk is particularly severe. As such, the heuristic is + disabled for "git am -3" to keep the machinery "more stupid but + predictable". + + * "git merge-base" in 2.19-rc1 has performance regression when the + (experimental) commit-graph feature is in use, which has been + mitigated. + * Code cleanup, docfix, build fix, etc. (merge aee9be2ebe sg/update-ref-stdin-cleanup later to maint). (merge 037714252f jc/clean-after-sanity-tests later to maint). @@ -598,3 +612,4 @@ Fixes since v2.18 (merge 79b04f9b60 sg/t3903-missing-fix later to maint). (merge 2745817028 sg/t3420-autostash-fix later to maint). (merge 7afb0d6777 sg/test-rebase-editor-fix later to maint). + (merge 6c6ce21baa es/freebsd-iconv-portability later to maint). diff --git a/Documentation/SubmittingPatches b/Documentation/SubmittingPatches index b44fd51f27..ec8b205145 100644 --- a/Documentation/SubmittingPatches +++ b/Documentation/SubmittingPatches @@ -80,7 +80,9 @@ GitHub-Travis CI hints section for details. Do not forget to update the documentation to describe the updated behavior and make sure that the resulting documentation set formats -well. It is currently a liberal mixture of US and UK English norms for +well (try the Documentation/doc-diff script). + +We currently have a liberal mixture of US and UK English norms for spelling and grammar, which is somewhat unfortunate. A huge patch that touches the files all over the place only to correct the inconsistency is not welcome, though. Potential clashes with other changes that can diff --git a/Documentation/config.txt b/Documentation/config.txt index eb66a11975..112041f407 100644 --- a/Documentation/config.txt +++ b/Documentation/config.txt @@ -937,6 +937,11 @@ core.useReplaceRefs:: option was given on the command line. See linkgit:git[1] and linkgit:git-replace[1] for more information. +core.multiPackIndex:: + Use the multi-pack-index file to track multiple packfiles using a + single index. See link:technical/multi-pack-index.html[the + multi-pack-index design document]. + core.sparseCheckout:: Enable "sparse checkout" feature. See section "Sparse checkout" in linkgit:git-read-tree[1] for more information. @@ -1154,6 +1159,14 @@ and by linkgit:git-worktree[1] when 'git worktree add' refers to a remote branch. This setting might be used for other checkout-like commands or functionality in the future. +checkout.optimizeNewBranch + Optimizes the performance of "git checkout -b <new_branch>" when + using sparse-checkout. When set to true, git will not update the + repo based on the current sparse-checkout settings. This means it + will not update the skip-worktree bit in the index nor add/remove + files in the working directory to reflect the current sparse checkout + settings nor will it show the local changes. + clean.requireForce:: A boolean to make git-clean do nothing unless given -f, -i or -n. Defaults to true. @@ -2671,6 +2684,21 @@ Note that changing the compression level will not automatically recompress all existing objects. You can force recompression by passing the -F option to linkgit:git-repack[1]. +pack.island:: + An extended regular expression configuring a set of delta + islands. See "DELTA ISLANDS" in linkgit:git-pack-objects[1] + for details. + +pack.islandCore:: + Specify an island name which gets to have its objects be + packed first. This creates a kind of pseudo-pack at the front + of one pack, so that the objects from the specified island are + hopefully faster to copy into any pack that should be served + to a user requesting these objects. In practice this means + that the island specified should likely correspond to what is + the most commonly cloned in the repo. See also "DELTA ISLANDS" + in linkgit:git-pack-objects[1]. + pack.deltaCacheSize:: The maximum memory in bytes used for caching deltas in linkgit:git-pack-objects[1] before writing them out to a pack. @@ -3205,6 +3233,10 @@ repack.packKeptObjects:: index is being written (either via `--write-bitmap-index` or `repack.writeBitmaps`). +repack.useDeltaIslands:: + If set to true, makes `git repack` act as if `--delta-islands` + was passed. Defaults to `false`. + repack.writeBitmaps:: When true, git will write a bitmap index when packing all objects to disk (e.g., when `git repack -a` is run). This diff --git a/Documentation/doc-diff b/Documentation/doc-diff index f483fe427c..dfd9418778 100755 --- a/Documentation/doc-diff +++ b/Documentation/doc-diff @@ -1,21 +1,34 @@ #!/bin/sh +# +# Build two documentation trees and diff the resulting formatted output. +# Compared to a source diff, this can reveal mistakes in the formatting. +# For example: +# +# ./doc-diff origin/master HEAD +# +# would show the differences introduced by a branch based on master. OPTIONS_SPEC="\ doc-diff [options] <from> <to> [-- <diff-options>] +doc-diff (-c|--clean) -- j=n parallel argument to pass to make f force rebuild; do not rely on cached results +c,clean cleanup temporary working files " SUBDIRECTORY_OK=1 . "$(git --exec-path)/git-sh-setup" parallel= force= +clean= while test $# -gt 0 do case "$1" in -j) parallel=$2; shift ;; + -c|--clean) + clean=t ;; -f) force=t ;; --) @@ -26,6 +39,17 @@ do shift done +cd_to_toplevel +tmp=Documentation/tmp-doc-diff + +if test -n "$clean" +then + test $# -eq 0 || usage + git worktree remove --force "$tmp/worktree" 2>/dev/null + rm -rf "$tmp" + exit 0 +fi + if test -z "$parallel" then parallel=$(getconf _NPROCESSORS_ONLN 2>/dev/null) @@ -42,9 +66,6 @@ to=$1; shift from_oid=$(git rev-parse --verify "$from") || exit 1 to_oid=$(git rev-parse --verify "$to") || exit 1 -cd_to_toplevel -tmp=Documentation/tmp-doc-diff - if test -n "$force" then rm -rf "$tmp" @@ -54,7 +75,7 @@ fi # results that don't differ between the two trees. if ! test -d "$tmp/worktree" then - git worktree add --detach "$tmp/worktree" "$from" && + git worktree add -f --detach "$tmp/worktree" "$from" && dots=$(echo "$tmp/worktree" | sed 's#[^/]*#..#g') && ln -s "$dots/config.mak" "$tmp/worktree/config.mak" fi @@ -69,12 +90,12 @@ generate_render_makefile () { printf '%s: %s\n' "$dst" "$src" printf '\t@echo >&2 " RENDER $(notdir $@)" && \\\n' printf '\tmkdir -p $(dir $@) && \\\n' - printf '\tMANWIDTH=80 man -l $< >$@+ && \\\n' + printf '\tMANWIDTH=80 man $< >$@+ && \\\n' printf '\tmv $@+ $@\n' done } -# render_tree <dirname> <committish> +# render_tree <committish_oid> render_tree () { # Skip install-man entirely if we already have an installed directory. # We can't rely on make here, since "install-man" unconditionally @@ -84,7 +105,7 @@ render_tree () { # through. if ! test -d "$tmp/installed/$1" then - git -C "$tmp/worktree" checkout "$2" && + git -C "$tmp/worktree" checkout --detach "$1" && make -j$parallel -C "$tmp/worktree" \ GIT_VERSION=omitted \ SOURCE_DATE_EPOCH=0 \ @@ -104,6 +125,6 @@ render_tree () { fi } -render_tree $from_oid "$from" && -render_tree $to_oid "$to" && +render_tree $from_oid && +render_tree $to_oid && git -C $tmp/rendered diff --no-index "$@" $from_oid $to_oid diff --git a/Documentation/fetch-options.txt b/Documentation/fetch-options.txt index 8bc36af4b1..fa0a3151b3 100644 --- a/Documentation/fetch-options.txt +++ b/Documentation/fetch-options.txt @@ -68,11 +68,16 @@ endif::git-pull[] -f:: --force:: - When 'git fetch' is used with `<rbranch>:<lbranch>` - refspec, it refuses to update the local branch - `<lbranch>` unless the remote branch `<rbranch>` it - fetches is a descendant of `<lbranch>`. This option - overrides that check. + When 'git fetch' is used with `<src>:<dst>` refspec it may + refuse to update the local branch as discussed +ifdef::git-pull[] + in the `<refspec>` part of the linkgit:git-fetch[1] + documentation. +endif::git-pull[] +ifndef::git-pull[] + in the `<refspec>` part below. +endif::git-pull[] + This option overrides that check. -k:: --keep:: diff --git a/Documentation/git-branch.txt b/Documentation/git-branch.txt index 9767b2b483..bf5316ffa9 100644 --- a/Documentation/git-branch.txt +++ b/Documentation/git-branch.txt @@ -14,7 +14,7 @@ SYNOPSIS [(--merged | --no-merged) [<commit>]] [--contains [<commit]] [--no-contains [<commit>]] [--points-at <object>] [--format=<format>] [<pattern>...] -'git branch' [--track | --no-track] [-l] [-f] <branchname> [<start-point>] +'git branch' [--track | --no-track] [-f] <branchname> [<start-point>] 'git branch' (--set-upstream-to=<upstream> | -u <upstream>) [<branchname>] 'git branch' --unset-upstream [<branchname>] 'git branch' (-m | -M) [<oldbranch>] <newbranch> @@ -100,8 +100,6 @@ OPTIONS The negated form `--no-create-reflog` only overrides an earlier `--create-reflog`, but currently does not negate the setting of `core.logAllRefUpdates`. -+ -The `-l` option is a deprecated synonym for `--create-reflog`. -f:: --force:: @@ -156,14 +154,11 @@ This option is only applicable in non-verbose mode. --all:: List both remote-tracking branches and local branches. +-l:: --list:: List branches. With optional `<pattern>...`, e.g. `git branch --list 'maint-*'`, list only the branches that match the pattern(s). -+ -This should not be confused with `git branch -l <branchname>`, -which creates a branch named `<branchname>` with a reflog. -See `--create-reflog` above for details. -v:: -vv:: diff --git a/Documentation/git-format-patch.txt b/Documentation/git-format-patch.txt index b41e1329a7..aba4c5febe 100644 --- a/Documentation/git-format-patch.txt +++ b/Documentation/git-format-patch.txt @@ -23,6 +23,8 @@ SYNOPSIS [(--reroll-count|-v) <n>] [--to=<email>] [--cc=<email>] [--[no-]cover-letter] [--quiet] [--notes[=<ref>]] + [--interdiff=<previous>] + [--range-diff=<previous> [--creation-factor=<percent>]] [--progress] [<common diff options>] [ <since> | <revision range> ] @@ -228,6 +230,33 @@ feeding the result to `git send-email`. containing the branch description, shortlog and the overall diffstat. You can fill in a description in the file before sending it out. +--interdiff=<previous>:: + As a reviewer aid, insert an interdiff into the cover letter, + or as commentary of the lone patch of a 1-patch series, showing + the differences between the previous version of the patch series and + the series currently being formatted. `previous` is a single revision + naming the tip of the previous series which shares a common base with + the series being formatted (for example `git format-patch + --cover-letter --interdiff=feature/v1 -3 feature/v2`). + +--range-diff=<previous>:: + As a reviewer aid, insert a range-diff (see linkgit:git-range-diff[1]) + into the cover letter, or as commentary of the lone patch of a + 1-patch series, showing the differences between the previous + version of the patch series and the series currently being formatted. + `previous` can be a single revision naming the tip of the previous + series if it shares a common base with the series being formatted (for + example `git format-patch --cover-letter --range-diff=feature/v1 -3 + feature/v2`), or a revision range if the two versions of the series are + disjoint (for example `git format-patch --cover-letter + --range-diff=feature/v1~3..feature/v1 -3 feature/v2`). + +--creation-factor=<percent>:: + Used with `--range-diff`, tweak the heuristic which matches up commits + between the previous and current series of patches by adjusting the + creation/deletion cost fudge factor. See linkgit:git-range-diff[1]) + for details. + --notes[=<ref>]:: Append the notes (see linkgit:git-notes[1]) for the commit after the three-dash line. diff --git a/Documentation/git-interpret-trailers.txt b/Documentation/git-interpret-trailers.txt index b8fafb1e8b..a5e8b36f62 100644 --- a/Documentation/git-interpret-trailers.txt +++ b/Documentation/git-interpret-trailers.txt @@ -56,8 +56,9 @@ least one Git-generated or user-configured trailer and consists of at least 25% trailers. The group must be preceded by one or more empty (or whitespace-only) lines. The group must either be at the end of the message or be the last -non-whitespace lines before a line that starts with '---'. Such three -minus signs start the patch part of the message. +non-whitespace lines before a line that starts with '---' (followed by a +space or the end of the line). Such three minus signs start the patch +part of the message. See also `--no-divider` below. When reading trailers, there can be whitespaces after the token, the separator and the value. There can also be whitespaces @@ -125,6 +126,11 @@ OPTIONS A convenience alias for `--only-trailers --only-input --unfold`. +--no-divider:: + Do not treat `---` as the end of the commit message. Use this + when you know your input contains just the commit message itself + (and not an email or the output of `git format-patch`). + CONFIGURATION VARIABLES ----------------------- diff --git a/Documentation/git-multi-pack-index.txt b/Documentation/git-multi-pack-index.txt new file mode 100644 index 0000000000..1f97e79912 --- /dev/null +++ b/Documentation/git-multi-pack-index.txt @@ -0,0 +1,56 @@ +git-multi-pack-index(1) +======================= + +NAME +---- +git-multi-pack-index - Write and verify multi-pack-indexes + + +SYNOPSIS +-------- +[verse] +'git multi-pack-index' [--object-dir=<dir>] <verb> + +DESCRIPTION +----------- +Write or verify a multi-pack-index (MIDX) file. + +OPTIONS +------- + +--object-dir=<dir>:: + Use given directory for the location of Git objects. We check + `<dir>/packs/multi-pack-index` for the current MIDX file, and + `<dir>/packs` for the pack-files to index. + +write:: + When given as the verb, write a new MIDX file to + `<dir>/packs/multi-pack-index`. + + +EXAMPLES +-------- + +* Write a MIDX file for the packfiles in the current .git folder. ++ +----------------------------------------------- +$ git multi-pack-index write +----------------------------------------------- + +* Write a MIDX file for the packfiles in an alternate object store. ++ +----------------------------------------------- +$ git multi-pack-index --object-dir <alt> write +----------------------------------------------- + + +SEE ALSO +-------- +See link:technical/multi-pack-index.html[The Multi-Pack-Index Design +Document] and link:technical/pack-format.html[The Multi-Pack-Index +Format] for more information on the multi-pack-index feature. + + +GIT +--- +Part of the linkgit:git[1] suite diff --git a/Documentation/git-pack-objects.txt b/Documentation/git-pack-objects.txt index d95b472d16..40c825c381 100644 --- a/Documentation/git-pack-objects.txt +++ b/Documentation/git-pack-objects.txt @@ -289,6 +289,103 @@ Unexpected missing object will raise an error. --unpack-unreachable:: Keep unreachable objects in loose form. This implies `--revs`. +--delta-islands:: + Restrict delta matches based on "islands". See DELTA ISLANDS + below. + + +DELTA ISLANDS +------------- + +When possible, `pack-objects` tries to reuse existing on-disk deltas to +avoid having to search for new ones on the fly. This is an important +optimization for serving fetches, because it means the server can avoid +inflating most objects at all and just send the bytes directly from +disk. This optimization can't work when an object is stored as a delta +against a base which the receiver does not have (and which we are not +already sending). In that case the server "breaks" the delta and has to +find a new one, which has a high CPU cost. Therefore it's important for +performance that the set of objects in on-disk delta relationships match +what a client would fetch. + +In a normal repository, this tends to work automatically. The objects +are mostly reachable from the branches and tags, and that's what clients +fetch. Any deltas we find on the server are likely to be between objects +the client has or will have. + +But in some repository setups, you may have several related but separate +groups of ref tips, with clients tending to fetch those groups +independently. For example, imagine that you are hosting several "forks" +of a repository in a single shared object store, and letting clients +view them as separate repositories through `GIT_NAMESPACE` or separate +repos using the alternates mechanism. A naive repack may find that the +optimal delta for an object is against a base that is only found in +another fork. But when a client fetches, they will not have the base +object, and we'll have to find a new delta on the fly. + +A similar situation may exist if you have many refs outside of +`refs/heads/` and `refs/tags/` that point to related objects (e.g., +`refs/pull` or `refs/changes` used by some hosting providers). By +default, clients fetch only heads and tags, and deltas against objects +found only in those other groups cannot be sent as-is. + +Delta islands solve this problem by allowing you to group your refs into +distinct "islands". Pack-objects computes which objects are reachable +from which islands, and refuses to make a delta from an object `A` +against a base which is not present in all of `A`'s islands. This +results in slightly larger packs (because we miss some delta +opportunities), but guarantees that a fetch of one island will not have +to recompute deltas on the fly due to crossing island boundaries. + +When repacking with delta islands the delta window tends to get +clogged with candidates that are forbidden by the config. Repacking +with a big --window helps (and doesn't take as long as it otherwise +might because we can reject some object pairs based on islands before +doing any computation on the content). + +Islands are configured via the `pack.island` option, which can be +specified multiple times. Each value is a left-anchored regular +expressions matching refnames. For example: + +------------------------------------------- +[pack] +island = refs/heads/ +island = refs/tags/ +------------------------------------------- + +puts heads and tags into an island (whose name is the empty string; see +below for more on naming). Any refs which do not match those regular +expressions (e.g., `refs/pull/123`) is not in any island. Any object +which is reachable only from `refs/pull/` (but not heads or tags) is +therefore not a candidate to be used as a base for `refs/heads/`. + +Refs are grouped into islands based on their "names", and two regexes +that produce the same name are considered to be in the same +island. The names are computed from the regexes by concatenating any +capture groups from the regex, with a '-' dash in between. (And if +there are no capture groups, then the name is the empty string, as in +the above example.) This allows you to create arbitrary numbers of +islands. Only up to 14 such capture groups are supported though. + +For example, imagine you store the refs for each fork in +`refs/virtual/ID`, where `ID` is a numeric identifier. You might then +configure: + +------------------------------------------- +[pack] +island = refs/virtual/([0-9]+)/heads/ +island = refs/virtual/([0-9]+)/tags/ +island = refs/virtual/([0-9]+)/(pull)/ +------------------------------------------- + +That puts the heads and tags for each fork in their own island (named +"1234" or similar), and the pull refs for each go into their own +"1234-pull". + +Note that we pick a single island for each regex to go into, using "last +one wins" ordering (which allows repo-specific config to take precedence +over user-wide config, and so forth). + SEE ALSO -------- linkgit:git-rev-list[1] diff --git a/Documentation/git-push.txt b/Documentation/git-push.txt index 55277a9781..f345bd30fc 100644 --- a/Documentation/git-push.txt +++ b/Documentation/git-push.txt @@ -74,22 +74,57 @@ without any `<refspec>` on the command line. Otherwise, missing `:<dst>` means to update the same ref as the `<src>`. + The object referenced by <src> is used to update the <dst> reference -on the remote side. By default this is only allowed if <dst> is not -a tag (annotated or lightweight), and then only if it can fast-forward -<dst>. By having the optional leading `+`, you can tell Git to update -the <dst> ref even if it is not allowed by default (e.g., it is not a -fast-forward.) This does *not* attempt to merge <src> into <dst>. See -EXAMPLES below for details. -+ -`tag <tag>` means the same as `refs/tags/<tag>:refs/tags/<tag>`. -+ -Pushing an empty <src> allows you to delete the <dst> ref from -the remote repository. +on the remote side. Whether this is allowed depends on where in +`refs/*` the <dst> reference lives as described in detail below, in +those sections "update" means any modifications except deletes, which +as noted after the next few sections are treated differently. ++ +The `refs/heads/*` namespace will only accept commit objects, and +updates only if they can be fast-forwarded. ++ +The `refs/tags/*` namespace will accept any kind of object (as +commits, trees and blobs can be tagged), and any updates to them will +be rejected. ++ +It's possible to push any type of object to any namespace outside of +`refs/{tags,heads}/*`. In the case of tags and commits, these will be +treated as if they were the commits inside `refs/heads/*` for the +purposes of whether the update is allowed. ++ +I.e. a fast-forward of commits and tags outside `refs/{tags,heads}/*` +is allowed, even in cases where what's being fast-forwarded is not a +commit, but a tag object which happens to point to a new commit which +is a fast-forward of the commit the last tag (or commit) it's +replacing. Replacing a tag with an entirely different tag is also +allowed, if it points to the same commit, as well as pushing a peeled +tag, i.e. pushing the commit that existing tag object points to, or a +new tag object which an existing commit points to. ++ +Tree and blob objects outside of `refs/{tags,heads}/*` will be treated +the same way as if they were inside `refs/tags/*`, any update of them +will be rejected. ++ +All of the rules described above about what's not allowed as an update +can be overridden by adding an the optional leading `+` to a refspec +(or using `--force` command line option). The only exception to this +is that no amount of forcing will make the `refs/heads/*` namespace +accept a non-commit object. Hooks and configuration can also override +or amend these rules, see e.g. `receive.denyNonFastForwards` in +linkgit:git-config[1] and`pre-receive` and `update` in +linkgit:githooks[5]. ++ +Pushing an empty <src> allows you to delete the <dst> ref from the +remote repository. Deletions are always accepted without a leading `+` +in the refspec (or `--force`), except when forbidden by configuration +or hooks. See `receive.denyDeletes` in linkgit:git-config[1] and +`pre-receive` and `update` in linkgit:githooks[5]. + The special refspec `:` (or `+:` to allow non-fast-forward updates) directs Git to push "matching" branches: for every branch that exists on the local side, the remote side is updated if a branch of the same name already exists on the remote side. ++ +`tag <tag>` means the same as `refs/tags/<tag>:refs/tags/<tag>`. --all:: Push all branches (i.e. refs under `refs/heads/`); cannot be diff --git a/Documentation/git-repack.txt b/Documentation/git-repack.txt index d056250968..aa0cc8bd44 100644 --- a/Documentation/git-repack.txt +++ b/Documentation/git-repack.txt @@ -160,6 +160,11 @@ depth is 4095. being removed. In addition, any unreachable loose objects will be packed (and their loose counterparts removed). +-i:: +--delta-islands:: + Pass the `--delta-islands` option to `git-pack-objects`, see + linkgit:git-pack-objects[1]. + Configuration ------------- diff --git a/Documentation/git-rerere.txt b/Documentation/git-rerere.txt index 031f31fa47..df310d2a58 100644 --- a/Documentation/git-rerere.txt +++ b/Documentation/git-rerere.txt @@ -211,6 +211,12 @@ would conflict the same way as the test merge you resolved earlier. 'git rerere' will be run by 'git rebase' to help you resolve this conflict. +[NOTE] 'git rerere' relies on the conflict markers in the file to +detect the conflict. If the file already contains lines that look the +same as lines with conflict markers, 'git rerere' may fail to record a +conflict resolution. To work around this, the `conflict-marker-size` +setting in linkgit:gitattributes[5] can be used. + GIT --- Part of the linkgit:git[1] suite diff --git a/Documentation/git-worktree.txt b/Documentation/git-worktree.txt index 29a5b7e252..e2ee9fc21b 100644 --- a/Documentation/git-worktree.txt +++ b/Documentation/git-worktree.txt @@ -120,8 +120,16 @@ OPTIONS --force:: By default, `add` refuses to create a new working tree when `<commit-ish>` is a branch name and is already checked out by - another working tree and `remove` refuses to remove an unclean - working tree. This option overrides these safeguards. + another working tree, or if `<path>` is already assigned to some + working tree but is missing (for instance, if `<path>` was deleted + manually). This option overrides these safeguards. To add a missing but + locked working tree path, specify `--force` twice. ++ +`move` refuses to move a locked working tree unless `--force` is specified +twice. ++ +`remove` refuses to remove an unclean working tree unless `--force` is used. +To remove a locked working tree, specify `--force` twice. -b <new-branch>:: -B <new-branch>:: diff --git a/Documentation/git.txt b/Documentation/git.txt index dba7f0c18e..74a9d7edb4 100644 --- a/Documentation/git.txt +++ b/Documentation/git.txt @@ -599,8 +599,8 @@ trace messages into this file descriptor. + Alternatively, if the variable is set to an absolute path (starting with a '/' character), Git will interpret this -as a file path and will try to write the trace messages -into it. +as a file path and will try to append the trace messages +to it. + Unsetting the variable, or setting it to empty, "0" or "false" (case insensitive) disables trace messages. diff --git a/Documentation/gitrevisions.txt b/Documentation/gitrevisions.txt index 1f6cceaefb..d407b7dee1 100644 --- a/Documentation/gitrevisions.txt +++ b/Documentation/gitrevisions.txt @@ -19,9 +19,10 @@ walk the revision graph (such as linkgit:git-log[1]), all commits which are reachable from that commit. For commands that walk the revision graph one can also specify a range of revisions explicitly. -In addition, some Git commands (such as linkgit:git-show[1]) also take -revision parameters which denote other objects than commits, e.g. blobs -("files") or trees ("directories of files"). +In addition, some Git commands (such as linkgit:git-show[1] and +linkgit:git-push[1]) can also take revision parameters which denote +other objects than commits, e.g. blobs ("files") or trees +("directories of files"). include::revisions.txt[] diff --git a/Documentation/pull-fetch-param.txt b/Documentation/pull-fetch-param.txt index f1fb08dc68..293c6b967d 100644 --- a/Documentation/pull-fetch-param.txt +++ b/Documentation/pull-fetch-param.txt @@ -33,11 +33,40 @@ name. it requests fetching everything up to the given tag. + The remote ref that matches <src> -is fetched, and if <dst> is not an empty string, the local -ref that matches it is fast-forwarded using <src>. -If the optional plus `+` is used, the local ref -is updated even if it does not result in a fast-forward -update. +is fetched, and if <dst> is not an empty string, an attempt +is made to update the local ref that matches it. ++ +Whether that update is allowed without `--force` depends on the ref +namespace it's being fetched to, the type of object being fetched, and +whether the update is considered to be a fast-forward. Generally, the +same rules apply for fetching as when pushing, see the `<refspec>...` +section of linkgit:git-push[1] for what those are. Exceptions to those +rules particular to 'git fetch' are noted below. ++ +Until Git version 2.20, and unlike when pushing with +linkgit:git-push[1], any updates to `refs/tags/*` would be accepted +without `+` in the refspec (or `--force`). The receiving promiscuously +considered all tag updates from a remote to be forced fetches. Since +Git version 2.20, fetching to update `refs/tags/*` work the same way +as when pushing. I.e. any updates will be rejected without `+` in the +refspec (or `--force`). ++ +Unlike when pushing with linkgit:git-push[1], any updates outside of +`refs/{tags,heads}/*` will be accepted without `+` in the refspec (or +`--force`), whether that's swapping e.g. a tree object for a blob, or +a commit for another commit that's doesn't have the previous commit as +an ancestor etc. ++ +Unlike when pushing with linkgit:git-push[1], there is no +configuration which'll amend these rules, and nothing like a +`pre-fetch` hook analogous to the `pre-receive` hook. ++ +As with pushing with linkgit:git-push[1], all of the rules described +above about what's not allowed as an update can be overridden by +adding an the optional leading `+` to a refspec (or using `--force` +command line option). The only exception to this is that no amount of +forcing will make the `refs/heads/*` namespace accept a non-commit +object. + [NOTE] When the remote branch you want to fetch is known to diff --git a/Documentation/technical/multi-pack-index.txt b/Documentation/technical/multi-pack-index.txt new file mode 100644 index 0000000000..d7e57639f7 --- /dev/null +++ b/Documentation/technical/multi-pack-index.txt @@ -0,0 +1,109 @@ +Multi-Pack-Index (MIDX) Design Notes +==================================== + +The Git object directory contains a 'pack' directory containing +packfiles (with suffix ".pack") and pack-indexes (with suffix +".idx"). The pack-indexes provide a way to lookup objects and +navigate to their offset within the pack, but these must come +in pairs with the packfiles. This pairing depends on the file +names, as the pack-index differs only in suffix with its pack- +file. While the pack-indexes provide fast lookup per packfile, +this performance degrades as the number of packfiles increases, +because abbreviations need to inspect every packfile and we are +more likely to have a miss on our most-recently-used packfile. +For some large repositories, repacking into a single packfile +is not feasible due to storage space or excessive repack times. + +The multi-pack-index (MIDX for short) stores a list of objects +and their offsets into multiple packfiles. It contains: + +- A list of packfile names. +- A sorted list of object IDs. +- A list of metadata for the ith object ID including: + - A value j referring to the jth packfile. + - An offset within the jth packfile for the object. +- If large offsets are required, we use another list of large + offsets similar to version 2 pack-indexes. + +Thus, we can provide O(log N) lookup time for any number +of packfiles. + +Design Details +-------------- + +- The MIDX is stored in a file named 'multi-pack-index' in the + .git/objects/pack directory. This could be stored in the pack + directory of an alternate. It refers only to packfiles in that + same directory. + +- The pack.multiIndex config setting must be on to consume MIDX files. + +- The file format includes parameters for the object ID hash + function, so a future change of hash algorithm does not require + a change in format. + +- The MIDX keeps only one record per object ID. If an object appears + in multiple packfiles, then the MIDX selects the copy in the most- + recently modified packfile. + +- If there exist packfiles in the pack directory not registered in + the MIDX, then those packfiles are loaded into the `packed_git` + list and `packed_git_mru` cache. + +- The pack-indexes (.idx files) remain in the pack directory so we + can delete the MIDX file, set core.midx to false, or downgrade + without any loss of information. + +- The MIDX file format uses a chunk-based approach (similar to the + commit-graph file) that allows optional data to be added. + +Future Work +----------- + +- Add a 'verify' subcommand to the 'git midx' builtin to verify the + contents of the multi-pack-index file match the offsets listed in + the corresponding pack-indexes. + +- The multi-pack-index allows many packfiles, especially in a context + where repacking is expensive (such as a very large repo), or + unexpected maintenance time is unacceptable (such as a high-demand + build machine). However, the multi-pack-index needs to be rewritten + in full every time. We can extend the format to be incremental, so + writes are fast. By storing a small "tip" multi-pack-index that + points to large "base" MIDX files, we can keep writes fast while + still reducing the number of binary searches required for object + lookups. + +- The reachability bitmap is currently paired directly with a single + packfile, using the pack-order as the object order to hopefully + compress the bitmaps well using run-length encoding. This could be + extended to pair a reachability bitmap with a multi-pack-index. If + the multi-pack-index is extended to store a "stable object order" + (a function Order(hash) = integer that is constant for a given hash, + even as the multi-pack-index is updated) then a reachability bitmap + could point to a multi-pack-index and be updated independently. + +- Packfiles can be marked as "special" using empty files that share + the initial name but replace ".pack" with ".keep" or ".promisor". + We can add an optional chunk of data to the multi-pack-index that + records flags of information about the packfiles. This allows new + states, such as 'repacked' or 'redeltified', that can help with + pack maintenance in a multi-pack environment. It may also be + helpful to organize packfiles by object type (commit, tree, blob, + etc.) and use this metadata to help that maintenance. + +- The partial clone feature records special "promisor" packs that + may point to objects that are not stored locally, but available + on request to a server. The multi-pack-index does not currently + track these promisor packs. + +Related Links +------------- +[0] https://bugs.chromium.org/p/git/issues/detail?id=6 + Chromium work item for: Multi-Pack Index (MIDX) + +[1] https://public-inbox.org/git/20180107181459.222909-1-dstolee@microsoft.com/ + An earlier RFC for the multi-pack-index feature + +[2] https://public-inbox.org/git/alpine.DEB.2.20.1803091557510.23109@alexmv-linux/ + Git Merge 2018 Contributor's summit notes (includes discussion of MIDX) diff --git a/Documentation/technical/pack-format.txt b/Documentation/technical/pack-format.txt index 70a99fd142..cab5bdd2ff 100644 --- a/Documentation/technical/pack-format.txt +++ b/Documentation/technical/pack-format.txt @@ -252,3 +252,80 @@ Pack file entry: <+ corresponding packfile. 20-byte SHA-1-checksum of all of the above. + +== multi-pack-index (MIDX) files have the following format: + +The multi-pack-index files refer to multiple pack-files and loose objects. + +In order to allow extensions that add extra data to the MIDX, we organize +the body into "chunks" and provide a lookup table at the beginning of the +body. The header includes certain length values, such as the number of packs, +the number of base MIDX files, hash lengths and types. + +All 4-byte numbers are in network order. + +HEADER: + + 4-byte signature: + The signature is: {'M', 'I', 'D', 'X'} + + 1-byte version number: + Git only writes or recognizes version 1. + + 1-byte Object Id Version + Git only writes or recognizes version 1 (SHA1). + + 1-byte number of "chunks" + + 1-byte number of base multi-pack-index files: + This value is currently always zero. + + 4-byte number of pack files + +CHUNK LOOKUP: + + (C + 1) * 12 bytes providing the chunk offsets: + First 4 bytes describe chunk id. Value 0 is a terminating label. + Other 8 bytes provide offset in current file for chunk to start. + (Chunks are provided in file-order, so you can infer the length + using the next chunk position if necessary.) + + The remaining data in the body is described one chunk at a time, and + these chunks may be given in any order. Chunks are required unless + otherwise specified. + +CHUNK DATA: + + Packfile Names (ID: {'P', 'N', 'A', 'M'}) + Stores the packfile names as concatenated, null-terminated strings. + Packfiles must be listed in lexicographic order for fast lookups by + name. This is the only chunk not guaranteed to be a multiple of four + bytes in length, so should be the last chunk for alignment reasons. + + OID Fanout (ID: {'O', 'I', 'D', 'F'}) + The ith entry, F[i], stores the number of OIDs with first + byte at most i. Thus F[255] stores the total + number of objects. + + OID Lookup (ID: {'O', 'I', 'D', 'L'}) + The OIDs for all objects in the MIDX are stored in lexicographic + order in this chunk. + + Object Offsets (ID: {'O', 'O', 'F', 'F'}) + Stores two 4-byte values for every object. + 1: The pack-int-id for the pack storing this object. + 2: The offset within the pack. + If all offsets are less than 2^31, then the large offset chunk + will not exist and offsets are stored as in IDX v1. + If there is at least one offset value larger than 2^32-1, then + the large offset chunk must exist. If the large offset chunk + exists and the 31st bit is on, then removing that bit reveals + the row in the large offsets containing the 8-byte offset of + this object. + + [Optional] Object Large Offsets (ID: {'L', 'O', 'F', 'F'}) + 8-byte offsets into large packfiles. + +TRAILER: + + 20-byte SHA1-checksum of the above contents. diff --git a/Documentation/technical/rerere.txt b/Documentation/technical/rerere.txt new file mode 100644 index 0000000000..aa22d7ace8 --- /dev/null +++ b/Documentation/technical/rerere.txt @@ -0,0 +1,186 @@ +Rerere +====== + +This document describes the rerere logic. + +Conflict normalization +---------------------- + +To ensure recorded conflict resolutions can be looked up in the rerere +database, even when branches are merged in a different order, +different branches are merged that result in the same conflict, or +when different conflict style settings are used, rerere normalizes the +conflicts before writing them to the rerere database. + +Different conflict styles and branch names are normalized by stripping +the labels from the conflict markers, and removing the common ancestor +version from the `diff3` conflict style. Branches that are merged +in different order are normalized by sorting the conflict hunks. More +on each of those steps in the following sections. + +Once these two normalization operations are applied, a conflict ID is +calculated based on the normalized conflict, which is later used by +rerere to look up the conflict in the rerere database. + +Removing the common ancestor version +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Say we have three branches AB, AC and AC2. The common ancestor of +these branches has a file with a line containing the string "A" (for +brevity this is called "line A" in the rest of the document). In +branch AB this line is changed to "B", in AC, this line is changed to +"C", and branch AC2 is forked off of AC, after the line was changed to +"C". + +Forking a branch ABAC off of branch AB and then merging AC into it, we +get a conflict like the following: + + <<<<<<< HEAD + B + ======= + C + >>>>>>> AC + +Doing the analogous with AC2 (forking a branch ABAC2 off of branch AB +and then merging branch AC2 into it), using the diff3 conflict style, +we get a conflict like the following: + + <<<<<<< HEAD + B + ||||||| merged common ancestors + A + ======= + C + >>>>>>> AC2 + +By resolving this conflict, to leave line D, the user declares: + + After examining what branches AB and AC did, I believe that making + line A into line D is the best thing to do that is compatible with + what AB and AC wanted to do. + +As branch AC2 refers to the same commit as AC, the above implies that +this is also compatible what AB and AC2 wanted to do. + +By extension, this means that rerere should recognize that the above +conflicts are the same. To do this, the labels on the conflict +markers are stripped, and the common ancestor version is removed. The above +examples would both result in the following normalized conflict: + + <<<<<<< + B + ======= + C + >>>>>>> + +Sorting hunks +~~~~~~~~~~~~~ + +As before, lets imagine that a common ancestor had a file with line A +its early part, and line X in its late part. And then four branches +are forked that do these things: + + - AB: changes A to B + - AC: changes A to C + - XY: changes X to Y + - XZ: changes X to Z + +Now, forking a branch ABAC off of branch AB and then merging AC into +it, and forking a branch ACAB off of branch AC and then merging AB +into it, would yield the conflict in a different order. The former +would say "A became B or C, what now?" while the latter would say "A +became C or B, what now?" + +As a reminder, the act of merging AC into ABAC and resolving the +conflict to leave line D means that the user declares: + + After examining what branches AB and AC did, I believe that + making line A into line D is the best thing to do that is + compatible with what AB and AC wanted to do. + +So the conflict we would see when merging AB into ACAB should be +resolved the same way---it is the resolution that is in line with that +declaration. + +Imagine that similarly previously a branch XYXZ was forked from XY, +and XZ was merged into it, and resolved "X became Y or Z" into "X +became W". + +Now, if a branch ABXY was forked from AB and then merged XY, then ABXY +would have line B in its early part and line Y in its later part. +Such a merge would be quite clean. We can construct 4 combinations +using these four branches ((AB, AC) x (XY, XZ)). + +Merging ABXY and ACXZ would make "an early A became B or C, a late X +became Y or Z" conflict, while merging ACXY and ABXZ would make "an +early A became C or B, a late X became Y or Z". We can see there are +4 combinations of ("B or C", "C or B") x ("X or Y", "Y or X"). + +By sorting, the conflict is given its canonical name, namely, "an +early part became B or C, a late part becames X or Y", and whenever +any of these four patterns appear, and we can get to the same conflict +and resolution that we saw earlier. + +Without the sorting, we'd have to somehow find a previous resolution +from combinatorial explosion. + +Conflict ID calculation +~~~~~~~~~~~~~~~~~~~~~~~ + +Once the conflict normalization is done, the conflict ID is calculated +as the sha1 hash of the conflict hunks appended to each other, +separated by <NUL> characters. The conflict markers are stripped out +before the sha1 is calculated. So in the example above, where we +merge branch AC which changes line A to line C, into branch AB, which +changes line A to line C, the conflict ID would be +SHA1('B<NUL>C<NUL>'). + +If there are multiple conflicts in one file, the sha1 is calculated +the same way with all hunks appended to each other, in the order in +which they appear in the file, separated by a <NUL> character. + +Nested conflicts +~~~~~~~~~~~~~~~~ + +Nested conflicts are handled very similarly to "simple" conflicts. +Similar to simple conflicts, the conflict is first normalized by +stripping the labels from conflict markers, stripping the common ancestor +version, and the sorting the conflict hunks, both for the outer and the +inner conflict. This is done recursively, so any number of nested +conflicts can be handled. + +Note that this only works for conflict markers that "cleanly nest". If +there are any unmatched conflict markers, rerere will fail to handle +the conflict and record a conflict resolution. + +The only difference is in how the conflict ID is calculated. For the +inner conflict, the conflict markers themselves are not stripped out +before calculating the sha1. + +Say we have the following conflict for example: + + <<<<<<< HEAD + 1 + ======= + <<<<<<< HEAD + 3 + ======= + 2 + >>>>>>> branch-2 + >>>>>>> branch-3~ + +After stripping out the labels of the conflict markers, and sorting +the hunks, the conflict would look as follows: + + <<<<<<< + 1 + ======= + <<<<<<< + 2 + ======= + 3 + >>>>>>> + >>>>>>> + +and finally the conflict ID would be calculated as: +`sha1('1<NUL><<<<<<<\n3\n=======\n2\n>>>>>>><NUL>')` |