summaryrefslogtreecommitdiff
path: root/Documentation
AgeCommit message (Collapse)AuthorFilesLines
2020-10-27Merge branch 'sb/clone-origin'Libravatar Junio C Hamano3-2/+9
"git clone" learned clone.defaultremotename configuration variable to customize what nickname to use to call the remote the repository was cloned from. * sb/clone-origin: clone: allow configurable default for `-o`/`--origin` clone: read new remote name from remote_name instead of option_origin clone: validate --origin option before use refs: consolidate remote name validation remote: add tests for add and rename with invalid names clone: use more conventional config/option layering clone: add tests for --template and some disallowed option pairs
2020-10-27Merge branch 'sk/force-if-includes'Libravatar Junio C Hamano3-4/+37
"git push --force-with-lease[=<ref>]" can easily be misused to lose commits unless the user takes good care of their own "git fetch". A new option "--force-if-includes" attempts to ensure that what is being force-pushed was created after examining the commit at the tip of the remote ref that is about to be force-replaced. * sk/force-if-includes: t, doc: update tests, reference for "--force-if-includes" push: parse and set flag for "--force-if-includes" push: add reflog check for "--force-if-includes"
2020-10-27Merge branch 'ds/maintenance-part-2'Libravatar Junio C Hamano3-2/+68
"git maintenance", an extended big brother of "git gc", continues to evolve. * ds/maintenance-part-2: maintenance: add incremental-repack auto condition maintenance: auto-size incremental-repack batch maintenance: add incremental-repack task midx: use start_delayed_progress() midx: enable core.multiPackIndex by default maintenance: create auto condition for loose-objects maintenance: add loose-objects task maintenance: add prefetch task
2020-10-27Merge branch 'rs/worktree-list-show-locked'Libravatar Junio C Hamano1-2/+3
"git worktree list" now shows if each worktree is locked. This possibly may open us to show other kinds of states in the future. * rs/worktree-list-show-locked: worktree: teach `list` to annotate locked worktree
2020-10-26other small fixes for 2.29.2Libravatar Junio C Hamano1-0/+22
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-10-26Merge branch 'cc/doc-filter-branch-typofix'Libravatar Junio C Hamano1-1/+1
Docfix. * cc/doc-filter-branch-typofix: filter-branch doc: fix filter-repo typo
2020-10-22Git 2.29.1Libravatar Junio C Hamano1-0/+11
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-10-20filter-branch doc: fix filter-repo typoLibravatar Christian Couder1-1/+1
The name of the tool is 'git-filter-repo' not 'git-repo-filter'. Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Reviewed-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-10-12worktree: teach `list` to annotate locked worktreeLibravatar Rafael Silva1-2/+3
The "git worktree list" shows the absolute path to the working tree, the commit that is checked out and the name of the branch. It is not immediately obvious which of the worktrees, if any, are locked. "git worktree remove" refuses to remove a locked worktree with an error message. If "git worktree list" told which worktrees are locked in its output, the user would not even attempt to remove such a worktree, or would realize that "git worktree remove -f -f <path>" is required. Teach "git worktree list" to append "locked" to its output. The output from the command becomes like so: $ git worktree list /path/to/main abc123 [master] /path/to/worktree 456def (detached HEAD) /path/to/locked-worktree 123abc (detached HEAD) locked Helped-by: Junio C Hamano <gitster@pobox.com> Helped-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Rafael Silva <rafaeloliveira.cs@gmail.com> Reviewed-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-10-08Git 2.29-rc1Libravatar Junio C Hamano1-13/+21
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-10-08doc: fix the bnf like style of some commandsLibravatar Jean-Noël Avila1-1/+1
In command line options, variables are entered between < and > Signed-off-by: Jean-Noël Avila <jn.avila@free.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-10-08doc: git-remote fix upsLibravatar Jean-Noël Avila1-10/+10
Signed-off-by: Jean-Noël Avila <jn.avila@free.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-10-08doc: use linkgit macro where needed.Libravatar Jean-Noël Avila1-1/+1
Signed-off-by: Jean-Noël Avila <jn.avila@free.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-10-08git-bisect-lk2009: make continuation of list indentedLibravatar Jean-Noël Avila1-8/+8
That's clearer asciidoc formatting. Signed-off-by: Jean-Noël Avila <jn.avila@free.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-10-05Git 2.29-rc0Libravatar Junio C Hamano1-0/+31
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-10-05Merge branch 'sn/fast-import-doc'Libravatar Junio C Hamano1-1/+1
Doc update. * sn/fast-import-doc: fast-import: fix typo in documentation
2020-10-05Merge branch 'pb/submodule-doc-fix'Libravatar Junio C Hamano1-10/+13
Doc update. * pb/submodule-doc-fix: gitsubmodules doc: invoke 'ls-files' with '--recurse-submodules'
2020-10-05Merge branch 'jk/format-auto-base-when-able'Libravatar Junio C Hamano1-1/+3
"git format-patch" learns to take "whenAble" as a possible value for the format.useAutoBase configuration variable to become no-op when the automatically computed base does not make sense. * jk/format-auto-base-when-able: format-patch: teach format.useAutoBase "whenAble" option
2020-10-05Merge branch 'jk/refspecs-negative'Libravatar Junio C Hamano1-0/+16
"git fetch" and "git push" support negative refspecs. * jk/refspecs-negative: refspec: add support for negative refspecs
2020-10-05Merge branch 'rs/archive-add-file'Libravatar Junio C Hamano1-0/+6
"git archive" learns the "--add-file" option to include untracked files into a snapshot from a tree-ish. * rs/archive-add-file: Makefile: use git-archive --add-file archive: add --add-file archive: read short blobs in archive.c::write_archive_entry()
2020-10-04fast-import: fix typo in documentationLibravatar Samanta Navarro1-1/+1
Signed-off-by: Samanta Navarro <ferivoz@riseup.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-10-04gitsubmodules doc: invoke 'ls-files' with '--recurse-submodules'Libravatar Philippe Blain1-10/+13
`git ls-files` was never taught to respect the `submodule.recurse` configuration variable, and it is too late now to change that [1], but still the command is mentioned in 'gitsubmodules(7)' as if it does respect that config. Adjust the call in 'gitsubmodules(7)' by calling 'ls-files' with the '--recurse-submodules' option. While at it, uniformize the capitalization in that file, and use backticks instead of quotes for Git commands and configuration variables. [1] https://lore.kernel.org/git/pull.732.git.1599707259907.gitgitgadget@gmail.com/T/#u Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-10-04Nineteenth batchLibravatar Junio C Hamano1-0/+33
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-10-04Merge branch 'jk/shortlog-group-by-trailer'Libravatar Junio C Hamano1-1/+30
"git shortlog" has been taught to group commits by the contents of the trailer lines, like "Reviewed-by:", "Coauthored-by:", etc. * jk/shortlog-group-by-trailer: shortlog: allow multiple groups to be specified shortlog: parse trailer idents shortlog: rename parse_stdin_ident() shortlog: de-duplicate trailer values shortlog: match commit trailers with --group trailer: add interface for iterating over commit trailers shortlog: add grouping option shortlog: change "author" variables to "ident"
2020-10-04Merge branch 'jc/fmt-merge-msg-suppress-destination'Libravatar Junio C Hamano1-1/+1
Docfix. * jc/fmt-merge-msg-suppress-destination: config/fmt-merge-msg.txt: drop space in quote
2020-10-04Merge branch 'tb/upload-pack-filters'Libravatar Junio C Hamano1-1/+1
Hotfix. * tb/upload-pack-filters: config/uploadpack.txt: fix typo in `--filter=tree:<n>`
2020-10-04Merge branch 'eg/mailinfo-doc-scissors'Libravatar Junio C Hamano1-4/+3
The explanation of the "scissors line" has been clarified. * eg/mailinfo-doc-scissors: Doc: show example scissors line
2020-10-03t, doc: update tests, reference for "--force-if-includes"Libravatar Srinidhi Kaushik3-4/+37
Update test cases for the new option, and document its usage and update related references. Update test cases for the new option, and document its usage and update related references. - t/t5533-push-cas.sh: Update test cases for "compare-and-swap" when used along with "--force-if-includes" helps mitigate overwrites when remote refs are updated in the background; allows forced updates when changes from remote are integrated locally. - Documentation: Add reference for the new option, configuration setting ("push.useForceIfIncludes") and advise messages. Signed-off-by: Srinidhi Kaushik <shrinidhi.kaushik@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-10-01format-patch: teach format.useAutoBase "whenAble" optionLibravatar Jacob Keller1-1/+3
The format.useAutoBase configuration option exists to allow users to enable '--base=auto' for format-patch by default. This can sometimes lead to poor workflow, due to unexpected failures when attempting to format an ancient patch: $ git format-patch -1 <an old commit> fatal: base commit shouldn't be in revision list This can be very confusing, as it is not necessarily immediately obvious that the user requested a --base (since this was in the configuration, not on the command line). We do want --base=auto to fail when it cannot provide a suitable base, as it would be equally confusing if a formatted patch did not include the base information when it was requested. Teach format.useAutoBase a new mode, "whenAble". This mode will cause format-patch to attempt to include a base commit when it can. However, if no valid base commit can be found, then format-patch will continue formatting the patch without a base commit. In order to avoid making yet another branch name unusable with --base, do not teach --base=whenAble or --base=whenable. Instead, refactor the base_commit option to use a callback, and rely on the global configuration variable auto_base. This does mean that a user cannot request this optional base commit generation from the command line. However, this is likely not too valuable. If the user requests base information manually, they will be immediately informed of the failure to acquire a suitable base commit. This allows the user to make an informed choice about whether to continue the format. Add tests to cover the new mode of operation for --base. Signed-off-by: Jacob Keller <jacob.keller@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-30clone: allow configurable default for `-o`/`--origin`Libravatar Sean Barag3-2/+9
While the default remote name of "origin" can be changed at clone-time with `git clone`'s `--origin` option, it was previously not possible to specify a default value for the name of that remote. Add support for a new `clone.defaultRemoteName` config, with the newly-created remote name resolved in priority order: 1. (Highest priority) A remote name passed directly to `git clone -o` 2. A `clone.defaultRemoteName=new_name` in config `git clone -c` 3. A `clone.defaultRemoteName` value set in `/path/to/template/config`, where `--template=/path/to/template` is provided 4. A `clone.defaultRemoteName` value set in a non-template config file 5. The default value of `origin` Helped-by: Junio C Hamano <gitster@pobox.com> Helped-by: Johannes Schindelin <johannes.schindelin@gmx.de> Helped-by: Derrick Stolee <stolee@gmail.com> Helped-by: Andrei Rybak <rybak.a.v@gmail.com> Signed-off-by: Sean Barag <sean@barag.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-30refspec: add support for negative refspecsLibravatar Jacob Keller1-0/+16
Both fetch and push support pattern refspecs which allow fetching or pushing references that match a specific pattern. Because these patterns are globs, they have somewhat limited ability to express more complex situations. For example, suppose you wish to fetch all branches from a remote except for a specific one. To allow this, you must setup a set of refspecs which match only the branches you want. Because refspecs are either explicit name matches, or simple globs, many patterns cannot be expressed. Add support for a new type of refspec, referred to as "negative" refspecs. These are prefixed with a '^' and mean "exclude any ref matching this refspec". They can only have one "side" which always refers to the source. During a fetch, this refers to the name of the ref on the remote. During a push, this refers to the name of the ref on the local side. With negative refspecs, users can express more complex patterns. For example: git fetch origin refs/heads/*:refs/remotes/origin/* ^refs/heads/dontwant will fetch all branches on origin into remotes/origin, but will exclude fetching the branch named dontwant. Refspecs today are commutative, meaning that order doesn't expressly matter. Rather than forcing an implied order, negative refspecs will always be applied last. That is, in order to match, a ref must match at least one positive refspec, and match none of the negative refspecs. This is similar to how negative pathspecs work. Signed-off-by: Jacob Keller <jacob.keller@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-29Eighteenth batchLibravatar Junio C Hamano1-0/+24
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-29Merge branch 'jk/make-protocol-v2-the-default'Libravatar Junio C Hamano2-6/+1
The transport protocol v2 has become the default again. * jk/make-protocol-v2-the-default: protocol: re-enable v2 protocol by default
2020-09-29Merge branch 'tb/bloom-improvements'Libravatar Junio C Hamano4-1/+18
"git commit-graph write" learned to limit the number of bloom filters that are computed from scratch with the --max-new-filters option. * tb/bloom-improvements: commit-graph: introduce 'commitGraph.maxNewFilters' builtin/commit-graph.c: introduce '--max-new-filters=<n>' commit-graph: rename 'split_commit_graph_opts' bloom: encode out-of-bounds filters as non-empty bloom/diff: properly short-circuit on max_changes bloom: use provided 'struct bloom_filter_settings' bloom: split 'get_bloom_filter()' in two commit-graph.c: store maximum changed paths commit-graph: respect 'commitGraph.readChangedPaths' t/helper/test-read-graph.c: prepare repo settings commit-graph: pass a 'struct repository *' in more places t4216: use an '&&'-chain commit-graph: introduce 'get_bloom_filter_settings()'
2020-09-29Merge branch 'bc/faq-misc'Libravatar Junio C Hamano1-0/+86
More FAQ entries. * bc/faq-misc: docs: explain how to deal with files that are always modified docs: explain why reverts are not always applied on merge docs: explain why squash merges are broken with long-running branches
2020-09-28Doc: show example scissors lineLibravatar Evan Gates1-4/+3
The text tries to say the code accepts many variations that look remotely like scissors and perforation marks, but gives too little detail for users to decide what is and what is not taken as a scissors line for themselves. Instead of describing the heuristics more, just spell out what will always be accepted, namely "-- >8 --", as it would not help users to give them more choices and flexibility and be "creative" in their scissors line. Signed-off-by: Evan Gates <evan.gates@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-27config/uploadpack.txt: fix typo in `--filter=tree:<n>`Libravatar Martin Ågren1-1/+1
That should be a ":", not a second "=". While at it, refer to the placeholder "<n>" as "<n>", not "n" (see, e.g., the entry just before this one). Signed-off-by: Martin Ågren <martin.agren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-27config/fmt-merge-msg.txt: drop space in quoteLibravatar Martin Ågren1-1/+1
We document how `merge.suppressDest` can be used to omit " into <branch name>" from the title of the merge message. It is true that we omit the space character before "into", but that lone double quote character risks ending up on the wrong side of a line break, looking a bit out of place. This currently happens with, e.g., 80-character terminals. Drop that leading quoted space. The result should be just as clear about how this option affects the formatted message. Signed-off-by: Martin Ågren <martin.agren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-27shortlog: allow multiple groups to be specifiedLibravatar Jeff King1-0/+7
Now that shortlog supports reading from trailers, it can be useful to combine counts from multiple trailers, or between trailers and authors. This can be done manually by post-processing the output from multiple runs, but it's non-trivial to make sure that each name/commit pair is counted only once. This patch teaches shortlog to accept multiple --group options on the command line, and pull data from all of them. That makes it possible to run: git shortlog -ns --group=author --group=trailer:co-authored-by to get a shortlog that counts authors and co-authors equally. The implementation is mostly straightforward. The "group" enum becomes a bitfield, and the trailer key becomes a list. I didn't bother implementing the multi-group semantics for reading from stdin. It would be possible to do, but the existing matching code makes it awkward, and I doubt anybody cares. The duplicate suppression we used for trailers now covers authors and committers as well (though in non-trailer single-group mode we can skip the hash insertion and lookup, since we only see one value per commit). There is one subtlety: we now care about the case when no group bit is set (in which case we default to showing the author). The caller in builtin/log.c needs to be adapted to ask explicitly for authors, rather than relying on shortlog_init(). It would be possible with some gymnastics to make this keep working as-is, but it's not worth it for a single caller. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-27shortlog: parse trailer identsLibravatar Jeff King1-3/+4
Trailers don't necessarily contain name/email identity values, so shortlog has so far treated them as opaque strings. However, since many trailers do contain identities, it's useful to treat them as such when they can be parsed. That lets "-e" work as usual, as well as mailmap. When they can't be parsed, we'll continue with the old behavior of treating them as a single string (there's no new test for that here, since the existing tests cover a trailer like this). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-27shortlog: de-duplicate trailer valuesLibravatar Jeff King1-1/+2
The current documentation is vague about what happens with --group=trailer:signed-off-by when we see a commit with: Signed-off-by: One Signed-off-by: Two Signed-off-by: One We clearly should credit both "One" and "Two", but should "One" get credited twice? The current code does so, but mostly because that was the easiest thing to do. It's probably more useful to count each commit at most once. This will become especially important when we allow values from multiple sources in a future patch. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-27shortlog: match commit trailers with --groupLibravatar Jeff King1-0/+13
If a project uses commit trailers, this patch lets you use shortlog to see who is performing each action. For example, running: git shortlog -ns --group=trailer:reviewed-by in git.git shows who has reviewed. You can even use a custom format to see things like who has helped whom: git shortlog --format="...helped %an (%ad)" \ --group=trailer:helped-by Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-27shortlog: add grouping optionLibravatar Jeff King1-1/+8
In preparation for adding more grouping types, let's refactor the committer/author grouping code and add a user-facing option that binds them together. In particular: - the main option is now "--group", to make it clear that the various group types are mutually exclusive. The "--committer" option is an alias for "--group=committer". - we keep an enum rather than a binary flag, to prepare for more values - we prefer switch statements to ternary assignment, since other group types will need more custom code Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-25Seventeenth batchLibravatar Junio C Hamano1-0/+19
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-25Merge branch 'jx/proc-receive-hook'Libravatar Junio C Hamano4-6/+136
"git receive-pack" that accepts requests by "git push" learned to outsource most of the ref updates to the new "proc-receive" hook. * jx/proc-receive-hook: doc: add documentation for the proc-receive hook transport: parse report options for tracking refs t5411: test updates of remote-tracking branches receive-pack: new config receive.procReceiveRefs doc: add document for capability report-status-v2 New capability "report-status-v2" for git-push receive-pack: feed report options to post-receive receive-pack: add new proc-receive hook t5411: add basic test cases for proc-receive hook transport: not report a non-head push as a branch
2020-09-25Merge branch 'ds/maintenance-part-1'Libravatar Junio C Hamano5-5/+104
A "git gc"'s big brother has been introduced to take care of more repository maintenance tasks, not limited to the object database cleaning. * ds/maintenance-part-1: maintenance: add trace2 regions for task execution maintenance: add auto condition for commit-graph task maintenance: use pointers to check --auto maintenance: create maintenance.<task>.enabled config maintenance: take a lock on the objects directory maintenance: add --task option maintenance: add commit-graph task maintenance: initialize task array maintenance: replace run_auto_gc() maintenance: add --quiet option maintenance: create basic maintenance runner
2020-09-25protocol: re-enable v2 protocol by defaultLibravatar Jeff King2-6/+1
Protocol v2 became the default in v2.26.0 via 684ceae32d (fetch: default to protocol version 2, 2019-12-23). More widespread use turned up a regression in negotiation. That was fixed in v2.27.0 via 4fa3f00abb (fetch-pack: in protocol v2, in_vain only after ACK, 2020-04-27), but we also reverted the default to v0 as a precuation in 11c7f2a30b (Revert "fetch: default to protocol version 2", 2020-04-22). In v2.28.0, we re-enabled it for experimental users with 3697caf4b9 (config: let feature.experimental imply protocol.version=2, 2020-05-20) and haven't heard any complaints. v2.28 has only been out for 2 months, but I'd generally expect people turning on feature.experimental to also stay pretty up-to-date. So we're not likely to collect much more data by waiting. In addition, we have no further reports from people running v2.26.0, and of course some people have been setting protocol.version manually for ages. Let's move forward with v2 as the default again. It's possible there are still lurking bugs, but we won't know until it gets more widespread use. And we can find and squash them just like any other bug at this point. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-25maintenance: add incremental-repack auto conditionLibravatar Derrick Stolee1-0/+9
The incremental-repack task updates the multi-pack-index by deleting pack- files that have been replaced with new packs, then repacking a batch of small pack-files into a larger pack-file. This incremental repack is faster than rewriting all object data, but is slower than some other maintenance activities. The 'maintenance.incremental-repack.auto' config option specifies how many pack-files should exist outside of the multi-pack-index before running the step. These pack-files could be created by 'git fetch' commands or by the loose-objects task. The default value is 10. Setting the option to zero disables the task with the '--auto' option, and a negative value makes the task run every time. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-25maintenance: add incremental-repack taskLibravatar Derrick Stolee1-0/+18
The previous change cleaned up loose objects using the 'loose-objects' that can be run safely in the background. Add a similar job that performs similar cleanups for pack-files. One issue with running 'git repack' is that it is designed to repack all pack-files into a single pack-file. While this is the most space-efficient way to store object data, it is not time or memory efficient. This becomes extremely important if the repo is so large that a user struggles to store two copies of the pack on their disk. Instead, perform an "incremental" repack by collecting a few small pack-files into a new pack-file. The multi-pack-index facilitates this process ever since 'git multi-pack-index expire' was added in 19575c7 (multi-pack-index: implement 'expire' subcommand, 2019-06-10) and 'git multi-pack-index repack' was added in ce1e4a1 (midx: implement midx_repack(), 2019-06-10). The 'incremental-repack' task runs the following steps: 1. 'git multi-pack-index write' creates a multi-pack-index file if one did not exist, and otherwise will update the multi-pack-index with any new pack-files that appeared since the last write. This is particularly relevant with the background fetch job. When the multi-pack-index sees two copies of the same object, it stores the offset data into the newer pack-file. This means that some old pack-files could become "unreferenced" which I will use to mean "a pack-file that is in the pack-file list of the multi-pack-index but none of the objects in the multi-pack-index reference a location inside that pack-file." 2. 'git multi-pack-index expire' deletes any unreferenced pack-files and updaes the multi-pack-index to drop those pack-files from the list. This is safe to do as concurrent Git processes will see the multi-pack-index and not open those packs when looking for object contents. (Similar to the 'loose-objects' job, there are some Git commands that open pack-files regardless of the multi-pack-index, but they are rarely used. Further, a user that self-selects to use background operations would likely refrain from using those commands.) 3. 'git multi-pack-index repack --bacth-size=<size>' collects a set of pack-files that are listed in the multi-pack-index and creates a new pack-file containing the objects whose offsets are listed by the multi-pack-index to be in those objects. The set of pack- files is selected greedily by sorting the pack-files by modified time and adding a pack-file to the set if its "expected size" is smaller than the batch size until the total expected size of the selected pack-files is at least the batch size. The "expected size" is calculated by taking the size of the pack-file divided by the number of objects in the pack-file and multiplied by the number of objects from the multi-pack-index with offset in that pack-file. The expected size approximates how much data from that pack-file will contribute to the resulting pack-file size. The intention is that the resulting pack-file will be close in size to the provided batch size. The next run of the incremental-repack task will delete these repacked pack-files during the 'expire' step. In this version, the batch size is set to "0" which ignores the size restrictions when selecting the pack-files. It instead selects all pack-files and repacks all packed objects into a single pack-file. This will be updated in the next change, but it requires doing some calculations that are better isolated to a separate change. These steps are based on a similar background maintenance step in Scalar (and VFS for Git) [1]. This was incredibly effective for users of the Windows OS repository. After using the same VFS for Git repository for over a year, some users had _thousands_ of pack-files that combined to up to 250 GB of data. We noticed a few users were running into the open file descriptor limits (due in part to a bug in the multi-pack-index fixed by af96fe3 (midx: add packs to packed_git linked list, 2019-04-29). These pack-files were mostly small since they contained the commits and trees that were pushed to the origin in a given hour. The GVFS protocol includes a "prefetch" step that asks for pre-computed pack- files containing commits and trees by timestamp. These pack-files were grouped into "daily" pack-files once a day for up to 30 days. If a user did not request prefetch packs for over 30 days, then they would get the entire history of commits and trees in a new, large pack-file. This led to a large number of pack-files that had poor delta compression. By running this pack-file maintenance step once per day, these repos with thousands of packs spanning 200+ GB dropped to dozens of pack- files spanning 30-50 GB. This was done all without removing objects from the system and using a constant batch size of two gigabytes. Once the work was done to reduce the pack-files to small sizes, the batch size of two gigabytes means that not every run triggers a repack operation, so the following run will not expire a pack-file. This has kept these repos in a "clean" state. [1] https://github.com/microsoft/scalar/blob/master/Scalar.Common/Maintenance/PackfileMaintenanceStep.cs Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-25midx: enable core.multiPackIndex by defaultLibravatar Derrick Stolee1-2/+2
The core.multiPackIndex setting has been around since c4d25228ebb (config: create core.multiPackIndex setting, 2018-07-12), but has been disabled by default. If a user wishes to use the multi-pack-index feature, then they must enable this config and run 'git multi-pack-index write'. The multi-pack-index feature is relatively stable now, so make the config option true by default. For users that do not use a multi-pack-index, the only extra cost will be a file lookup to see if a multi-pack-index file exists (once per process, per object directory). Also, this config option will be referenced by an upcoming "incremental-repack" task in the maintenance builtin, so move the config option into the repository settings struct. Note that if GIT_TEST_MULTI_PACK_INDEX=1, then we want to ignore the config option and treat core.multiPackIndex as enabled. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>