summaryrefslogtreecommitdiff
path: root/Documentation/config.txt
AgeCommit message (Collapse)AuthorFilesLines
2017-02-03Merge branch 'cw/log-updates-for-all-refs-really'Libravatar Junio C Hamano1-3/+5
The "core.logAllRefUpdates" that used to be boolean has been enhanced to take 'always' as well, to record ref updates to refs other than the ones that are expected to be updated (i.e. branches, remote-tracking branches and notes). * cw/log-updates-for-all-refs-really: doc: add note about ignoring '--no-create-reflog' update-ref: add test cases for bare repository refs: add option core.logAllRefUpdates = always config: add markup to core.logAllRefUpdates doc
2017-02-02Merge branch 'nd/log-graph-configurable-colors'Libravatar Junio C Hamano1-0/+7
Some people feel the default set of colors used by "git log --graph" rather limiting. A mechanism to customize the set of colors has been introduced. * nd/log-graph-configurable-colors: document behavior of empty color name color_parse_mem: allow empty color spec log --graph: customize the graph lines with config log.graphColors color.c: trim leading spaces in color_parse_mem() color.c: fix color_parse_mem() with value_len == 0
2017-02-02document behavior of empty color nameLibravatar Jeff King1-0/+3
Commit 55cccf4bb (color_parse_mem: allow empty color spec, 2017-02-01) clearly defined the behavior of an empty color config variable. Let's document that, and give a hint about why it might be useful. It's important not to say that it makes the item uncolored, because it doesn't. It just sets no attributes, which means that any previous attributes continue to take effect. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-01-31refs: add option core.logAllRefUpdates = alwaysLibravatar Cornelius Weig1-0/+2
When core.logallrefupdates is true, we only create a new reflog for refs that are under certain well-known hierarchies. The reason is that we know that some hierarchies (like refs/tags) are not meant to change, and that unknown hierarchies might not want reflogs at all (e.g., a hypothetical refs/foo might be meant to change often and drop old history immediately). However, sometimes it is useful to override this decision and simply log for all refs, because the safety and audit trail is more important than the performance implications of keeping the log around. This patch introduces a new "always" mode for the core.logallrefupdates option which will log updates to everything under refs/, regardless where in the hierarchy it is (we still will not log things like ORIG_HEAD and FETCH_HEAD, which are known to be transient). Based-on-patch-by: Jeff King <peff@peff.net> Signed-off-by: Cornelius Weig <cornelius.weig@tngtech.com> Reviewed-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-01-30config: add markup to core.logAllRefUpdates docLibravatar Cornelius Weig1-3/+3
Signed-off-by: Cornelius Weig <cornelius.weig@tngtech.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-01-23log --graph: customize the graph lines with config log.graphColorsLibravatar Nguyễn Thái Ngọc Duy1-0/+4
If you have a 256 colors terminal (or one with true color support), then the predefined 12 colors seem limited. On the other hand, you don't want to draw graph lines with every single color in this mode because the two colors could look extremely similar. This option allows you to hand pick the colors you want. Even with standard terminal, if your background color is neither black or white, then the graph line may match your background and become hidden. You can exclude your background color (or simply the colors you hate) with this. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-01-23Merge branch 'sg/fix-versioncmp-with-common-suffix'Libravatar Junio C Hamano1-11/+33
The prereleaseSuffix feature of version comparison that is used in "git tag -l" did not correctly when two or more prereleases for the same release were present (e.g. when 2.0, 2.0-beta1, and 2.0-beta2 are there and the code needs to compare 2.0-beta1 and 2.0-beta2). * sg/fix-versioncmp-with-common-suffix: versioncmp: generalize version sort suffix reordering versioncmp: factor out helper for suffix matching versioncmp: use earliest-longest contained suffix to determine sorting order versioncmp: cope with common part overlapping with prerelease suffix versioncmp: pass full tagnames to swap_prereleases() t7004-tag: add version sort tests to show prerelease reordering issues t7004-tag: use test_config helper t7004-tag: delete unnecessary tags with test_when_finished
2017-01-12versioncmp: generalize version sort suffix reorderingLibravatar SZEDER Gábor1-10/+26
The 'versionsort.prereleaseSuffix' configuration variable, as its name suggests, is supposed to only deal with tagnames with prerelease suffixes, and allows sorting those prerelease tags in a user-defined order before the suffixless main release tag, instead of sorting them simply lexicographically. However, the previous changes in this series resulted in an interesting and useful property of version sort: - The empty string as a configured suffix matches all tagnames, including tagnames without any suffix, but - tagnames containing a "real" configured suffix are still ordered according to that real suffix, because any longer suffix takes precedence over the empty string. Exploiting this property we can easily generalize suffix reordering and specify the order of tags with given suffixes not only before but even after a main release tag by using the empty suffix to denote the position of the main release tag, without any algorithm changes: $ git -c versionsort.prereleaseSuffix=-alpha \ -c versionsort.prereleaseSuffix=-beta \ -c versionsort.prereleaseSuffix="" \ -c versionsort.prereleaseSuffix=-gamma \ -c versionsort.prereleaseSuffix=-delta \ tag -l --sort=version:refname 'v3.0*' v3.0-alpha1 v3.0-beta1 v3.0 v3.0-gamma1 v3.0-delta1 Since 'versionsort.prereleaseSuffix' is not a fitting name for a configuration variable to control this more general suffix reordering, introduce the new variable 'versionsort.suffix'. Still keep the old configuration variable name as a deprecated alias, though, to avoid suddenly breaking setups already using it. Ignore the old variable if both old and new configuration variables are set, but emit a warning so users will be aware of it and can fix their configuration. Extend the documentation to describe and add a test to check this more general behavior. Note: since the empty suffix matches all tagnames, tagnames with suffixes not included in the configuration are listed together with the suffixless main release tag, ordered lexicographically right after that, i.e. before tags with suffixes listed in the configuration following the empty suffix. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-01-10Merge branch 'jc/abbrev-autoscale-config'Libravatar Junio C Hamano1-4/+5
Recent update to the default abbreviation length that auto-scales lacked documentation update, which has been corrected. * jc/abbrev-autoscale-config: config.abbrev: document the new default that auto-scales
2017-01-10Merge branch 'dt/smart-http-detect-server-going-away'Libravatar Junio C Hamano1-0/+5
When the http server gives an incomplete response to a smart-http rpc call, it could lead to client waiting for a full response that will never come. Teach the client side to notice this condition and abort the transfer. An improvement counterproposal has failed. cf. <20161114194049.mktpsvgdhex2f4zv@sigill.intra.peff.net> * dt/smart-http-detect-server-going-away: upload-pack: optionally allow fetching any sha1 remote-curl: don't hang when a server dies before any output
2017-01-10Merge branch 'mm/gc-safety-doc'Libravatar Junio C Hamano1-1/+3
Doc update. * mm/gc-safety-doc: git-gc.txt: expand discussion of races with other processes
2017-01-10Merge branch 'mm/push-social-engineering-attack-doc'Libravatar Junio C Hamano1-3/+14
Doc update on fetching and pushing. * mm/push-social-engineering-attack-doc: doc: mention transfer data leaks in more places
2016-12-27Merge branch 'bw/transport-protocol-policy'Libravatar Junio C Hamano1-0/+46
Finer-grained control of what protocols are allowed for transports during clone/fetch/push have been enabled via a new configuration mechanism. * bw/transport-protocol-policy: http: respect protocol.*.allow=user for http-alternates transport: add from_user parameter to is_transport_allowed http: create function to get curl allowed protocols transport: add protocol policy config option http: always warn if libcurl version is too old lib-proto-disable: variable name fix
2016-12-22config.abbrev: document the new default that auto-scalesLibravatar Junio C Hamano1-4/+5
We somehow forgot to update the "default is 7" in the documentation. Also give a way to explicitly ask the auto-scaling by setting config.abbrev to "auto". Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-19Merge branch 'jk/http-walker-limit-redirect-2.9'Libravatar Junio C Hamano1-0/+10
Transport with dumb http can be fooled into following foreign URLs that the end user does not intend to, especially with the server side redirects and http-alternates mechanism, which can lead to security issues. Tighten the redirection and make it more obvious to the end user when it happens. * jk/http-walker-limit-redirect-2.9: http: treat http-alternates like redirects http: make redirects more obvious remote-curl: rename shadowed options variable http: always update the base URL for redirects http: simplify update_url_from_redirect
2016-12-15transport: add protocol policy config optionLibravatar Brandon Williams1-0/+46
Previously the `GIT_ALLOW_PROTOCOL` environment variable was used to specify a whitelist of protocols to be used in clone/fetch/push commands. This patch introduces new configuration options for more fine-grained control for allowing/disallowing protocols. This also has the added benefit of allowing easier construction of a protocol whitelist on systems where setting an environment variable is non-trivial. Now users can specify a policy to be used for each type of protocol via the 'protocol.<name>.allow' config option. A default policy for all unconfigured protocols can be set with the 'protocol.allow' config option. If no user configured default is made git will allow known-safe protocols (http, https, git, ssh, file), disallow known-dangerous protocols (ext), and have a default policy of `user` for all other protocols. The supported policies are `always`, `never`, and `user`. The `user` policy can be used to configure a protocol to be usable when explicitly used by a user, while disallowing it for commands which run clone/fetch/push commands without direct user intervention (e.g. recursive initialization of submodules). Commands which can potentially clone/fetch/push from untrusted repositories without user intervention can export `GIT_PROTOCOL_FROM_USER` with a value of '0' to prevent protocols configured to the `user` policy from being used. Fix remote-ext tests to use the new config to allow the ext protocol to be tested. Based on a patch by Jeff King <peff@peff.net> Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-08versioncmp: use earliest-longest contained suffix to determine sorting orderLibravatar SZEDER Gábor1-2/+4
When comparing tagnames, it is possible that a tagname contains more than one of the configured prerelease suffixes around the first different character. After fixing a bug in the previous commit such a tagname is sorted according to the contained suffix which comes first in the configuration. This is, however, not quite the right thing to do in the following corner cases: 1. $ git -c versionsort.suffix=-bar -c versionsort.suffix=-foo-baz -c versionsort.suffix=-foo-bar tag -l --sort=version:refname 'v1*' v1.0-foo-bar v1.0-foo-baz The suffix of the tagname 'v1.0-foo-bar' is clearly '-foo-bar', so it should be listed last. However, as it also contains '-bar' around the first different character, it is listed first instead, because that '-bar' suffix comes first the configuration. 2. One of the configured suffixes starts with the other: $ git -c versionsort.prereleasesuffix=-pre \ -c versionsort.prereleasesuffix=-prerelease \ tag -l --sort=version:refname 'v2*' v2.0-prerelease1 v2.0-pre1 v2.0-pre2 Here the tagname 'v2.0-prerelease1' should be the last. When comparing 'v2.0-pre1' and 'v2.0-prerelease1' the first different characters are '1' and 'r', respectively. Since this first different character must be part of the configured suffix, the '-pre' suffix is not recognized in the first tagname. OTOH, the '-prerelease' suffix is properly recognized in 'v2.0-prerelease1', thus it is listed first. Improve version sort in these corner cases, and - look for a configured prerelease suffix containing the first different character or ending right before it, so the '-pre' suffixes are recognized in case (2). This also means that when comparing tagnames 'v2.0-pre1' and 'v2.0-pre2', swap_prereleases() would find the '-pre' suffix in both, but then it will return "undecided" and the caller will do the right thing by sorting based in '1' and '2'. - If the tagname contains more than one suffix, then give precedence to the contained suffix that starts at the earliest offset in the tagname to address (1). - If there are more than one suffixes starting at that earliest position, then give precedence to the longest of those suffixes, thus ensuring that in (2) the tagname 'v2.0-prerelease1' won't be sorted based on the '-pre' suffix. Add tests for these corner cases and adjust the documentation accordingly. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-08versioncmp: cope with common part overlapping with prerelease suffixLibravatar SZEDER Gábor1-2/+6
Version sort with prerelease reordering sometimes puts tagnames in the wrong order, when the common part of two compared tagnames overlaps with the leading character(s) of one or more configured prerelease suffixes. Note the position of "v2.1.0-beta-1": $ git -c versionsort.prereleaseSuffix=-beta \ tag -l --sort=version:refname v2.1.* v2.1.0-beta-2 v2.1.0-beta-3 v2.1.0 v2.1.0-RC1 v2.1.0-RC2 v2.1.0-beta-1 v2.1.1 v2.1.2 The reason is that when comparing a pair of tagnames, first versioncmp() looks for the first different character in a pair of tagnames, and then the swap_prereleases() helper function looks for a configured prerelease suffix _starting at_ that character. Thus, when in the above example the sorting algorithm happens to compare the tagnames "v2.1.0-beta-1" and "v2.1.0-RC2", swap_prereleases() tries to match the suffix "-beta" against "beta-1" to no avail, and the two tagnames erroneously end up being ordered lexicographically. To fix this issue change swap_prereleases() to look for configured prerelease suffixes _containing_ the position of that first different character. Care must be taken, when a configured suffix is longer than the tagnames' common part up to the first different character, to avoid reading memory before the beginning of the tagnames. Add a test that uses an exceptionally long prerelease suffix to check for this, in the hope that in case of a regression the illegal memory access causes a segfault in 'git tag' on one of the commonly used platforms (the test happens to pass successfully on my Linux system with the safety check removed), or at least makes valgrind complain. Under some circumstances it's possible that more than one prerelease suffixes can be found in the same tagname around that first different character. With this simple bugfix patch such a tagname is sorted according to the contained suffix that comes first in the configuration for now. This is less than ideal in some cases, and the following patch will take care of those. Reported-by: Leho Kraav <leho@conversionready.com> Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-06http: make redirects more obviousLibravatar Jeff King1-0/+10
We instruct curl to always follow HTTP redirects. This is convenient, but it creates opportunities for malicious servers to create confusing situations. For instance, imagine Alice is a git user with access to a private repository on Bob's server. Mallory runs her own server and wants to access objects from Bob's repository. Mallory may try a few tricks that involve asking Alice to clone from her, build on top, and then push the result: 1. Mallory may simply redirect all fetch requests to Bob's server. Git will transparently follow those redirects and fetch Bob's history, which Alice may believe she got from Mallory. The subsequent push seems like it is just feeding Mallory back her own objects, but is actually leaking Bob's objects. There is nothing in git's output to indicate that Bob's repository was involved at all. The downside (for Mallory) of this attack is that Alice will have received Bob's entire repository, and is likely to notice that when building on top of it. 2. If Mallory happens to know the sha1 of some object X in Bob's repository, she can instead build her own history that references that object. She then runs a dumb http server, and Alice's client will fetch each object individually. When it asks for X, Mallory redirects her to Bob's server. The end result is that Alice obtains objects from Bob, but they may be buried deep in history. Alice is less likely to notice. Both of these attacks are fairly hard to pull off. There's a social component in getting Mallory to convince Alice to work with her. Alice may be prompted for credentials in accessing Bob's repository (but not always, if she is using a credential helper that caches). Attack (1) requires a certain amount of obliviousness on Alice's part while making a new commit. Attack (2) requires that Mallory knows a sha1 in Bob's repository, that Bob's server supports dumb http, and that the object in question is loose on Bob's server. But we can probably make things a bit more obvious without any loss of functionality. This patch does two things to that end. First, when we encounter a whole-repo redirect during the initial ref discovery, we now inform the user on stderr, making attack (1) much more obvious. Second, the decision to follow redirects is now configurable. The truly paranoid can set the new http.followRedirects to false to avoid any redirection entirely. But for a more practical default, we will disallow redirects only after the initial ref discovery. This is enough to thwart attacks similar to (2), while still allowing the common use of redirects at the repository level. Since c93c92f30 (http: update base URLs when we see redirects, 2013-09-28) we re-root all further requests from the redirect destination, which should generally mean that no further redirection is necessary. As an escape hatch, in case there really is a server that needs to redirect individual requests, the user can set http.followRedirects to "true" (and this can be done on a per-server basis via http.*.followRedirects config). Reported-by: Jann Horn <jannh@google.com> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-11-18upload-pack: optionally allow fetching any sha1Libravatar David Turner1-0/+5
It seems a little silly to do a reachabilty check in the case where we trust the user to access absolutely everything in the repository. Also, it's racy in a distributed system -- perhaps one server advertises a ref, but another has since had a force-push to that ref, and perhaps the two HTTP requests end up directed to these different servers. Signed-off-by: David Turner <dturner@twosigma.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-11-16git-gc.txt: expand discussion of races with other processesLibravatar Matt McCutchen1-1/+3
In general, "git gc" may delete objects that another concurrent process is using but hasn't created a reference to. Git has some mitigations, but they fall short of a complete solution. Document this in the git-gc(1) man page and add a reference from the documentation of the gc.pruneExpire config variable. Based on a write-up by Jeff King: http://marc.info/?l=git&m=147922960131779&w=2 Signed-off-by: Matt McCutchen <matt@mattmccutchen.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-11-14doc: mention transfer data leaks in more placesLibravatar Matt McCutchen1-3/+14
The "SECURITY" section of the gitnamespaces(7) man page described two ways for a client to steal data from a server that wasn't intended to be shared. Similar attacks can be performed by a server on a client, so adapt the section to cover both directions and add it to the git-fetch(1), git-pull(1), and git-push(1) man pages. Also add references to this section from the documentation of server configuration options that attempt to control data leakage but may not be fully effective. Signed-off-by: Matt McCutchen <matt@mattmccutchen.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-10-31Merge branch 'jk/rebase-config-insn-fmt-docfix'Libravatar Junio C Hamano1-1/+1
Documentation fix. * jk/rebase-config-insn-fmt-docfix: doc: fix missing "::" in config list
2016-10-30doc: fix missing "::" in config listLibravatar Jeff King1-1/+1
The rebase.instructionFormat option is missing its "::" to tell AsciiDoc that it's a list entry. As a result, the option name gets lumped into the description in one big paragraph. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-10-17Merge branch 'sb/submodule-config-doc-drop-path'Libravatar Junio C Hamano1-5/+6
The "submodule.<name>.path" stored in .gitmodules is never copied to .git/config and such a key in .git/config has no meaning, but the documentation described it and submodule.<name>.url next to each other as if both belong to .git/config. This has been fixed. * sb/submodule-config-doc-drop-path: documentation: improve submodule.<name>.{url, path} description
2016-10-11documentation: improve submodule.<name>.{url, path} descriptionLibravatar Stefan Beller1-5/+6
Unlike the url variable a user cannot override the the path variable, as it is part of the content together with the gitlink at the given path. To avoid confusion do not mention the .path variable in the config section and rely on the documentation provided in gitmodules[5]. Enhance the description of submodule.<name>.url and mention its two use cases separately. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-10-06Merge branch 'ps/http-gssapi-cred-delegation'Libravatar Junio C Hamano1-0/+14
In recent versions of cURL, GSSAPI credential delegation is disabled by default due to CVE-2011-2192; introduce a configuration to selectively allow enabling this. * ps/http-gssapi-cred-delegation: http: control GSSAPI credential delegation
2016-09-29http: control GSSAPI credential delegationLibravatar Petr Stodulka1-0/+14
Delegation of credentials is disabled by default in libcurl since version 7.21.7 due to security vulnerability CVE-2011-2192. Which makes troubles with GSS/kerberos authentication when delegation of credentials is required. This can be changed with option CURLOPT_GSSAPI_DELEGATION in libcurl with set expected parameter since libcurl version 7.22.0. This patch provides new configuration variable http.delegation which corresponds to curl parameter "--delegation" (see man 1 curl). The following values are supported: * none (default). * policy * always Signed-off-by: Petr Stodulka <pstodulk@redhat.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-09-26Merge branch 'mm/config-color-ui-default-to-auto'Libravatar Junio C Hamano1-6/+12
Documentation for individual configuration variables to control use of color (like `color.grep`) said that their default value is 'false', instead of saying their default is taken from `color.ui`. When we updated the default value for color.ui from 'false' to 'auto' quite a while ago, all of them broke. This has been corrected. * mm/config-color-ui-default-to-auto: Documentation/config: default for color.* is color.ui
2016-09-21Merge branch 'jk/reduce-gc-aggressive-depth'Libravatar Junio C Hamano1-1/+1
"git gc --aggressive" used to limit the delta-chain length to 250, which is way too deep for gaining additional space savings and is detrimental for runtime performance. The limit has been reduced to 50. * jk/reduce-gc-aggressive-depth: gc: default aggressive depth to 50
2016-09-16Documentation/config: default for color.* is color.uiLibravatar Matthieu Moy1-6/+12
Since 4c7f181 (make color.ui default to 'auto', 2013-06-10), the default for color.* when nothing is set is 'auto' and we still claimed that the default was 'false'. Be more precise by saying explicitly that the default is to follow color.ui, and recall that the default is 'auto' to avoid one indirection for the reader. Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-09-08Merge branch 'sb/submodule-clone-rr'Libravatar Junio C Hamano1-0/+12
"git clone --resurse-submodules --reference $path $URL" is a way to reduce network transfer cost by borrowing objects in an existing $path repository when cloning the superproject from $URL; it learned to also peek into $path for presense of corresponding repositories of submodules and borrow objects from there when able. * sb/submodule-clone-rr: clone: recursive and reference option triggers submodule alternates clone: implement optional references clone: clarify option_reference as required clone: factor out checking for an alternate path submodule--helper update-clone: allow multiple references submodule--helper module-clone: allow multiple references t7408: merge short tests, factor out testing method t7408: modernize style
2016-09-08Merge branch 'jc/renormalize-merge-kill-safer-crlf' into maintLibravatar Junio C Hamano1-7/+5
"git merge" with renormalization did not work well with merge-recursive, due to "safer crlf" conversion kicking in when it shouldn't. * jc/renormalize-merge-kill-safer-crlf: merge: avoid "safer crlf" during recording of merge results convert: unify the "auto" handling of CRLF
2016-08-24receive-pack: allow a maximum input size to be specifiedLibravatar Jeff King1-0/+6
Receive-pack feeds its input to either index-pack or unpack-objects, which will happily accept as many bytes as a sender is willing to provide. Let's allow an arbitrary cutoff point where we will stop writing bytes to disk. Cleaning up what has already been written to disk is a related problem that is not addressed by this patch. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-17clone: recursive and reference option triggers submodule alternatesLibravatar Stefan Beller1-0/+12
When `--recursive` and `--reference` is given, it is reasonable to expect that the submodules are created with references to the submodules of the given alternate for the superproject. An initial attempt to do this was presented to the mailing list, which used flags that are passed around ("--super-reference") that instructed the submodule clone to look for a reference in the submodules of the referenced superproject. This is not well thought out, as any further `submodule update` should also respect the initial setup. When a new submodule is added to the superproject and the alternate of the superproject does not know about that submodule yet, we rather error out informing the user instead of being unclear if we did or did not use a submodules alternate. To solve this problem introduce new options that store the configuration for what the user wanted originally. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-11gc: default aggressive depth to 50Libravatar Jeff King1-1/+1
This commit message is long and has lots of background and numbers. The summary is: the current default of 250 doesn't save much space, and costs CPU. It's not a good tradeoff. Read on for details. The "--aggressive" flag to git-gc does three things: 1. use "-f" to throw out existing deltas and recompute from scratch 2. use "--window=250" to look harder for deltas 3. use "--depth=250" to make longer delta chains Items (1) and (2) are good matches for an "aggressive" repack. They ask the repack to do more computation work in the hopes of getting a better pack. You pay the costs during the repack, and other operations see only the benefit. Item (3) is not so clear. Allowing longer chains means fewer restrictions on the deltas, which means potentially finding better ones and saving some space. But it also means that operations which access the deltas have to follow longer chains, which affects their performance. So it's a tradeoff, and it's not clear that the tradeoff is even a good one. The existing "250" numbers for "--aggressive" come originally from this thread: http://public-inbox.org/git/alpine.LFD.0.9999.0712060803430.13796@woody.linux-foundation.org/ where Linus says: So when I said "--depth=250 --window=250", I chose those numbers more as an example of extremely aggressive packing, and I'm not at all sure that the end result is necessarily wonderfully usable. It's going to save disk space (and network bandwidth - the delta's will be re-used for the network protocol too!), but there are definitely downsides too, and using long delta chains may simply not be worth it in practice. There are some numbers in that thread, but they're mostly focused on the improved window size, and measure the improvement from --depth=250 and --window=250 together. E.g.: http://public-inbox.org/git/9e4733910712062006l651571f3w7f76ce64c6650dff@mail.gmail.com/ talks about the improved run-time of "git-blame", which comes from the reduced pack size. But most of that reduction is coming from --window=250, whereas most of the extra costs come from --depth=250. There's a link in that thread showing that increasing the depth beyond 50 doesn't seem to help much with the size: https://vcscompare.blogspot.com/2008/06/git-repack-parameters.html but again, no discussion of the timing impact. In an earlier thread from Ted Ts'o which discussed setting the non-aggressive default (from 10 to 50): http://public-inbox.org/git/20070509134958.GA21489%40thunk.org/ we have more numbers, with the conclusion that going past 50 does not help size much, and hurts the speed of normal operations. So from that, we might guess that 50 is actually a sweet spot, even for aggressive, if we interpret aggressive to "spend time now to make a better pack". It is not clear that "--depth=250" is actually a better pack. It may be slightly _smaller_, but it carries a run-time penalty. Here are some more recent timings I did to verify that. They show three things: - the size of the resulting pack (so disk saved to store, bandwidth saved on clones/fetches) - the cost of "rev-list --objects --all", which shows the effect of the delta chains on trees (commits typically don't delta, and the command doesn't touch the blobs at all) - the cost of "log -Sfoo", which will additionally access each blob All cases were repacked with "git repack -adf --depth=$d --window=250" (so basically, what would happen if we tweaked the "gc --aggressive" default depth). The timings are all wall-clock best-of-3. The machine itself has plenty of RAM compared to the repositories (which is probably typical of most workstations these days), so we're really measuring CPU usage, as the whole thing will be in disk cache after the first run. The core.deltaBaseCacheLimit is at its default of 96MiB. It's possible that tweaking it would have some impact on the tests, as some of them (especially "log -S" on a large repo) are likely to overflow that. But bumping that carries a run-time memory cost, so for these tests, I focused on what we could do just with the on-disk pack tradeoffs. Each test is done for four depths: 250 (the current value), 50 (the current default that tested well previously), 100 (to show something on the larger side, which previous tests showed was not a good tradeoff), and 10 (the very old default, which previous tests showed was worse than 50). Here are the numbers for linux.git: depth | size | % | rev-list | % | log -Sfoo | % -------+-------+-------+----------+--------+-----------+------- 250 | 967MB | n/a | 48.159s | n/a | 378.088 | n/a 100 | 971MB | +0.4% | 41.471s | -13.9% | 342.060 | -9.5% 50 | 979MB | +1.2% | 37.778s | -21.6% | 311.040s | -17.7% 10 | 1.1GB | +6.6% | 32.518s | -32.5% | 279.890s | -25.9% and for git.git: depth | size | % | rev-list | % | log -Sfoo | % -------+-------+-------+----------+--------+-----------+------- 250 | 48MB | n/a | 2.215s | n/a | 20.922s | n/a 100 | 49MB | +0.5% | 2.140s | -3.4% | 17.736s | -15.2% 50 | 49MB | +1.7% | 2.099s | -5.2% | 15.418s | -26.3% 10 | 53MB | +9.3% | 2.001s | -9.7% | 12.677s | -39.4% You can see that that the CPU savings for regular operations improves as we decrease the depth. The savings are less for "rev-list" on a smaller repository than they are for blob-accessing operations, or even rev-list on a larger repository. This may mean that a larger delta cache would help (though setting core.deltaBaseCacheLimit by itself doesn't). But we can also see that the space savings are not that great as the depth goes higher. Saving 5-10% between 10 and 50 is probably worth the CPU tradeoff. Saving 1% to go from 50 to 100, or another 0.5% to go from 100 to 250 is probably not. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-10Merge branch 'jt/format-patch-from-config'Libravatar Junio C Hamano1-0/+10
"git format-patch" learned format.from configuration variable to specify the default settings for its "--from" option. * jt/format-patch-from-config: format-patch: format.from gives the default for --from
2016-08-03Merge branch 'jk/push-progress'Libravatar Junio C Hamano1-0/+9
"git push" and "git clone" learned to give better progress meters to the end user who is waiting on the terminal. * jk/push-progress: receive-pack: send keepalives during quiet periods receive-pack: turn on connectivity progress receive-pack: relay connectivity errors to sideband receive-pack: turn on index-pack resolving progress index-pack: add flag for showing delta-resolution progress clone: use a real progress meter for connectivity check check_connected: add progress flag check_connected: relay errors to alternate descriptor check_everything_connected: use a struct with named options check_everything_connected: convert to argv_array rev-list: add optional progress reporting check_everything_connected: always pass --quiet to rev-list
2016-08-03Merge branch 'sb/push-options'Libravatar Junio C Hamano1-2/+7
"git push" learned to accept and pass extra options to the receiving end so that hooks can read and react to them. * sb/push-options: add a test for push options push: accept push options receive-pack: implement advertising and receiving push options push options: {pre,post}-receive hook learns about push options
2016-08-01format-patch: format.from gives the default for --fromLibravatar Josh Triplett1-0/+10
This helps users who would prefer format-patch to default to --from, and makes it easier to change the default in the future. Signed-off-by: Josh Triplett <josh@joshtriplett.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-28Merge branch 'mm/doc-tt' into maintLibravatar Junio C Hamano1-20/+20
More mark-up updates to typeset strings that are expected to literally typed by the end user in fixed-width font. * mm/doc-tt: doc: typeset HEAD and variants as literal CodingGuidelines: formatting HEAD in documentation doc: typeset long options with argument as literal doc: typeset '--' as literal doc: typeset long command-line options as literal doc: typeset short command-line options as literal Documentation/git-mv.txt: fix whitespace indentation
2016-07-25Merge branch 'jc/renormalize-merge-kill-safer-crlf'Libravatar Junio C Hamano1-7/+5
"git merge" with renormalization did not work well with merge-recursive, due to "safer crlf" conversion kicking in when it shouldn't. * jc/renormalize-merge-kill-safer-crlf: merge: avoid "safer crlf" during recording of merge results convert: unify the "auto" handling of CRLF
2016-07-20receive-pack: send keepalives during quiet periodsLibravatar Jeff King1-0/+9
After a client has sent us the complete pack, we may spend some time processing the data and running hooks. If the client asked us to be quiet, receive-pack won't send any progress data during the index-pack or connectivity-check steps. And hooks may or may not produce their own progress output. In these cases, the network connection is totally silent from both ends. Git itself doesn't care about this (it will wait forever), but other parts of the system (e.g., firewalls, load-balancers, etc) might hang up the connection. So we'd like to send some sort of keepalive to let the network and the client side know that we're still alive and processing. We can use the same trick we did in 05e9515 (upload-pack: send keepalive packets during pack computation, 2013-09-08). Namely, we will send an empty sideband data packet every `N` seconds that we do not relay any stderr data over the sideband channel. As with 05e9515, this means that we won't bother sending keepalives when there's actual progress data, but will kick in when it has been disabled (or if there is a lull in the progress data). The concept is simple, but the details are subtle enough that they need discussing here. Before the client sends us the pack, we don't want to do any keepalives. We'll have sent our ref advertisement, and we're waiting for them to send us the pack (and tell us that they support sidebands at all). While we're receiving the pack from the client (or waiting for it to start), there's no need for keepalives; it's up to them to keep the connection active by sending data. Moreover, it would be wrong for us to do so. When we are the server in the smart-http protocol, we must treat our connection as half-duplex. So any keepalives we send while receiving the pack would potentially be buffered by the webserver. Not only does this make them useless (since they would not be delivered in a timely manner), but it could actually cause a deadlock if we fill up the buffer with keepalives. (It wouldn't be wrong to send keepalives in this phase for a full-duplex connection like ssh; it's simply pointless, as it is the client's responsibility to speak). As soon as we've gotten all of the pack data, then the client is waiting for us to speak, and we should start keepalives immediately. From here until the end of the connection, we send one any time we are not otherwise sending data. But there's a catch. Receive-pack doesn't know the moment we've gotten all the data. It passes the descriptor to index-pack, who reads all of the data, and then starts resolving the deltas. We have to communicate that back. To make this work, we instruct the sideband muxer to enable keepalives in three phases: 1. In the beginning, not at all. 2. While reading from index-pack, wait for a signal indicating end-of-input, and then start them. 3. Afterwards, always. The signal from index-pack in phase 2 has to come over the stderr channel which the muxer is reading. We can't use an extra pipe because the portable run-command interface only gives us stderr and stdout. Stdout is already used to pass the .keep filename back to receive-pack. We could also send a signal there, but then we would find out about it in the main thread. And the keepalive needs to be done by the async muxer thread (since it's the one writing sideband data back to the client). And we can't reliably signal the async thread from the main thread, because the async code sometimes uses threads and sometimes uses forked processes. Therefore the signal must come over the stderr channel, where it may be interspersed with other random human-readable messages from index-pack. This patch makes the signal a single NUL byte. This is easy to parse, should not appear in any normal stderr output, and we don't have to worry about any timing issues (like seeing half the signal bytes in one read(), and half in a subsequent one). This is a bit ugly, but it's simple to code and should work reliably. Another option would be to stop using an async thread for muxing entirely, and just poll() both stderr and stdout of index-pack from the main thread. This would work for index-pack (because we aren't doing anything useful in the main thread while it runs anyway). But it would make the connectivity check and the hook muxers much more complicated, as they need to simultaneously feed the sub-programs while reading their stderr. The index-pack phase is the only one that needs this signaling, so it could simply behave differently than the other two. That would mean having two separate implementations of copy_to_sideband (and the keepalive code), though. And it still doesn't get rid of the signaling; it just means we can write a nicer message like "END_OF_INPUT" or something on stdout, since we don't have to worry about separating it from the stderr cruft. One final note: this signaling trick is only done with index-pack, not with unpack-objects. There's no point in doing it for the latter, because by definition it only kicks in for a small number of objects, where keepalives are not as useful (and this conveniently lets us avoid duplicating the implementation). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-19Merge branch 'nd/fetch-ref-summary'Libravatar Junio C Hamano1-0/+5
Improve the look of the way "git fetch" reports what happened to each ref that was fetched. * nd/fetch-ref-summary: fetch: reduce duplicate in ref update status lines with placeholder fetch: align all "remote -> local" output fetch: change flag code for displaying tag update and deleted ref fetch: refactor ref update status formatting code git-fetch.txt: document fetch output
2016-07-19Merge branch 'nd/connect-ssh-command-config'Libravatar Junio C Hamano1-0/+7
A new configuration variable core.sshCommand has been added to specify what value for GIT_SSH_COMMAND to use per repository. * nd/connect-ssh-command-config: connect: read $GIT_SSH_COMMAND from config file
2016-07-14receive-pack: implement advertising and receiving push optionsLibravatar Stefan Beller1-2/+7
The pre/post receive hook may be interested in more information from the user. This information can be transmitted when both client and server support the "push-options" capability, which when used is a phase directly after update commands ended by a flush pkt. Similar to the atomic option, the server capability can be disabled via the `receive.advertisePushOptions` config variable. While documenting this, fix a nit in the `receive.advertiseAtomic` wording. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-13Merge branch 'mm/doc-tt'Libravatar Junio C Hamano1-20/+20
More mark-up updates to typeset strings that are expected to literally typed by the end user in fixed-width font. * mm/doc-tt: doc: typeset HEAD and variants as literal CodingGuidelines: formatting HEAD in documentation doc: typeset long options with argument as literal doc: typeset '--' as literal doc: typeset long command-line options as literal doc: typeset short command-line options as literal Documentation/git-mv.txt: fix whitespace indentation
2016-07-11Merge branch 'jk/ansi-color'Libravatar Junio C Hamano1-21/+28
The output coloring scheme learned two new attributes, italic and strike, in addition to existing bold, reverse, etc. * jk/ansi-color: color: support strike-through attribute color: support "italic" attribute color: allow "no-" for negating attributes color: refactor parse_attr add skip_prefix_mem helper doc: refactor description of color format color: fix max-size comment
2016-07-06connect: read $GIT_SSH_COMMAND from config fileLibravatar Nguyễn Thái Ngọc Duy1-0/+7
Similar to $GIT_ASKPASS or $GIT_PROXY_COMMAND, we also read from config file first then fall back to $GIT_SSH_COMMAND. This is useful for selecting different private keys targetting the same host (e.g. github) Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-06Merge branch 'jk/upload-pack-hook'Libravatar Junio C Hamano1-0/+15
"upload-pack" allows a custom "git pack-objects" replacement when responding to "fetch/clone" via the uploadpack.packObjectsHook. * jk/upload-pack-hook: upload-pack: provide a hook for running pack-objects t1308: do not get fooled by symbolic links to the source tree config: add a notion of "scope" config: return configset value for current_config_ functions config: set up config_source for command-line config git_config_parse_parameter: refactor cleanup code git_config_with_options: drop "found" counting