summaryrefslogtreecommitdiff
path: root/builtin/fetch.c
AgeCommit message (Collapse)AuthorFilesLines
2020-03-02Merge branch 'ds/partial-clone-fixes'Libravatar Junio C Hamano1-5/+5
Fix for a bug revealed by a recent change to make the protocol v2 the default. * ds/partial-clone-fixes: partial-clone: avoid fetching when looking for objects partial-clone: demonstrate bugs in partial fetch
2020-02-22partial-clone: avoid fetching when looking for objectsLibravatar Derrick Stolee1-5/+5
When using partial clone, find_non_local_tags() in builtin/fetch.c checks each remote tag to see if its object also exists locally. There is no expectation that the object exist locally, but this function nevertheless triggers a lazy fetch if the object does not exist. This can be extremely expensive when asking for a commit, as we are completely removed from the context of the non-existent object and thus supply no "haves" in the request. 6462d5eb9a (fetch: remove fetch_if_missing=0, 2019-11-05) removed a global variable that prevented these fetches in favor of a bitflag. However, some object existence checks were not updated to use this flag. Update find_non_local_tags() to use OBJECT_INFO_SKIP_FETCH_OBJECT in addition to OBJECT_INFO_QUICK. The _QUICK option only prevents repreparing the pack-file structures. We need to be extremely careful about supplying _SKIP_FETCH_OBJECT when we expect an object to not exist due to updated refs. This resolves a broken test in t5616-partial-clone.sh. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-02-14Merge branch 'tb/commit-graph-object-dir'Libravatar Junio C Hamano1-1/+1
The code to compute the commit-graph has been taught to use a more robust way to tell if two object directories refer to the same thing. * tb/commit-graph-object-dir: commit-graph.h: use odb in 'load_commit_graph_one_fd_st' commit-graph.c: remove path normalization, comparison commit-graph.h: store object directory in 'struct commit_graph' commit-graph.h: store an odb in 'struct write_commit_graph_context' t5318: don't pass non-object directory to '--object-dir'
2020-02-04commit-graph.h: store an odb in 'struct write_commit_graph_context'Libravatar Taylor Blau1-1/+1
There are lots of places in 'commit-graph.h' where a function either has (or almost has) a full 'struct object_directory *', accesses '->path', and then throws away the rest of the struct. This can cause headaches when comparing the locations of object directories across alternates (e.g., in the case of deciding if two commit-graph layers can be merged). These paths are normalized with 'normalize_path_copy()' which mitigates some comparison issues, but not all [1]. Replace usage of 'char *object_dir' with 'odb->path' by storing a 'struct object_directory *' in the 'write_commit_graph_context' structure. This is an intermediate step towards getting rid of all path normalization in 'commit-graph.c'. Resolving a user-provided '--object-dir' argument now requires that we compare it to the known alternates for equality. Prior to this patch, an unknown '--object-dir' argument would silently exit with status zero. This can clearly lead to unintended behavior, such as verifying commit-graphs that aren't in a repository's own object store (or one of its alternates), or causing a typo to mask a legitimate commit-graph verification failure. Make this error non-silent by 'die()'-ing when the given '--object-dir' does not match any known alternate object store. [1]: In my testing, for example, I can get one side of the commit-graph code to fill object_dir with "./objects" and the other with just "objects". Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-30fetch: forgo full connectivity check if --filterLibravatar Jonathan Tan1-1/+10
If a filter is specified, we do not need a full connectivity check on the contents of the packfile we just fetched; we only need to check that the objects referenced are promisor objects. This significantly speeds up fetches into repositories that have many promisor objects, because during the connectivity check, all promisor objects are enumerated (to mark them UNINTERESTING), and that takes a significant amount of time. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Reviewed-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-06Merge branch 'ds/commit-graph-set-size-mult'Libravatar Junio C Hamano1-3/+1
The code to write split commit-graph file(s) upon fetching computed bogus value for the parameter used in splitting the resulting files, which has been corrected. * ds/commit-graph-set-size-mult: commit-graph: prefer default size_mult when given zero
2020-01-02commit-graph: prefer default size_mult when given zeroLibravatar Derrick Stolee1-3/+1
In 50f26bd ("fetch: add fetch.writeCommitGraph config setting", 2019-09-02), the fetch builtin added the capability to write a commit-graph using the "--split" feature. This feature creates multiple commit-graph files, and those can merge based on a set of "split options" including a size multiple. The default size multiple is 2, which intends to provide a log_2 N depth of the commit-graph chain where N is the number of commits. However, I noticed during dogfooding that my commit-graph chains were becoming quite large when left only to builds by 'git fetch'. It turns out that in split_graph_merge_strategy(), we default the size_mult variable to 2 except we override it with the context's split_opts if they exist. In builtin/fetch.c, we create such a split_opts, but do not populate it with values. This problem is due to two failures: 1. It is unclear that we can add the flag COMMIT_GRAPH_WRITE_SPLIT with a NULL split_opts. 2. If we have a non-NULL split_opts, then we override the default values even if a zero value is given. Correct both of these issues. First, do not override size_mult when the options provide a zero value. Second, stop creating a split_opts in the fetch builtin. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-06Merge branch 'rs/use-skip-prefix-more'Libravatar Junio C Hamano1-9/+3
Code cleanup. * rs/use-skip-prefix-more: name-rev: use skip_prefix() instead of starts_with() push: use skip_prefix() instead of starts_with() shell: use skip_prefix() instead of starts_with() fmt-merge-msg: use skip_prefix() instead of starts_with() fetch: use skip_prefix() instead of starts_with()
2019-12-01Merge branch 'jt/fetch-remove-lazy-fetch-plugging'Libravatar Junio C Hamano1-3/+2
"git fetch" codepath had a big "do not lazily fetch missing objects when I ask if something exists" switch. This has been corrected by marking the "does this thing exist?" calls with "if not please do not lazily fetch it" flag. * jt/fetch-remove-lazy-fetch-plugging: promisor-remote: remove fetch_if_missing=0 clone: remove fetch_if_missing=0 fetch: remove fetch_if_missing=0
2019-12-01Merge branch 'en/doc-typofix'Libravatar Junio C Hamano1-1/+1
Docfix. * en/doc-typofix: Fix spelling errors in no-longer-updated-from-upstream modules multimail: fix a few simple spelling errors sha1dc: fix trivial comment spelling error Fix spelling errors in test commands Fix spelling errors in messages shown to users Fix spelling errors in names of tests Fix spelling errors in comments of testcases Fix spelling errors in code comments Fix spelling errors in documentation outside of Documentation/ Documentation: fix a bunch of typos, both old and new
2019-12-01Merge branch 'js/fetch-multi-lockfix'Libravatar Junio C Hamano1-2/+8
Fetching from multiple remotes into the same repository in parallel had a bad interaction with the recent change to (optionally) update the commit-graph after a fetch job finishes, as these parallel fetches compete with each other. Which has been corrected. * js/fetch-multi-lockfix: fetch: avoid locking issues between fetch.jobs/fetch.writeCommitGraph fetch: add the command-line option `--write-commit-graph`
2019-12-01Merge branch 'rt/fetch-message-fix'Libravatar Junio C Hamano1-1/+1
A small message update. * rt/fetch-message-fix: fetch.c: fix typo in a warning message
2019-11-27fetch: use skip_prefix() instead of starts_with()Libravatar René Scharfe1-9/+3
Get rid of magic numbers by letting skip_prefix() set the pointer "what". Signed-off-by: René Scharfe <l.s.r@web.de> Acked-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-10Fix spelling errors in code commentsLibravatar Elijah Newren1-1/+1
Reported-by: Jens Schleusener <Jens.Schleusener@fossies.org> Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-08fetch: remove fetch_if_missing=0Libravatar Jonathan Tan1-3/+2
In fetch_pack() (and all functions it calls), pass OBJECT_INFO_SKIP_FETCH_OBJECT whenever we query an object that could be a tree or blob that we do not want to be lazy-fetched even if it is absent. Thus, the only lazy-fetches occurring for trees and blobs are when resolving deltas. Thus, we can remove fetch_if_missing=0 from builtin/fetch.c. Remove this, and also add a test ensuring that such objects are not lazy-fetched. (We might be able to remove fetch_if_missing=0 from other places too, but I have limited myself to builtin/fetch.c in this commit because I have not written tests for the other commands yet.) Note that commits and tags may still be lazy-fetched. I limited myself to objects that could be trees or blobs here because Git does not support creating such commit- and tag-excluding clones yet, and even if such a clone were manually created, Git does not have good support for fetching a single commit (when fetching a commit, it and all its ancestors would be sent). Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-06fetch: avoid locking issues between fetch.jobs/fetch.writeCommitGraphLibravatar Johannes Schindelin1-1/+2
When both `fetch.jobs` and `fetch.writeCommitGraph` is set, we currently try to write the commit graph in each of the concurrent fetch jobs, which frequently leads to error messages like this one: fatal: Unable to create '.../.git/objects/info/commit-graphs/commit-graph-chain.lock': File exists. Let's avoid this by holding off from writing the commit graph until all fetch jobs are done. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-06fetch: add the command-line option `--write-commit-graph`Libravatar Johannes Schindelin1-1/+6
This option overrides the config setting `fetch.writeCommitGraph`, if both are set. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-02fetch.c: fix typo in a warning messageLibravatar Ralf Thielow1-1/+1
Signed-off-by: Ralf Thielow <ralf.thielow@gmail.com> Reviewed-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-24fetch: delay fetch_if_missing=0 until after configLibravatar Jonathan Tan1-2/+2
Suppose, from a repository that has ".gitmodules", we clone with --filter=blob:none: git clone --filter=blob:none --no-checkout \ https://kernel.googlesource.com/pub/scm/git/git Then we fetch: git -C git fetch This will cause a "unable to load config blob object", because the fetch_config_from_gitmodules() invocation in cmd_fetch() will attempt to load ".gitmodules" (which Git knows to exist because the client has the tree of HEAD) while fetch_if_missing is set to 0. fetch_if_missing is set to 0 too early - ".gitmodules" here should be lazily fetched. Git must set fetch_if_missing to 0 before the fetch because as part of the fetch, packfile negotiation happens (and we do not want to fetch any missing objects when checking existence of objects), but we do not need to set it so early. Move the setting of fetch_if_missing to the earliest possible point in cmd_fetch(), right before any fetching happens. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-15Merge branch 'js/trace2-fetch-push'Libravatar Junio C Hamano1-7/+15
Dev support. * js/trace2-fetch-push: transport: push codepath can take arbitrary repository push: add trace2 instrumentation fetch: add trace2 instrumentation
2019-10-15Merge branch 'ew/hashmap'Libravatar Junio C Hamano1-14/+18
Code clean-up of the hashmap API, both users and implementation. * ew/hashmap: hashmap_entry: remove first member requirement from docs hashmap: remove type arg from hashmap_{get,put,remove}_entry OFFSETOF_VAR macro to simplify hashmap iterators hashmap: introduce hashmap_free_entries hashmap: hashmap_{put,remove} return hashmap_entry * hashmap: use *_entry APIs for iteration hashmap_cmp_fn takes hashmap_entry params hashmap_get{,_from_hash} return "struct hashmap_entry *" hashmap: use *_entry APIs to wrap container_of hashmap_get_next returns "struct hashmap_entry *" introduce container_of macro hashmap_put takes "struct hashmap_entry *" hashmap_remove takes "const struct hashmap_entry *" hashmap_get takes "const struct hashmap_entry *" hashmap_add takes "struct hashmap_entry *" hashmap_get_next takes "const struct hashmap_entry *" hashmap_entry_init takes "struct hashmap_entry *" packfile: use hashmap_entry in delta_base_cache_entry coccicheck: detect hashmap_entry.hash assignment diff: use hashmap_entry_init on moved_entry.ent
2019-10-15Merge branch 'js/fetch-jobs'Libravatar Junio C Hamano1-17/+107
"git fetch --jobs=<n>" allowed <n> parallel jobs when fetching submodules, but this did not apply to "git fetch --multiple" that fetches from multiple remote repositories. It now does. * js/fetch-jobs: fetch: let --jobs=<n> parallelize --multiple, too
2019-10-07Merge branch 'ms/fetch-follow-tag-optim'Libravatar Junio C Hamano1-8/+10
The code used in following tags in "git fetch" has been optimized. * ms/fetch-follow-tag-optim: fetch: use oidset to keep the want OIDs for faster lookup
2019-10-07hashmap_entry: remove first member requirement from docsLibravatar Eric Wong1-1/+1
Comments stating that "struct hashmap_entry" must be the first member in a struct are no longer valid. Suggested-by: Phillip Wood <phillip.wood123@gmail.com> Signed-off-by: Eric Wong <e@80x24.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-07hashmap: introduce hashmap_free_entriesLibravatar Eric Wong1-3/+3
`hashmap_free_entries' behaves like `container_of' and passes the offset of the hashmap_entry struct to the internal `hashmap_free_' function, allowing the function to free any struct pointer regardless of where the hashmap_entry field is located. `hashmap_free' no longer takes any arguments aside from the hashmap itself. Signed-off-by: Eric Wong <e@80x24.org> Reviewed-by: Derrick Stolee <stolee@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-07hashmap_cmp_fn takes hashmap_entry paramsLibravatar Eric Wong1-4/+5
Another step in eliminating the requirement of hashmap_entry being the first member of a struct. Signed-off-by: Eric Wong <e@80x24.org> Reviewed-by: Derrick Stolee <stolee@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-07hashmap_get{,_from_hash} return "struct hashmap_entry *"Libravatar Eric Wong1-4/+7
Update callers to use hashmap_get_entry, hashmap_get_entry_from_hash or container_of as appropriate. This is another step towards eliminating the requirement of hashmap_entry being the first field in a struct. Signed-off-by: Eric Wong <e@80x24.org> Reviewed-by: Derrick Stolee <stolee@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-07hashmap_add takes "struct hashmap_entry *"Libravatar Eric Wong1-1/+1
This is less error-prone than "void *" as the compiler now detects invalid types being passed. Signed-off-by: Eric Wong <e@80x24.org> Reviewed-by: Derrick Stolee <stolee@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-07hashmap_entry_init takes "struct hashmap_entry *"Libravatar Eric Wong1-1/+1
C compilers do type checking to make life easier for us. So rely on that and update all hashmap_entry_init callers to take "struct hashmap_entry *" to avoid future bugs while improving safety and readability. Signed-off-by: Eric Wong <e@80x24.org> Reviewed-by: Derrick Stolee <stolee@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-06fetch: let --jobs=<n> parallelize --multiple, tooLibravatar Johannes Schindelin1-17/+107
So far, `--jobs=<n>` only parallelizes submodule fetches/clones, not `--multiple` fetches, which is unintuitive, given that the option's name does not say anything about submodules in particular. Let's change that. With this patch, also fetches from multiple remotes are parallelized. For backwards-compatibility (and to prepare for a use case where submodule and multiple-remote fetches may need different parallelization limits), the config setting `submodule.fetchJobs` still only controls the submodule part of `git fetch`, while the newly-introduced setting `fetch.parallel` controls both (but can be overridden for submodules with `submodule.fetchJobs`). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-03fetch: add trace2 instrumentationLibravatar Josh Steadmon1-7/+15
Add trace2 regions to fetch-pack.c and builtins/fetch.c to better track time spent in the various phases of a fetch: * listing refs * negotiation for protocol versions v0-v2 * fetching refs * consuming refs Signed-off-by: Josh Steadmon <steadmon@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-09-30Merge branch 'ds/commit-graph-on-fetch'Libravatar Junio C Hamano1-0/+15
A configuration variable tells "git fetch" to write the commit graph after finishing. * ds/commit-graph-on-fetch: fetch: add fetch.writeCommitGraph config setting
2019-09-18Merge branch 'md/list-objects-filter-combo'Libravatar Junio C Hamano1-6/+3
The list-objects-filter API (used to create a sparse/lazy clone) learned to take a combined filter specification. * md/list-objects-filter-combo: list-objects-filter-options: make parser void list-objects-filter-options: clean up use of ALLOC_GROW list-objects-filter-options: allow mult. --filter strbuf: give URL-encoding API a char predicate fn list-objects-filter-options: make filter_spec a string_list list-objects-filter-options: move error check up list-objects-filter: implement composite filters list-objects-filter-options: always supply *errbuf list-objects-filter: put omits set in filter struct list-objects-filter: encapsulate filter components
2019-09-18Merge branch 'cc/multi-promisor'Libravatar Junio C Hamano1-19/+10
Teach the lazy clone machinery that there can be more than one promisor remote and consult them in order when downloading missing objects on demand. * cc/multi-promisor: Move core_partial_clone_filter_default to promisor-remote.c Move repository_format_partial_clone to promisor-remote.c Remove fetch-object.{c,h} in favor of promisor-remote.{c,h} remote: add promisor and partial clone config to the doc partial-clone: add multiple remotes in the doc t0410: test fetching from many promisor remotes builtin/fetch: remove unique promisor remote limitation promisor-remote: parse remote.*.partialclonefilter Use promisor_remote_get_direct() and has_promisor_remote() promisor-remote: use repository_format_partial_clone promisor-remote: add promisor_remote_reinit() promisor-remote: implement promisor_remote_get_direct() Add initial support for many promisor remotes fetch-object: make functions return an error code t0410: remove pipes after git commands
2019-09-16fetch: use oidset to keep the want OIDs for faster lookupLibravatar Masaya Suzuki1-8/+10
During git-fetch, the client checks if the advertised tags' OIDs are already in the fetch request's want OID set. This check is done in a linear scan. For a repository that has a lot of refs, repeating this scan takes 15+ minutes. In order to speed this up, create a oid_set for other refs' OIDs. Signed-off-by: Masaya Suzuki <masayasuzuki@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-09-03fetch: add fetch.writeCommitGraph config settingLibravatar Derrick Stolee1-0/+15
The commit-graph feature is now on by default, and is being written during 'git gc' by default. Typically, Git only writes a commit-graph when a 'git gc --auto' command passes the gc.auto setting to actualy do work. This means that a commit-graph will typically fall behind the commits that are being used every day. To stay updated with the latest commits, add a step to 'git fetch' to write a commit-graph after fetching new objects. The fetch.writeCommitGraph config setting enables writing a split commit-graph, so on average the cost of writing this file is very small. Occasionally, the commit-graph chain will collapse to a single level, and this could be slow for very large repos. For additional use, adjust the default to be true when feature.experimental is enabled. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-19pull, fetch: add --set-upstream optionLibravatar Corentin BOMPARD1-1/+50
Add the --set-upstream option to git pull/fetch which lets the user set the upstream configuration (branch.<current-branch-name>.merge and branch.<current-branch-name>.remote) for the current branch. A typical use-case is: git clone http://example.com/my-public-fork git remote add main http://example.com/project-main-repo git pull --set-upstream main master or, instead of the last line: git fetch --set-upstream main master git merge # or git rebase This is mostly equivalent to cloning project-main-repo (which sets upsteam) and then "git remote add" my-public-fork, but may feel more natural for people using a hosting system which allows forking from the web UI. This functionality is analog to "git push --set-upstream". Signed-off-by: Corentin BOMPARD <corentin.bompard@etu.univ-lyon1.fr> Signed-off-by: Nathan BERBEZIER <nathan.berbezier@etu.univ-lyon1.fr> Signed-off-by: Pablo CHABANNE <pablo.chabanne@etu.univ-lyon1.fr> Signed-off-by: Matthieu Moy <git@matthieu-moy.fr> Patch-edited-by: Matthieu Moy <git@matthieu-moy.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-06l10n: reformat some localized strings for v2.23.0Libravatar Jean-Noël Avila1-4/+11
Signed-off-by: Jean-Noël Avila <jn.avila@free.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-07-09Merge branch 'ds/fetch-disable-force-notice'Libravatar Junio C Hamano1-1/+33
"git fetch" and "git pull" reports when a fetch results in non-fast-forward updates to let the user notice unusual situation. The commands learned "--no-shown-forced-updates" option to disable this safety feature. * ds/fetch-disable-force-notice: pull: add --[no-]show-forced-updates passthrough fetch: warn about forced updates in branch listing fetch: add --[no-]show-forced-updates argument
2019-07-09Merge branch 'nd/fetch-multi-gc-once'Libravatar Junio C Hamano1-6/+11
"git fetch" that grabs from a group of remotes learned to run the auto-gc only once at the very end. * nd/fetch-multi-gc-once: fetch: only run 'gc' once when fetching multiple remotes
2019-07-09Merge branch 'ds/close-object-store'Libravatar Junio C Hamano1-1/+1
The commit-graph file is now part of the "files that the runtime may keep open file descriptors on, all of which would need to be closed when done with the object store", and the file descriptor to an existing commit-graph file now is closed before "gc" finalizes a new instance to replace it. * ds/close-object-store: packfile: rename close_all_packs to close_object_store packfile: close commit-graph in close_all_packs commit-graph: use raw_object_store when closing
2019-07-09Merge branch 'fc/fetch-with-import-fix'Libravatar Junio C Hamano1-10/+18
Code restructuring during 2.20 period broke fetching tags via "import" based transports. * fc/fetch-with-import-fix: fetch: fix regression with transport helpers fetch: make the code more understandable fetch: trivial cleanup t5801 (remote-helpers): add test to fetch tags t5801 (remote-helpers): cleanup refspec stuff
2019-06-28list-objects-filter-options: make filter_spec a string_listLibravatar Matthew DeVore1-6/+3
Make the filter_spec string a string_list rather than a raw C string. The list of strings must be concatted together to make a complete filter_spec. A future patch will use this capability to build "combine:" filter specs gradually. A strbuf would seem to be a more natural choice for this object, but it unfortunately requires initialization besides just zero'ing out the memory. This results in all container structs, and all containers of those structs, etc., to also require initialization. Initializing them all would be more cumbersome that simply using a string_list, which behaves properly when its contents are zero'd. For the purposes of code simplification, change behavior in how filter specs are conveyed over the protocol: do not normalize the tree:<depth> filter specs since there should be no server in existence that supports tree:# but not tree:#k etc. Helped-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Matthew DeVore <matvore@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-06-25builtin/fetch: remove unique promisor remote limitationLibravatar Christian Couder1-15/+5
As the infrastructure for more than one promisor remote has been introduced in previous patches, we can remove code that forbids the registration of more than one promisor remote. Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-06-25promisor-remote: parse remote.*.partialclonefilterLibravatar Christian Couder1-1/+1
This makes it possible to specify a different partial clone filter for each promisor remote. Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-06-25Use promisor_remote_get_direct() and has_promisor_remote()Libravatar Christian Couder1-5/+6
Instead of using the repository_format_partial_clone global and fetch_objects() directly, let's use has_promisor_remote() and promisor_remote_get_direct(). This way all the configured promisor remotes will be taken into account, not only the one specified by extensions.partialClone. Also when cloning or fetching using a partial clone filter, remote.origin.promisor will be set to "true" instead of setting extensions.partialClone to "origin". This makes it possible to use many promisor remote just by fetching from them. Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-06-21fetch: warn about forced updates in branch listingLibravatar Derrick Stolee1-1/+24
The --[no-]show-forced-updates option in 'git fetch' can be confusing for some users, especially if it is enabled via config setting and not by argument. Add advice to warn the user that the (forced update) messages were not listed. Additionally, warn users when the forced update check takes longer than ten seconds, and recommend that they disable the check. These messages can be disabled by the advice.fetchShowForcedUpdates config setting. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-06-21fetch: add --[no-]show-forced-updates argumentLibravatar Derrick Stolee1-1/+10
After updating a set of remove refs during a 'git fetch', we walk the commits in the new ref value and not in the old ref value to discover if the update was a forced update. This results in two things happening during the command: 1. The line including the ref update has an additional "(forced-update)" marker at the end. 2. The ref log for that remote branch includes a bit saying that update is a forced update. For many situations, this forced-update message happens infrequently, or is a small bit of information among many ref updates. Many users ignore these messages, but the calculation required here slows down their fetches significantly. Keep in mind that they do not have the opportunity to calculate a commit-graph file containing the newly-fetched commits, so these comparisons can be very slow. Add a '--[no-]show-forced-updates' option that allows a user to skip this calculation. The only permanent result is dropping the forced-update bit in the reflog. Include a new fetch.showForcedUpdates config setting that allows this behavior without including the argument in every command. The config setting is overridden by the command-line arguments. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-06-19fetch: only run 'gc' once when fetching multiple remotesLibravatar Nguyễn Thái Ngọc Duy1-6/+11
In multiple remotes mode, git-fetch is launched for n-1 remotes and the last remote is handled by the current process. Each of these processes will in turn run 'gc' at the end. This is not really a problem because even if multiple 'gc --auto' is run at the same time we still handle it correctly. It does show multiple "auto packing in the background" messages though. And we may waste some resources when gc actually runs because we still do some stuff before checking the lock and moving it to background. So let's try to avoid that. We should only need one 'gc' run after all objects and references are added anyway. Add a new option --no-auto-gc that will be used by those n-1 processes. 'gc --auto' will always run on the main fetch process (*). (*) even if we fetch remotes in parallel at some point in future, this should still be fine because we should "join" all those processes before this step. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Acked-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-06-12packfile: rename close_all_packs to close_object_storeLibravatar Derrick Stolee1-1/+1
The close_all_packs() method is now responsible for more than just pack-files. It also closes the commit-graph and the multi-pack-index. Rename the function to be more descriptive of its larger role. The name also fits because the input parameter is a raw_object_store. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>