summaryrefslogtreecommitdiff
path: root/builtin/fetch.c
AgeCommit message (Collapse)AuthorFilesLines
2017-01-17Merge branch 'jt/fetch-no-redundant-tag-fetch-map' into maintLibravatar Junio C Hamano1-3/+0
Code cleanup to avoid using redundant refspecs while fetching with the --tags option. * jt/fetch-no-redundant-tag-fetch-map: fetch: do not redundantly calculate tag refmap
2016-11-11fetch: do not redundantly calculate tag refmapLibravatar Jonathan Tan1-3/+0
builtin/fetch.c redundantly calculates refmaps for tags twice. Remove the first calculation. This is only a code simplification and slight performance improvement - the result is unchanged, as the redundant refmaps are subsequently removed by the invocation to "ref_remove_duplicates" anyway. This was introduced in commit c5a84e9 ("fetch --tags: fetch tags *in addition to* other stuff", 2013-10-29) when modifying the effect of the --tags parameter to "git fetch". The refmap-for-tag calculation was copied instead of moved. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Reviewed-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-10-27Merge branch 'jc/abbrev-auto'Libravatar Junio C Hamano1-16/+21
"git push" and "git fetch" reports from what old object to what new object each ref was updated, using abbreviated refnames, and they attempt to align the columns for this and other pieces of information. The way these codepaths compute how many display columns to allocate for the object names portion of this output has been updated to match the recent "auto scale the default abbreviation length" change. * jc/abbrev-auto: transport: compute summary-width dynamically transport: allow summary-width to be computed dynamically fetch: pass summary_width down the callchain transport: pass summary_width down the callchain
2016-10-27Merge branch 'lt/abbrev-auto'Libravatar Junio C Hamano1-0/+3
Allow the default abbreviation length, which has historically been 7, to scale as the repository grows. The logic suggests to use 12 hexdigits for the Linux kernel, and 9 to 10 for Git itself. * lt/abbrev-auto: abbrev: auto size the default abbreviation abbrev: prepare for new world order abbrev: add FALLBACK_DEFAULT_ABBREV to prepare for auto sizing
2016-10-26Merge branch 'jk/fetch-quick-tag-following'Libravatar Junio C Hamano1-4/+7
When fetching from a remote that has many tags that are irrelevant to branches we are following, we used to waste way too many cycles when checking if the object pointed at by a tag (that we are not going to fetch!) exists in our repository too carefully. * jk/fetch-quick-tag-following: fetch: use "quick" has_sha1_file for tag following
2016-10-21transport: allow summary-width to be computed dynamicallyLibravatar Junio C Hamano1-2/+2
Now we have identified three callchains that have a set of refs that they want to show their <old, new> object names in an aligned output, we can replace their reference to the constant TRANSPORT_SUMMARY_WIDTH with a helper function call to transport_summary_width() that takes the set of ref as a parameter. This step does not yet iterate over the refs and compute, which is left as an exercise to the readers. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-10-21fetch: pass summary_width down the callchainLibravatar Junio C Hamano1-16/+21
The leaf function on the "fetch" side that uses TRANSPORT_SUMMARY_WIDTH constant is builtin/fetch.c::format_display() and it has two distinct callchains. The one that reports the primary result of fetch originates at store_updated_refs(); the other one that reports the pruning of the remote-tracking refs originates at prune_refs(). Teach these two places to pass summary_width down the callchain, just like we did for the "push" side in the previous commit. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-10-14fetch: use "quick" has_sha1_file for tag followingLibravatar Jeff King1-4/+7
When we auto-follow tags in a fetch, we look at all of the tags advertised by the remote and fetch ones where we don't already have the tag, but we do have the object it peels to. This involves a lot of calls to has_sha1_file(), some of which we can reasonably expect to fail. Since 45e8a74 (has_sha1_file: re-check pack directory before giving up, 2013-08-30), this may cause many calls to reprepare_packed_git(), which is potentially expensive. This has gone unnoticed for several years because it requires a fairly unique setup to matter: 1. You need to have a lot of packs on the client side to make reprepare_packed_git() expensive (the most expensive part is finding duplicates in an unsorted list, which is currently quadratic). 2. You need a large number of tag refs on the server side that are candidates for auto-following (i.e., that the client doesn't have). Each one triggers a re-read of the pack directory. 3. Under normal circumstances, the client would auto-follow those tags and after one large fetch, (2) would no longer be true. But if those tags point to history which is disconnected from what the client otherwise fetches, then it will never auto-follow, and those candidates will impact it on every fetch. So when all three are true, each fetch pays an extra O(nr_tags * nr_packs^2) cost, mostly in string comparisons on the pack names. This was exacerbated by 47bf4b0 (prepare_packed_git_one: refactor duplicate-pack check, 2014-06-30) which uses a slightly more expensive string check, under the assumption that the duplicate check doesn't happen very often (and it shouldn't; the real problem here is how often we are calling reprepare_packed_git()). This patch teaches fetch to use HAS_SHA1_QUICK to sacrifice accuracy for speed, in cases where we might be racy with a simultaneous repack. This is similar to the fix in 0eeb077 (index-pack: avoid excessive re-reading of pack directory, 2015-06-09). As with that case, it's OK for has_sha1_file() occasionally say "no I don't have it" when we do, because the worst case is not a corruption, but simply that we may fail to auto-follow a tag that points to it. Here are results from the included perf script, which sets up a situation similar to the one described above: Test HEAD^ HEAD ---------------------------------------------------------- 5550.4: fetch 11.21(10.42+0.78) 0.08(0.04+0.02) -99.3% Reported-by: Vegard Nossum <vegard.nossum@oracle.com> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-10-10Merge branch 'nd/shallow-deepen'Libravatar Junio C Hamano1-7/+43
The existing "git fetch --depth=<n>" option was hard to use correctly when making the history of an existing shallow clone deeper. A new option, "--deepen=<n>", has been added to make this easier to use. "git clone" also learned "--shallow-since=<date>" and "--shallow-exclude=<tag>" options to make it easier to specify "I am interested only in the recent N months worth of history" and "Give me only the history since that version". * nd/shallow-deepen: (27 commits) fetch, upload-pack: --deepen=N extends shallow boundary by N commits upload-pack: add get_reachable_list() upload-pack: split check_unreachable() in two, prep for get_reachable_list() t5500, t5539: tests for shallow depth excluding a ref clone: define shallow clone boundary with --shallow-exclude fetch: define shallow boundary with --shallow-exclude upload-pack: support define shallow boundary by excluding revisions refs: add expand_ref() t5500, t5539: tests for shallow depth since a specific date clone: define shallow clone boundary based on time with --shallow-since fetch: define shallow boundary with --shallow-since upload-pack: add deepen-since to cut shallow repos based on time shallow.c: implement a generic shallow boundary finder based on rev-list fetch-pack: use a separate flag for fetch in deepening mode fetch-pack.c: mark strings for translating fetch-pack: use a common function for verbose printing fetch-pack: use skip_prefix() instead of starts_with() upload-pack: move rev-list code out of check_non_tip() upload-pack: make check_non_tip() clean things up on error upload-pack: tighten number parsing at "deepen" lines ...
2016-10-03abbrev: add FALLBACK_DEFAULT_ABBREV to prepare for auto sizingLibravatar Junio C Hamano1-0/+3
We'll be introducing a new way to decide the default abbreviation length by initialising DEFAULT_ABBREV to -1 to signal the first call to "find unique abbreviation" codepath to compute a reasonable value based on the number of objects we have to avoid collisions. We have long relied on DEFAULT_ABBREV being a positive concrete value that is used as the abbreviation length when no extra configuration or command line option has overridden it. Some codepaths wants to use such a positive concrete default value even before making their first request to actually trigger the computation for the auto sized default. Introduce FALLBACK_DEFAULT_ABBREV and use it to the code that attempts to align the report from "git fetch". For now, this macro is also used to initialize the default_abbrev variable, but the auto-sizing code will use -1 and then use the value of FALLBACK_DEFAULT_ABBREV as the starting point of auto-sizing. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-03Merge branch 'jk/push-progress'Libravatar Junio C Hamano1-2/+4
"git push" and "git clone" learned to give better progress meters to the end user who is waiting on the terminal. * jk/push-progress: receive-pack: send keepalives during quiet periods receive-pack: turn on connectivity progress receive-pack: relay connectivity errors to sideband receive-pack: turn on index-pack resolving progress index-pack: add flag for showing delta-resolution progress clone: use a real progress meter for connectivity check check_connected: add progress flag check_connected: relay errors to alternate descriptor check_everything_connected: use a struct with named options check_everything_connected: convert to argv_array rev-list: add optional progress reporting check_everything_connected: always pass --quiet to rev-list
2016-07-25Merge branch 'mh/ref-iterators'Libravatar Junio C Hamano1-1/+1
The API to iterate over all the refs (i.e. for_each_ref(), etc.) has been revamped. * mh/ref-iterators: for_each_reflog(): reimplement using iterators dir_iterator: new API for iterating over a directory tree for_each_reflog(): don't abort for bad references do_for_each_ref(): reimplement using reference iteration refs: introduce an iterator interface ref_resolves_to_object(): new function entry_resolves_to_object(): rename function from ref_resolves_to_object() get_ref_cache(): only create an instance if there is a submodule remote rm: handle symbolic refs correctly delete_refs(): add a flags argument refs: use name "prefix" consistently do_for_each_ref(): move docstring to the header file refs: remove unnecessary "extern" keywords
2016-07-20check_everything_connected: use a struct with named optionsLibravatar Jeff King1-2/+4
The number of variants of check_everything_connected has grown over the years, so that the "real" function takes several possibly-zero, possibly-NULL arguments. We hid the complexity behind some wrapper functions, but this doesn't scale well when we want to add new options. If we add more wrapper variants to handle the new options, then we can get a combinatorial explosion when those options might be used together (right now nobody wants to use both "shallow" and "transport" together, so we get by with just a few wrappers). If instead we add new parameters to each function, each of which can have a default value, then callers who want the defaults end up with confusing invocations like: check_everything_connected(fn, 0, data, -1, 0, NULL); where it is unclear which parameter is which (and every caller needs updated when we add new options). Instead, let's add a struct to hold all of the optional parameters. This is a little more verbose for the callers (who have to declare the struct and fill it in), but it makes their code much easier to follow, because every option is named as it is set (and unused options do not have to be mentioned at all). Note that we could also stick the iteration function and its callback data into the option struct, too. But since those are required for each call, by avoiding doing so, we can let very simple callers just pass "NULL" for the options and not worry about the struct at all. While we're touching each site, let's also rename the function to check_connected(). The existing name was quite long, and not all of the wrappers even used the full name. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-19Merge branch 'nd/fetch-ref-summary'Libravatar Junio C Hamano1-40/+157
Improve the look of the way "git fetch" reports what happened to each ref that was fetched. * nd/fetch-ref-summary: fetch: reduce duplicate in ref update status lines with placeholder fetch: align all "remote -> local" output fetch: change flag code for displaying tag update and deleted ref fetch: refactor ref update status formatting code git-fetch.txt: document fetch output
2016-07-11Merge branch 'km/fetch-do-not-free-remote-name' into maintLibravatar Junio C Hamano1-4/+2
The ownership rule for the piece of memory that hold references to be fetched in "git fetch" was screwy, which has been cleaned up. * km/fetch-do-not-free-remote-name: builtin/fetch.c: don't free remote->name after fetch
2016-07-06Merge branch 'km/fetch-do-not-free-remote-name'Libravatar Junio C Hamano1-4/+2
The ownership rule for the piece of memory that hold references to be fetched in "git fetch" was screwy, which has been cleaned up. * km/fetch-do-not-free-remote-name: builtin/fetch.c: don't free remote->name after fetch
2016-07-06fetch: reduce duplicate in ref update status lines with placeholderLibravatar Nguyễn Thái Ngọc Duy1-1/+76
In the "remote -> local" line, if either ref is a substring of the other, the common part in the other string is replaced with "*". For example abc -> origin/abc refs/pull/123/head -> pull/123 become abc -> origin/* refs/*/head -> pull/123 Activated with fetch.output=compact. For the record, this output is not perfect. A single giant ref can push all refs very far to the right and likely be wrapped around. We may have a few options: - exclude these long lines smarter - break the line after "->", exclude it from column width calculation - implement a new format, { -> origin/}foo, which makes the problem go away at the cost of a bit harder to read - reverse all the arrows so we have "* <- looong-ref", again still hard to read. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-06fetch: align all "remote -> local" outputLibravatar Nguyễn Thái Ngọc Duy1-2/+45
We do align "remote -> local" output by allocating 10 columns to "remote". That produces aligned output only for short refs. An extra pass is performed to find the longest remote ref name (that does not produce a line longer than terminal width) to produce better aligned output. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-27fetch: change flag code for displaying tag update and deleted refLibravatar Nguyễn Thái Ngọc Duy1-2/+2
This makes the fetch flag code consistent with push, where '-' means deleted ref. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-27fetch: refactor ref update status formatting codeLibravatar Nguyễn Thái Ngọc Duy1-39/+38
This makes it easier to change the formatting later. And it makes sure translators cannot mess up format specifiers and break Git. There are a couple call sites where the length of the second column is TRANSPORT_SUMMARY_WIDTH instead of calculated by TRANSPORT_SUMMARY(), which is enforced now. The result should be the same because these call sites do not contain characters outside ASCII range. The two strbuf_addf() calls instead of one is mostly to reduce diff-noise in a future patch where "ref -> ref" is reformatted differently. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-20delete_refs(): add a flags argumentLibravatar Michael Haggerty1-1/+1
This will be useful for passing REF_NODEREF through. Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-14builtin/fetch.c: don't free remote->name after fetchLibravatar Keith McGuigan1-4/+2
Make fetch's string_list of remote names own all of its string items (strdup'ing when necessary) so that it can deallocate them safely when clearing. Signed-off-by: Keith McGuigan <kmcguigan@twopensource.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-13fetch, upload-pack: --deepen=N extends shallow boundary by N commitsLibravatar Nguyễn Thái Ngọc Duy1-1/+13
In git-fetch, --depth argument is always relative with the latest remote refs. This makes it a bit difficult to cover this use case, where the user wants to make the shallow history, say 3 levels deeper. It would work if remote refs have not moved yet, but nobody can guarantee that, especially when that use case is performed a couple months after the last clone or "git fetch --depth". Also, modifying shallow boundary using --depth does not work well with clones created by --since or --not. This patch fixes that. A new argument --deepen=<N> will add <N> more (*) parent commits to the current history regardless of where remote refs are. Have/Want negotiation is still respected. So if remote refs move, the server will send two chunks: one between "have" and "want" and another to extend shallow history. In theory, the client could send no "want"s in order to get the second chunk only. But the protocol does not allow that. Either you send no want lines, which means ls-remote; or you have to send at least one want line that carries deep-relative to the server.. The main work was done by Dongcan Jiang. I fixed it up here and there. And of course all the bugs belong to me. (*) We could even support --deepen=<N> where <N> is negative. In that case we can cut some history from the shallow clone. This operation (and --depth=<shorter depth>) does not require interaction with remote side (and more complicated to implement as a result). Helped-by: Duy Nguyen <pclouds@gmail.com> Helped-by: Eric Sunshine <sunshine@sunshineco.com> Helped-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Dongcan Jiang <dongcan.jiang@gmail.com> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-13fetch: define shallow boundary with --shallow-excludeLibravatar Nguyễn Thái Ngọc Duy1-3/+10
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-13fetch: define shallow boundary with --shallow-sinceLibravatar Nguyễn Thái Ngọc Duy1-6/+23
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-17Merge branch 'nd/error-errno'Libravatar Junio C Hamano1-2/+2
The code for warning_errno/die_errno has been refactored and a new error_errno() reporting helper is introduced. * nd/error-errno: (41 commits) wrapper.c: use warning_errno() vcs-svn: use error_errno() upload-pack.c: use error_errno() unpack-trees.c: use error_errno() transport-helper.c: use error_errno() sha1_file.c: use {error,die,warning}_errno() server-info.c: use error_errno() sequencer.c: use error_errno() run-command.c: use error_errno() rerere.c: use error_errno() and warning_errno() reachable.c: use error_errno() mailmap.c: use error_errno() ident.c: use warning_errno() http.c: use error_errno() and warning_errno() grep.c: use error_errno() gpg-interface.c: use error_errno() fast-import.c: use error_errno() entry.c: use error_errno() editor.c: use error_errno() diff-no-index.c: use error_errno() ...
2016-05-09builtin/fetch.c: use error_errno()Libravatar Nguyễn Thái Ngọc Duy1-2/+2
A couple of newlines are also removed, because both error() and error_errno() automatically append a newline. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-04-06Merge branch 'sb/submodule-parallel-update'Libravatar Junio C Hamano1-1/+1
A major part of "git submodule update" has been ported to C to take advantage of the recently added framework to run download tasks in parallel. * sb/submodule-parallel-update: clone: allow an explicit argument for parallel submodule clones submodule update: expose parallelism to the user submodule helper: remove double 'fatal: ' prefix git submodule update: have a dedicated helper for cloning run_processes_parallel: rename parameters for the callbacks run_processes_parallel: treat output of children as byte array submodule update: direct error message to stderr fetching submodules: respect `submodule.fetchJobs` config option submodule-config: drop check against NULL submodule-config: keep update strategy around
2016-03-01fetching submodules: respect `submodule.fetchJobs` config optionLibravatar Stefan Beller1-1/+1
This allows to configure fetching and updating in parallel without having the command line option. This moved the responsibility to determine how many parallel processes to start from builtin/fetch to submodule.c as we need a way to communicate "The user did not specify the number of parallel processes in the command line options" in the builtin fetch. The submodule code takes care of the precedence (CLI > config > default). Reviewed-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-02-26Merge branch 'jk/tighten-alloc'Libravatar Junio C Hamano1-1/+1
Update various codepaths to avoid manually-counted malloc(). * jk/tighten-alloc: (22 commits) ewah: convert to REALLOC_ARRAY, etc convert ewah/bitmap code to use xmalloc diff_populate_gitlink: use a strbuf transport_anonymize_url: use xstrfmt git-compat-util: drop mempcpy compat code sequencer: simplify memory allocation of get_message test-path-utils: fix normalize_path_copy output buffer size fetch-pack: simplify add_sought_entry fast-import: simplify allocation in start_packfile write_untracked_extension: use FLEX_ALLOC helper prepare_{git,shell}_cmd: use argv_array use st_add and st_mult for allocation size computation convert trivial cases to FLEX_ARRAY macros use xmallocz to avoid size arithmetic convert trivial cases to ALLOC_ARRAY convert manual allocations to argv_array argv-array: add detach function add helpers for allocating flex-array structs harden REALLOC_ARRAY and xcalloc against size_t overflow tree-diff: catch integer overflow in combine_diff_path allocation ...
2016-02-26Merge branch 'tg/git-remote'Libravatar Junio C Hamano1-3/+2
The internal API to interact with "remote.*" configuration variables has been streamlined. * tg/git-remote: remote: use remote_is_configured() for add and rename remote: actually check if remote exits remote: simplify remote_is_configured() remote: use parse_config_key
2016-02-24Merge branch 'ew/force-ipv4'Libravatar Junio C Hamano1-0/+6
"git fetch" and friends that make network connections can now be told to only use ipv4 (or ipv6). * ew/force-ipv4: connect & http: support -4 and -6 switches for remote operations
2016-02-22use st_add and st_mult for allocation size computationLibravatar Jeff King1-1/+1
If our size computation overflows size_t, we may allocate a much smaller buffer than we expected and overflow it. It's probably impossible to trigger an overflow in most of these sites in practice, but it is easy enough convert their additions and multiplications into overflow-checking variants. This may be fixing real bugs, and it makes auditing the code easier. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-02-16remote: simplify remote_is_configured()Libravatar Thomas Gummerer1-3/+2
The remote_is_configured() function allows checking whether a remote exists or not. The function however only works if remote_get() wasn't called before calling it. In addition, it only checks the configuration for remotes, but not remotes or branches files. Make use of the origin member of struct remote instead, which indicates where the remote comes from. It will be set to some value if the remote is configured in any file in the repository, but is initialized to 0 if the remote is only created in make_remote(). Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com> Reviewed-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-02-12connect & http: support -4 and -6 switches for remote operationsLibravatar Eric Wong1-0/+6
Sometimes it is necessary to force IPv4-only or IPv6-only operation on networks where name lookups may return a non-routable address and stall remote operations. The ssh(1) command has an equivalent switches which we may pass when we run them. There may be old ssh(1) implementations out there which do not support these switches; they should report the appropriate error in that case. rsync support is untouched for now since it is deprecated and scheduled to be removed. Signed-off-by: Eric Wong <normalperson@yhbt.net> Reviewed-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-02-05Merge branch 'js/close-packs-before-gc' into maintLibravatar Junio C Hamano1-0/+2
Many codepaths that run "gc --auto" before exiting kept packfiles mapped and left the file descriptors to them open, which was not friendly to systems that cannot remove files that are open. They now close the packs before doing so. * js/close-packs-before-gc: receive-pack: release pack files before garbage-collecting merge: release pack files before garbage-collecting am: release pack files before garbage-collecting fetch: release pack files before garbage-collecting
2016-02-05Merge branch 'js/fopen-harder' into maintLibravatar Junio C Hamano1-1/+1
Some codepaths used fopen(3) when opening a fixed path in $GIT_DIR (e.g. COMMIT_EDITMSG) that is meant to be left after the command is done. This however did not work well if the repository is set to be shared with core.sharedRepository and the umask of the previous user is tighter. They have been made to work better by calling unlink(2) and retrying after fopen(3) fails with EPERM. * js/fopen-harder: Handle more file writes correctly in shared repos commit: allow editing the commit message even in shared repos
2016-01-26Merge branch 'js/close-packs-before-gc'Libravatar Junio C Hamano1-0/+2
Many codepaths that run "gc --auto" before exiting kept packfiles mapped and left the file descriptors to them open, which was not friendly to systems that cannot remove files that are open. They now close the packs before doing so. * js/close-packs-before-gc: receive-pack: release pack files before garbage-collecting merge: release pack files before garbage-collecting am: release pack files before garbage-collecting fetch: release pack files before garbage-collecting
2016-01-20Merge branch 'js/fopen-harder'Libravatar Junio C Hamano1-1/+1
Some codepaths used fopen(3) when opening a fixed path in $GIT_DIR (e.g. COMMIT_EDITMSG) that is meant to be left after the command is done. This however did not work well if the repository is set to be shared with core.sharedRepository and the umask of the previous user is tighter. They have been made to work better by calling unlink(2) and retrying after fopen(3) fails with EPERM. * js/fopen-harder: Handle more file writes correctly in shared repos commit: allow editing the commit message even in shared repos
2016-01-13fetch: release pack files before garbage-collectingLibravatar Johannes Schindelin1-0/+2
Before auto-gc'ing, we need to make sure that the pack files are released in case they need to be repacked and garbage-collected. This fixes https://github.com/git-for-windows/git/issues/500 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Reviewed-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-01-11Handle more file writes correctly in shared reposLibravatar Johannes Schindelin1-1/+1
In shared repositories, we have to be careful when writing files whose permissions do not allow users other than the owner to write them. In particular, we force the marks file of fast-export and the FETCH_HEAD when fetching to be rewritten from scratch. This commit does not touch other calls to fopen() that want to write files: - commands that write to working tree files (core.sharedRepository does not affect permission bits of working tree files), e.g. .rej file created by "apply --reject", result of applying a previous conflict resolution by "rerere", "git merge-file". - git am, when splitting mails (git-am correctly cleans up its directory after finishing, so there is no need to share those files between users) - git submodule clone, when writing the .git file, because the file will not be overwritten - git_terminal_prompt() in compat/terminal.c, because it is not writing to a file at all - git diff --output, because the output file is clearly not intended to be shared between the users of the current repository - git fast-import, when writing a crash report, because the reports' file names are unique due to an embedded process ID - mailinfo() in mailinfo.c, because the output is clearly not intended to be shared between the users of the current repository - check_or_regenerate_marks() in remote-testsvn.c, because this is only used for Git's internal testing - git fsck, when writing lost&found blobs (this should probably be changed, but left as a low-hanging fruit for future contributors). Note that this patch does not touch callers of write_file() and write_file_gently(), which would benefit from the same scrutiny as to usage in shared repositories. Most notable users are branch, daemon, submodule & worktree, and a worrisome call in transport.c when updating one ref (which ignores the shared flag). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-12-16submodules: allow parallel fetching, add tests and documentationLibravatar Stefan Beller1-1/+5
This enables the work of the previous patches. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-11-20Remove get_object_hash.Libravatar brian m. carlson1-2/+2
Convert all instances of get_object_hash to use an appropriate reference to the hash member of the oid member of struct object. This provides no functional change, as it is essentially a macro substitution. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Jeff King <peff@peff.net>
2015-11-20Add several uses of get_object_hash.Libravatar brian m. carlson1-2/+2
Convert most instances where the sha1 member of struct object is dereferenced to use get_object_hash. Most instances that are passed to functions that have versions taking struct object_id, such as get_sha1_hex/get_oid_hex, or instances that can be trivially converted to use struct object_id instead, are not converted. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Jeff King <peff@peff.net>
2015-11-20Convert struct ref to use object_id.Libravatar brian m. carlson1-25/+25
Use struct object_id in three fields in struct ref and convert all the necessary places that use it. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Jeff King <peff@peff.net>
2015-10-05use alloc_ref rather than hand-allocating "struct ref"Libravatar Jeff King1-2/+1
This saves us some manual computation, and eliminates a call to strcpy. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-10-05transport: use strbufs for status table "quickref" stringsLibravatar Jeff King1-10/+12
We generate range strings like "1234abcd...5678efab" for use in the the fetch and push status tables. We use fixed-size buffers along with strcat to do so. These aren't buggy, as our manual size computation is correct, but there's nothing checking that this is so. Let's switch them to strbufs instead, which are obviously correct, and make it easier to audit the code base for problematic calls to strcat(). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-09-25fetch: replace static buffer with xstrfmtLibravatar Jeff King1-5/+2
We parse the INFINITE_DEPTH constant into a static, fixed-size buffer using sprintf. This buffer is sufficiently large for the current constant, but it's a suspicious pattern, as the constant is defined far away, and it's not immediately obvious that 12 bytes are large enough to hold it. We can just use xstrfmt here, which gets rid of any question of the buffer size. It also removes any concerns with object lifetime, which means we do not have to wonder why this buffer deep within a conditional is marked "static" (we never free our newly allocated result, of course, but that's OK; it's global that lasts the lifetime of the whole program anyway). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-08-31Merge branch 'hv/submodule-config'Libravatar Junio C Hamano1-0/+1
The gitmodules API accessed from the C code learned to cache stuff lazily. * hv/submodule-config: submodule: allow erroneous values for the fetchRecurseSubmodules option submodule: use new config API for worktree configurations submodule: extract functions for config set and lookup submodule: implement a config API for lookup of .gitmodules values
2015-08-19Merge branch 'jk/git-path'Libravatar Junio C Hamano1-2/+2
git_path() and mkpath() are handy helper functions but it is easy to misuse, as the callers need to be careful to keep the number of active results below 4. Their uses have been reduced. * jk/git-path: memoize common git-path "constant" files get_repo_path: refactor path-allocation find_hook: keep our own static buffer refs.c: remove_empty_directories can take a strbuf refs.c: avoid git_path assignment in lock_ref_sha1_basic refs.c: avoid repeated git_path calls in rename_tmp_log refs.c: simplify strbufs in reflog setup and writing path.c: drop git_path_submodule refs.c: remove extra git_path calls from read_loose_refs remote.c: drop extraneous local variable from migrate_file prefer mkpathdup to mkpath in assignments prefer git_pathdup to git_path in some possibly-dangerous cases add_to_alternates_file: don't add duplicate entries t5700: modernize style cache.h: complete set of git_path_submodule helpers cache.h: clarify documentation for git_path, et al