summaryrefslogtreecommitdiff
path: root/t/t5500-fetch-pack.sh
AgeCommit message (Collapse)AuthorFilesLines
2018-08-02Merge branch 'jt/fetch-pack-negotiator'Libravatar Junio C Hamano1-0/+33
Code restructuring and a small fix to transport protocol v2 during fetching. * jt/fetch-pack-negotiator: fetch-pack: introduce negotiator API fetch-pack: move common check and marking together fetch-pack: make negotiation-related vars local fetch-pack: use ref adv. to prune "have" sent fetch-pack: directly end negotiation if ACK ready fetch-pack: clear marks before re-marking fetch-pack: split up everything_local()
2018-08-02Merge branch 'es/test-fixes'Libravatar Junio C Hamano1-1/+1
Test clean-up and corrections. * es/test-fixes: (26 commits) t5608: fix broken &&-chain t9119: fix broken &&-chains t9000-t9999: fix broken &&-chains t7000-t7999: fix broken &&-chains t6000-t6999: fix broken &&-chains t5000-t5999: fix broken &&-chains t4000-t4999: fix broken &&-chains t3030: fix broken &&-chains t3000-t3999: fix broken &&-chains t2000-t2999: fix broken &&-chains t1000-t1999: fix broken &&-chains t0000-t0999: fix broken &&-chains t9814: simplify convoluted check that command correctly errors out t9001: fix broken "invoke hook" test t7810: use test_expect_code() instead of hand-rolled comparison t7400: fix broken "submodule add/reconfigure --force" test t7201: drop pointless "exit 0" at end of subshell t6036: fix broken "merge fails but has appropriate contents" tests t5505: modernize and simplify hard-to-digest test t5406: use write_script() instead of birthing shell script manually ...
2018-07-24Merge branch 'jk/fetch-all-peeled-fix'Libravatar Junio C Hamano1-6/+13
Test modernization. * jk/fetch-all-peeled-fix: t5500: prettify non-commit tag tests
2018-07-16t5000-t5999: fix broken &&-chainsLibravatar Eric Sunshine1-1/+1
Signed-off-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-06t5500: prettify non-commit tag testsLibravatar Jeff King1-6/+13
We don't need to use backslash continuation, as the "&&" already provides continuation (and happily soaks up empty lines between commands). We can also expand the multi-line printf into a here-document, which lets us use line breaks more naturally (and avoids another continuation that required us to break the natural indentation). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-06-28Merge branch 'jk/fetch-all-peeled-fix'Libravatar Junio C Hamano1-0/+41
"git fetch-pack --all" used to unnecessarily fail upon seeing an annotated tag that points at an object other than a commit. * jk/fetch-all-peeled-fix: fetch-pack: test explicitly that --all can fetch tag references pointing to non-commits fetch-pack: don't try to fetch peel values with --all
2018-06-25Merge branch 'nd/reject-empty-shallow-request'Libravatar Junio C Hamano1-0/+11
"git fetch --shallow-since=<cutoff>" that specifies the cut-off point that is newer than the existing history used to end up grabbing the entire history. Such a request now errors out. * nd/reject-empty-shallow-request: upload-pack: reject shallow requests that would return nothing
2018-06-15fetch-pack: use ref adv. to prune "have" sentLibravatar Jonathan Tan1-0/+33
In negotiation using protocol v2, fetch-pack sometimes does not make full use of the information obtained in the ref advertisement: specifically, that if the server advertises a commit that the client also has, the client never needs to inform the server that it has the commit's parents, since it can just tell the server that it has the advertised commit and it knows that the server can and will infer the rest. This is because, in do_fetch_pack_v2(), rev_list_insert_ref_oid() is invoked before mark_complete_and_common_ref(). This means that if we have a commit that is both our ref and their ref, it would be enqueued by rev_list_insert_ref_oid() as SEEN, and since it is thus already SEEN, mark_complete_and_common_ref() would not enqueue it. If mark_complete_and_common_ref() were invoked first, as it is in do_fetch_pack() for protocol v0, then mark_complete_and_common_ref() would enqueue it with COMMON_REF | SEEN. The addition of COMMON_REF ensures that its parents are not sent as "have" lines. Change the order in do_fetch_pack_v2() to be consistent with do_fetch_pack(), and to avoid sending unnecessary "have" lines. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-06-13fetch-pack: test explicitly that --all can fetch tag references pointing to ↵Libravatar Kirill Smelkov1-0/+31
non-commits Fetch-pack --all became broken with respect to unusual tags in 5f0fc64513 (fetch-pack: eliminate spurious error messages, 2012-09-09), and was fixed only recently in e9502c0a7f (fetch-pack: don't try to fetch peel values with --all, 2018-06-11). However the test added in e9502c0a7f does not explicitly cover all funky cases. In order to be sure fetching funky tags will never break, let's explicitly test all relevant cases with 4 tag objects pointing to 1) a blob, 2) a tree, 3) a commit, and 4) another tag objects. The referenced tag objects themselves are referenced from under regular refs/tags/* namespace. Before e9502c0a7f `fetch-pack --all` was failing e.g. this way: .../git/t/trash directory.t5500-fetch-pack/fetchall$ git ls-remote .. 44085874... HEAD ... bc4e9e1f... refs/tags/tag-to-blob 038f48ad... refs/tags/tag-to-blob^{} # peeled 520db1f5... refs/tags/tag-to-tree 7395c100... refs/tags/tag-to-tree^{} # peeled .../git/t/trash directory.t5500-fetch-pack/fetchall$ git fetch-pack --all .. fatal: A git upload-pack: not our ref 038f48ad... fatal: The remote end hung up unexpectedly Helped-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Kirill Smelkov <kirr@nexedi.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-06-11fetch-pack: don't try to fetch peel values with --allLibravatar Jeff King1-0/+10
When "fetch-pack --all" sees a tag-to-blob on the remote, it tries to fetch both the tag itself ("refs/tags/foo") and the peeled value that the remote advertises ("refs/tags/foo^{}"). Asking for the object pointed to by the latter can cause upload-pack to complain with "not our ref", since it does not mark the peeled objects with the OUR_REF (unless they were at the tip of some other ref). Arguably upload-pack _should_ be marking those peeled objects. But it never has in the past, since clients would generally just ask for the tag and expect to get the peeled value along with it. And that's how "git fetch" works, as well as older versions of "fetch-pack --all". The problem was introduced by 5f0fc64513 (fetch-pack: eliminate spurious error messages, 2012-09-09). Before then, the matching logic was something like: if (refname is ill-formed) do nothing else if (doing --all) always consider it matched else look through list of sought refs for a match That commit wanted to flip the order of the second two arms of that conditional. But we ended up with: if (refname is ill-formed) do nothing else look through list of sought refs for a match if (--all and no match so far) always consider it matched That means tha an ill-formed ref will trigger the --all conditional block, even though we should just be ignoring it. We can fix that by having a single "else" with all of the well-formed logic, that checks the sought refs and "--all" in the correct order. Reported-by: Kirill Smelkov <kirr@nexedi.com> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-06-04upload-pack: reject shallow requests that would return nothingLibravatar Nguyễn Thái Ngọc Duy1-0/+11
Shallow clones with --shallow-since or --shalow-exclude work by running rev-list to get all reachable commits, then draw a boundary between reachable and unreachable and send "shallow" requests based on that. The code does miss one corner case: if rev-list returns nothing, we'll have no border and we'll send no shallow requests back to the client (i.e. no history cuts). This essentially means a full clone (or a full branch if the client requests just one branch). One example is the oldest commit is older than what is specified by --shallow-since. To avoid this, if rev-list returns nothing, we abort the clone/fetch. The user could adjust their request (e.g. --shallow-since further back in the past) and retry. Another possible option for this case is to fall back to a default depth (like depth 1). But I don't like too much magic that way because we may return something unexpected to the user. If they request "history since 2008" and we return a single depth at 2000, that might break stuff for them. It is better to tell them that something is wrong and let them take the best course of action. Note that we need to die() in get_shallow_commits_by_rev_list() instead of just checking for empty result from its caller deepen_by_rev_list() and handling the error there. The reason is, empty result could be a valid case: if you have commits in year 2013 and you request --shallow-since=year.2000 then you should get a full clone (i.e. empty result). Reported-by: Andreas Krey <a.krey@gmx.de> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-05-23t: make many tests depend less on the refs being filesLibravatar David Turner1-5/+5
Many tests are very focused on the file system representation of the loose and packed refs code. As there are plans to implement other ref storage systems, let's migrate these tests to a form that test the intent of the refs storage system instead of it internals. This will make clear to readers that these tests do not depend on which ref backend is used. The internals of the loose refs backend are still tested in t1400-update-ref.sh, whereas the tests changed in this patch focus on testing other aspects. This patch just takes care of many low hanging fruits. It does not try to completely solves the issue. Helped-by: Stefan Beller <sbeller@google.com> Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de> Signed-off-by: David Turner <dturner@twopensource.com> Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-02-28t5500-fetch-pack: don't check the stderr of a subshellLibravatar SZEDER Gábor1-6/+6
Three "missing reference" tests in 't5500-fetch-pack.sh' fail when the test script is run with '-x' tracing (and using a shell other than a Bash version supporting BASH_XTRACEFD). The reason for those failures is that the tests check a subshell's stderr, which includes the trace of executing commands in that subshell as well, throwing off the comparison with the expected output. Save the stderr of 'git fetch-pack' only instead of the whole subshell, so it remains free from tracing output. After this change t5500 passes with '-x', even when running with /bin/sh. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-12-08fetch: support filtersLibravatar Jeff Hostetler1-0/+36
Teach fetch to support filters. This is only allowed for the remote configured in extensions.partialcloneremote. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-12-08fetch-pack: test support excluding large blobsLibravatar Jonathan Tan1-0/+27
Created tests to verify fetch-pack and upload-pack support for excluding large blobs using --filter=blobs:limit=<n> parameter. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-16fetch-pack: always allow fetching of literal SHA1sLibravatar Jonathan Tan1-0/+35
fetch-pack, when fetching a literal SHA-1 from a server that is not configured with uploadpack.allowtipsha1inwant (or similar), always returns an error message of the form "Server does not allow request for unadvertised object %s". However, it is sometimes the case that such object is advertised. This situation would occur, for example, if a user or a script was provided a SHA-1 instead of a branch or tag name for fetching, and wanted to invoke "git fetch" or "git fetch-pack" using that SHA-1. Teach fetch-pack to also check the SHA-1s of the refs in the received ref advertisement if a literal SHA-1 was given by the user. Helped-by: Jeff King <peff@peff.net> Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Reviewed-by: Jonathan Nieder <jrnieder@gmail.com> Reviewed-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-02fetch-pack: move code to report unmatched refs to a functionLibravatar Matt McCutchen1-3/+3
Prepare to reuse this code in transport.c for "git fetch". While we're here, internationalize the existing error message. Signed-off-by: Matt McCutchen <matt@mattmccutchen.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-10-10Merge branch 'nd/shallow-deepen'Libravatar Junio C Hamano1-0/+68
The existing "git fetch --depth=<n>" option was hard to use correctly when making the history of an existing shallow clone deeper. A new option, "--deepen=<n>", has been added to make this easier to use. "git clone" also learned "--shallow-since=<date>" and "--shallow-exclude=<tag>" options to make it easier to specify "I am interested only in the recent N months worth of history" and "Give me only the history since that version". * nd/shallow-deepen: (27 commits) fetch, upload-pack: --deepen=N extends shallow boundary by N commits upload-pack: add get_reachable_list() upload-pack: split check_unreachable() in two, prep for get_reachable_list() t5500, t5539: tests for shallow depth excluding a ref clone: define shallow clone boundary with --shallow-exclude fetch: define shallow boundary with --shallow-exclude upload-pack: support define shallow boundary by excluding revisions refs: add expand_ref() t5500, t5539: tests for shallow depth since a specific date clone: define shallow clone boundary based on time with --shallow-since fetch: define shallow boundary with --shallow-since upload-pack: add deepen-since to cut shallow repos based on time shallow.c: implement a generic shallow boundary finder based on rev-list fetch-pack: use a separate flag for fetch in deepening mode fetch-pack.c: mark strings for translating fetch-pack: use a common function for verbose printing fetch-pack: use skip_prefix() instead of starts_with() upload-pack: move rev-list code out of check_non_tip() upload-pack: make check_non_tip() clean things up on error upload-pack: tighten number parsing at "deepen" lines ...
2016-06-13fetch, upload-pack: --deepen=N extends shallow boundary by N commitsLibravatar Nguyễn Thái Ngọc Duy1-0/+23
In git-fetch, --depth argument is always relative with the latest remote refs. This makes it a bit difficult to cover this use case, where the user wants to make the shallow history, say 3 levels deeper. It would work if remote refs have not moved yet, but nobody can guarantee that, especially when that use case is performed a couple months after the last clone or "git fetch --depth". Also, modifying shallow boundary using --depth does not work well with clones created by --since or --not. This patch fixes that. A new argument --deepen=<N> will add <N> more (*) parent commits to the current history regardless of where remote refs are. Have/Want negotiation is still respected. So if remote refs move, the server will send two chunks: one between "have" and "want" and another to extend shallow history. In theory, the client could send no "want"s in order to get the second chunk only. But the protocol does not allow that. Either you send no want lines, which means ls-remote; or you have to send at least one want line that carries deep-relative to the server.. The main work was done by Dongcan Jiang. I fixed it up here and there. And of course all the bugs belong to me. (*) We could even support --deepen=<N> where <N> is negative. In that case we can cut some history from the shallow clone. This operation (and --depth=<shorter depth>) does not require interaction with remote side (and more complicated to implement as a result). Helped-by: Duy Nguyen <pclouds@gmail.com> Helped-by: Eric Sunshine <sunshine@sunshineco.com> Helped-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Dongcan Jiang <dongcan.jiang@gmail.com> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-13t5500, t5539: tests for shallow depth excluding a refLibravatar Nguyễn Thái Ngọc Duy1-0/+21
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-13t5500, t5539: tests for shallow depth since a specific dateLibravatar Nguyễn Thái Ngọc Duy1-0/+24
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-10Merge branch 'jk/shell-portability'Libravatar Junio C Hamano1-1/+0
test fixes. * jk/shell-portability: t5500 & t7403: lose bash-ism "local" test-lib: add in-shell "env" replacement
2016-06-01t5500 & t7403: lose bash-ism "local"Libravatar Junio C Hamano1-1/+0
In t5500::check_prot_host_port_path(), diagport is not a variable used elsewhere and the function is not recursively called so this can simply lose the "local", which may not be supported by shell (besides, the function liberally clobbers other variables without making them "local"). t7403::reset_submodule_urls() overrides the "root" variable used in the test framework for no good reason; its use is not about temporarily relocating where the test repositories are created. This assignment can be made not to clobber the variable by moving them into the subshells it already uses. Its value is always $TRASH_DIRECTORY, so we could use it instead there, and this function that is called only once and its two subshells may not be necessary (instead, the caller can use "git -C $there config" and set a value that is derived from $TRASH_DIRECTORY), but this is a minimum fix that is needed to lose "local". Helped-by: John Keeping <john@keeping.me.uk> Helped-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-04-08Merge branch 'jc/merge-refuse-new-root'Libravatar Junio C Hamano1-2/+4
"git merge" used to allow merging two branches that have no common base by default, which led to a brand new history of an existing project created and then get pulled by an unsuspecting maintainer, which allowed an unnecessary parallel history merged into the existing project. The command has been taught not to allow this by default, with an escape hatch "--allow-unrelated-histories" option to be used in a rare event that merges histories of two projects that started their lives independently. * jc/merge-refuse-new-root: merge: refuse to create too cool a merge by default
2016-03-23merge: refuse to create too cool a merge by defaultLibravatar Junio C Hamano1-2/+4
While it makes sense to allow merging unrelated histories of two projects that started independently into one, in the way "gitk" was merged to "git" itself aka "the coolest merge ever", such a merge is still an unusual event. Worse, if somebody creates an independent history by starting from a tarball of an established project and sends a pull request to the original project, "git merge" however happily creates such a merge without any sign of something unusual is happening. Teach "git merge" to refuse to create such a merge by default, unless the user passes a new "--allow-unrelated-histories" option to tell it that the user is aware that two unrelated projects are merged. Because such a "two project merge" is a rare event, a configuration option to always allow such a merge is not added. We could add the same option to "git pull" and have it passed through to underlying "git merge". I do not have a fundamental opposition against such a feature, but this commit does not do so and instead leaves it as low-hanging fruit for others, because such a "two project merge" would be done after fetching the other project into some location in the working tree of an existing project and making sure how well they fit together, it is sufficient to allow a local merge without such an option pass-through from "git pull" to "git merge". Many tests that are updated by this patch does the pass-through manually by turning: git pull something into its equivalent: git fetch something && git merge --allow-unrelated-histories FETCH_HEAD If somebody is inclined to add such an option, updated tests in this change need to be adjusted back to: git pull --allow-unrelated-histories something Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-01fetch-pack: fix object_id of exact sha1Libravatar Gabriel Souza Franco1-0/+14
Commit 58f2ed0 (remote-curl: pass ref SHA-1 to fetch-pack as well, 2013-12-05) added support for specifying a SHA-1 as well as a ref name. Add support for specifying just a SHA-1 and set the ref name to the same value in this case. Signed-off-by: Gabriel Souza Franco <gabrielfrancosouza@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-12-28t/t5500-fetch-pack.sh: use the $( ... ) construct for command substitutionLibravatar Elia Pinto1-10/+10
The Git CodingGuidelines prefer the $(...) construct for command substitution instead of using the backquotes `...`. The backquoted form is the traditional method for command substitution, and is supported by POSIX. However, all but the simplest uses become complicated quickly. In particular, embedded command substitutions and/or the use of double quotes require careful escaping with the backslash character. The patch was generated by: for _f in $(find . -name "*.sh") do perl -i -pe 'BEGIN{undef $/;} s/`(.+?)`/\$(\1)/smg' "${_f}" done and then carefully proof-read. Signed-off-by: Elia Pinto <gitter.spiros@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-04-20Merge branch 'tb/connect-ipv6-parse-fix'Libravatar Junio C Hamano1-7/+10
An earlier update to the parser that disects an address broke an address, followed by a colon, followed by an empty string (instead of the port number). * tb/connect-ipv6-parse-fix: connect.c: ignore extra colon after hostname
2015-04-08connect.c: ignore extra colon after hostnameLibravatar Torsten Bögershausen1-7/+10
Ignore an extra ':' at the end of the hostname in URL's like "ssh://example.com:/path/to/repo" The colon is meant to separate a port number from the hostname. If the port is empty, the colon should be ignored, see RFC 3986. It had been working for URLs with ssh:// scheme, but was unintentionally broken in 86ceb3, "allow ssh://user@[2001:db8::1]/repo.git" Reported-by: Reid Woodbury Jr. <reidw@rawsound.com> Signed-off-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-03-20t: fix trivial &&-chain breakageLibravatar Jeff King1-1/+1
These are tests which are missing a link in their &&-chain, but during a setup phase. We may fail to notice failure in commands that build the test environment, but these are typically not expected to fail at all (but it's still good to double-check that our test environment is what we expect). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-02-22t5500: show user name and host in diag-urlLibravatar Torsten Bögershausen1-18/+33
The URL for ssh may have include a username before the hostname, like ssh://user@host/repo. When literal IPV6 addresses are used together with a username, the substring "user@[::1]" must be converted into "user@::1". Make that conversion visible for the user, and write userandhost in the diagnostics Signed-off-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-12-09connect.c: refactor url parsingLibravatar Torsten Bögershausen1-0/+7
Make the function is_local() in transport.c public, rename it into url_is_local_not_ssh() and use it in both transport.c and connect.c Use a protocol "local" for URLs for the local file system. One note about using file:// under Windows: The (absolute) path on Unix like system typically starts with "/". When the host is empty, it can be omitted, so that a shell scriptlet url=file://$pwd will give a URL like "file:///home/user/repo". Windows does not have the same concept of a root directory located in "/". When parsing the URL allow "file://C:/user/repo" (even if RFC1738 indicates that "file:///C:/user/repo" should be used). Signed-off-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-12-09git_connect(): refactor the port handling for sshLibravatar Torsten Bögershausen1-6/+3
Use get_host_and_port() even for ssh. Remove the variable port git_connect(), and simplify parse_connect_url() Use only one return point in git_connect(), doing the free() and return conn. t5601 had 2 corner test cases which now pass. Signed-off-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-12-09git fetch: support host:/~repoLibravatar Torsten Bögershausen1-0/+24
The documentation (in urls.txt) says that "ssh://host:/~repo", "host:/~repo" or "host:~repo" specify the repository "repo" in the home directory at "host". This has not been working for "host:/~repo". Before commit 356bec "Support [address] in URLs", the comparison "url != hostname" could be used to determine if the URL had a scheme or not: "ssh://host/host" != "host". However, after 356bec "[::1]" was converted into "::1", yielding url != hostname as well. To fix this regression, don't use "if (url != hostname)", but look at the separator instead. Rename the variable "c" into "separator" to make it easier to read. Signed-off-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-12-09t5500: add test cases for diag-urlLibravatar Torsten Bögershausen1-0/+59
Add test cases using git fetch-pack --diag-url: - parse out host and path for URLs with a scheme (git:// file:// ssh://) - parse host names embedded by [] correctly - extract the port number, if present - separate URLs like "file" (which are local) from URLs like "host:repo" which should use ssh Signed-off-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-09-20Merge branch 'nd/fetch-into-shallow'Libravatar Junio C Hamano1-0/+11
When there is no sufficient overlap between old and new history during a fetch into a shallow repository, we unnecessarily sent objects the sending side knows the receiving end has. * nd/fetch-into-shallow: Add testcase for needless objects during a shallow fetch list-objects: mark more commits as edges in mark_edges_uninteresting list-objects: reduce one argument in mark_edges_uninteresting upload-pack: delegate rev walking in shallow fetch to pack-objects shallow: add setup_temporary_shallow() shallow: only add shallow graft points to new shallow file move setup_alternate_shallow and write_shallow_commits to shallow.c
2013-08-28Add testcase for needless objects during a shallow fetchLibravatar Matthijs Kooijman1-0/+11
This is a testcase that checks for a problem where, during a specific shallow fetch where the client does not have any commits that are a successor of the new shallow root (i.e., the fetch creates a new detached piece of history), the server would simply send over _all_ objects, instead of taking into account the objects already present in the client. The actual problem was fixed by a recent patch series by Nguyễn Thái Ngọc Duy already. Signed-off-by: Matthijs Kooijman <matthijs@stdin.nl> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-08-25fetch-pack: do not remove .git/shallow file when --depth is not specifiedLibravatar Nguyễn Thái Ngọc Duy1-0/+16
fetch_pack() can remove .git/shallow file when a shallow repository becomes a full one again. This behavior is triggered incorrectly when tags are also fetched because fetch_pack() will be called twice. At the first fetch_pack() call: - shallow_lock is set up - alternate_shallow_file points to shallow_lock.filename, which is "shallow.lock" - commit_lock_file is called, which sets shallow_lock.filename to "". alternate_shallow_file also becomes "" because it points to the same memory. At the second call, setup_alternate_shallow() is not called and alternate_shallow_file remains "". It's mistaken as unshallow case and .git/shallow is removed. The end result is a broken repository. Fix this by always initializing alternate_shallow_file when fetch_pack() is called. As an extra measure, check if args->depth > 0 before commit/rollback shallow file. Reported-by: Kacper Kornet <kornet@camk.edu.pl> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-06-28Merge branch 'nd/clone-connectivity-shortcut' (early part) into maintLibravatar Junio C Hamano1-0/+7
Cloning with "git clone --depth N" while fetch.fsckobjects (or transfer.fsckobjects) is set to true did not tell the cut-off points of the shallow history to the process that validates the objects and the history received, causing the validation to fail. * 'nd/clone-connectivity-shortcut' (early part): fetch-pack: prepare updated shallow file before fetching the pack clone: let the user know when check_everything_connected is run
2013-06-06Merge branch 'nd/clone-connectivity-shortcut'Libravatar Junio C Hamano1-0/+7
Special case "git clone" and use lighter-weight implementation to check the completeness of the history behind refs. * nd/clone-connectivity-shortcut: clone: open a shortcut for connectivity check index-pack: remove dead code (it should never happen) fetch-pack: prepare updated shallow file before fetching the pack clone: let the user know when check_everything_connected is run
2013-05-28fetch-pack: prepare updated shallow file before fetching the packLibravatar Nguyễn Thái Ngọc Duy1-0/+7
index-pack --strict looks up and follows parent commits. If shallow information is not ready by the time index-pack is run, index-pack may be led to non-existent objects. Make fetch-pack save shallow file to disk before invoking index-pack. git learns new global option --shallow-file to pass on the alternate shallow file path. Undocumented (and not even support --shallow-file= syntax) because it's unlikely to be used again elsewhere. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-05-02t5500: add test for fetching with an unknown 'shallow'Libravatar Michael Heemskerk1-0/+14
When the client sends a 'shallow' line for an object that the server does not have, the server should just ignore it and let the client keep that unknown shallow boundary. Signed-off-by: Michael Heemskerk <mheemskerk@atlassian.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-03-25Merge branch 'jk/peel-ref'Libravatar Junio C Hamano1-0/+9
Recent optimization broke shallow clones. * jk/peel-ref: upload-pack: load non-tip "want" objects from disk upload-pack: make sure "want" objects are parsed upload-pack: drop lookup-before-parse optimization
2013-03-16upload-pack: make sure "want" objects are parsedLibravatar Jeff King1-0/+9
When upload-pack receives a "want" line from the client, it adds it to an object array. We call lookup_object to find the actual object, which will only check for objects already in memory. This works because we are expecting to find objects that we already loaded during the ref advertisement. We use the resulting object structs for a variety of purposes. Some of them care only about the object flags, but others care about the type of the object (e.g., ok_to_give_up), or even feed them to the revision parser (when --depth is used), which assumes that objects it receives are fully parsed. Once upon a time, this was OK; any object we loaded into memory would also have been parsed. But since 435c833 (upload-pack: use peel_ref for ref advertisements, 2012-10-04), we try to avoid parsing objects during the ref advertisement. This means that lookup_object may return an object with a type of OBJ_NONE. The resulting mess depends on the exact set of objects, but can include the revision parser barfing, or the shallow code sending the wrong set of objects. This patch teaches upload-pack to parse each "want" object as we receive it. We do not replace the lookup_object call with parse_object, as the current code is careful not to let just any object appear on a "want" line, but rather only one we have previously advertised (whereas parse_object would actually load any arbitrary object from disk). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-01-11upload-pack: fix off-by-one depth calculation in shallow cloneLibravatar Nguyễn Thái Ngọc Duy1-6/+19
get_shallow_commits() is used to determine the cut points at a given depth (i.e. the number of commits in a chain that the user likes to get). However we count current depth up to the commit "commit" but we do the cutting at its parents (i.e. current depth + 1). This makes upload-pack always return one commit more than requested. This patch fixes it. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-01-11fetch: add --unshallow for turning shallow repo into complete oneLibravatar Nguyễn Thái Ngọc Duy1-0/+20
The user can do --depth=2147483647 (*) for restoring complete repo now. But it's hard to remember. Any other numbers larger than the longest commit chain in the repository would also do, but some guessing may be involved. Make easy-to-remember --unshallow an alias for --depth=2147483647. Make upload-pack recognize this special number as infinite depth. The effect is essentially the same as before, except that upload-pack is more efficient because it does not have to traverse to the bottom anymore. The chance of a user actually wanting exactly 2147483647 commits depth, not infinite, on a repository with a history that long, is probably too small to consider. The client can learn to add or subtract one commit to avoid the special treatment when that actually happens. (*) This is the largest positive number a 32-bit signed integer can contain. JGit and older C Git store depth as "int" so both are OK with this number. Dulwich does not support shallow clone. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-09-12fetch-pack: eliminate spurious error messagesLibravatar Michael Haggerty1-2/+2
It used to be that if "--all", "--depth", and also explicit references were sought, then the explicit references were not handled correctly in filter_refs() because the "--all --depth" code took precedence over the explicit reference handling, and the explicit references were never noted as having been found. So check for explicitly sought references before proceeding to the "--all --depth" logic. This fixes two test cases in t5500. Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-09-12fetch-pack: report missing refs even if no existing refs were receivedLibravatar Michael Haggerty1-1/+1
This fixes a test in t5500. Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-09-12fetch_pack(): update sought->nr to reflect number of unique entriesLibravatar Michael Haggerty1-1/+1
fetch_pack() removes duplicates from the "sought" list, thereby shrinking the list. But previously, the caller was not informed about the shrinkage. This would cause a spurious error message to be emitted by cmd_fetch_pack() if "git fetch-pack" is called with duplicate refnames. Instead, remove duplicates using string_list_remove_duplicates(), which adjusts sought->nr to reflect the new length of the list. The last test of t5500 inexplicably *required* "git fetch-pack" to fail when fetching a list of references that contains duplicates; i.e., it insisted on the buggy behavior. So change the test to expect the correct behavior. Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-09-12t5500: add tests of fetch-pack --all --depth=N $URL $REFLibravatar Michael Haggerty1-0/+15
Document some bugs in "git fetch-pack": 1. If "git fetch-pack" is called with "--all", "--depth", and an explicit existing non-tag reference to fetch, then it falsely reports that the reference was not found, even though it was fetched correctly. 2. If "git fetch-pack" is called with "--all", "--depth", and an explicit existing tag reference to fetch, then it segfaults in filter_refs() because return_refs is used without having been initialized. Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>