summaryrefslogtreecommitdiff
path: root/t/t5702-protocol-v2.sh
AgeCommit message (Collapse)AuthorFilesLines
2020-02-27t/lib-httpd: avoid using macOS' sedLibravatar Johannes Schindelin1-6/+6
Among other differences relative to GNU sed, macOS' sed always ends its output with a trailing newline, even if the input did not have such a trailing newline. Surprisingly, this makes three httpd-based tests fail on macOS: t5616, t5702 and t5703. ("Surprisingly" because those tests have been around for some time, but apparently nobody runs them on macOS with a working Apache2 setup.) The reason is that we use `sed` in those tests to filter the response of the web server. Apart from the fact that we use GNU constructs (such as using a space after the `c` command instead of a backslash and a newline), we have another problem: macOS' sed LF-only newlines while webservers are supposed to use CR/LF ones. Even worse, t5616 uses `sed` to replace a binary part of the response with a new binary part (kind of hoping that the replaced binary part does not contain a 0x0a byte which would be interpreted as a newline). To that end, it calls on Perl to read the binary pack file and hex-encode it, then calls on `sed` to prefix every hex digit pair with a `\x` in order to construct the text that the `c` statement of the `sed` invocation is supposed to insert. So we call Perl and sed to construct a sed statement. The final nail in the coffin is that macOS' sed does not even interpret those `\x<hex>` constructs. Let's just replace all of that by Perl snippets. With Perl, at least, we do not have to deal with GNU vs macOS semantics, we do not have to worry about unwanted trailing newlines, and we do not have to spawn commands to construct arguments for other commands to be spawned (i.e. we can avoid a whole lot of shell scripting complexity). The upshot is that this fixes t5616, t5702 and t5703 on macOS with Apache2. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-02-14Merge branch 'jk/no-flush-upon-disconnecting-slrpc-transport' into maintLibravatar Junio C Hamano1-0/+12
Reduce unnecessary round-trip when running "ls-remote" over the stateless RPC mechanism. * jk/no-flush-upon-disconnecting-slrpc-transport: transport: don't flush when disconnecting stateless-rpc helper
2020-01-08transport: don't flush when disconnecting stateless-rpc helperLibravatar Jeff King1-0/+12
Since ba227857d2 (Reduce the number of connects when fetching, 2008-02-04), when we disconnect a git transport, we send a final flush packet. This cleanly tells the other side that we're done, and avoids the other side complaining "the remote end hung up unexpectedly" (though we'd only see that for transports that pass along the server stderr, like ssh or local-host). But when we've initiated a v2 stateless-connect session over a transport helper, there's no point in sending this flush packet. Each operation we've performed is self-contained, and the other side is fine with us hanging up between operations. But much worse, by sending the flush packet we may cause the helper to issue an entirely new request _just_ to send the flush packet. So we can incur an extra network request just to say "by the way, we have nothing more to send". Let's drop this extra flush packet. As the test shows, this reduces the number of POSTs required for a v2 ls-remote over http from 2 to 1. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-10Fix spelling errors in names of testsLibravatar Elijah Newren1-3/+3
Reported-by: Jens Schleusener <Jens.Schleusener@fossies.org> Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-10Fix spelling errors in comments of testcasesLibravatar Elijah Newren1-2/+2
Reported-by: Jens Schleusener <Jens.Schleusener@fossies.org> Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-22transport-helper: skip ls-refs if unnecessaryLibravatar Jonathan Tan1-0/+13
Commit e70a3030e7 ("fetch: do not list refs if fetching only hashes", 2018-10-07) and its ancestors taught Git, as an optimization, to skip the ls-refs step when it is not necessary during a protocol v2 fetch (for example, when lazy fetching a missing object in a partial clone, or when running "git fetch --no-tags <remote> <SHA-1>"). But that was only done for natively supported protocols; in particular, HTTP was not supported. Teach Git to skip ls-refs when using remote helpers that support connect or stateless-connect. To do this, fetch() is made an acceptable entry point. Because fetch() can now be the first function in the vtable called, "get_helper(transport);" has to be added to the beginning of that function to set the transport up (if not yet set up) before process_connect() is invoked. When fetch() is called, the transport could be taken over (this happens if "connect" or "stateless-connect" is successfully run without any "fallback" response), or not. If the transport is taken over, execution continues like execution for natively supported protocols (fetch_refs_via_pack() is executed, which will fetch refs using ls-refs if needed). If not, the remote helper interface will invoke get_refs_list() if it hasn't been invoked yet, preserving existing behavior. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-02t: warn against adding non-httpd-specific tests after sourcing 'lib-httpd'Libravatar SZEDER Gábor1-0/+3
We have a couple of test scripts that are not completely httpd-specific, but do run a few httpd-specific tests at the end. These test scripts source 'lib-httpd.sh' somewhere mid-script, which then skips all the rest of the test script if the dependencies for running httpd tests are not fulfilled. As the previous two patches in this series show, already on two occasions non-httpd-specific tests were appended at the end of such test scripts, and, consequently, they were skipped as well when httpd tests couldn't be run. Add a comment at the end of these test scripts to warn against adding non-httpd-specific tests at the end, in the hope that they will help prevent similar issues in the future. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-07-02t5702: use test_commit_bulkLibravatar Jeff King1-8/+2
There are two loops that create 32 commits each using test_commit. Using test_commit_bulk speeds this up from: Benchmark #1: ./t5702-protocol-v2.sh --root=/var/ram/git-tests Time (mean ± σ): 5.409 s ± 0.513 s [User: 2.382 s, System: 2.466 s] Range (min … max): 4.633 s … 5.927 s 10 runs to: Benchmark #1: ./t5702-protocol-v2.sh --root=/var/ram/git-tests Time (mean ± σ): 3.956 s ± 0.242 s [User: 1.775 s, System: 1.627 s] Range (min … max): 3.449 s … 4.239 s 10 runs for an average savings of over 25%. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-05-09Merge branch 'js/misc-doc-fixes'Libravatar Junio C Hamano1-2/+3
"make check-docs", "git help -a", etc. did not account for cases where a particular build may deliberately omit some subcommands, which has been corrected. * js/misc-doc-fixes: Turn `git serve` into a test helper test-tool: handle the `-C <directory>` option just like `git` check-docs: do not bother checking for legacy scripts' documentation docs: exclude documentation for commands that have been excluded check-docs: allow command-list.txt to contain excluded commands help -a: do not list commands that are excluded from the build Makefile: drop the NO_INSTALL variable remote-testgit: move it into the support directory for t5801
2019-05-09Merge branch 'jt/clone-server-option'Libravatar Junio C Hamano1-0/+41
"git clone" learned a new --server-option option when talking over the protocol version 2. * jt/clone-server-option: clone: send server options when using protocol v2 transport: die if server options are unsupported
2019-04-19Turn `git serve` into a test helperLibravatar Johannes Schindelin1-2/+3
The `git serve` built-in was introduced in ed10cb952d31 (serve: introduce git-serve, 2018-03-15) as a backend to serve Git protocol v2, probably originally intended to be spawned by `git upload-pack`. However, in the version that the protocol v2 patches made it into core Git, `git upload-pack` calls the `serve()` function directly instead of spawning `git serve`; The only reason in life for `git serve` to survive as a built-in command is to provide a way to test the protocol v2 functionality. Meaning that it does not even have to be a built-in that is installed with end-user facing Git installations, but it can be a test helper instead. Let's make it so. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-18clone: send server options when using protocol v2Libravatar Jonathan Tan1-0/+22
Commit 5e3548ef16 ("fetch: send server options when using protocol v2", 2018-04-24) taught "fetch" the ability to send server options when using protocol v2, but not "clone". This ability is triggered by "-o" or "--server-option". Teach "clone" the same ability, except that because "clone" already has "-o" for another parameter, teach "clone" only to receive "--server-option". Explain in the documentation, both for clone and for fetch, that server handling of server options are server-specific. This is similar to receive-pack's handling of push options - currently, they are just sent to hooks to interpret as they see fit. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Reviewed-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-18transport: die if server options are unsupportedLibravatar Jonathan Tan1-0/+19
Server options were added in commit 5e3548ef16 ("fetch: send server options when using protocol v2", 2018-04-24), supported only for protocol version 2. But if the user specifies server options, and the protocol version being used doesn't support them, the server options are silently ignored. Teach any transport users to die instead in this situation, just like how "push" dies if push options are provided when the server doesn't support them. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-14tests: use 'test_atexit' to stop httpdLibravatar SZEDER Gábor1-2/+0
Use 'test_atexit' to run cleanup commands to stop httpd at the end of the test script or upon interrupt or failure, as it is shorter, simpler, and more robust than registering such cleanup commands in the trap on EXIT in the test scripts. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-03remote-curl: use post_rpc() for protocol v2 alsoLibravatar Jonathan Tan1-1/+32
When transmitting and receiving POSTs for protocol v0 and v1, remote-curl uses post_rpc() (and associated functions), but when doing the same for protocol v2, it uses a separate set of functions (proxy_rpc() and others). Besides duplication of code, this has caused at least one bug: the auth retry mechanism that was implemented in v0/v1 was not implemented in v2. To fix this issue and avoid it in the future, make remote-curl also use post_rpc() when handling protocol v2. Because line lengths are written to the HTTP request in protocol v2 (unlike in protocol v0/v1), this necessitates changes in post_rpc() and some of the functions it uses; perform these changes too. A test has been included to ensure that the code for both the unchunked and chunked variants of the HTTP request is exercised. Note: stateless_connect() has been updated to use the lower-level packet reading functions instead of struct packet_reader. The low-level control is necessary here because we cannot change the destination buffer of struct packet_reader while it is being used; struct packet_buffer has a peeking mechanism which relies on the destination buffer being present in between a peek and a read. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-05Merge branch 'jt/namespaced-ls-refs-fix'Libravatar Junio C Hamano1-0/+21
Fix namespace support in protocol v2. * jt/namespaced-ls-refs-fix: ls-refs: filter refs using namespace-stripped name
2019-02-05Merge branch 'jt/fetch-v2-sideband'Libravatar Junio C Hamano1-2/+2
"git fetch" and "git upload-pack" learned to send all exchange over the sideband channel while talking the v2 protocol. * jt/fetch-v2-sideband: tests: define GIT_TEST_SIDEBAND_ALL {fetch,upload}-pack: sideband v2 fetch response sideband: reverse its dependency on pkt-line pkt-line: introduce struct packet_writer pack-protocol.txt: accept error packets in any context Use packet_reader instead of packet_read_line
2019-01-18ls-refs: filter refs using namespace-stripped nameLibravatar Jonathan Tan1-0/+21
If a user fetches refs/heads/master from a repo with namespace "ns", the remote is expected to (1) not send the real refs/heads/master, and (2) send refs/namespaces/ns/refs/heads/master with the name refs/heads/master. (1) indeed happens now, but not (2) - Git only sends refs that have the user-given prefix, but it checks them against the full name of the ref (the one starting with refs/namespaces), and not the namespace-stripped one. This is demonstrated by the patch in the test. Currently, it results in "fatal: couldn't find remote ref refs/heads/master" despite both unnamespaced and namespaced master being present. With the code change, it produces the expected result. Check the ref prefixes against the namespace-stripped name. This bug was discovered through applying patches [1] that override protocol.version to 2 in repositories when running tests, allowing us to notice differences in behavior across different protocol versions. [1] https://public-inbox.org/git/cover.1547677183.git.jonathantanmy@google.com/ Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-17tests: define GIT_TEST_SIDEBAND_ALLLibravatar Jonathan Tan1-2/+2
Define a GIT_TEST_SIDEBAND_ALL environment variable meant to be used from tests. When set to true, this overrides uploadpack.allowsidebandall to true, allowing the entire test suite to be run as if this configuration is in place for all repositories. As of this patch, all tests pass whether GIT_TEST_SIDEBAND_ALL is unset or set to 1. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-10upload-pack: teach deepen-relative in protocol v2Libravatar Jonathan Tan1-0/+29
Commit 685fbd3291 ("fetch-pack: perform a fetch using v2", 2018-03-15) attempted to teach Git deepen-relative in protocol v2 (among other things), but it didn't work: (1) fetch-pack.c needs to emit "deepen-relative". (2) upload-pack.c needs to ensure that the correct deepen_relative variable is passed to deepen() (there are two - the static variable and the one in struct upload_pack_data). (3) Before deepen() computes the list of reachable shallows, it first needs to mark all "our" refs as OUR_REF. v2 currently does not do this, because unlike v0, it is not needed otherwise. Fix all this and include a test demonstrating that it works now. For (2), the static variable deepen_relative is also eliminated, removing a source of confusion. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Reviewed-by: Josh Steadmon <steadmon@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-10fetch-pack: do not take shallow lock unnecessarilyLibravatar Jonathan Tan1-0/+18
When fetching using protocol v2, the remote may send a "shallow-info" section if the client is shallow. If so, Git as the client currently takes the shallow file lock, even if the "shallow-info" section is empty. This is not a problem except that Git does not support taking the shallow file lock after modifying the shallow file, because is_repository_shallow() stores information that is never cleared. And this take-after-modify occurs when Git does a tag-following fetch from a shallow repository on a transport that does not support tag following (since in this case, 2 fetches are performed). To solve this issue, take the shallow file lock (and perform all other shallow processing) only if the "shallow-info" section is non-empty; otherwise, behave as if it were empty. A full solution (probably, ensuring that any action of committing shallow file locks also includes clearing the information stored by is_repository_shallow()) would solve the issue without need for this patch, but this patch is independently useful (as an optimization to prevent writing a file in an unnecessary case), hence why I wrote it. I have included a NEEDSWORK outlining the full solution. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-11-13Merge branch 'jt/tighten-fetch-proto-v2-response'Libravatar Junio C Hamano1-0/+50
"git fetch" was a bit loose in parsing resposes from the other side when talking over the protocol v2. * jt/tighten-fetch-proto-v2-response: fetch-pack: be more precise in parsing v2 response
2018-11-06Merge branch 'jt/upload-pack-v2-fix-shallow'Libravatar Junio C Hamano1-0/+25
"git fetch" over protocol v2 into a shallow repository failed to fetch full history behind a new tip of history that was diverged before the cut-off point of the history that was previously fetched shallowly. * jt/upload-pack-v2-fix-shallow: upload-pack: clear flags before each v2 request upload-pack: make want_obj not global upload-pack: make have_obj not global
2018-11-01fetch-pack: be more precise in parsing v2 responseLibravatar Jonathan Tan1-0/+50
Each section in a protocol v2 response is followed by either a DELIM packet (indicating more sections to follow) or a FLUSH packet (indicating none to follow). But when parsing the "acknowledgments" section, do_fetch_pack_v2() is liberal in accepting both, but determines whether to continue reading or not based solely on the contents of the "acknowledgments" section, not on whether DELIM or FLUSH was read. There is no issue with a protocol-compliant server, but can result in confusing error messages when communicating with a server that serves unexpected additional sections. Consider a server that sends "new-section" after "acknowledgments": - client writes request - client reads the "acknowledgments" section which contains no "ready", then DELIM - since there was no "ready", client needs to continue negotiation, and writes request - client reads "new-section", and reports to the end user "expected 'acknowledgments', received 'new-section'" For the person debugging the involved Git implementation(s), the error message is confusing in that "new-section" was not received in response to the latest request, but to the first one. One solution is to always continue reading after DELIM, but in this case, we can do better. We know from the protocol that "ready" means at least the packfile section is coming (hence, DELIM) and that no "ready" means that no sections are to follow (hence, FLUSH). So teach process_acks() to enforce this. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-19Merge branch 'jt/avoid-ls-refs'Libravatar Junio C Hamano1-0/+17
Over some transports, fetching objects with an exact commit object name can be done without first seeing the ref advertisements. The code has been optimized to exploit this. * jt/avoid-ls-refs: fetch: do not list refs if fetching only hashes transport: list refs before fetch if necessary transport: do not list refs if possible transport: allow skipping of ref listing
2018-10-19upload-pack: clear flags before each v2 requestLibravatar Jonathan Tan1-0/+25
Suppose a server has the following commit graph: A B \ / O We create a client by cloning A from the server with depth 1, and add many commits to it (so that future fetches span multiple requests due to lengthy negotiation). If it then fetches B using protocol v2, the fetch spanning multiple requests, the resulting packfile does not contain O even though the client did report that A is shallow. This is because upload_pack_v2() can be called multiple times while processing the same session. During the 2nd and all subsequent invocations, some object flags remain from the previous invocations. In particular, CLIENT_SHALLOW remains, preventing process_shallow() from adding client-reported shallows to the "shallows" array, and hence pack-objects not knowing about these client-reported shallows. Therefore, teach upload_pack_v2() to clear object flags at the start of each invocation. This has some other results: - THEY_HAVE gates addition of objects to have_obj in process_haves(). Previously in upload_pack_v2(), have_obj needed to be static because once an object is added to have_obj, it is never readded and thus we needed to retain the contents of have_obj between invocations. Now that flags are cleared, this is no longer necessary. This patch does not change the behavior of ok_to_give_up() (THEY_HAVE is still set on each "have") and got_oid() (used only in non-v2)); THEY_HAVE is not used in any other function. - WANTED gates addition of objects to want_obj in parse_want() and parse_want_ref(). It is also used in receive_needs(), but that is only used in non-v2. For the same reasons as THEY_HAVE, want_obj no longer needs to be static in upload_pack_v2(). - CLIENT_SHALLOW is changed as discussed above. Clearing of the other 5 flags does not affect functionality in v2. (Note that in non-v2, upload_pack() is only called once per process, so each invocation starts with blank flags anyway.) - OUR_REF is only used in non-v2. - COMMON_KNOWN is only used as a scratch flag in ok_to_give_up(). - SHALLOW is passed to invocations in deepen() and deepen_by_rev_list(), but upload-pack doesn't use it. - NOT_SHALLOW is used by send_shallow() and send_unshallow(), but invocations of those functions are always preceded by code that sets NOT_SHALLOW on the appropriate objects. - HIDDEN_REF is only used in non-v2. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-16Merge branch 'md/test-cleanup'Libravatar Junio C Hamano1-7/+7
Various test scripts have been updated for style and also correct handling of exit status of various commands. * md/test-cleanup: tests: order arguments to git-rev-list properly t9109: don't swallow Git errors upstream of pipes tests: don't swallow Git errors upstream of pipes t/*: fix ordering of expected/observed arguments tests: standardize pipe placement Documentation: add shell guidelines t/README: reformat Do, Don't, Keep in mind lists
2018-10-07fetch: do not list refs if fetching only hashesLibravatar Jonathan Tan1-0/+13
If only hash literals are given on a "git fetch" command-line, tag following is not requested, and the fetch is done using protocol v2, a list of refs is not required from the remote. Therefore, optimize by invoking transport_get_remote_refs() only if we need the refs. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-07transport: do not list refs if possibleLibravatar Jonathan Tan1-0/+4
When all refs to be fetched are exact OIDs, it is possible to perform a fetch without requiring the remote to list refs if protocol v2 is used. Teach Git to do this. This currently has an effect only for lazy fetches done from partial clones. The change necessary to likewise optimize "git fetch <remote> <sha-1>" will be done in a subsequent patch. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-07tests: order arguments to git-rev-list properlyLibravatar Matthew DeVore1-2/+2
It is a common mistake to put positional arguments before flags when invoking git-rev-list. Order the positional arguments last. This patch skips git-rev-list invocations which include the --not flag, since the ordering of flags and positional arguments affects the behavior. This patch also skips invocations of git-rev-list that occur in command substitution in which the exit code is discarded, since fixing those properly will require a more involved cleanup. Signed-off-by: Matthew DeVore <matvore@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-07t/*: fix ordering of expected/observed argumentsLibravatar Matthew DeVore1-5/+5
Fix various places where the ordering was obviously wrong, meaning it was easy to find with grep. Signed-off-by: Matthew DeVore <matvore@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-11t/helper: merge test-pkt-line into test-toolLibravatar Nguyễn Thái Ngọc Duy1-1/+1
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-15Merge branch 'jt/tag-following-with-proto-v2-fix'Libravatar Junio C Hamano1-3/+68
The wire-protocol v2 relies on the client to send "ref prefixes" to limit the bandwidth spent on the initial ref advertisement. "git fetch $remote branch:branch" that asks tags that point into the history leading to the "branch" automatically followed sent to narrow prefix and broke the tag following, which has been fixed. * jt/tag-following-with-proto-v2-fix: fetch: send "refs/tags/" prefix upon CLI refspecs t5702: test fetch with multiple refspecs at a time
2018-07-24fetch: send "refs/tags/" prefix upon CLI refspecsLibravatar Jonathan Tan1-3/+21
When performing tag following, in addition to using the server's "include-tag" capability to send tag objects (and emulating it if the server does not support that capability), "git fetch" relies upon the presence of refs/tags/* entries in the initial ref advertisement to locally create refs pointing to the aforementioned tag objects. When using protocol v2, refs/tags/* entries in the initial ref advertisement may be suppressed by a ref-prefix argument, leading to the tag object being downloaded, but the ref not being created. Commit dcc73cf7ff ("fetch: generate ref-prefixes when using a configured refspec", 2018-05-18) ensured that "refs/tags/" is always sent as a ref prefix when "git fetch" is invoked with no refspecs, but not when "git fetch" is invoked with refspecs. Extend that functionality to make it work in both situations. This also necessitates a change another test which tested ref advertisement filtering using tag refs - since tag refs are sent by default now, the test has been switched to using branch refs instead. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-24t5702: test fetch with multiple refspecs at a timeLibravatar Jonathan Tan1-0/+47
Extend the protocol v2 tests to also test fetches with multiple refspecs specified. This also covers the previously uncovered cases of fetching with prefix matching and fetching by SHA-1. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-20clone: send ref-prefixes when using protocol v2Libravatar Brandon Williams1-1/+6
Teach clone to send a list of ref-prefixes, when using protocol v2, to allow the server to filter out irrelevant references from the ref-advertisement. This reduces wasted time and bandwidth when cloning repositories with a larger number of references. Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-05-30Merge branch 'bw/ref-prefix-for-configured-refspec'Libravatar Junio C Hamano1-0/+14
"git fetch $there $refspec" that talks over protocol v2 can take advantage of server-side ref filtering; the code has been extended so that this mechanism triggers also when fetching with configured refspec. * bw/ref-prefix-for-configured-refspec: (38 commits) fetch: generate ref-prefixes when using a configured refspec refspec: consolidate ref-prefix generation logic submodule: convert push_unpushed_submodules to take a struct refspec remote: convert check_push_refs to take a struct refspec remote: convert match_push_refs to take a struct refspec http-push: store refspecs in a struct refspec transport: remove transport_verify_remote_names send-pack: store refspecs in a struct refspec transport: convert transport_push to take a struct refspec push: convert to use struct refspec push: check for errors earlier remote: convert match_explicit_refs to take a struct refspec remote: convert get_ref_match to take a struct refspec remote: convert query_refspecs to take a struct refspec remote: convert apply_refspecs to take a struct refspec remote: convert get_stale_heads to take a struct refspec fetch: convert prune_refs to take a struct refspec fetch: convert get_ref_map to take a struct refspec fetch: convert do_fetch to take a struct refspec refspec: remove the deprecated functions ...
2018-05-30Merge branch 'jt/partial-clone-proto-v2'Libravatar Junio C Hamano1-0/+112
Transfer protocol v2 learned to support the partial clone. * jt/partial-clone-proto-v2: {fetch,upload}-pack: support filter in protocol v2 upload-pack: read config when serving protocol v2 upload-pack: fix error message typo
2018-05-18fetch: generate ref-prefixes when using a configured refspecLibravatar Brandon Williams1-0/+14
Teach fetch to generate ref-prefixes, to be used for server-side filtering of the ref-advertisement, based on the configured fetch refspec ('remote.<name>.fetch') when no user provided refspec exists. Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-05-06{fetch,upload}-pack: support filter in protocol v2Libravatar Jonathan Tan1-0/+98
The fetch-pack/upload-pack protocol v2 was developed independently of the filter parameter (used in partial fetches), thus it did not include support for it. Add support for the filter parameter. Like in the legacy protocol, the server advertises and supports "filter" only if uploadpack.allowfilter is configured. Like in the legacy protocol, the client continues with a warning if "--filter" is specified, but the server does not advertise it. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-05-06upload-pack: read config when serving protocol v2Libravatar Jonathan Tan1-0/+14
The upload-pack code paths never call git_config() with upload_pack_config() when protocol v2 is used, causing options like uploadpack.packobjectshook to not take effect. Ensure that this function is called. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-04-24fetch: send server options when using protocol v2Libravatar Brandon Williams1-0/+16
Teach fetch to optionally accept server options by specifying them on the cmdline via '-o' or '--server-option'. These server options are sent to the remote end when performing a fetch communicating using protocol version 2. If communicating using a protocol other than v2 the provided options are ignored and not sent to the remote end. Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-04-24ls-remote: send server options when using protocol v2Libravatar Brandon Williams1-0/+16
Teach ls-remote to optionally accept server options by specifying them on the cmdline via '-o' or '--server-option'. These server options are sent to the remote end when querying for the remote end's refs using protocol version 2. If communicating using a protocol other than v2 the provided options are ignored and not sent to the remote end. Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-03-15remote-curl: don't request v2 when pushingLibravatar Brandon Williams1-0/+24
In order to be able to ship protocol v2 with only supporting fetch, we need clients to not issue a request to use protocol v2 when pushing (since the client currently doesn't know how to push using protocol v2). This allows a client to have protocol v2 configured in `protocol.version` and take advantage of using v2 for fetch and falling back to using v0 when pushing while v2 for push is being designed. We could run into issues if we didn't fall back to protocol v2 when pushing right now. This is because currently a server will ignore a request to use v2 when contacting the 'receive-pack' endpoint and fall back to using v0, but when push v2 is rolled out to servers, the 'receive-pack' endpoint will start responding using v2. So we don't want to get into a state where a client is requesting to push with v2 before they actually know how to push using v2. Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-03-15remote-curl: implement stateless-connect commandLibravatar Brandon Williams1-0/+45
Teach remote-curl the 'stateless-connect' command which is used to establish a stateless connection with servers which support protocol version 2. This allows remote-curl to act as a proxy, allowing the git client to communicate natively with a remote end, simply using remote-curl as a pass through to convert requests to http. Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-03-15connect: don't request v2 when pushingLibravatar Brandon Williams1-0/+24
In order to be able to ship protocol v2 with only supporting fetch, we need clients to not issue a request to use protocol v2 when pushing (since the client currently doesn't know how to push using protocol v2). This allows a client to have protocol v2 configured in `protocol.version` and take advantage of using v2 for fetch and falling back to using v0 when pushing while v2 for push is being designed. We could run into issues if we didn't fall back to protocol v2 when pushing right now. This is because currently a server will ignore a request to use v2 when contacting the 'receive-pack' endpoint and fall back to using v0, but when push v2 is rolled out to servers, the 'receive-pack' endpoint will start responding using v2. So we don't want to get into a state where a client is requesting to push with v2 before they actually know how to push using v2. Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-03-15fetch-pack: perform a fetch using v2Libravatar Brandon Williams1-0/+97
When communicating with a v2 server, perform a fetch by requesting the 'fetch' command. Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-03-15ls-remote: pass ref prefixes when requesting a remote's refsLibravatar Brandon Williams1-0/+26
Construct an argv_array of ref prefixes based on the patterns supplied via the command line and pass them to 'transport_get_remote_refs()' to be used when communicating protocol v2 so that the server can limit the ref advertisement based on those prefixes. Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-03-15connect: request remote refs using v2Libravatar Brandon Williams1-0/+57
Teach the client to be able to request a remote's refs using protocol v2. This is done by having a client issue a 'ls-refs' request to a v2 server. Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>