summaryrefslogtreecommitdiff
path: root/remote-curl.c
AgeCommit message (Collapse)AuthorFilesLines
2012-10-02Merge branch 'sp/maint-http-enable-gzip' into maintLibravatar Junio C Hamano1-2/+2
"git fetch" over http advertised that it supports "deflate", which is much less common, and did not advertise more common "gzip" on its Accept-Encoding header. * sp/maint-http-enable-gzip: Enable info/refs gzip decompression in HTTP client
2012-09-20Enable info/refs gzip decompression in HTTP clientLibravatar Shawn O. Pearce1-2/+2
Some HTTP servers try to use gzip compression on the /info/refs request to save transfer bandwidth. Repositories with many tags may find the /info/refs request can be gzipped to be 50% of the original size due to the few but often repeated bytes used (hex SHA-1 and commonly digits in tag names). For most HTTP requests enable "Accept-Encoding: gzip" ensuring the /info/refs payload can use this encoding format. Only request gzip encoding from servers. Although deflate is supported by libcurl, most servers have standardized on gzip encoding for compression as that is what most browsers support. Asking for deflate increases request sizes by a few bytes, but is unlikely to ever be used by a server. Disable the Accept-Encoding header on probe RPCs as response bodies are supposed to be exactly 4 bytes long, "0000". The HTTP headers requesting and indicating compression use more space than the data transferred in the body. Signed-off-by: Shawn O. Pearce <spearce@spearce.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-09-20Revert "retry request without query when info/refs?query fails"Libravatar Shawn O. Pearce1-16/+2
This reverts commit 703e6e76a14825e5b0c960d525f34e607154b4f7. Retrying without the query parameter was added as a workaround for a single broken HTTP server at git.debian.org[1]. The server was misconfigured to route every request with a query parameter into gitweb.cgi. Admins fixed the server's configuration within 16 hours of the bug report to the Git mailing list, but we still patched Git with this fallback and have been paying for it since. Most Git hosting services configure the smart HTTP protocol and the retry logic confuses users when there is a transient HTTP error as Git dropped the real error from the smart HTTP request. Removing the retry makes root causes easier to identify. [1] http://thread.gmane.org/gmane.comp.version-control.git/137609 Signed-off-by: Shawn O. Pearce <spearce@spearce.org> Acked-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-08-27http: prompt for credentials on failed POSTLibravatar Jeff King1-8/+15
All of the smart-http GET requests go through the http_get_* functions, which will prompt for credentials and retry if we see an HTTP 401. POST requests, however, do not go through any central point. Moreover, it is difficult to retry in the general case; we cannot assume the request body fits in memory or is even seekable, and we don't know how much of it was consumed during the attempt. Most of the time, this is not a big deal; for both fetching and pushing, we make a GET request before doing any POSTs, so typically we figure out the credentials during the first request, then reuse them during the POST. However, some servers may allow a client to get the list of refs from receive-pack without authentication, and then require authentication when the client actually tries to POST the pack. This is not ideal, as the client may do a non-trivial amount of work to generate the pack (e.g., delta-compressing objects). However, for a long time it has been the recommended example configuration in git-http-backend(1) for setting up a repository with anonymous fetch and authenticated push. This setup has always been broken without putting a username into the URL. Prior to commit 986bbc0, it did work with a username in the URL, because git would prompt for credentials before making any requests at all. However, post-986bbc0, it is totally broken. Since it has been advertised in the manpage for some time, we should make sure it works. Unfortunately, it is not as easy as simply calling post_rpc again when it fails, due to the input issue mentioned above. However, we can still make this specific case work by retrying in two specific instances: 1. If the request is large (bigger than LARGE_PACKET_MAX), we will first send a probe request with a single flush packet. Since this request is static, we can freely retry it. 2. If the request is small and we are not using gzip, then we have the whole thing in-core, and we can freely retry. That means we will not retry in some instances, including: 1. If we are using gzip. However, we only do so when calling git-upload-pack, so it does not apply to pushes. 2. If we have a large request, the probe succeeds, but then the real POST wants authentication. This is an extremely unlikely configuration and not worth worrying about. While it might be nice to cover those instances, doing so would be significantly more complex for very little real-world gain. In the long run, we will be much better off when curl learns to internally handle authentication as a callback, and we can cleanly handle all cases that way. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-05-10Merge branch 'jk/maint-push-progress' into maintLibravatar Junio C Hamano1-0/+1
"git push" over smart-http lost progress output a few releases ago. By Jeff King * jk/maint-push-progress: t5541: test more combinations of --progress teach send-pack about --[no-]progress send-pack: show progress when isatty(2)
2012-05-01Merge branch 'it/fetch-pack-many-refs' into maintLibravatar Junio C Hamano1-5/+13
When "git fetch" encounters repositories with too many references, the command line of "fetch-pack" that is run by a helper e.g. remote-curl, may fail to hold all of them. Now such an internal invocation can feed the references through the standard input of "fetch-pack". By Ivan Todoroski * it/fetch-pack-many-refs: remote-curl: main test case for the OS command line overflow fetch-pack: test cases for the new --stdin option remote-curl: send the refs to fetch-pack on stdin fetch-pack: new --stdin option to read refs from stdin Conflicts: t/t5500-fetch-pack.sh
2012-05-01teach send-pack about --[no-]progressLibravatar Jeff King1-0/+1
The send_pack function gets a "progress" flag saying "yes, definitely show progress" or "no, definitely do not show progress". This gets set properly by transport_push when send_pack is called directly. However, when the send-pack command is executed separately (as it is for the remote-curl helper), there is no way to tell it "definitely do this". As a result, we do not properly respect "git push --no-progress" for smart-http remotes; you will still get progress if stderr is a tty. This patch teaches send-pack --progress and --no-progress, and teaches remote-curl to pass the appropriate option to override send-pack's isatty check. This fixes the --no-progress case above, and as a bonus, also makes "git push --progress" work when stderr is not a tty. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-04-10remote-curl: send the refs to fetch-pack on stdinLibravatar Ivan Todoroski1-5/+13
Now that we can throw an arbitrary number of refs at fetch-pack using its --stdin option, we use it in the remote-curl helper to bypass the OS command line length limit. Signed-off-by: Ivan Todoroski <grnch@gmx.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-02-05Merge branch 'sp/smart-http-failure-to-push' into maintLibravatar Junio C Hamano1-4/+5
* sp/smart-http-failure-to-push: remote-curl: Fix push status report when all branches fail
2012-01-29Merge branch 'sp/smart-http-failure-to-push'Libravatar Junio C Hamano1-4/+5
* sp/smart-http-failure-to-push: remote-curl: Fix push status report when all branches fail
2012-01-20remote-curl: Fix push status report when all branches failLibravatar Shawn O. Pearce1-4/+5
The protocol between transport-helper.c and remote-curl requires remote-curl to always print a blank line after the push command has run. If the blank line is ommitted, transport-helper kills its container process (the git push the user started) with exit(128) and no message indicating a problem, assuming the helper already printed reasonable error text to the console. However if the remote rejects all branches with "ng" commands in the report-status reply, send-pack terminates with non-zero status, and in turn remote-curl exited with non-zero status before outputting the blank line after the helper status printed by send-pack. No error messages reach the user. This caused users to see the following from git push over HTTP when the remote side's update hook rejected the branch: $ git push http://... master Counting objects: 4, done. Delta compression using up to 6 threads. Compressing objects: 100% (2/2), done. Writing objects: 100% (3/3), 301 bytes, done. Total 3 (delta 0), reused 0 (delta 0) $ Always print a blank line after the send-pack process terminates, ensuring the helper status report (if it was output) will be correctly parsed by the calling transport-helper.c. This ensures the helper doesn't abort before the status report can be shown to the user. Signed-off-by: Shawn O. Pearce <spearce@spearce.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-01-08fix push --quiet: add 'quiet' capability to receive-packLibravatar Clemens Buchacher1-1/+3
Currently, git push --quiet produces some non-error output, e.g.: $ git push --quiet Unpacking objects: 100% (3/3), done. This fixes a bug reported for the fedora git package: https://bugzilla.redhat.com/show_bug.cgi?id=725593 Reported-by: Jesse Keating <jkeating@redhat.com> Cc: Todd Zullinger <tmz@pobox.com> Commit 90a6c7d4 (propagate --quiet to send-pack/receive-pack) introduced the --quiet option to receive-pack and made send-pack pass that option. Older versions of receive-pack do not recognize the option, however, and terminate immediately. The commit was therefore reverted. This change instead adds a 'quiet' capability to receive-pack, which is a backwards compatible. In addition, this fixes push --quiet via http: A verbosity of 0 means quiet for remote helpers. Reported-by: Tobias Ulmer <tobiasu@tmux.org> Signed-off-by: Clemens Buchacher <drizzd@aon.at> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-12-22Merge branch 'jk/http-push-to-empty'Libravatar Junio C Hamano1-3/+4
* jk/http-push-to-empty: remote-curl: don't pass back fake refs Conflicts: remote-curl.c
2011-12-19Merge branch 'jk/maint-push-over-dav'Libravatar Junio C Hamano1-1/+1
* jk/maint-push-over-dav: http-push: enable "proactive auth" t5540: test DAV push with authentication Conflicts: http.c
2011-12-19remote-curl: don't pass back fake refsLibravatar Jeff King1-3/+4
When receive-pack advertises its list of refs, it generally hides the capabilities information after a NUL at the end of the first ref. However, when we have an empty repository, there are no refs, and therefore receive-pack writes a fake ref "capabilities^{}" with the capabilities afterwards. On the client side, git reads the result with get_remote_heads(). We pick the capabilities from the end of the line, and then call check_ref() to make sure the ref name is valid. We see that it isn't, and don't bother adding it to our list of refs. However, the call to check_ref() is enabled by passing the REF_NORMAL flag to get_remote_heads. For the regular git transport, we pass REF_NORMAL in get_refs_via_connect() if we are doing a push (since only receive-pack uses this fake ref). But in remote-curl, we never use this flag, and we accept the fake ref as a real one, passing it back from the helper to the parent git-push. Most of the time this bug goes unnoticed, as the fake ref won't match our refspecs. However, if "--mirror" is used, then we see it as remote cruft to be pruned, and try to pass along a deletion refspec for it. Of course this refspec has bogus syntax (because of the ^{}), and the helper complains, aborting the push. Let's have remote-curl mirror what the builtin get_refs_via_connect() does (at least for the case of using git protocol; we can leave the dumb info/refs reader as it is). This also fixes pushing with --mirror to a smart-http remote that uses alternates. The fake ".have" refs the server gives to avoid unnecessary network transfer has a similar bad interactions with the machinery. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-12-13http-push: enable "proactive auth"Libravatar Jeff King1-1/+1
Before commit 986bbc08, git was proactive about asking for http passwords. It assumed that if you had a username in your URL, you would also want a password, and asked for it before making any http requests. However, this could interfere with the use of .netrc (see 986bbc08 for details). And it was also unnecessary, since the http fetching code had learned to recognize an HTTP 401 and prompt the user then. Furthermore, the proactive prompt could interfere with the usage of .netrc (see 986bbc08 for details). Unfortunately, the http push-over-DAV code never learned to recognize HTTP 401, and so was broken by this change. This patch does a quick fix of re-enabling the "proactive auth" strategy only for http-push, leaving the dumb http fetch and smart-http as-is. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-12-13drop "match" parameter from get_remote_headsLibravatar Jeff King1-1/+1
The get_remote_heads function reads the list of remote refs during git protocol session. It dates all the way back to def88e9 (Commit first cut at "git-fetch-pack", 2005-07-04). At that time, the idea was to come up with a list of refs we were interested in, and then filter the list as we got it from the remote side. Later, 1baaae5 (Make maximal use of the remote refs, 2005-10-28) stopped filtering at the get_remote_heads layer, letting us use the non-matching refs to find common history. As a result, all callers now simply pass an empty match list (and any future callers will want to do the same). So let's drop these now-useless parameters. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-10-17Merge branch 'jk/http-auth'Libravatar Junio C Hamano1-2/+2
* jk/http-auth: http_init: accept separate URL parameter http: use hostname in credential description http: retry authentication failures for all http requests remote-curl: don't retry auth failures with dumb protocol improve httpd auth tests url: decode buffers that are not NUL-terminated
2011-10-15http_init: accept separate URL parameterLibravatar Jeff King1-1/+1
The http_init function takes a "struct remote". Part of its initialization procedure is to look at the remote's url and grab some auth-related parameters. However, using the url included in the remote is: - wrong; the remote-curl helper may have a separate, unrelated URL (e.g., from remote.*.pushurl). Looking at the remote's configured url is incorrect. - incomplete; http-fetch doesn't have a remote, so passes NULL. So http_init never gets to see the URL we are actually going to use. - cumbersome; http-push has a similar problem to http-fetch, but actually builds a fake remote just to pass in the URL. Instead, let's just add a separate URL parameter to http_init, and all three callsites can pass in the appropriate information. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-10-12Merge branch 'sp/smart-http-failure'Libravatar Junio C Hamano1-1/+8
* sp/smart-http-failure: remote-curl: Fix warning after HTTP failure
2011-10-04remote-curl: Fix warning after HTTP failureLibravatar Shawn O. Pearce1-1/+8
If the HTTP connection is broken in the middle of a fetch or clone body, the client presented a useless error message due to part of the upload-pack->remote-curl pkt-line protocol leaking out of the helper as the helper's "fetch result": error: RPC failed; result=18, HTTP code = 200 fatal: The remote end hung up unexpectedly fatal: early EOF fatal: unpack-objects failed warning: https unexpectedly said: '0000' Instead when the HTTP RPC fails discard all remaining data from upload-pack and report nothing to the transport helper. Errors were already sent to stderr. Signed-off-by: Shawn O. Pearce <spearce@spearce.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-09-06Sync with 1.7.6.2Libravatar Junio C Hamano1-3/+1
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-09-06Revert "Merge branch 'cb/maint-quiet-push' into maint"Libravatar Junio C Hamano1-3/+1
This reverts commit ffa69e61d3c5730bd4b65a465efc130b0ef3c7df, reversing changes made to 4a13c4d14841343d7caad6ed41a152fee550261d. Adding a new command line option to receive-pack and feed it from send-pack is not an acceptable way to add features, as there is no guarantee that your updated send-pack will be talking to updated receive-pack. New features need to be added via the capability mechanism negotiated over the protocol. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-08-23Merge branch 'cb/maint-quiet-push' into maintLibravatar Junio C Hamano1-1/+3
* cb/maint-quiet-push: receive-pack: do not overstep command line argument array propagate --quiet to send-pack/receive-pack Conflicts: Documentation/git-receive-pack.txt Documentation/git-send-pack.txt
2011-08-17Merge branch 'cb/maint-quiet-push'Libravatar Junio C Hamano1-1/+3
* cb/maint-quiet-push: receive-pack: do not overstep command line argument array propagate --quiet to send-pack/receive-pack Conflicts: Documentation/git-receive-pack.txt Documentation/git-send-pack.txt
2011-08-16Merge branch 'jc/zlib-wrap' into maintLibravatar Junio C Hamano1-9/+5
* jc/zlib-wrap: zlib: allow feeding more than 4GB in one go zlib: zlib can only process 4GB at a time zlib: wrap deflateBound() too zlib: wrap deflate side of the API zlib: wrap inflateInit2 used to accept only for gzip format zlib: wrap remaining calls to direct inflate/inflateEnd zlib wrapper: refactor error message formatter
2011-08-01Merge branch 'sr/transport-helper-fix'Libravatar Junio C Hamano1-1/+9
* sr/transport-helper-fix: (21 commits) transport-helper: die early on encountering deleted refs transport-helper: implement marks location as capability transport-helper: Use capname for refspec capability too transport-helper: change import semantics transport-helper: update ref status after push with export transport-helper: use the new done feature where possible transport-helper: check status code of finish_command transport-helper: factor out push_update_refs_status fast-export: support done feature fast-import: introduce 'done' command git-remote-testgit: fix error handling git-remote-testgit: only push for non-local repositories remote-curl: accept empty line as terminator remote-helpers: export GIT_DIR variable to helpers git_remote_helpers: push all refs during a non-local export transport-helper: don't feed bogus refs to export push git-remote-testgit: import non-HEAD refs t5800: document some non-functional parts of remote helpers t5800: use skip_all instead of prereq t5800: factor out some ref tests ...
2011-07-31propagate --quiet to send-pack/receive-packLibravatar Clemens Buchacher1-1/+3
Currently, git push --quiet produces some non-error output, e.g.: $ git push --quiet Unpacking objects: 100% (3/3), done. Add the --quiet option to send-pack/receive-pack and pass it to unpack-objects in the receive-pack codepath and to receive-pack in the push codepath. This fixes a bug reported for the fedora git package: https://bugzilla.redhat.com/show_bug.cgi?id=725593 Reported-by: Jesse Keating <jkeating@redhat.com> Cc: Todd Zullinger <tmz@pobox.com> Signed-off-by: Clemens Buchacher <drizzd@aon.at> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-07-22Merge branch 'maint'Libravatar Junio C Hamano1-0/+2
* maint: doc/fast-import: clarify notemodify command Documentation: minor grammatical fix in rev-list-options.txt Documentation: git-filter-branch honors replacement refs remote-curl: Add a format check to parsing of info/refs git-config: Remove extra whitespaces
2011-07-20remote-curl: Add a format check to parsing of info/refsLibravatar Julian Phillips1-0/+2
When parsing info/refs, no checks were applied that the file was in the requried format. Since the file is read from a remote webserver, this isn't guarenteed to be true. Add a check that the file at least only contains lines that consist of 40 characters followed by a tab and then the ref name. Signed-off-by: Julian Phillips <julian@quantumfyre.co.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-07-20remote-curl: don't retry auth failures with dumb protocolLibravatar Jeff King1-1/+1
When fetching an http URL, we first try fetching info/refs with an extra "service" parameter. This will work for a smart-http server, or a dumb server which ignores extra parameters when fetching files. If that fails, we retry without the extra parameter to remain compatible with dumb servers which didn't like our first request. If the server returned a "401 Unauthorized", indicating that the credentials we provided were not good, there is not much point in retrying. With the current code, we just waste an extra round trip to the HTTP server before failing. But as the http code becomes smarter about throwing away rejected credentials and re-prompting the user for new ones (which it will later in this series), this will become more confusing. At some point we will stop asking for credentials to retry smart http, and will be asking for credentials to retry dumb http. So now we're not only wasting an extra HTTP round trip for something that is unlikely to work, but we're making the user re-type their password for it. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-07-19remote-curl: accept empty line as terminatorLibravatar Sverre Rabbelier1-1/+9
This went unnoticed because the transport helper infrastructore did not check the return value of the helper, nor did the helper print anything before exiting. While at it also make sure that the stream doesn't end unexpectedly. Signed-off-by: Sverre Rabbelier <srabbelier@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-07-19Merge branch 'jc/zlib-wrap'Libravatar Junio C Hamano1-9/+5
* jc/zlib-wrap: zlib: allow feeding more than 4GB in one go zlib: zlib can only process 4GB at a time zlib: wrap deflateBound() too zlib: wrap deflate side of the API zlib: wrap inflateInit2 used to accept only for gzip format zlib: wrap remaining calls to direct inflate/inflateEnd zlib wrapper: refactor error message formatter Conflicts: sha1_file.c
2011-06-20plug a few coverity-spotted leaksLibravatar Jim Meyering1-4/+6
Signed-off-by: Jim Meyering <meyering@redhat.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-06-10zlib: zlib can only process 4GB at a timeLibravatar Junio C Hamano1-1/+1
The size of objects we read from the repository and data we try to put into the repository are represented in "unsigned long", so that on larger architectures we can handle objects that weigh more than 4GB. But the interface defined in zlib.h to communicate with inflate/deflate limits avail_in (how many bytes of input are we calling zlib with) and avail_out (how many bytes of output from zlib are we ready to accept) fields effectively to 4GB by defining their type to be uInt. In many places in our code, we allocate a large buffer (e.g. mmap'ing a large loose object file) and tell zlib its size by assigning the size to avail_in field of the stream, but that will truncate the high octets of the real size. The worst part of this story is that we often pass around z_stream (the state object used by zlib) to keep track of the number of used bytes in input/output buffer by inspecting these two fields, which practically limits our callchain to the same 4GB limit. Wrap z_stream in another structure git_zstream that can express avail_in and avail_out in unsigned long. For now, just die() when the caller gives a size that cannot be given to a single zlib call. In later patches in the series, we would make git_inflate() and git_deflate() internally loop to give callers an illusion that our "improved" version of zlib interface can operate on a buffer larger than 4GB in one go. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-06-10zlib: wrap deflateBound() tooLibravatar Junio C Hamano1-1/+1
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-06-10zlib: wrap deflate side of the APILibravatar Junio C Hamano1-7/+3
Wrap deflateInit, deflate, and deflateEnd for everybody, and the sole use of deflateInit2 in remote-curl.c to tell the library to use gzip header and trailer in git_deflate_init_gzip(). There is only one caller that cares about the status from deflateEnd(). Introduce git_deflate_end_gently() to let that sole caller retrieve the status and act on it (i.e. die) for now, but we would probably want to make inflate_end/deflate_end die when they ran out of memory and get rid of the _gently() kind. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-05-04http: make curl callbacks match contracts from curl headerLibravatar Dan McGee1-1/+1
Yes, these don't match perfectly with the void* first parameter of the fread/fwrite in the standard library, but they do match the curl expected method signature. This is needed when a refactor passes a curl_write_callback around, which would otherwise give incorrect parameter warnings. Signed-off-by: Dan McGee <dpmcgee@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-03-14Merge branch 'sp/maint-smart-http-sans-100-continue'Libravatar Junio C Hamano1-1/+1
* sp/maint-smart-http-sans-100-continue: smart-http: Really never use Expect: 100-continue
2011-03-14smart-http: Really never use Expect: 100-continueLibravatar Shawn O. Pearce1-1/+1
libcurl may choose to try and use Expect: 100-continue for any type of POST, not just a Transfer: chunked-encoding type. Force it to disable this feature, as not all proxy servers support 100-continue and leaving it enabled can cause 1 second stalls during the negotiation phase of fetch-pack/upload-pack. In ("206b099d26 smart-http: Don't use Expect: 100-Continue") we tried to disable this for only large POST bodies, but it should be disabled for every POST body. Signed-off-by: Shawn O. Pearce <spearce@spearce.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-02-27Merge branch 'sp/maint-smart-http-sans-100-continue'Libravatar Junio C Hamano1-11/+55
* sp/maint-smart-http-sans-100-continue: smart-http: Don't use Expect: 100-Continue
2011-02-15smart-http: Don't use Expect: 100-ContinueLibravatar Shawn O. Pearce1-11/+55
Some HTTP/1.1 servers or proxies don't correctly implement the 100-Continue feature of HTTP/1.1. Its a difficult feature to implement right, and isn't commonly used by browsers, so many developers may not even be aware that their server (or proxy) doesn't honor it. Within the smart HTTP protocol for Git we only use this newer "Expect: 100-Continue" feature to probe for missing authentication before uploading a large payload like a pack file during push. If authentication is necessary, we expect the server to send the 401 Not Authorized response before the bulk data transfer starts, thus saving the client bandwidth during the retry. A different method to probe for working authentication is to send an empty command list (that is just "0000") to $URL/git-receive-pack. or $URL/git-upload-pack. All versions of both receive-pack and upload-pack since the introduction of smart HTTP in Git 1.6.6 cleanly accept just a flush-pkt under --stateless-rpc mode, and exit with success. If HTTP level authentication is successful, the backend will return an empty response, but with HTTP status code 200. This enables the client to continue with the transfer. Signed-off-by: Shawn O. Pearce <spearce@spearce.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-08-12Merge branch 'sp/fix-smart-http-deadlock-on-error'Libravatar Junio C Hamano1-2/+3
* sp/fix-smart-http-deadlock-on-error: smart-http: Don't deadlock on server failure
2010-08-06smart-http: Don't deadlock on server failureLibravatar Shawn O. Pearce1-2/+3
If the remote HTTP server fails (e.g. returns 404 or 500) when we posted the RPC to it, we won't have sent anything to the background Git process that is supposed to handle the stream. Because we didn't send anything, its waiting for input from remote-curl, and remote-curl cannot read its response payload because doing so would lead to a deadlock. Send the background task EOF on its input before we try to read its response back, that way it will break out of its read loop and terminate. Signed-off-by: Shawn O. Pearce <spearce@spearce.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-05-08Merge branch 'rc/maint-curl-helper'Libravatar Junio C Hamano1-6/+8
* rc/maint-curl-helper: remote-curl: ensure that URLs have a trailing slash http: make end_url_with_slash() public t5541-http-push: add test for URLs with trailing slash Conflicts: remote-curl.c
2010-04-09remote-curl: ensure that URLs have a trailing slashLibravatar Tay Ray Chuan1-6/+8
Previously, we blindly assumed that URLs passed to the remote-curl helper did not end with a trailing slash. Use the convenience function end_url_with_slash() from http.[ch] to ensure that URLs have a trailing slash on invocation of the remote-curl helper, and use the URL as one with a trailing slash throughout. It is possible for users to pass a URL with a trailing slash to remote-curl, by, say, setting it in remote.<name>.url in their git config. The resulting requests have an empty path component (//) and may break implementations of the http git protocol. Signed-off-by: Tay Ray Chuan <rctay89@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-04-01Prompt for a username when an HTTP request 401sLibravatar Scott Chacon1-0/+2
When an HTTP request returns a 401, Git will currently fail with a confusing message saying that it got a 401, which is not very descriptive. Currently if a user wants to use Git over HTTP, they have to use one URL with the username in the URL (e.g. "http://user@host.com/repo.git") for write access and another without the username for unauthenticated read access (unless they want to be prompted for the password each time). However, since the HTTP servers will return a 401 if an action requires authentication, we can prompt for username and password if we see this, allowing us to use a single URL for both purposes. This patch changes http_request to prompt for the username and password, then return HTTP_REAUTH so http_get_strbuf can try again. If it gets a 401 even when a user/pass is supplied, http_request will now return HTTP_NOAUTH which remote_curl can then use to display a more intelligent error message that is less confusing. Signed-off-by: Scott Chacon <schacon@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-03-15Merge branch 'tc/http-cleanup'Libravatar Junio C Hamano1-12/+9
* tc/http-cleanup: remote-curl: init walker only when needed remote-curl: use http_fetch_ref() instead of walker wrapper http: init and cleanup separately from http-walker http-walker: cleanup more thoroughly http-push: remove "|| 1" to enable verbose check t554[01]-http-push: refactor, add non-ff tests t5541-http-push: check that ref is unchanged for non-ff test
2010-03-02Merge branch 'sp/maint-push-sideband' into maintLibravatar Junio C Hamano1-3/+4
* sp/maint-push-sideband: receive-pack: Send internal errors over side-band #2 t5401: Use a bare repository for the remote peer receive-pack: Send hook output over side band #2 receive-pack: Wrap status reports inside side-band-64k receive-pack: Refactor how capabilities are shown to the client send-pack: demultiplex a sideband stream with status data run-command: support custom fd-set in async run-command: Allow stderr to be a caller supplied pipe Conflicts: builtin-receive-pack.c run-command.c t/t5401-update-hooks.sh
2010-03-02remote-curl: init walker only when neededLibravatar Tay Ray Chuan1-10/+3
Invoke get_http_walker() only when fetching with the dumb protocol. Additionally, add an invocation to walker_free() after we're done using the walker. Signed-off-by: Tay Ray Chuan <rctay89@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>