summaryrefslogtreecommitdiff
path: root/http.h
AgeCommit message (Collapse)AuthorFilesLines
2011-12-28Merge branch 'jk/maint-push-over-dav' into maintLibravatar Junio C Hamano1-1/+2
* jk/maint-push-over-dav: http-push: enable "proactive auth" t5540: test DAV push with authentication
2011-12-13Merge branch 'mf/curl-select-fdset' into maintLibravatar Junio C Hamano1-2/+0
* mf/curl-select-fdset: http: drop "local" member from request struct http.c: Rely on select instead of tracking whether data was received http.c: Use timeout suggested by curl instead of fixed 50ms timeout http.c: Use curl_multi_fdset to select on curl fds instead of just sleeping
2011-12-13http-push: enable "proactive auth"Libravatar Jeff King1-1/+2
Before commit 986bbc08, git was proactive about asking for http passwords. It assumed that if you had a username in your URL, you would also want a password, and asked for it before making any http requests. However, this could interfere with the use of .netrc (see 986bbc08 for details). And it was also unnecessary, since the http fetching code had learned to recognize an HTTP 401 and prompt the user then. Furthermore, the proactive prompt could interfere with the usage of .netrc (see 986bbc08 for details). Unfortunately, the http push-over-DAV code never learned to recognize HTTP 401, and so was broken by this change. This patch does a quick fix of re-enabling the "proactive auth" strategy only for http-push, leaving the dumb http fetch and smart-http as-is. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-11-04http: drop "local" member from request structLibravatar Jeff King1-1/+0
This is a FILE pointer in the case that we are sending our output to a file. We originally used it to run ftell() to determine whether data had been written to our file during our last call to curl. However, as of the last patch, we no longer care about that flag anymore. All uses of this struct member are now just book-keeping that can go away. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-11-04http.c: Rely on select instead of tracking whether data was receivedLibravatar Mika Fischer1-1/+0
Since now select is used with the file descriptors of the http connections, tracking whether data was received recently (and trying to read more in that case) is no longer necessary. Instead, always call select and rely on it to return as soon as new data can be read. Signed-off-by: Mika Fischer <mika.fischer@zoopnet.de> Helped-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-10-15http_init: accept separate URL parameterLibravatar Jeff King1-1/+1
The http_init function takes a "struct remote". Part of its initialization procedure is to look at the remote's url and grab some auth-related parameters. However, using the url included in the remote is: - wrong; the remote-curl helper may have a separate, unrelated URL (e.g., from remote.*.pushurl). Looking at the remote's configured url is incorrect. - incomplete; http-fetch doesn't have a remote, so passes NULL. So http_init never gets to see the URL we are actually going to use. - cumbersome; http-push has a similar problem to http-fetch, but actually builds a fake remote just to pass in the URL. Instead, let's just add a separate URL parameter to http_init, and all three callsites can pass in the appropriate information. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-07-19Merge branch 'jc/zlib-wrap'Libravatar Junio C Hamano1-1/+1
* jc/zlib-wrap: zlib: allow feeding more than 4GB in one go zlib: zlib can only process 4GB at a time zlib: wrap deflateBound() too zlib: wrap deflate side of the API zlib: wrap inflateInit2 used to accept only for gzip format zlib: wrap remaining calls to direct inflate/inflateEnd zlib wrapper: refactor error message formatter Conflicts: sha1_file.c
2011-06-10zlib: zlib can only process 4GB at a timeLibravatar Junio C Hamano1-1/+1
The size of objects we read from the repository and data we try to put into the repository are represented in "unsigned long", so that on larger architectures we can handle objects that weigh more than 4GB. But the interface defined in zlib.h to communicate with inflate/deflate limits avail_in (how many bytes of input are we calling zlib with) and avail_out (how many bytes of output from zlib are we ready to accept) fields effectively to 4GB by defining their type to be uInt. In many places in our code, we allocate a large buffer (e.g. mmap'ing a large loose object file) and tell zlib its size by assigning the size to avail_in field of the stream, but that will truncate the high octets of the real size. The worst part of this story is that we often pass around z_stream (the state object used by zlib) to keep track of the number of used bytes in input/output buffer by inspecting these two fields, which practically limits our callchain to the same 4GB limit. Wrap z_stream in another structure git_zstream that can express avail_in and avail_out in unsigned long. For now, just die() when the caller gives a size that cannot be given to a single zlib call. In later patches in the series, we would make git_inflate() and git_deflate() internally loop to give callers an illusion that our "improved" version of zlib interface can operate on a buffer larger than 4GB in one go. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-05-04http: make curl callbacks match contracts from curl headerLibravatar Dan McGee1-3/+3
Yes, these don't match perfectly with the void* first parameter of the fread/fwrite in the standard library, but they do match the curl expected method signature. This is needed when a refactor passes a curl_write_callback around, which would otherwise give incorrect parameter warnings. Signed-off-by: Dan McGee <dpmcgee@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-03-16standardize brace placement in struct definitionsLibravatar Jonathan Nieder1-10/+5
In a struct definitions, unlike functions, the prevailing style is for the opening brace to go on the same line as the struct name, like so: struct foo { int bar; char *baz; }; Indeed, grepping for 'struct [a-z_]* {$' yields about 5 times as many matches as 'struct [a-z_]*$'. Linus sayeth: Heretic people all over the world have claimed that this inconsistency is ... well ... inconsistent, but all right-thinking people know that (a) K&R are _right_ and (b) K&R are right. Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-11-26shift end_url_with_slash() from http.[ch] to url.[ch]Libravatar Tay Ray Chuan1-1/+1
This allows non-http/curl users to access it too (eg. http-backend.c). Update include headers in end_url_with_slash() users too. Signed-off-by: Tay Ray Chuan <rctay89@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-08-12Standardize do { ... } while (0) styleLibravatar Jonathan Nieder1-2/+2
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-05-21Merge branch 'sp/maint-dumb-http-pack-reidx'Libravatar Junio C Hamano1-2/+0
* sp/maint-dumb-http-pack-reidx: http.c::new_http_pack_request: do away with the temp variable filename http-fetch: Use temporary files for pack-*.idx until verified http-fetch: Use index-pack rather than verify-pack to check packs Allow parse_pack_index on temporary files Extract verify_pack_index for reuse from verify_pack Introduce close_pack_index to permit replacement http.c: Remove unnecessary strdup of sha1_to_hex result http.c: Don't store destination name in request structures http.c: Drop useless != NULL test in finish_http_pack_request http.c: Tiny refactoring of finish_http_pack_request t5550-http-fetch: Use subshell for repository operations http.c: Remove bad free of static block
2010-05-08Merge branch 'rc/maint-curl-helper'Libravatar Junio C Hamano1-0/+1
* rc/maint-curl-helper: remote-curl: ensure that URLs have a trailing slash http: make end_url_with_slash() public t5541-http-push: add test for URLs with trailing slash Conflicts: remote-curl.c
2010-04-17http.c: Don't store destination name in request structuresLibravatar Shawn O. Pearce1-2/+0
The destination name within the object store is easily computed on demand, reusing a static buffer held by sha1_file.c. We don't need to copy the entire path into the request structure for safe keeping, when it can be easily reformatted after the download has been completed. This reduces the size of the per-request structure, and removes yet another PATH_MAX based limit. Signed-off-by: Shawn O. Pearce <spearce@spearce.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-04-09http: make end_url_with_slash() publicLibravatar Tay Ray Chuan1-0/+1
Signed-off-by: Tay Ray Chuan <rctay89@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-04-01Prompt for a username when an HTTP request 401sLibravatar Scott Chacon1-0/+2
When an HTTP request returns a 401, Git will currently fail with a confusing message saying that it got a 401, which is not very descriptive. Currently if a user wants to use Git over HTTP, they have to use one URL with the username in the URL (e.g. "http://user@host.com/repo.git") for write access and another without the username for unauthenticated read access (unless they want to be prompted for the password each time). However, since the HTTP servers will return a 401 if an action requires authentication, we can prompt for username and password if we see this, allowing us to use a single URL for both purposes. This patch changes http_request to prompt for the username and password, then return HTTP_REAUTH so http_get_strbuf can try again. If it gets a 401 even when a user/pass is supplied, http_request will now return HTTP_NOAUTH which remote_curl can then use to display a more intelligent error message that is less confusing. Signed-off-by: Scott Chacon <schacon@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-01-12http.c: mark file-local functions staticLibravatar Junio C Hamano1-9/+0
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-11-04Smart push over HTTP: client sideLibravatar Shawn O. Pearce1-0/+2
The git-remote-curl backend detects if the remote server supports the git-receive-pack service, and if so, runs git-send-pack in a pipe to dump the command and pack data as a single POST request. The advertisements from the server that were obtained during the discovery are passed into git-send-pack before the POST request starts. This permits git-send-pack to operate largely unmodified. For smaller packs (those under 1 MiB) a HTTP/1.0 POST with a Content-Length is used, permitting interaction with any server. The 1 MiB limit is arbitrary, but is sufficent to fit most deltas created by human authors against text sources with the occasional small binary file (e.g. few KiB icon image). The configuration option http.postBuffer can be used to increase (or shink) this buffer if the default is not sufficient. For larger packs which cannot be spooled entirely into the helper's memory space (due to http.postBuffer being too small), the POST request requires HTTP/1.1 and sets "Transfer-Encoding: chunked". This permits the client to upload an unknown amount of data in one HTTP transaction without needing to pregenerate the entire pack file locally. Signed-off-by: Shawn O. Pearce <spearce@spearce.org> CC: Daniel Barkalow <barkalow@iabervon.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-06-06http*: add helper methods for fetching objects (loose)Libravatar Tay Ray Chuan1-4/+33
The code handling the fetching of loose objects in http-push.c and http-walker.c have been refactored into new methods and a new struct (object_http_request) in http.c. They are not meant to be invoked elsewhere. The new methods in http.c are - new_http_object_request - process_http_object_request - finish_http_object_request - abort_http_object_request - release_http_object_request and the new struct is http_object_request. RANGER_HEADER_SIZE and no_pragma_header is no longer made available outside of http.c, since after the above changes, there are no other instances of usage outside of http.c. Remove members of the transfer_request struct in http-push.c and http-walker.c, including filename, real_sha1 and zret, as they are used no longer used. Move the methods append_remote_object_url() and get_remote_object_url() from http-push.c to http.c. Additionally, get_remote_object_url() is no longer defined only when USE_CURL_MULTI is defined, since non-USE_CURL_MULTI code in http.c uses it (namely, in new_http_object_request()). Refactor code from http-push.c::start_fetch_loose() and http-walker.c::start_object_fetch_request() that deals with the details of coming up with the filename to store the retrieved object, resuming a previously aborted request, and making a new curl request, into a new function, new_http_object_request(). Refactor code from http-walker.c::process_object_request() into the function, process_http_object_request(). Refactor code from http-push.c::finish_request() and http-walker.c::finish_object_request() into a new function, finish_http_object_request(). It returns the result of the move_temp_to_file() invocation. Add a function, release_http_object_request(), which cleans up object request data. http-push.c and http-walker.c invoke this function separately; http-push.c::release_request() and http-walker.c::release_object_request() do not invoke this function. Add a function, abort_http_object_request(), which unlink()s the object file and invokes release_http_object_request(). Update http-walker.c::abort_object_request() to use this. Signed-off-by: Tay Ray Chuan <rctay89@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-06-06http*: add helper methods for fetching packsLibravatar Tay Ray Chuan1-0/+17
The code handling the fetching of packs in http-push.c and http-walker.c have been refactored into new methods and a new struct (http_pack_request) in http.c. They are not meant to be invoked elsewhere. The new methods in http.c are - new_http_pack_request - finish_http_pack_request - release_http_pack_request and the new struct is http_pack_request. Add a function, new_http_pack_request(), that deals with the details of coming up with the filename to store the retrieved packfile, resuming a previously aborted request, and making a new curl request. Update http-push.c::start_fetch_packed() and http-walker.c::fetch_pack() to use this. Add a function, finish_http_pack_request(), that deals with renaming the pack, advancing the pack list, and installing the pack. Update http-push.c::finish_request() and http-walker.c::fetch_pack to use this. Update release_request() in http-push.c and http-walker.c to invoke release_http_pack_request() to clean up pack request helper data. The local_stream member of the transfer_request struct in http-push.c has been removed, as the packfile pointer will be managed in the struct http_pack_request. Signed-off-by: Tay Ray Chuan <rctay89@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-06-06http*: add http_get_info_packsLibravatar Tay Ray Chuan1-0/+4
http-push.c and http-walker.c no longer have to use fetch_index or setup_index; they simply need to use http_get_info_packs, a new http method, in their fetch_indices implementations. Move fetch_index() and rename to fetch_pack_index() in http.c; this method is not meant to be used outside of http.c. It invokes end_url_with_slash with base_url; apart from that change, the code is identical. Move setup_index() and rename to fetch_and_setup_pack_index() in http.c; this method is not meant to be used outside of http.c. Do not immediately set ret to 0 in http-walker.c::fetch_indices(); instead do it in the HTTP_MISSING_TARGET case, to make it clear that the HTTP_OK and HTTP_MISSING_TARGET cases both return 0. Signed-off-by: Tay Ray Chuan <rctay89@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-06-06http.c: new functions for the http APILibravatar Mike Hommey1-0/+30
The new functions added are: - http_request() (internal function) - http_get_strbuf() - http_get_file() - http_error() http_get_strbuf and http_get_file allow respectively to retrieve contents of an URL to a strbuf or an opened file handle. http_error prints out an error message containing the URL and the curl error (in curl_errorstr). Signed-off-by: Mike Hommey <mh@glandium.org> Signed-off-by: Tay Ray Chuan <rctay89@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-06-06http*: move common variables and macros to http.[ch]Libravatar Tay Ray Chuan1-0/+5
Move RANGE_HEADER_SIZE to http.h. Create no_pragma_header, the curl header list containing the header "Pragma:" in http.[ch]. It is allocated in http_init, and freed in http_cleanup. This replaces the no_pragma_header in http-push.c, and the no_pragma_header member in walker_data in http-walker.c. Create http_is_verbose. It is to be used by methods in http.c, and is modified at the entry points of http.c's users, namely http-push.c (when parsing options) and http-walker.c (in get_http_walker). Signed-off-by: Tay Ray Chuan <rctay89@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-04-02Allow curl to rewind the read buffersLibravatar Martin Storsjö1-0/+7
When using multi-pass authentication methods, the curl library may need to rewind the read buffers (depending on how much already has been fed to the server) used for providing data to HTTP PUT, POST or PROPFIND, and in order to allow the library to do so, we need to tell it how by providing either an ioctl callback or a seek callback. This patch adds an ioctl callback, which should be usable on older curl versions (since 7.12.3) than the seek callback (introduced in curl 7.18.0). Some HTTP servers (such as Apache) give an 401 error reply immediately after receiving the headers (so no data has been read from the read buffers, and thus no rewinding is needed), but other servers (such as Lighttpd) only replies after the whole request has been sent and all data has been read from the read buffers, making rewinding necessary. Signed-off-by: Martin Storsjo <martin@martin.st> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-07-04Work around gcc warnings from curl headersLibravatar Junio C Hamano1-6/+3
After master.k.org upgrade, I started seeing these warning messages: transport.c: In function 'get_refs_via_curl': transport.c:458: error: call to '_curl_easy_setopt_err_write_callback' declared with attribute warning: curl_easy_setopt expects a curl_write_callback argument for this option It appears that the curl header wants to enforce the function signature for callback function given to curl_easy_setopt() to be compatible with that of (*curl_write_callback) or fwrite. This patch seems to work the issue around. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-04-26Make walker.fetch_ref() take a struct ref.Libravatar Daniel Barkalow1-1/+1
This simplifies a few things, makes a few things slightly more complicated, but, more importantly, allows that, when struct ref can represent a symref, http_fetch_ref() can return one. Incidentally makes the string that http_fetch_ref() gets include "refs/" (if appropriate), because that's how the name field of struct ref works. As far as I can tell, the usage in walker:interpret_target() wouldn't have worked previously, if it ever would have been used, which it wouldn't (since the fetch process uses the hash instead of the name of the ref there). Signed-off-by: Daniel Barkalow <barkalow@iabervon.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-02-27Set proxy override with http_init()Libravatar Mike Hommey1-1/+2
In transport.c, proxy setting (the one from the remote conf) was set through curl_easy_setopt() call, while http.c already does the same with the http.proxy setting. We now just use this infrastructure instead, and make http_init() now take the struct remote as argument so that it can take the http_proxy setting from there, and any other property that would be added later. At the same time, we make get_http_walker() take a struct remote argument too, and pass it to http_init(), which makes remote defined proxy be used for more than get_refs_via_curl(). We leave out http-fetch and http-push, which don't use remotes for the moment, purposefully. Signed-off-by: Mike Hommey <mh@glandium.org> Acked-by: Daniel Barkalow <barkalow@iabervon.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-01-22Clarify that http-push being temporarily disabled with older cURLLibravatar Junio C Hamano1-0/+8
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-12-14Move fetch_ref from http-push.c and http-walker.c to http.cLibravatar Mike Hommey1-0/+2
Make the necessary changes to be ok with their difference, and rename the function http_fetch_ref. Signed-off-by: Mike Hommey <mh@glandium.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-12-14Use strbuf in http codeLibravatar Mike Hommey1-5/+6
Also, replace whitespaces with tabs in some places Signed-off-by: Mike Hommey <mh@glandium.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-12-14Avoid redundant declaration of missing_target()Libravatar Mike Hommey1-0/+13
Signed-off-by: Mike Hommey <mh@glandium.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-12-09Cleanup variables in http.[ch]Libravatar Mike Hommey1-18/+0
Quite some variables defined as extern in http.h are only used in http.c, and some others, only defined in http.c, were not static. Signed-off-by: Mike Hommey <mh@glandium.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-09-19Make function to refill http queue a callbackLibravatar Daniel Barkalow1-3/+1
This eliminates the last function provided by the code using http.h as a global symbol, so it should be possible to have multiple programs using http.h in the same executable, and it also adds an argument to that callback, so that info can be passed into the callback without being global. Signed-off-by: Daniel Barkalow <barkalow@iabervon.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-09-19Refactor http.h USE_CURL_MULTI fill_active_slots().Libravatar Daniel Barkalow1-6/+3
This removes all of the boilerplate and http-internal stuff from fill_active_slots() and makes it easy to turn into a callback. Signed-off-by: Daniel Barkalow <barkalow@iabervon.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-05-03http-fetch: Disable use of curl multi support for libcurl < 7.16.Libravatar Alexandre Julliard1-1/+1
curl_multi_remove_handle() is broken in libcurl < 7.16, in that it doesn't correctly update the active handles count when a request is aborted. This causes the transfer to hang forever waiting for the handle count to become less than the number of active requests. Signed-off-by: Alexandre Julliard <julliard@winehq.org> Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-12-27Work around http-fetch built with cURL 7.16.0Libravatar Junio C Hamano1-1/+1
It appears that curl_easy_duphandle() from libcurl 7.16.0 returns a curl session handle which fails GOOD_MULTI_HANDLE() check in curl_multi_add_handle(). This causes fetch_ref() to fail because start_active_slot() cannot start the request. For now, check for 7.16.0 to work this issue around. Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-09-20Patch for http-fetch.c and older curl releasesLibravatar Art Haas1-0/+4
Older curl releases do not define CURLE_HTTP_RETURNED_ERROR, they use CURLE_HTTP_NOT_FOUND instead. Newer curl releases keep the CURLE_HTTP_NOT_FOUND definition but using a -DCURL_NO_OLDIES preprocessor flag the old name will not be present in the 'curl.h' header. This patch makes our code written for newer releases of the curl library but allow compiling against an older curl (older than 0x070a03) by defining the missing CURLE_HTTP_RETURNED_ERROR as a synonym for CURLE_HTTP_NOT_FOUND. Signed-off-by: Art Haas <ahaas@airmail.net> Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-03-10HTTP slot reuse fixesLibravatar Nick Hengeveld1-0/+1
Incorporate into http-push a fix related to accessing slot results after the slot was reused, and fix a case in run_active_slot where a finished slot wasn't detected if the slot was reused. Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-02-07http-fetch: Abort requests for objects which arrived in packsLibravatar Mark Wooding1-0/+1
In fetch_object, there's a call to release an object request if the object mysteriously arrived, say in a pack. Unfortunately, the fetch attempt for this object might already be in progress, and we'll leak the descriptor. Instead, try to tidy away the request. Signed-off-by: Mark Wooding <mdw@distorted.org.uk> Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-01-31Fix HTTP request result processing after slot reuseLibravatar Nick Hengeveld1-0/+7
Add a way to store the results of an HTTP request when a slot finishes so the results can be processed after the slot has been reused. Signed-off-by: Nick Hengeveld <nickh@reactrix.com> Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-11-19Isolate shared HTTP request functionalityLibravatar Nick Hengeveld1-0/+95
Move shared HTTP request functionality out of http-fetch and http-push, and replace the two fwrite_buffer/fwrite_buffer_dynamic functions with one fwrite_buffer function that does dynamic buffering. Use slot callbacks to process responses to fetch object transfer requests and push transfer requests, and put all of http-push into an #ifdef check for curl multi support. Signed-off-by: Nick Hengeveld <nickh@reactrix.com> Signed-off-by: Junio C Hamano <junkio@cox.net>