summaryrefslogtreecommitdiff
path: root/t/lib-httpd
AgeCommit message (Collapse)AuthorFilesLines
2020-05-24remote-curl: error on incomplete packetLibravatar Denton Liu3-0/+14
Currently, remote-curl acts as a proxy and blindly forwards packets between an HTTP server and fetch-pack. In the case of a stateless RPC connection where the connection is terminated with a partially written packet, remote-curl will blindly send the partially written packet before waiting on more input from fetch-pack. Meanwhile, fetch-pack will read the partial packet and continue reading, expecting more input. This results in a deadlock between the two processes. For a stateless connection, inspect packets before sending them and error out if a packet line packet is incomplete. Helped-by: Jeff King <peff@peff.net> Signed-off-by: Denton Liu <liu.denton@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-02-27t/lib-httpd: avoid using macOS' sedLibravatar Johannes Schindelin3-27/+30
Among other differences relative to GNU sed, macOS' sed always ends its output with a trailing newline, even if the input did not have such a trailing newline. Surprisingly, this makes three httpd-based tests fail on macOS: t5616, t5702 and t5703. ("Surprisingly" because those tests have been around for some time, but apparently nobody runs them on macOS with a working Apache2 setup.) The reason is that we use `sed` in those tests to filter the response of the web server. Apart from the fact that we use GNU constructs (such as using a space after the `c` command instead of a backslash and a newline), we have another problem: macOS' sed LF-only newlines while webservers are supposed to use CR/LF ones. Even worse, t5616 uses `sed` to replace a binary part of the response with a new binary part (kind of hoping that the replaced binary part does not contain a 0x0a byte which would be interpreted as a newline). To that end, it calls on Perl to read the binary pack file and hex-encode it, then calls on `sed` to prefix every hex digit pair with a `\x` in order to construct the text that the `c` statement of the `sed` invocation is supposed to insert. So we call Perl and sed to construct a sed statement. The final nail in the coffin is that macOS' sed does not even interpret those `\x<hex>` constructs. Let's just replace all of that by Perl snippets. With Perl, at least, we do not have to deal with GNU vs macOS semantics, we do not have to worry about unwanted trailing newlines, and we do not have to spawn commands to construct arguments for other commands to be spawned (i.e. we can avoid a whole lot of shell scripting complexity). The upshot is that this fixes t5616, t5702 and t5703 on macOS with Apache2. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-29apply-one-time-sed.sh: modernize styleLibravatar Denton Liu1-3/+5
Convert `[ ... ]` to use `test` and test for the existence of a regular file (`-f`) instead of any file (`-e`). Move the `then`s onto their own lines so that it conforms with the general test style. Instead of redirecting input into sed, allow it to open its own input. Use `cmp -s` instead of `diff` since we only care about whether the two files are equal and `diff` is overkill for this. Signed-off-by: Denton Liu <liu.denton@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-05-08t/lib-httpd: pass LSAN_OPTIONS through apacheLibravatar Jeff King1-0/+1
Just as we instruct Apache to pass through ASAN_OPTIONS (so that server-side Git programs it spawns will respect our options while running the tests), we should do the same with LSAN_OPTIONS. Otherwise trying to collect a list of leaks like: export LSAN_OPTIONS=exitcode=0:log_path=/tmp/lsan make SANITIZE=leak test won't work for http tests (the server-side programs won't log their leaks to the right place, and they'll prematurely die, producing a spurious test failure). Signed-off-by: Jeff King <peff@peff.net> Acked-by: Josh Steadmon <steadmon@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-06t5551: test server-side ERR packetLibravatar Josh Steadmon2-0/+7
When a smart HTTP server sends an error message via pkt-line, we detect the error due to using PACKET_READ_DIE_ON_ERR_PACKET. This case was added by 2d103c31c2 (pack-protocol.txt: accept error packets in any context, 2018-12-29), but not covered by tests. Signed-off-by: Josh Steadmon <steadmon@google.com> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-05Merge branch 'jt/fetch-v2-sideband'Libravatar Junio C Hamano1-0/+1
"git fetch" and "git upload-pack" learned to send all exchange over the sideband channel while talking the v2 protocol. * jt/fetch-v2-sideband: tests: define GIT_TEST_SIDEBAND_ALL {fetch,upload}-pack: sideband v2 fetch response sideband: reverse its dependency on pkt-line pkt-line: introduce struct packet_writer pack-protocol.txt: accept error packets in any context Use packet_reader instead of packet_read_line
2019-01-17tests: define GIT_TEST_SIDEBAND_ALLLibravatar Jonathan Tan1-0/+1
Define a GIT_TEST_SIDEBAND_ALL environment variable meant to be used from tests. When set to true, this overrides uploadpack.allowsidebandall to true, allowing the entire test suite to be run as if this configuration is in place for all repositories. As of this patch, all tests pass whether GIT_TEST_SIDEBAND_ALL is unset or set to 1. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-10test: test GIT_CURL_VERBOSE=1 shows an errorLibravatar Masaya Suzuki1-0/+1
This tests GIT_CURL_VERBOSE shows an error when an URL returns 500. This exercises the code in remote_curl. Signed-off-by: Masaya Suzuki <masayasuzuki@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-06-28upload-pack: test negotiation with changing repositoryLibravatar Brandon Williams2-0/+30
Add tests to check the behavior of fetching from a repository which changes between rounds of negotiation (for example, when different servers in a load-balancing agreement participate in the same stateless RPC negotiation). This forms a baseline of comparison to the ref-in-want functionality (which will be introduced to the client in subsequent commits), and ensures that subsequent commits do not change existing behavior. As part of this effort, a mechanism to substitute strings in a single HTTP response is added. Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-01-04http: fix v1 protocol tests with apache httpd < 2.4Libravatar Todd Zullinger1-6/+4
The apache config used by tests was updated to use the SetEnvIf directive to set the Git-Protocol header in 19113a26b6 ("http: tell server that the client understands v1", 2017-10-16). Setting the Git-Protocol header is restricted to httpd >= 2.4, but mod_setenvif and the SetEnvIf directive work with lower versions, at least as far back as 2.0, according to the httpd documentation: https://httpd.apache.org/docs/2.0/mod/mod_setenvif.html Drop the restriction. Tested with httpd 2.2 and 2.4. Signed-off-by: Todd Zullinger <tmz@pobox.com> Acked-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-10-17http: tell server that the client understands v1Libravatar Brandon Williams1-0/+7
Tell a server that protocol v1 can be used by sending the http header 'Git-Protocol' with 'version=1' indicating this. Also teach the apache http server to pass through the 'Git-Protocol' header as an environment variable 'GIT_PROTOCOL'. Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-02-28http: attempt updating base URL only if no errorLibravatar Jonathan Tan1-0/+9
http.c supports HTTP redirects of the form http://foo/info/refs?service=git-upload-pack -> http://anything -> http://bar/info/refs?service=git-upload-pack (that is to say, as long as the Git part of the path and the query string is preserved in the final redirect destination, the intermediate steps can have any URL). However, if one of the intermediate steps results in an HTTP exception, a confusing "unable to update url base from redirection" message is printed instead of a Curl error message with the HTTP exception code. This was introduced by 2 commits. Commit c93c92f ("http: update base URLs when we see redirects", 2013-09-28) introduced a best-effort optimization that required checking if only the "base" part of the URL differed between the initial request and the final redirect destination, but it performed the check before any HTTP status checking was done. If something went wrong, the normal code path was still followed, so this did not cause any confusing error messages until commit 6628eb4 ("http: always update the base URL for redirects", 2016-12-06), which taught http to die if the non-"base" part of the URL differed. Therefore, teach http to check the HTTP status before attempting to check if only the "base" part of the URL differed. This commit teaches http_request_reauth to return early without updating options->base_url upon an error; the only invoker of this function that passes a non-NULL "options" is remote-curl.c (through "http_get_strbuf"), which only uses options->base_url for an informational message in the situations that this commit cares about (that is, when the return value is not HTTP_OK). The included test checks that the redirect scheme at the beginning of this commit message works, and that returning a 502 in the middle of the redirect scheme produces the correct result. Note that this is different from the test in commit 6628eb4 ("http: always update the base URL for redirects", 2016-12-06) in that this commit tests that a Git-shaped URL (http://.../info/refs?service=git-upload-pack) works, whereas commit 6628eb4 tests that a non-Git-shaped URL (http://.../info/refs/foo?service=git-upload-pack) does not work (even though Git is processing that URL) and is an error that is fatal, not silently swallowed. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Acked-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-19Merge branch 'jk/http-walker-limit-redirect-2.9'Libravatar Junio C Hamano1-0/+14
Transport with dumb http can be fooled into following foreign URLs that the end user does not intend to, especially with the server side redirects and http-alternates mechanism, which can lead to security issues. Tighten the redirection and make it more obvious to the end user when it happens. * jk/http-walker-limit-redirect-2.9: http: treat http-alternates like redirects http: make redirects more obvious remote-curl: rename shadowed options variable http: always update the base URL for redirects http: simplify update_url_from_redirect
2016-12-06http: make redirects more obviousLibravatar Jeff King1-0/+6
We instruct curl to always follow HTTP redirects. This is convenient, but it creates opportunities for malicious servers to create confusing situations. For instance, imagine Alice is a git user with access to a private repository on Bob's server. Mallory runs her own server and wants to access objects from Bob's repository. Mallory may try a few tricks that involve asking Alice to clone from her, build on top, and then push the result: 1. Mallory may simply redirect all fetch requests to Bob's server. Git will transparently follow those redirects and fetch Bob's history, which Alice may believe she got from Mallory. The subsequent push seems like it is just feeding Mallory back her own objects, but is actually leaking Bob's objects. There is nothing in git's output to indicate that Bob's repository was involved at all. The downside (for Mallory) of this attack is that Alice will have received Bob's entire repository, and is likely to notice that when building on top of it. 2. If Mallory happens to know the sha1 of some object X in Bob's repository, she can instead build her own history that references that object. She then runs a dumb http server, and Alice's client will fetch each object individually. When it asks for X, Mallory redirects her to Bob's server. The end result is that Alice obtains objects from Bob, but they may be buried deep in history. Alice is less likely to notice. Both of these attacks are fairly hard to pull off. There's a social component in getting Mallory to convince Alice to work with her. Alice may be prompted for credentials in accessing Bob's repository (but not always, if she is using a credential helper that caches). Attack (1) requires a certain amount of obliviousness on Alice's part while making a new commit. Attack (2) requires that Mallory knows a sha1 in Bob's repository, that Bob's server supports dumb http, and that the object in question is loose on Bob's server. But we can probably make things a bit more obvious without any loss of functionality. This patch does two things to that end. First, when we encounter a whole-repo redirect during the initial ref discovery, we now inform the user on stderr, making attack (1) much more obvious. Second, the decision to follow redirects is now configurable. The truly paranoid can set the new http.followRedirects to false to avoid any redirection entirely. But for a more practical default, we will disallow redirects only after the initial ref discovery. This is enough to thwart attacks similar to (2), while still allowing the common use of redirects at the repository level. Since c93c92f30 (http: update base URLs when we see redirects, 2013-09-28) we re-root all further requests from the redirect destination, which should generally mean that no further redirection is necessary. As an escape hatch, in case there really is a server that needs to redirect individual requests, the user can set http.followRedirects to "true" (and this can be done on a per-server basis via http.*.followRedirects config). Reported-by: Jann Horn <jannh@google.com> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-06http: always update the base URL for redirectsLibravatar Jeff King1-0/+8
If a malicious server redirects the initial ref advertisement, it may be able to leak sha1s from other, unrelated servers that the client has access to. For example, imagine that Alice is a git user, she has access to a private repository on a server hosted by Bob, and Mallory runs a malicious server and wants to find out about Bob's private repository. Mallory asks Alice to clone an unrelated repository from her over HTTP. When Alice's client contacts Mallory's server for the initial ref advertisement, the server issues an HTTP redirect for Bob's server. Alice contacts Bob's server and gets the ref advertisement for the private repository. If there is anything to fetch, she then follows up by asking the server for one or more sha1 objects. But who is the server? If it is still Mallory's server, then Alice will leak the existence of those sha1s to her. Since commit c93c92f30 (http: update base URLs when we see redirects, 2013-09-28), the client usually rewrites the base URL such that all further requests will go to Bob's server. But this is done by textually matching the URL. If we were originally looking for "http://mallory/repo.git/info/refs", and we got pointed at "http://bob/other.git/info/refs", then we know that the right root is "http://bob/other.git". If the redirect appears to change more than just the root, we punt and continue to use the original server. E.g., imagine the redirect adds a URL component that Bob's server will ignore, like "http://bob/other.git/info/refs?dummy=1". We can solve this by aborting in this case rather than silently continuing to use Mallory's server. In addition to protecting from sha1 leakage, it's arguably safer and more sane to refuse a confusing redirect like that in general. For example, part of the motivation in c93c92f30 is avoiding accidentally sending credentials over clear http, just to get a response that says "try again over https". So even in a non-malicious case, we'd prefer to err on the side of caution. The downside is that it's possible this will break a legitimate but complicated server-side redirection scheme. The setup given in the newly added test does work, but it's convoluted enough that we don't need to care about it. A more plausible case would be a server which redirects a request for "info/refs?service=git-upload-pack" to just "info/refs" (because it does not do smart HTTP, and for some reason really dislikes query parameters). Right now we would transparently downgrade to dumb-http, but with this patch, we'd complain (and the user would have to set GIT_SMART_HTTP=0 to fetch). Reported-by: Jann Horn <jannh@google.com> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-08Merge branch 'ew/git-svn-http-tests'Libravatar Junio C Hamano1-2/+2
Tests for "git svn" have been taught to reuse the lib-httpd test infrastructure when testing the subversion integration that interacts with subversion repositories served over the http:// protocol. * ew/git-svn-http-tests: git svn: migrate tests to use lib-httpd t/t91*: do not say how to avoid the tests
2016-07-25git svn: migrate tests to use lib-httpdLibravatar Eric Wong1-2/+2
This allows us to use common test infrastructure and parallelize the tests. For now, GIT_SVN_TEST_HTTPD=true needs to be set to enable the SVN HTTP tests because we reuse the same test cases for both file:// and http:// SVN repositories. SVN_HTTPD_PORT is no longer honored. Tested under Apache 2.2 and 2.4 on Debian 7.x (wheezy) and 8.x (jessie), respectively. Cc: Clemens Buchacher <drizzd@aon.at> Cc: Michael J Gruber <git@drmicha.warpmail.net> Signed-off-by: Eric Wong <e@80x24.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-17Merge branch 'js/http-custom-headers'Libravatar Junio C Hamano1-4/+12
Update tests for "http.extraHeaders=<header>" to be portable back to Apache 2.2 (the original depended on <RequireAll/> which is a more recent feature). * js/http-custom-headers: submodule: ensure that -c http.extraheader is heeded t5551: make the test for extra HTTP headers more robust tests: adjust the configuration for Apache 2.2
2016-05-10tests: adjust the configuration for Apache 2.2Libravatar Johannes Schindelin1-4/+12
Lars Schneider noticed that the configuration introduced to test the extra HTTP headers cannot be used with Apache 2.2 (which is still actively maintained, as pointed out by Junio Hamano). To let the tests pass with Apache 2.2 again, let's substitute the offending <RequireAll> and `expr` by using old school RewriteCond statements. As RewriteCond does not allow testing for *non*-matches, we simply match the desired case first and let it pass by marking the RewriteRule as '[L]' ("last rule, do not process any other matching RewriteRules after this"), and then have another RewriteRule that matches all other cases and lets them fail via '[F]' ("fail"). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Tested-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-06Merge branch 'js/http-custom-headers'Libravatar Junio C Hamano1-0/+8
HTTP transport clients learned to throw extra HTTP headers at the server, specified via http.extraHeader configuration variable. * js/http-custom-headers: http: support sending custom HTTP headers
2016-04-27http: support sending custom HTTP headersLibravatar Johannes Schindelin1-0/+8
We introduce a way to send custom HTTP headers with all requests. This allows us, for example, to send an extra token from build agents for temporary access to private repositories. (This is the use case that triggered this patch.) This feature can be used like this: git -c http.extraheader='Secret: sssh!' fetch $URL $REF Note that `curl_easy_setopt(..., CURLOPT_HTTPHEADER, ...)` takes only a single list, overriding any previous call. This means we have to collect _all_ of the headers we want to use into a single list, and feed it to cURL in one shot. Since we already unconditionally set a "pragma" header when initializing the curl handles, we can add our new headers to that list. For callers which override the default header list (like probe_rpc), we provide `http_copy_default_headers()` so they can do the same trick. Big thanks to Jeff King and Junio Hamano for their outstanding help and patient reviews. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Reviewed-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-04-14Merge branch 'jk/test-httpd-config-nosystem' into maintLibravatar Junio C Hamano1-0/+1
The tests that involve running httpd leaked the system-wide configuration in /etc/gitconfig to the tested environment. * jk/test-httpd-config-nosystem: t/lib-httpd: pass through GIT_CONFIG_NOSYSTEM env
2016-04-06Merge branch 'jk/test-httpd-config-nosystem'Libravatar Junio C Hamano1-0/+1
The tests that involve running httpd leaked the system-wide configuration in /etc/gitconfig to the tested environment. * jk/test-httpd-config-nosystem: t/lib-httpd: pass through GIT_CONFIG_NOSYSTEM env
2016-03-18t/lib-httpd: pass through GIT_CONFIG_NOSYSTEM envLibravatar Jeff King1-0/+1
We set GIT_CONFIG_NOSYSTEM in our test scripts so that we do not accidentally read /etc/gitconfig and have it influence the outcome of the tests. But when running smart-http tests, Apache will clean the environment, including this variable, and the "server" side of our http operations will read it. You can see this breakage by doing something like: make ./git config --system http.getanyfile false make test which will cause t5561 to fail when it tests the fallback-to-dumb operation. We can fix this by instructing Apache to pass through the variable. Unlike with other variables (e.g., 89c57ab3's GIT_TRACE), we don't need to set a dummy value to prevent warnings from Apache. test-lib.sh already makes sure that GIT_CONFIG_NOSYSTEM is set and exported. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-02-25t/lib-httpd: load mod_unixdLibravatar Michael J Gruber1-0/+3
In contrast to apache 2.2, apache 2.4 does not load mod_unixd in its default configuration (because there are choices). Thus, with the current config, apache 2.4.10 will not be started and the httpd tests will not run on distros with default apache config (RedHat type). Enable mod_unixd to make the httpd tests run. This does not affect distros negatively which have that config already in their default (Debian type). httpd tests will run on these before and after this patch. Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-09-25http: limit redirection depthLibravatar Blake Burkhart1-0/+3
By default, libcurl will follow circular http redirects forever. Let's put a cap on this so that somebody who can trigger an automated fetch of an arbitrary repository (e.g., for CI) cannot convince git to loop infinitely. The value chosen is 20, which is the same default that Firefox uses. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-09-25http: limit redirection to protocol-whitelistLibravatar Blake Burkhart1-0/+1
Previously, libcurl would follow redirection to any protocol it was compiled for support with. This is desirable to allow redirection from HTTP to HTTPS. However, it would even successfully allow redirection from HTTP to SFTP, a protocol that git does not otherwise support at all. Furthermore git's new protocol-whitelisting could be bypassed by following a redirect within the remote helper, as it was only enforced at transport selection time. This patch limits redirects within libcurl to HTTP, HTTPS, FTP and FTPS. If there is a protocol-whitelist present, this list is limited to those also allowed by the whitelist. As redirection happens from within libcurl, it is impossible for an HTTP redirect to a protocol implemented within another remote helper. When the curl version git was compiled with is too old to support restrictions on protocol redirection, we warn the user if GIT_ALLOW_PROTOCOL restrictions were requested. This is a little inaccurate, as even without that variable in the environment, we would still restrict SFTP, etc, and we do not warn in that case. But anything else means we would literally warn every time git accesses an http remote. This commit includes a test, but it is not as robust as we would hope. It redirects an http request to ftp, and checks that curl complained about the protocol, which means that we are relying on curl's specific error message to know what happened. Ideally we would redirect to a working ftp server and confirm that we can clone without protocol restrictions, and not with them. But we do not have a portable way of providing an ftp server, nor any other protocol that curl supports (https is the closest, but we would have to deal with certificates). [jk: added test and version warning] Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-03-12t: pass GIT_TRACE through ApacheLibravatar Jeff King1-0/+1
Apache removes GIT_TRACE from the environment before running git-http-backend. This can make it hard to debug the server side of an http session. Let's let it through. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-12-11t: support clang/gcc AddressSanitizerLibravatar Jeff King1-0/+1
When git is compiled with "-fsanitize=address" (using clang or gcc >= 4.8), all invocations of git will check for buffer overflows. This is similar to running with valgrind, except that it is more thorough (because of the compiler support, function-local buffers can be checked, too) and runs much faster (making it much less painful to run the whole test suite with the checks turned on). Unlike valgrind, the magic happens at compile-time, so we don't need the same infrastructure in the test suite that we did to support --valgrind. But there are two things we can help with: 1. On some platforms, the leak-detector is on by default, and causes every invocation of "git init" (and thus every test script) to fail. Since running git with the leak detector is pointless, let's shut it off automatically in the tests, unless the user has already configured it. 2. When apache runs a CGI, it clears the environment of unknown variables. This means that the $ASAN_OPTIONS config doesn't make it to git-http-backend, and it dies due to the leak detector. Let's mark the variable as OK for apache to pass. With these two changes, running make CC=clang CFLAGS=-fsanitize=address test works out of the box. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-09-17signed push: teach smart-HTTP to pass "git push --signed" aroundLibravatar Junio C Hamano1-0/+1
The "--signed" option received by "git push" is first passed to the transport layer, which the native transport directly uses to notice that a push certificate needs to be sent. When the transport-helper is involved, however, the option needs to be told to the helper with set_helper_option(), and the helper needs to take necessary action. For the smart-HTTP helper, the "necessary action" involves spawning the "git send-pack" subprocess with the "--signed" option. Once the above all gets wired in, the smart-HTTP transport now can use the push certificate mechanism to authenticate its pushes. Add a test that is modeled after tests for the native transport in t5534-push-signed.sh to t5541-http-push-smart.sh. Update the test Apache configuration to pass GNUPGHOME environment variable through. As PassEnv would trigger warnings for an environment variable that is not set, export it from test-lib.sh set to a harmless value when GnuPG is not being used in the tests. Note that the added test is deliberately loose and does not check the nonce in this step. This is because the stateless RPC mode is inevitably flaky and a nonce that comes back in the actual push processing is one issued by a different process; if the two interactions with the server crossed a second boundary, the nonces will not match and such a check will fail. A later patch in the series will work around this shortcoming. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-17http: fix charset detection of extract_content_type()Libravatar Yi EungJun1-0/+4
extract_content_type() could not extract a charset parameter if the parameter is not the first one and there is a whitespace and a following semicolon just before the parameter. For example: text/plain; format=fixed ;charset=utf-8 And it also could not handle correctly some other cases, such as: text/plain; charset=utf-8; format=fixed text/plain; some-param="a long value with ;semicolons;"; charset=utf-8 Thanks-to: Jeff King <peff@peff.net> Signed-off-by: Yi EungJun <eungjun.yi@navercorp.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-05-27remote-curl: reencode http error messagesLibravatar Jeff King1-0/+4
We currently recognize an error message with a content-type "text/plain; charset=utf-16" as text, but we ignore the charset parameter entirely. Let's encode it to log_output_encoding, which is presumably something the user's terminal can handle. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-05-27http: extract type/subtype portion of content-typeLibravatar Jeff King1-1/+7
When we get a content-type from curl, we get the whole header line, including any parameters, and without any normalization (like downcasing or whitespace) applied. If we later try to match it with strcmp() or even strcasecmp(), we may get false negatives. This could cause two visible behaviors: 1. We might fail to recognize a smart-http server by its content-type. 2. We might fail to relay text/plain error messages to users (especially if they contain a charset parameter). This patch teaches the http code to extract and normalize just the type/subtype portion of the string. This is technically passing out less information to the callers, who can no longer see the parameters. But none of the current callers cares, and a future patch will add back an easier-to-use method for accessing those parameters. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-05-23t5550: test display of remote http error messagesLibravatar Jeff King2-0/+21
Since commit 426e70d (remote-curl: show server content on http errors, 2013-04-05), we relay any text/plain error messages from the remote server to the user. However, we never tested it. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-05-23t/lib-httpd: use write_script to copy CGI scriptsLibravatar Jeff King1-1/+0
Using write_script will set our shebang line appropriately with $SHELL_PATH. The script that is there now is quite simple and likely to succeed even with a non-POSIX /bin/sh, but it does not hurt to be defensive. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-01-02use distinct username/password for http auth testsLibravatar Jeff King1-1/+1
The httpd server we set up to test git's http client code knows about a single account, in which both the username and password are "user@host" (the unusual use of the "@" here is to verify that we handle the character correctly when URL escaped). This means that we may miss a certain class of errors in which the username and password are mixed up internally by git. We can make our tests more robust by having distinct values for the username and password. In addition to tweaking the server passwd file and the client URL, we must teach the "askpass" harness to accept multiple values. As a bonus, this makes the setup of some tests more obvious; when we are expecting git to ask only about the password, we can seed the username askpass response with a bogus value. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-10-30Merge branch 'jk/http-auth-redirects'Libravatar Junio C Hamano1-0/+2
Handle the case where http transport gets redirected during the authorization request better. * jk/http-auth-redirects: http.c: Spell the null pointer as NULL remote-curl: rewrite base url from info/refs redirects remote-curl: store url as a strbuf remote-curl: make refs_url a strbuf http: update base URLs when we see redirects http: provide effective url to callers http: hoist credential request out of handle_curl_result http: refactor options to http_get_* http_request: factor out curlinfo_strbuf http_get_file: style fixes
2013-10-14remote-curl: rewrite base url from info/refs redirectsLibravatar Jeff King1-0/+2
For efficiency and security reasons, an earlier commit in this series taught http_get_* to re-write the base url based on redirections we saw while making a specific request. This commit wires that option into the info/refs request, meaning that a redirect from http://example.com/foo.git/info/refs to https://example.com/bar.git/info/refs will behave as if "https://example.com/bar.git" had been provided to git in the first place. The tests bear some explanation. We introduce two new hierearchies into the httpd test config: 1. Requests to /smart-redir-limited will work only for the initial info/refs request, but not any subsequent requests. As a result, we can confirm whether the client is re-rooting its requests after the initial contact, since otherwise it will fail (it will ask for "repo.git/git-upload-pack", which is not redirected). 2. Requests to smart-redir-auth will redirect, and require auth after the redirection. Since we are using the redirected base for further requests, we also update the credential struct, in order not to mislead the user (or credential helpers) about which credential is needed. We can therefore check the GIT_ASKPASS prompts to make sure we are prompting for the new location. Because we have neither multiple servers nor https support in our test setup, we can only redirect between paths, meaning we need to turn on credential.useHttpPath to see the difference. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
2013-07-30http: add http.savecookies option to write out HTTP cookiesLibravatar Dave Borowitz1-0/+8
HTTP servers may send Set-Cookie headers in a response and expect them to be set on subsequent requests. By default, libcurl behavior is to store such cookies in memory and reuse them across requests within a single session. However, it may also make sense, depending on the server and the cookies, to store them across sessions. Provide users an option to enable this behavior, writing cookies out to the same file specified in http.cookiefile. Signed-off-by: Dave Borowitz <dborowitz@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-06-23Merge branch 'jk/apache-test-for-2.4'Libravatar Junio C Hamano1-1/+19
* jk/apache-test-for-2.4: lib-httpd/apache.conf: check version only after mod_version loads t/lib-httpd/apache.conf: configure an MPM module for apache 2.4 t/lib-httpd/apache.conf: load compat access module in apache 2.4 t/lib-httpd/apache.conf: load extra auth modules in apache 2.4 t/lib-httpd/apache.conf: do not use LockFile in apache >= 2.4
2013-06-21lib-httpd/apache.conf: check version only after mod_version loadsLibravatar Jeff King1-3/+4
Commit 0442743 introduced an <IfVersion> directive near the top of the apache config file. However, at that point we have not yet checked for and loaded the mod_version module. This means that the directive will behave oddly if mod_version is dynamically loaded, failing to match when it should. We can fix this by moving the whole block below the LoadModule directive for mod_version. Reported-by: Brian Gernhardt <brian@gernhardtsoftware.com> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-06-14t/lib-httpd/apache.conf: configure an MPM module for apache 2.4Libravatar Jeff King1-0/+3
Versions of Apache before 2.4 always had a "MultiProcessing Module" (MPM) statically built in, which manages the worker threads/processes. We do not care which one, as it is largely a performance issue, and we put only a light load on the server during our testing. As of Apache 2.4, the MPM module is loadable just like any other module, but exactly one such module must be loaded. On a system where the MPMs are compiled dynamically (e.g., Debian unstable), this means that our test Apache server will not start unless we provide the appropriate configuration. Unfortunately, we do not actually know which MPM modules are available or appropriate for the system on which the tests are running. This patch picks the "prefork" module, as it is likely to be available on all Unix-like systems. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-06-14t/lib-httpd/apache.conf: load compat access module in apache 2.4Libravatar Jeff King1-0/+3
In apache 2.4, the "Order" directive has gone away in favor of a new system in mod_authz_host. However, since we want our config file to remain compatible across multiple Apache versions, we can use mod_access_compat to keep using the older style. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-06-14t/lib-httpd/apache.conf: load extra auth modules in apache 2.4Libravatar Jeff King1-0/+9
In apache 2.4, the "Auth*" and "Require" directives have moved into the authn_core and authz_core modules, respectively. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-06-14t/lib-httpd/apache.conf: do not use LockFile in apache >= 2.4Libravatar Jeff King1-0/+2
The LockFile directive from earlier versions of apache has been replaced by the Mutex directive. The latter seems to give sane defaults and does not need any specific customization, so we can get away with just adding a version check to the use of LockFile. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-04-21Merge branch 'jk/doc-http-backend'Libravatar Junio C Hamano1-0/+18
Improve documentation to illustrate "push authenticated, fetch anonymous" configuration for smart HTTP servers. * jk/doc-http-backend: doc/http-backend: match query-string in apache half-auth example doc/http-backend: give some lighttpd config examples doc/http-backend: clarify "half-auth" repo configuration
2013-04-13doc/http-backend: match query-string in apache half-auth exampleLibravatar Jeff King1-0/+18
When setting up a "half-auth" repository in which reads can be done anonymously but writes require authentication, it is best if the server can require authentication for both the ref advertisement and the actual receive-pack POSTs. This alleviates the need for the admin to set http.receivepack in the repositories, and means that the client is challenged for credentials immediately, instead of partway through the push process (and git clients older than v1.7.11.7 had trouble handling these challenges). Since detecting a push during the ref advertisement requires matching the query string, and this is non-trivial to do in Apache, we have traditionally punted and instructed users to just protect "/git-receive-pack$". This patch provides the mod_rewrite recipe to actually match the ref advertisement, which is preferred. While we're at it, let's add the recipe to our test scripts so that we can be sure that it works, and doesn't get broken (either by our changes or by changes in Apache). Signed-off-by: Jeff King <peff@peff.net> Acked-by: Jakub Narębski <jnareb@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-04-09http-backend: respect GIT_NAMESPACE with dumb clientsLibravatar John Koleszar1-0/+5
Filter the list of refs returned via the dumb HTTP protocol according to the active namespace, consistent with other clients of the upload-pack service. Signed-off-by: John Koleszar <jkoleszar@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-02-04Verify Content-Type from smart HTTP serversLibravatar Shawn Pearce2-0/+15
Before parsing a suspected smart-HTTP response verify the returned Content-Type matches the standard. This protects a client from attempting to process a payload that smells like a smart-HTTP server response. JGit has been doing this check on all responses since the dawn of time. I mistakenly failed to include it in git-core when smart HTTP was introduced. At the time I didn't know how to get the Content-Type from libcurl. I punted, meant to circle back and fix this, and just plain forgot about it. Signed-off-by: Shawn Pearce <spearce@spearce.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-11-20Merge branch 'jk/maint-http-half-auth-fetch'Libravatar Junio C Hamano1-0/+7
Fixes fetch from servers that ask for auth only during the actual packing phase. This is not really a recommended configuration, but it cleans up the code at the same time. * jk/maint-http-half-auth-fetch: remote-curl: retry failed requests for auth even with gzip remote-curl: hoist gzip buffer size to top of post_rpc