summaryrefslogtreecommitdiff
path: root/t/lib-httpd/apache.conf
AgeCommit message (Collapse)AuthorFilesLines
2020-02-27t/lib-httpd: avoid using macOS' sedLibravatar Johannes Schindelin1-3/+3
Among other differences relative to GNU sed, macOS' sed always ends its output with a trailing newline, even if the input did not have such a trailing newline. Surprisingly, this makes three httpd-based tests fail on macOS: t5616, t5702 and t5703. ("Surprisingly" because those tests have been around for some time, but apparently nobody runs them on macOS with a working Apache2 setup.) The reason is that we use `sed` in those tests to filter the response of the web server. Apart from the fact that we use GNU constructs (such as using a space after the `c` command instead of a backslash and a newline), we have another problem: macOS' sed LF-only newlines while webservers are supposed to use CR/LF ones. Even worse, t5616 uses `sed` to replace a binary part of the response with a new binary part (kind of hoping that the replaced binary part does not contain a 0x0a byte which would be interpreted as a newline). To that end, it calls on Perl to read the binary pack file and hex-encode it, then calls on `sed` to prefix every hex digit pair with a `\x` in order to construct the text that the `c` statement of the `sed` invocation is supposed to insert. So we call Perl and sed to construct a sed statement. The final nail in the coffin is that macOS' sed does not even interpret those `\x<hex>` constructs. Let's just replace all of that by Perl snippets. With Perl, at least, we do not have to deal with GNU vs macOS semantics, we do not have to worry about unwanted trailing newlines, and we do not have to spawn commands to construct arguments for other commands to be spawned (i.e. we can avoid a whole lot of shell scripting complexity). The upshot is that this fixes t5616, t5702 and t5703 on macOS with Apache2. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-05-08t/lib-httpd: pass LSAN_OPTIONS through apacheLibravatar Jeff King1-0/+1
Just as we instruct Apache to pass through ASAN_OPTIONS (so that server-side Git programs it spawns will respect our options while running the tests), we should do the same with LSAN_OPTIONS. Otherwise trying to collect a list of leaks like: export LSAN_OPTIONS=exitcode=0:log_path=/tmp/lsan make SANITIZE=leak test won't work for http tests (the server-side programs won't log their leaks to the right place, and they'll prematurely die, producing a spurious test failure). Signed-off-by: Jeff King <peff@peff.net> Acked-by: Josh Steadmon <steadmon@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-06t5551: test server-side ERR packetLibravatar Josh Steadmon1-0/+4
When a smart HTTP server sends an error message via pkt-line, we detect the error due to using PACKET_READ_DIE_ON_ERR_PACKET. This case was added by 2d103c31c2 (pack-protocol.txt: accept error packets in any context, 2018-12-29), but not covered by tests. Signed-off-by: Josh Steadmon <steadmon@google.com> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-05Merge branch 'jt/fetch-v2-sideband'Libravatar Junio C Hamano1-0/+1
"git fetch" and "git upload-pack" learned to send all exchange over the sideband channel while talking the v2 protocol. * jt/fetch-v2-sideband: tests: define GIT_TEST_SIDEBAND_ALL {fetch,upload}-pack: sideband v2 fetch response sideband: reverse its dependency on pkt-line pkt-line: introduce struct packet_writer pack-protocol.txt: accept error packets in any context Use packet_reader instead of packet_read_line
2019-01-17tests: define GIT_TEST_SIDEBAND_ALLLibravatar Jonathan Tan1-0/+1
Define a GIT_TEST_SIDEBAND_ALL environment variable meant to be used from tests. When set to true, this overrides uploadpack.allowsidebandall to true, allowing the entire test suite to be run as if this configuration is in place for all repositories. As of this patch, all tests pass whether GIT_TEST_SIDEBAND_ALL is unset or set to 1. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-10test: test GIT_CURL_VERBOSE=1 shows an errorLibravatar Masaya Suzuki1-0/+1
This tests GIT_CURL_VERBOSE shows an error when an URL returns 500. This exercises the code in remote_curl. Signed-off-by: Masaya Suzuki <masayasuzuki@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-06-28upload-pack: test negotiation with changing repositoryLibravatar Brandon Williams1-0/+8
Add tests to check the behavior of fetching from a repository which changes between rounds of negotiation (for example, when different servers in a load-balancing agreement participate in the same stateless RPC negotiation). This forms a baseline of comparison to the ref-in-want functionality (which will be introduced to the client in subsequent commits), and ensures that subsequent commits do not change existing behavior. As part of this effort, a mechanism to substitute strings in a single HTTP response is added. Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-01-04http: fix v1 protocol tests with apache httpd < 2.4Libravatar Todd Zullinger1-6/+4
The apache config used by tests was updated to use the SetEnvIf directive to set the Git-Protocol header in 19113a26b6 ("http: tell server that the client understands v1", 2017-10-16). Setting the Git-Protocol header is restricted to httpd >= 2.4, but mod_setenvif and the SetEnvIf directive work with lower versions, at least as far back as 2.0, according to the httpd documentation: https://httpd.apache.org/docs/2.0/mod/mod_setenvif.html Drop the restriction. Tested with httpd 2.2 and 2.4. Signed-off-by: Todd Zullinger <tmz@pobox.com> Acked-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-10-17http: tell server that the client understands v1Libravatar Brandon Williams1-0/+7
Tell a server that protocol v1 can be used by sending the http header 'Git-Protocol' with 'version=1' indicating this. Also teach the apache http server to pass through the 'Git-Protocol' header as an environment variable 'GIT_PROTOCOL'. Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-02-28http: attempt updating base URL only if no errorLibravatar Jonathan Tan1-0/+9
http.c supports HTTP redirects of the form http://foo/info/refs?service=git-upload-pack -> http://anything -> http://bar/info/refs?service=git-upload-pack (that is to say, as long as the Git part of the path and the query string is preserved in the final redirect destination, the intermediate steps can have any URL). However, if one of the intermediate steps results in an HTTP exception, a confusing "unable to update url base from redirection" message is printed instead of a Curl error message with the HTTP exception code. This was introduced by 2 commits. Commit c93c92f ("http: update base URLs when we see redirects", 2013-09-28) introduced a best-effort optimization that required checking if only the "base" part of the URL differed between the initial request and the final redirect destination, but it performed the check before any HTTP status checking was done. If something went wrong, the normal code path was still followed, so this did not cause any confusing error messages until commit 6628eb4 ("http: always update the base URL for redirects", 2016-12-06), which taught http to die if the non-"base" part of the URL differed. Therefore, teach http to check the HTTP status before attempting to check if only the "base" part of the URL differed. This commit teaches http_request_reauth to return early without updating options->base_url upon an error; the only invoker of this function that passes a non-NULL "options" is remote-curl.c (through "http_get_strbuf"), which only uses options->base_url for an informational message in the situations that this commit cares about (that is, when the return value is not HTTP_OK). The included test checks that the redirect scheme at the beginning of this commit message works, and that returning a 502 in the middle of the redirect scheme produces the correct result. Note that this is different from the test in commit 6628eb4 ("http: always update the base URL for redirects", 2016-12-06) in that this commit tests that a Git-shaped URL (http://.../info/refs?service=git-upload-pack) works, whereas commit 6628eb4 tests that a non-Git-shaped URL (http://.../info/refs/foo?service=git-upload-pack) does not work (even though Git is processing that URL) and is an error that is fatal, not silently swallowed. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Acked-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-19Merge branch 'jk/http-walker-limit-redirect-2.9'Libravatar Junio C Hamano1-0/+14
Transport with dumb http can be fooled into following foreign URLs that the end user does not intend to, especially with the server side redirects and http-alternates mechanism, which can lead to security issues. Tighten the redirection and make it more obvious to the end user when it happens. * jk/http-walker-limit-redirect-2.9: http: treat http-alternates like redirects http: make redirects more obvious remote-curl: rename shadowed options variable http: always update the base URL for redirects http: simplify update_url_from_redirect
2016-12-06http: make redirects more obviousLibravatar Jeff King1-0/+6
We instruct curl to always follow HTTP redirects. This is convenient, but it creates opportunities for malicious servers to create confusing situations. For instance, imagine Alice is a git user with access to a private repository on Bob's server. Mallory runs her own server and wants to access objects from Bob's repository. Mallory may try a few tricks that involve asking Alice to clone from her, build on top, and then push the result: 1. Mallory may simply redirect all fetch requests to Bob's server. Git will transparently follow those redirects and fetch Bob's history, which Alice may believe she got from Mallory. The subsequent push seems like it is just feeding Mallory back her own objects, but is actually leaking Bob's objects. There is nothing in git's output to indicate that Bob's repository was involved at all. The downside (for Mallory) of this attack is that Alice will have received Bob's entire repository, and is likely to notice that when building on top of it. 2. If Mallory happens to know the sha1 of some object X in Bob's repository, she can instead build her own history that references that object. She then runs a dumb http server, and Alice's client will fetch each object individually. When it asks for X, Mallory redirects her to Bob's server. The end result is that Alice obtains objects from Bob, but they may be buried deep in history. Alice is less likely to notice. Both of these attacks are fairly hard to pull off. There's a social component in getting Mallory to convince Alice to work with her. Alice may be prompted for credentials in accessing Bob's repository (but not always, if she is using a credential helper that caches). Attack (1) requires a certain amount of obliviousness on Alice's part while making a new commit. Attack (2) requires that Mallory knows a sha1 in Bob's repository, that Bob's server supports dumb http, and that the object in question is loose on Bob's server. But we can probably make things a bit more obvious without any loss of functionality. This patch does two things to that end. First, when we encounter a whole-repo redirect during the initial ref discovery, we now inform the user on stderr, making attack (1) much more obvious. Second, the decision to follow redirects is now configurable. The truly paranoid can set the new http.followRedirects to false to avoid any redirection entirely. But for a more practical default, we will disallow redirects only after the initial ref discovery. This is enough to thwart attacks similar to (2), while still allowing the common use of redirects at the repository level. Since c93c92f30 (http: update base URLs when we see redirects, 2013-09-28) we re-root all further requests from the redirect destination, which should generally mean that no further redirection is necessary. As an escape hatch, in case there really is a server that needs to redirect individual requests, the user can set http.followRedirects to "true" (and this can be done on a per-server basis via http.*.followRedirects config). Reported-by: Jann Horn <jannh@google.com> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-06http: always update the base URL for redirectsLibravatar Jeff King1-0/+8
If a malicious server redirects the initial ref advertisement, it may be able to leak sha1s from other, unrelated servers that the client has access to. For example, imagine that Alice is a git user, she has access to a private repository on a server hosted by Bob, and Mallory runs a malicious server and wants to find out about Bob's private repository. Mallory asks Alice to clone an unrelated repository from her over HTTP. When Alice's client contacts Mallory's server for the initial ref advertisement, the server issues an HTTP redirect for Bob's server. Alice contacts Bob's server and gets the ref advertisement for the private repository. If there is anything to fetch, she then follows up by asking the server for one or more sha1 objects. But who is the server? If it is still Mallory's server, then Alice will leak the existence of those sha1s to her. Since commit c93c92f30 (http: update base URLs when we see redirects, 2013-09-28), the client usually rewrites the base URL such that all further requests will go to Bob's server. But this is done by textually matching the URL. If we were originally looking for "http://mallory/repo.git/info/refs", and we got pointed at "http://bob/other.git/info/refs", then we know that the right root is "http://bob/other.git". If the redirect appears to change more than just the root, we punt and continue to use the original server. E.g., imagine the redirect adds a URL component that Bob's server will ignore, like "http://bob/other.git/info/refs?dummy=1". We can solve this by aborting in this case rather than silently continuing to use Mallory's server. In addition to protecting from sha1 leakage, it's arguably safer and more sane to refuse a confusing redirect like that in general. For example, part of the motivation in c93c92f30 is avoiding accidentally sending credentials over clear http, just to get a response that says "try again over https". So even in a non-malicious case, we'd prefer to err on the side of caution. The downside is that it's possible this will break a legitimate but complicated server-side redirection scheme. The setup given in the newly added test does work, but it's convoluted enough that we don't need to care about it. A more plausible case would be a server which redirects a request for "info/refs?service=git-upload-pack" to just "info/refs" (because it does not do smart HTTP, and for some reason really dislikes query parameters). Right now we would transparently downgrade to dumb-http, but with this patch, we'd complain (and the user would have to set GIT_SMART_HTTP=0 to fetch). Reported-by: Jann Horn <jannh@google.com> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-08Merge branch 'ew/git-svn-http-tests'Libravatar Junio C Hamano1-2/+2
Tests for "git svn" have been taught to reuse the lib-httpd test infrastructure when testing the subversion integration that interacts with subversion repositories served over the http:// protocol. * ew/git-svn-http-tests: git svn: migrate tests to use lib-httpd t/t91*: do not say how to avoid the tests
2016-07-25git svn: migrate tests to use lib-httpdLibravatar Eric Wong1-2/+2
This allows us to use common test infrastructure and parallelize the tests. For now, GIT_SVN_TEST_HTTPD=true needs to be set to enable the SVN HTTP tests because we reuse the same test cases for both file:// and http:// SVN repositories. SVN_HTTPD_PORT is no longer honored. Tested under Apache 2.2 and 2.4 on Debian 7.x (wheezy) and 8.x (jessie), respectively. Cc: Clemens Buchacher <drizzd@aon.at> Cc: Michael J Gruber <git@drmicha.warpmail.net> Signed-off-by: Eric Wong <e@80x24.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-17Merge branch 'js/http-custom-headers'Libravatar Junio C Hamano1-4/+12
Update tests for "http.extraHeaders=<header>" to be portable back to Apache 2.2 (the original depended on <RequireAll/> which is a more recent feature). * js/http-custom-headers: submodule: ensure that -c http.extraheader is heeded t5551: make the test for extra HTTP headers more robust tests: adjust the configuration for Apache 2.2
2016-05-10tests: adjust the configuration for Apache 2.2Libravatar Johannes Schindelin1-4/+12
Lars Schneider noticed that the configuration introduced to test the extra HTTP headers cannot be used with Apache 2.2 (which is still actively maintained, as pointed out by Junio Hamano). To let the tests pass with Apache 2.2 again, let's substitute the offending <RequireAll> and `expr` by using old school RewriteCond statements. As RewriteCond does not allow testing for *non*-matches, we simply match the desired case first and let it pass by marking the RewriteRule as '[L]' ("last rule, do not process any other matching RewriteRules after this"), and then have another RewriteRule that matches all other cases and lets them fail via '[F]' ("fail"). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Tested-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-06Merge branch 'js/http-custom-headers'Libravatar Junio C Hamano1-0/+8
HTTP transport clients learned to throw extra HTTP headers at the server, specified via http.extraHeader configuration variable. * js/http-custom-headers: http: support sending custom HTTP headers
2016-04-27http: support sending custom HTTP headersLibravatar Johannes Schindelin1-0/+8
We introduce a way to send custom HTTP headers with all requests. This allows us, for example, to send an extra token from build agents for temporary access to private repositories. (This is the use case that triggered this patch.) This feature can be used like this: git -c http.extraheader='Secret: sssh!' fetch $URL $REF Note that `curl_easy_setopt(..., CURLOPT_HTTPHEADER, ...)` takes only a single list, overriding any previous call. This means we have to collect _all_ of the headers we want to use into a single list, and feed it to cURL in one shot. Since we already unconditionally set a "pragma" header when initializing the curl handles, we can add our new headers to that list. For callers which override the default header list (like probe_rpc), we provide `http_copy_default_headers()` so they can do the same trick. Big thanks to Jeff King and Junio Hamano for their outstanding help and patient reviews. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Reviewed-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-04-14Merge branch 'jk/test-httpd-config-nosystem' into maintLibravatar Junio C Hamano1-0/+1
The tests that involve running httpd leaked the system-wide configuration in /etc/gitconfig to the tested environment. * jk/test-httpd-config-nosystem: t/lib-httpd: pass through GIT_CONFIG_NOSYSTEM env
2016-04-06Merge branch 'jk/test-httpd-config-nosystem'Libravatar Junio C Hamano1-0/+1
The tests that involve running httpd leaked the system-wide configuration in /etc/gitconfig to the tested environment. * jk/test-httpd-config-nosystem: t/lib-httpd: pass through GIT_CONFIG_NOSYSTEM env
2016-03-18t/lib-httpd: pass through GIT_CONFIG_NOSYSTEM envLibravatar Jeff King1-0/+1
We set GIT_CONFIG_NOSYSTEM in our test scripts so that we do not accidentally read /etc/gitconfig and have it influence the outcome of the tests. But when running smart-http tests, Apache will clean the environment, including this variable, and the "server" side of our http operations will read it. You can see this breakage by doing something like: make ./git config --system http.getanyfile false make test which will cause t5561 to fail when it tests the fallback-to-dumb operation. We can fix this by instructing Apache to pass through the variable. Unlike with other variables (e.g., 89c57ab3's GIT_TRACE), we don't need to set a dummy value to prevent warnings from Apache. test-lib.sh already makes sure that GIT_CONFIG_NOSYSTEM is set and exported. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-02-25t/lib-httpd: load mod_unixdLibravatar Michael J Gruber1-0/+3
In contrast to apache 2.2, apache 2.4 does not load mod_unixd in its default configuration (because there are choices). Thus, with the current config, apache 2.4.10 will not be started and the httpd tests will not run on distros with default apache config (RedHat type). Enable mod_unixd to make the httpd tests run. This does not affect distros negatively which have that config already in their default (Debian type). httpd tests will run on these before and after this patch. Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-09-25http: limit redirection depthLibravatar Blake Burkhart1-0/+3
By default, libcurl will follow circular http redirects forever. Let's put a cap on this so that somebody who can trigger an automated fetch of an arbitrary repository (e.g., for CI) cannot convince git to loop infinitely. The value chosen is 20, which is the same default that Firefox uses. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-09-25http: limit redirection to protocol-whitelistLibravatar Blake Burkhart1-0/+1
Previously, libcurl would follow redirection to any protocol it was compiled for support with. This is desirable to allow redirection from HTTP to HTTPS. However, it would even successfully allow redirection from HTTP to SFTP, a protocol that git does not otherwise support at all. Furthermore git's new protocol-whitelisting could be bypassed by following a redirect within the remote helper, as it was only enforced at transport selection time. This patch limits redirects within libcurl to HTTP, HTTPS, FTP and FTPS. If there is a protocol-whitelist present, this list is limited to those also allowed by the whitelist. As redirection happens from within libcurl, it is impossible for an HTTP redirect to a protocol implemented within another remote helper. When the curl version git was compiled with is too old to support restrictions on protocol redirection, we warn the user if GIT_ALLOW_PROTOCOL restrictions were requested. This is a little inaccurate, as even without that variable in the environment, we would still restrict SFTP, etc, and we do not warn in that case. But anything else means we would literally warn every time git accesses an http remote. This commit includes a test, but it is not as robust as we would hope. It redirects an http request to ftp, and checks that curl complained about the protocol, which means that we are relying on curl's specific error message to know what happened. Ideally we would redirect to a working ftp server and confirm that we can clone without protocol restrictions, and not with them. But we do not have a portable way of providing an ftp server, nor any other protocol that curl supports (https is the closest, but we would have to deal with certificates). [jk: added test and version warning] Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-03-12t: pass GIT_TRACE through ApacheLibravatar Jeff King1-0/+1
Apache removes GIT_TRACE from the environment before running git-http-backend. This can make it hard to debug the server side of an http session. Let's let it through. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-12-11t: support clang/gcc AddressSanitizerLibravatar Jeff King1-0/+1
When git is compiled with "-fsanitize=address" (using clang or gcc >= 4.8), all invocations of git will check for buffer overflows. This is similar to running with valgrind, except that it is more thorough (because of the compiler support, function-local buffers can be checked, too) and runs much faster (making it much less painful to run the whole test suite with the checks turned on). Unlike valgrind, the magic happens at compile-time, so we don't need the same infrastructure in the test suite that we did to support --valgrind. But there are two things we can help with: 1. On some platforms, the leak-detector is on by default, and causes every invocation of "git init" (and thus every test script) to fail. Since running git with the leak detector is pointless, let's shut it off automatically in the tests, unless the user has already configured it. 2. When apache runs a CGI, it clears the environment of unknown variables. This means that the $ASAN_OPTIONS config doesn't make it to git-http-backend, and it dies due to the leak detector. Let's mark the variable as OK for apache to pass. With these two changes, running make CC=clang CFLAGS=-fsanitize=address test works out of the box. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-09-17signed push: teach smart-HTTP to pass "git push --signed" aroundLibravatar Junio C Hamano1-0/+1
The "--signed" option received by "git push" is first passed to the transport layer, which the native transport directly uses to notice that a push certificate needs to be sent. When the transport-helper is involved, however, the option needs to be told to the helper with set_helper_option(), and the helper needs to take necessary action. For the smart-HTTP helper, the "necessary action" involves spawning the "git send-pack" subprocess with the "--signed" option. Once the above all gets wired in, the smart-HTTP transport now can use the push certificate mechanism to authenticate its pushes. Add a test that is modeled after tests for the native transport in t5534-push-signed.sh to t5541-http-push-smart.sh. Update the test Apache configuration to pass GNUPGHOME environment variable through. As PassEnv would trigger warnings for an environment variable that is not set, export it from test-lib.sh set to a harmless value when GnuPG is not being used in the tests. Note that the added test is deliberately loose and does not check the nonce in this step. This is because the stateless RPC mode is inevitably flaky and a nonce that comes back in the actual push processing is one issued by a different process; if the two interactions with the server crossed a second boundary, the nonces will not match and such a check will fail. A later patch in the series will work around this shortcoming. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-05-23t5550: test display of remote http error messagesLibravatar Jeff King1-0/+4
Since commit 426e70d (remote-curl: show server content on http errors, 2013-04-05), we relay any text/plain error messages from the remote server to the user. However, we never tested it. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-10-30Merge branch 'jk/http-auth-redirects'Libravatar Junio C Hamano1-0/+2
Handle the case where http transport gets redirected during the authorization request better. * jk/http-auth-redirects: http.c: Spell the null pointer as NULL remote-curl: rewrite base url from info/refs redirects remote-curl: store url as a strbuf remote-curl: make refs_url a strbuf http: update base URLs when we see redirects http: provide effective url to callers http: hoist credential request out of handle_curl_result http: refactor options to http_get_* http_request: factor out curlinfo_strbuf http_get_file: style fixes
2013-10-14remote-curl: rewrite base url from info/refs redirectsLibravatar Jeff King1-0/+2
For efficiency and security reasons, an earlier commit in this series taught http_get_* to re-write the base url based on redirections we saw while making a specific request. This commit wires that option into the info/refs request, meaning that a redirect from http://example.com/foo.git/info/refs to https://example.com/bar.git/info/refs will behave as if "https://example.com/bar.git" had been provided to git in the first place. The tests bear some explanation. We introduce two new hierearchies into the httpd test config: 1. Requests to /smart-redir-limited will work only for the initial info/refs request, but not any subsequent requests. As a result, we can confirm whether the client is re-rooting its requests after the initial contact, since otherwise it will fail (it will ask for "repo.git/git-upload-pack", which is not redirected). 2. Requests to smart-redir-auth will redirect, and require auth after the redirection. Since we are using the redirected base for further requests, we also update the credential struct, in order not to mislead the user (or credential helpers) about which credential is needed. We can therefore check the GIT_ASKPASS prompts to make sure we are prompting for the new location. Because we have neither multiple servers nor https support in our test setup, we can only redirect between paths, meaning we need to turn on credential.useHttpPath to see the difference. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
2013-07-30http: add http.savecookies option to write out HTTP cookiesLibravatar Dave Borowitz1-0/+8
HTTP servers may send Set-Cookie headers in a response and expect them to be set on subsequent requests. By default, libcurl behavior is to store such cookies in memory and reuse them across requests within a single session. However, it may also make sense, depending on the server and the cookies, to store them across sessions. Provide users an option to enable this behavior, writing cookies out to the same file specified in http.cookiefile. Signed-off-by: Dave Borowitz <dborowitz@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-06-23Merge branch 'jk/apache-test-for-2.4'Libravatar Junio C Hamano1-1/+19
* jk/apache-test-for-2.4: lib-httpd/apache.conf: check version only after mod_version loads t/lib-httpd/apache.conf: configure an MPM module for apache 2.4 t/lib-httpd/apache.conf: load compat access module in apache 2.4 t/lib-httpd/apache.conf: load extra auth modules in apache 2.4 t/lib-httpd/apache.conf: do not use LockFile in apache >= 2.4
2013-06-21lib-httpd/apache.conf: check version only after mod_version loadsLibravatar Jeff King1-3/+4
Commit 0442743 introduced an <IfVersion> directive near the top of the apache config file. However, at that point we have not yet checked for and loaded the mod_version module. This means that the directive will behave oddly if mod_version is dynamically loaded, failing to match when it should. We can fix this by moving the whole block below the LoadModule directive for mod_version. Reported-by: Brian Gernhardt <brian@gernhardtsoftware.com> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-06-14t/lib-httpd/apache.conf: configure an MPM module for apache 2.4Libravatar Jeff King1-0/+3
Versions of Apache before 2.4 always had a "MultiProcessing Module" (MPM) statically built in, which manages the worker threads/processes. We do not care which one, as it is largely a performance issue, and we put only a light load on the server during our testing. As of Apache 2.4, the MPM module is loadable just like any other module, but exactly one such module must be loaded. On a system where the MPMs are compiled dynamically (e.g., Debian unstable), this means that our test Apache server will not start unless we provide the appropriate configuration. Unfortunately, we do not actually know which MPM modules are available or appropriate for the system on which the tests are running. This patch picks the "prefork" module, as it is likely to be available on all Unix-like systems. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-06-14t/lib-httpd/apache.conf: load compat access module in apache 2.4Libravatar Jeff King1-0/+3
In apache 2.4, the "Order" directive has gone away in favor of a new system in mod_authz_host. However, since we want our config file to remain compatible across multiple Apache versions, we can use mod_access_compat to keep using the older style. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-06-14t/lib-httpd/apache.conf: load extra auth modules in apache 2.4Libravatar Jeff King1-0/+9
In apache 2.4, the "Auth*" and "Require" directives have moved into the authn_core and authz_core modules, respectively. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-06-14t/lib-httpd/apache.conf: do not use LockFile in apache >= 2.4Libravatar Jeff King1-0/+2
The LockFile directive from earlier versions of apache has been replaced by the Mutex directive. The latter seems to give sane defaults and does not need any specific customization, so we can get away with just adding a version check to the use of LockFile. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-04-21Merge branch 'jk/doc-http-backend'Libravatar Junio C Hamano1-0/+18
Improve documentation to illustrate "push authenticated, fetch anonymous" configuration for smart HTTP servers. * jk/doc-http-backend: doc/http-backend: match query-string in apache half-auth example doc/http-backend: give some lighttpd config examples doc/http-backend: clarify "half-auth" repo configuration
2013-04-13doc/http-backend: match query-string in apache half-auth exampleLibravatar Jeff King1-0/+18
When setting up a "half-auth" repository in which reads can be done anonymously but writes require authentication, it is best if the server can require authentication for both the ref advertisement and the actual receive-pack POSTs. This alleviates the need for the admin to set http.receivepack in the repositories, and means that the client is challenged for credentials immediately, instead of partway through the push process (and git clients older than v1.7.11.7 had trouble handling these challenges). Since detecting a push during the ref advertisement requires matching the query string, and this is non-trivial to do in Apache, we have traditionally punted and instructed users to just protect "/git-receive-pack$". This patch provides the mod_rewrite recipe to actually match the ref advertisement, which is preferred. While we're at it, let's add the recipe to our test scripts so that we can be sure that it works, and doesn't get broken (either by our changes or by changes in Apache). Signed-off-by: Jeff King <peff@peff.net> Acked-by: Jakub Narębski <jnareb@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-04-09http-backend: respect GIT_NAMESPACE with dumb clientsLibravatar John Koleszar1-0/+5
Filter the list of refs returned via the dumb HTTP protocol according to the active namespace, consistent with other clients of the upload-pack service. Signed-off-by: John Koleszar <jkoleszar@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-02-04Verify Content-Type from smart HTTP serversLibravatar Shawn Pearce1-0/+4
Before parsing a suspected smart-HTTP response verify the returned Content-Type matches the standard. This protects a client from attempting to process a payload that smells like a smart-HTTP server response. JGit has been doing this check on all responses since the dawn of time. I mistakenly failed to include it in git-core when smart HTTP was introduced. At the time I didn't know how to get the Content-Type from libcurl. I punted, meant to circle back and fix this, and just plain forgot about it. Signed-off-by: Shawn Pearce <spearce@spearce.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-11-20Merge branch 'jk/maint-http-half-auth-fetch'Libravatar Junio C Hamano1-0/+7
Fixes fetch from servers that ask for auth only during the actual packing phase. This is not really a recommended configuration, but it cleans up the code at the same time. * jk/maint-http-half-auth-fetch: remote-curl: retry failed requests for auth even with gzip remote-curl: hoist gzip buffer size to top of post_rpc
2012-10-31remote-curl: retry failed requests for auth even with gzipLibravatar Jeff King1-0/+7
Commit b81401c taught the post_rpc function to retry the http request after prompting for credentials. However, it did not handle two cases: 1. If we have a large request, we do not retry. That's OK, since we would have sent a probe (with retry) already. 2. If we are gzipping the request, we do not retry. That was considered OK, because the intended use was for push (e.g., listing refs is OK, but actually pushing objects is not), and we never gzip on push. This patch teaches post_rpc to retry even a gzipped request. This has two advantages: 1. It is possible to configure a "half-auth" state for fetching, where the set of refs and their sha1s are advertised, but one cannot actually fetch objects. This is not a recommended configuration, as it leaks some information about what is in the repository (e.g., an attacker can try brute-forcing possible content in your repository and checking whether it matches your branch sha1). However, it can be slightly more convenient, since a no-op fetch will not require a password at all. 2. It future-proofs us should we decide to ever gzip more requests. Signed-off-by: Jeff King <peff@peff.net>
2012-09-07Merge branch 'jk/maint-http-half-auth-push'Libravatar Junio C Hamano1-10/+15
Pushing to smart HTTP server with recent Git fails without having the username in the URL to force authentication, if the server is configured to allow GET anonymously, while requiring authentication for POST. * jk/maint-http-half-auth-push: http: prompt for credentials on failed POST http: factor out http error code handling t: test http access to "half-auth" repositories t: test basic smart-http authentication t/lib-httpd: recognize */smart/* repos as smart-http t/lib-httpd: only route auth/dumb to dumb repos t5550: factor out http auth setup t5550: put auth-required repo in auth/dumb
2012-08-27t: test http access to "half-auth" repositoriesLibravatar Jeff King1-0/+7
Some sites set up http access to repositories such that fetching is anonymous and unauthenticated, but pushing is authenticated. While there are multiple ways to do this, the technique advertised in the git-http-backend manpage is to block access to locations matching "/git-receive-pack$". Let's emulate that advice in our test setup, which makes it clear that this advice does not actually work. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-08-27t/lib-httpd: recognize */smart/* repos as smart-httpLibravatar Jeff King1-9/+7
We do not currently test authentication for smart-http repos at all. Part of the infrastructure to do this is recognizing that auth/smart is indeed a smart-http repo. The current apache config recognizes only "^/smart/*" as smart-http. Let's instead treat anything with /smart/ in the URL as smart-http. This is obviously a stupid thing to do for a real production site, but for our test suite we know that our repositories will not have this magic string in the name. Note that we will route /foo/smart/bar.git directly to git-http-backend/bar.git; in other words, everything before the "/smart/" is irrelevant to finding the repo on disk (but may impact apache config, for example by triggering auth checks). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-08-27t/lib-httpd: only route auth/dumb to dumb reposLibravatar Jeff King1-1/+1
Our test apache config points all of auth/ directly to the on-disk repositories via an Alias directive. This works fine because everything authenticated is currently in auth/dumb, which is a subset. However, this would conflict with a ScriptAlias for auth/smart (which will come in future patches), so let's narrow the Alias. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-07-24t/lib-httpd: handle running under --valgrindLibravatar Jeff King1-1/+4
Running the http tests with valgrind does not work for two reasons: 1. Apache complains about following the symbolic link from git-http-backend to valgrind.sh. 2. Apache does not pass through the GIT_VALGRIND variable to the backend CGI. This patch fixes both problems. Unfortunately, there is a slight hack we need to handle passing environment variables through Apache. If we just tell it: PassEnv GIT_VALGRIND then Apache will complain when GIT_VALGRIND is not set. If we try: SetEnv GIT_VALGRIND ${GIT_VALGRIND} then when GIT_VALGRIND is not set, it will pass through the literal "${GIT_VALGRIND}". Instead, we now unconditionally pass through GIT_VALGRIND from lib-httpd.sh into apache, even if it is empty. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-03-30http-backend: respect existing GIT_COMMITTER_* variablesLibravatar Jeff King1-0/+7
The http-backend program sets default GIT_COMMITTER_NAME and GIT_COMMITTER_EMAIL variables based on the REMOTE_USER and REMOTE_ADDR variables provided by the webserver. However, it unconditionally overwrites any existing GIT_COMMITTER variables, which may have been customized by site-specific code in the webserver (or in a script wrapping http-backend). Let's leave those variables intact if they already exist, assuming that any such configuration was intentional. There is a slight chance of a regression if somebody has set GIT_COMMITTER_* for the entire webserver, not intending it to leak through http-backend. We could protect against this by passing the information in alternate variables. However, it seems unlikely that anyone will care about that regression, and there is value in the simplicity of using the common variable names that are used elsewhere in git. While we're tweaking the environment-handling in http-backend, let's switch it to use argv_array to handle the list of variables. That makes the memory management much simpler. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>