summaryrefslogtreecommitdiff
path: root/t/lib-httpd
AgeCommit message (Collapse)AuthorFilesLines
2016-12-19Merge branch 'jk/http-walker-limit-redirect-2.9'Libravatar Junio C Hamano1-0/+14
Transport with dumb http can be fooled into following foreign URLs that the end user does not intend to, especially with the server side redirects and http-alternates mechanism, which can lead to security issues. Tighten the redirection and make it more obvious to the end user when it happens. * jk/http-walker-limit-redirect-2.9: http: treat http-alternates like redirects http: make redirects more obvious remote-curl: rename shadowed options variable http: always update the base URL for redirects http: simplify update_url_from_redirect
2016-12-06http: make redirects more obviousLibravatar Jeff King1-0/+6
We instruct curl to always follow HTTP redirects. This is convenient, but it creates opportunities for malicious servers to create confusing situations. For instance, imagine Alice is a git user with access to a private repository on Bob's server. Mallory runs her own server and wants to access objects from Bob's repository. Mallory may try a few tricks that involve asking Alice to clone from her, build on top, and then push the result: 1. Mallory may simply redirect all fetch requests to Bob's server. Git will transparently follow those redirects and fetch Bob's history, which Alice may believe she got from Mallory. The subsequent push seems like it is just feeding Mallory back her own objects, but is actually leaking Bob's objects. There is nothing in git's output to indicate that Bob's repository was involved at all. The downside (for Mallory) of this attack is that Alice will have received Bob's entire repository, and is likely to notice that when building on top of it. 2. If Mallory happens to know the sha1 of some object X in Bob's repository, she can instead build her own history that references that object. She then runs a dumb http server, and Alice's client will fetch each object individually. When it asks for X, Mallory redirects her to Bob's server. The end result is that Alice obtains objects from Bob, but they may be buried deep in history. Alice is less likely to notice. Both of these attacks are fairly hard to pull off. There's a social component in getting Mallory to convince Alice to work with her. Alice may be prompted for credentials in accessing Bob's repository (but not always, if she is using a credential helper that caches). Attack (1) requires a certain amount of obliviousness on Alice's part while making a new commit. Attack (2) requires that Mallory knows a sha1 in Bob's repository, that Bob's server supports dumb http, and that the object in question is loose on Bob's server. But we can probably make things a bit more obvious without any loss of functionality. This patch does two things to that end. First, when we encounter a whole-repo redirect during the initial ref discovery, we now inform the user on stderr, making attack (1) much more obvious. Second, the decision to follow redirects is now configurable. The truly paranoid can set the new http.followRedirects to false to avoid any redirection entirely. But for a more practical default, we will disallow redirects only after the initial ref discovery. This is enough to thwart attacks similar to (2), while still allowing the common use of redirects at the repository level. Since c93c92f30 (http: update base URLs when we see redirects, 2013-09-28) we re-root all further requests from the redirect destination, which should generally mean that no further redirection is necessary. As an escape hatch, in case there really is a server that needs to redirect individual requests, the user can set http.followRedirects to "true" (and this can be done on a per-server basis via http.*.followRedirects config). Reported-by: Jann Horn <jannh@google.com> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-06http: always update the base URL for redirectsLibravatar Jeff King1-0/+8
If a malicious server redirects the initial ref advertisement, it may be able to leak sha1s from other, unrelated servers that the client has access to. For example, imagine that Alice is a git user, she has access to a private repository on a server hosted by Bob, and Mallory runs a malicious server and wants to find out about Bob's private repository. Mallory asks Alice to clone an unrelated repository from her over HTTP. When Alice's client contacts Mallory's server for the initial ref advertisement, the server issues an HTTP redirect for Bob's server. Alice contacts Bob's server and gets the ref advertisement for the private repository. If there is anything to fetch, she then follows up by asking the server for one or more sha1 objects. But who is the server? If it is still Mallory's server, then Alice will leak the existence of those sha1s to her. Since commit c93c92f30 (http: update base URLs when we see redirects, 2013-09-28), the client usually rewrites the base URL such that all further requests will go to Bob's server. But this is done by textually matching the URL. If we were originally looking for "http://mallory/repo.git/info/refs", and we got pointed at "http://bob/other.git/info/refs", then we know that the right root is "http://bob/other.git". If the redirect appears to change more than just the root, we punt and continue to use the original server. E.g., imagine the redirect adds a URL component that Bob's server will ignore, like "http://bob/other.git/info/refs?dummy=1". We can solve this by aborting in this case rather than silently continuing to use Mallory's server. In addition to protecting from sha1 leakage, it's arguably safer and more sane to refuse a confusing redirect like that in general. For example, part of the motivation in c93c92f30 is avoiding accidentally sending credentials over clear http, just to get a response that says "try again over https". So even in a non-malicious case, we'd prefer to err on the side of caution. The downside is that it's possible this will break a legitimate but complicated server-side redirection scheme. The setup given in the newly added test does work, but it's convoluted enough that we don't need to care about it. A more plausible case would be a server which redirects a request for "info/refs?service=git-upload-pack" to just "info/refs" (because it does not do smart HTTP, and for some reason really dislikes query parameters). Right now we would transparently downgrade to dumb-http, but with this patch, we'd complain (and the user would have to set GIT_SMART_HTTP=0 to fetch). Reported-by: Jann Horn <jannh@google.com> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-08Merge branch 'ew/git-svn-http-tests'Libravatar Junio C Hamano1-2/+2
Tests for "git svn" have been taught to reuse the lib-httpd test infrastructure when testing the subversion integration that interacts with subversion repositories served over the http:// protocol. * ew/git-svn-http-tests: git svn: migrate tests to use lib-httpd t/t91*: do not say how to avoid the tests
2016-07-25git svn: migrate tests to use lib-httpdLibravatar Eric Wong1-2/+2
This allows us to use common test infrastructure and parallelize the tests. For now, GIT_SVN_TEST_HTTPD=true needs to be set to enable the SVN HTTP tests because we reuse the same test cases for both file:// and http:// SVN repositories. SVN_HTTPD_PORT is no longer honored. Tested under Apache 2.2 and 2.4 on Debian 7.x (wheezy) and 8.x (jessie), respectively. Cc: Clemens Buchacher <drizzd@aon.at> Cc: Michael J Gruber <git@drmicha.warpmail.net> Signed-off-by: Eric Wong <e@80x24.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-17Merge branch 'js/http-custom-headers'Libravatar Junio C Hamano1-4/+12
Update tests for "http.extraHeaders=<header>" to be portable back to Apache 2.2 (the original depended on <RequireAll/> which is a more recent feature). * js/http-custom-headers: submodule: ensure that -c http.extraheader is heeded t5551: make the test for extra HTTP headers more robust tests: adjust the configuration for Apache 2.2
2016-05-10tests: adjust the configuration for Apache 2.2Libravatar Johannes Schindelin1-4/+12
Lars Schneider noticed that the configuration introduced to test the extra HTTP headers cannot be used with Apache 2.2 (which is still actively maintained, as pointed out by Junio Hamano). To let the tests pass with Apache 2.2 again, let's substitute the offending <RequireAll> and `expr` by using old school RewriteCond statements. As RewriteCond does not allow testing for *non*-matches, we simply match the desired case first and let it pass by marking the RewriteRule as '[L]' ("last rule, do not process any other matching RewriteRules after this"), and then have another RewriteRule that matches all other cases and lets them fail via '[F]' ("fail"). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Tested-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-06Merge branch 'js/http-custom-headers'Libravatar Junio C Hamano1-0/+8
HTTP transport clients learned to throw extra HTTP headers at the server, specified via http.extraHeader configuration variable. * js/http-custom-headers: http: support sending custom HTTP headers
2016-04-27http: support sending custom HTTP headersLibravatar Johannes Schindelin1-0/+8
We introduce a way to send custom HTTP headers with all requests. This allows us, for example, to send an extra token from build agents for temporary access to private repositories. (This is the use case that triggered this patch.) This feature can be used like this: git -c http.extraheader='Secret: sssh!' fetch $URL $REF Note that `curl_easy_setopt(..., CURLOPT_HTTPHEADER, ...)` takes only a single list, overriding any previous call. This means we have to collect _all_ of the headers we want to use into a single list, and feed it to cURL in one shot. Since we already unconditionally set a "pragma" header when initializing the curl handles, we can add our new headers to that list. For callers which override the default header list (like probe_rpc), we provide `http_copy_default_headers()` so they can do the same trick. Big thanks to Jeff King and Junio Hamano for their outstanding help and patient reviews. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Reviewed-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-04-14Merge branch 'jk/test-httpd-config-nosystem' into maintLibravatar Junio C Hamano1-0/+1
The tests that involve running httpd leaked the system-wide configuration in /etc/gitconfig to the tested environment. * jk/test-httpd-config-nosystem: t/lib-httpd: pass through GIT_CONFIG_NOSYSTEM env
2016-04-06Merge branch 'jk/test-httpd-config-nosystem'Libravatar Junio C Hamano1-0/+1
The tests that involve running httpd leaked the system-wide configuration in /etc/gitconfig to the tested environment. * jk/test-httpd-config-nosystem: t/lib-httpd: pass through GIT_CONFIG_NOSYSTEM env
2016-03-18t/lib-httpd: pass through GIT_CONFIG_NOSYSTEM envLibravatar Jeff King1-0/+1
We set GIT_CONFIG_NOSYSTEM in our test scripts so that we do not accidentally read /etc/gitconfig and have it influence the outcome of the tests. But when running smart-http tests, Apache will clean the environment, including this variable, and the "server" side of our http operations will read it. You can see this breakage by doing something like: make ./git config --system http.getanyfile false make test which will cause t5561 to fail when it tests the fallback-to-dumb operation. We can fix this by instructing Apache to pass through the variable. Unlike with other variables (e.g., 89c57ab3's GIT_TRACE), we don't need to set a dummy value to prevent warnings from Apache. test-lib.sh already makes sure that GIT_CONFIG_NOSYSTEM is set and exported. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-02-25t/lib-httpd: load mod_unixdLibravatar Michael J Gruber1-0/+3
In contrast to apache 2.2, apache 2.4 does not load mod_unixd in its default configuration (because there are choices). Thus, with the current config, apache 2.4.10 will not be started and the httpd tests will not run on distros with default apache config (RedHat type). Enable mod_unixd to make the httpd tests run. This does not affect distros negatively which have that config already in their default (Debian type). httpd tests will run on these before and after this patch. Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-09-25http: limit redirection depthLibravatar Blake Burkhart1-0/+3
By default, libcurl will follow circular http redirects forever. Let's put a cap on this so that somebody who can trigger an automated fetch of an arbitrary repository (e.g., for CI) cannot convince git to loop infinitely. The value chosen is 20, which is the same default that Firefox uses. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-09-25http: limit redirection to protocol-whitelistLibravatar Blake Burkhart1-0/+1
Previously, libcurl would follow redirection to any protocol it was compiled for support with. This is desirable to allow redirection from HTTP to HTTPS. However, it would even successfully allow redirection from HTTP to SFTP, a protocol that git does not otherwise support at all. Furthermore git's new protocol-whitelisting could be bypassed by following a redirect within the remote helper, as it was only enforced at transport selection time. This patch limits redirects within libcurl to HTTP, HTTPS, FTP and FTPS. If there is a protocol-whitelist present, this list is limited to those also allowed by the whitelist. As redirection happens from within libcurl, it is impossible for an HTTP redirect to a protocol implemented within another remote helper. When the curl version git was compiled with is too old to support restrictions on protocol redirection, we warn the user if GIT_ALLOW_PROTOCOL restrictions were requested. This is a little inaccurate, as even without that variable in the environment, we would still restrict SFTP, etc, and we do not warn in that case. But anything else means we would literally warn every time git accesses an http remote. This commit includes a test, but it is not as robust as we would hope. It redirects an http request to ftp, and checks that curl complained about the protocol, which means that we are relying on curl's specific error message to know what happened. Ideally we would redirect to a working ftp server and confirm that we can clone without protocol restrictions, and not with them. But we do not have a portable way of providing an ftp server, nor any other protocol that curl supports (https is the closest, but we would have to deal with certificates). [jk: added test and version warning] Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-03-12t: pass GIT_TRACE through ApacheLibravatar Jeff King1-0/+1
Apache removes GIT_TRACE from the environment before running git-http-backend. This can make it hard to debug the server side of an http session. Let's let it through. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-12-11t: support clang/gcc AddressSanitizerLibravatar Jeff King1-0/+1
When git is compiled with "-fsanitize=address" (using clang or gcc >= 4.8), all invocations of git will check for buffer overflows. This is similar to running with valgrind, except that it is more thorough (because of the compiler support, function-local buffers can be checked, too) and runs much faster (making it much less painful to run the whole test suite with the checks turned on). Unlike valgrind, the magic happens at compile-time, so we don't need the same infrastructure in the test suite that we did to support --valgrind. But there are two things we can help with: 1. On some platforms, the leak-detector is on by default, and causes every invocation of "git init" (and thus every test script) to fail. Since running git with the leak detector is pointless, let's shut it off automatically in the tests, unless the user has already configured it. 2. When apache runs a CGI, it clears the environment of unknown variables. This means that the $ASAN_OPTIONS config doesn't make it to git-http-backend, and it dies due to the leak detector. Let's mark the variable as OK for apache to pass. With these two changes, running make CC=clang CFLAGS=-fsanitize=address test works out of the box. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-09-17signed push: teach smart-HTTP to pass "git push --signed" aroundLibravatar Junio C Hamano1-0/+1
The "--signed" option received by "git push" is first passed to the transport layer, which the native transport directly uses to notice that a push certificate needs to be sent. When the transport-helper is involved, however, the option needs to be told to the helper with set_helper_option(), and the helper needs to take necessary action. For the smart-HTTP helper, the "necessary action" involves spawning the "git send-pack" subprocess with the "--signed" option. Once the above all gets wired in, the smart-HTTP transport now can use the push certificate mechanism to authenticate its pushes. Add a test that is modeled after tests for the native transport in t5534-push-signed.sh to t5541-http-push-smart.sh. Update the test Apache configuration to pass GNUPGHOME environment variable through. As PassEnv would trigger warnings for an environment variable that is not set, export it from test-lib.sh set to a harmless value when GnuPG is not being used in the tests. Note that the added test is deliberately loose and does not check the nonce in this step. This is because the stateless RPC mode is inevitably flaky and a nonce that comes back in the actual push processing is one issued by a different process; if the two interactions with the server crossed a second boundary, the nonces will not match and such a check will fail. A later patch in the series will work around this shortcoming. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-17http: fix charset detection of extract_content_type()Libravatar Yi EungJun1-0/+4
extract_content_type() could not extract a charset parameter if the parameter is not the first one and there is a whitespace and a following semicolon just before the parameter. For example: text/plain; format=fixed ;charset=utf-8 And it also could not handle correctly some other cases, such as: text/plain; charset=utf-8; format=fixed text/plain; some-param="a long value with ;semicolons;"; charset=utf-8 Thanks-to: Jeff King <peff@peff.net> Signed-off-by: Yi EungJun <eungjun.yi@navercorp.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-05-27remote-curl: reencode http error messagesLibravatar Jeff King1-0/+4
We currently recognize an error message with a content-type "text/plain; charset=utf-16" as text, but we ignore the charset parameter entirely. Let's encode it to log_output_encoding, which is presumably something the user's terminal can handle. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-05-27http: extract type/subtype portion of content-typeLibravatar Jeff King1-1/+7
When we get a content-type from curl, we get the whole header line, including any parameters, and without any normalization (like downcasing or whitespace) applied. If we later try to match it with strcmp() or even strcasecmp(), we may get false negatives. This could cause two visible behaviors: 1. We might fail to recognize a smart-http server by its content-type. 2. We might fail to relay text/plain error messages to users (especially if they contain a charset parameter). This patch teaches the http code to extract and normalize just the type/subtype portion of the string. This is technically passing out less information to the callers, who can no longer see the parameters. But none of the current callers cares, and a future patch will add back an easier-to-use method for accessing those parameters. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-05-23t5550: test display of remote http error messagesLibravatar Jeff King2-0/+21
Since commit 426e70d (remote-curl: show server content on http errors, 2013-04-05), we relay any text/plain error messages from the remote server to the user. However, we never tested it. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-05-23t/lib-httpd: use write_script to copy CGI scriptsLibravatar Jeff King1-1/+0
Using write_script will set our shebang line appropriately with $SHELL_PATH. The script that is there now is quite simple and likely to succeed even with a non-POSIX /bin/sh, but it does not hurt to be defensive. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-01-02use distinct username/password for http auth testsLibravatar Jeff King1-1/+1
The httpd server we set up to test git's http client code knows about a single account, in which both the username and password are "user@host" (the unusual use of the "@" here is to verify that we handle the character correctly when URL escaped). This means that we may miss a certain class of errors in which the username and password are mixed up internally by git. We can make our tests more robust by having distinct values for the username and password. In addition to tweaking the server passwd file and the client URL, we must teach the "askpass" harness to accept multiple values. As a bonus, this makes the setup of some tests more obvious; when we are expecting git to ask only about the password, we can seed the username askpass response with a bogus value. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-10-30Merge branch 'jk/http-auth-redirects'Libravatar Junio C Hamano1-0/+2
Handle the case where http transport gets redirected during the authorization request better. * jk/http-auth-redirects: http.c: Spell the null pointer as NULL remote-curl: rewrite base url from info/refs redirects remote-curl: store url as a strbuf remote-curl: make refs_url a strbuf http: update base URLs when we see redirects http: provide effective url to callers http: hoist credential request out of handle_curl_result http: refactor options to http_get_* http_request: factor out curlinfo_strbuf http_get_file: style fixes
2013-10-14remote-curl: rewrite base url from info/refs redirectsLibravatar Jeff King1-0/+2
For efficiency and security reasons, an earlier commit in this series taught http_get_* to re-write the base url based on redirections we saw while making a specific request. This commit wires that option into the info/refs request, meaning that a redirect from http://example.com/foo.git/info/refs to https://example.com/bar.git/info/refs will behave as if "https://example.com/bar.git" had been provided to git in the first place. The tests bear some explanation. We introduce two new hierearchies into the httpd test config: 1. Requests to /smart-redir-limited will work only for the initial info/refs request, but not any subsequent requests. As a result, we can confirm whether the client is re-rooting its requests after the initial contact, since otherwise it will fail (it will ask for "repo.git/git-upload-pack", which is not redirected). 2. Requests to smart-redir-auth will redirect, and require auth after the redirection. Since we are using the redirected base for further requests, we also update the credential struct, in order not to mislead the user (or credential helpers) about which credential is needed. We can therefore check the GIT_ASKPASS prompts to make sure we are prompting for the new location. Because we have neither multiple servers nor https support in our test setup, we can only redirect between paths, meaning we need to turn on credential.useHttpPath to see the difference. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
2013-07-30http: add http.savecookies option to write out HTTP cookiesLibravatar Dave Borowitz1-0/+8
HTTP servers may send Set-Cookie headers in a response and expect them to be set on subsequent requests. By default, libcurl behavior is to store such cookies in memory and reuse them across requests within a single session. However, it may also make sense, depending on the server and the cookies, to store them across sessions. Provide users an option to enable this behavior, writing cookies out to the same file specified in http.cookiefile. Signed-off-by: Dave Borowitz <dborowitz@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-06-23Merge branch 'jk/apache-test-for-2.4'Libravatar Junio C Hamano1-1/+19
* jk/apache-test-for-2.4: lib-httpd/apache.conf: check version only after mod_version loads t/lib-httpd/apache.conf: configure an MPM module for apache 2.4 t/lib-httpd/apache.conf: load compat access module in apache 2.4 t/lib-httpd/apache.conf: load extra auth modules in apache 2.4 t/lib-httpd/apache.conf: do not use LockFile in apache >= 2.4
2013-06-21lib-httpd/apache.conf: check version only after mod_version loadsLibravatar Jeff King1-3/+4
Commit 0442743 introduced an <IfVersion> directive near the top of the apache config file. However, at that point we have not yet checked for and loaded the mod_version module. This means that the directive will behave oddly if mod_version is dynamically loaded, failing to match when it should. We can fix this by moving the whole block below the LoadModule directive for mod_version. Reported-by: Brian Gernhardt <brian@gernhardtsoftware.com> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-06-14t/lib-httpd/apache.conf: configure an MPM module for apache 2.4Libravatar Jeff King1-0/+3
Versions of Apache before 2.4 always had a "MultiProcessing Module" (MPM) statically built in, which manages the worker threads/processes. We do not care which one, as it is largely a performance issue, and we put only a light load on the server during our testing. As of Apache 2.4, the MPM module is loadable just like any other module, but exactly one such module must be loaded. On a system where the MPMs are compiled dynamically (e.g., Debian unstable), this means that our test Apache server will not start unless we provide the appropriate configuration. Unfortunately, we do not actually know which MPM modules are available or appropriate for the system on which the tests are running. This patch picks the "prefork" module, as it is likely to be available on all Unix-like systems. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-06-14t/lib-httpd/apache.conf: load compat access module in apache 2.4Libravatar Jeff King1-0/+3
In apache 2.4, the "Order" directive has gone away in favor of a new system in mod_authz_host. However, since we want our config file to remain compatible across multiple Apache versions, we can use mod_access_compat to keep using the older style. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-06-14t/lib-httpd/apache.conf: load extra auth modules in apache 2.4Libravatar Jeff King1-0/+9
In apache 2.4, the "Auth*" and "Require" directives have moved into the authn_core and authz_core modules, respectively. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-06-14t/lib-httpd/apache.conf: do not use LockFile in apache >= 2.4Libravatar Jeff King1-0/+2
The LockFile directive from earlier versions of apache has been replaced by the Mutex directive. The latter seems to give sane defaults and does not need any specific customization, so we can get away with just adding a version check to the use of LockFile. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-04-21Merge branch 'jk/doc-http-backend'Libravatar Junio C Hamano1-0/+18
Improve documentation to illustrate "push authenticated, fetch anonymous" configuration for smart HTTP servers. * jk/doc-http-backend: doc/http-backend: match query-string in apache half-auth example doc/http-backend: give some lighttpd config examples doc/http-backend: clarify "half-auth" repo configuration
2013-04-13doc/http-backend: match query-string in apache half-auth exampleLibravatar Jeff King1-0/+18
When setting up a "half-auth" repository in which reads can be done anonymously but writes require authentication, it is best if the server can require authentication for both the ref advertisement and the actual receive-pack POSTs. This alleviates the need for the admin to set http.receivepack in the repositories, and means that the client is challenged for credentials immediately, instead of partway through the push process (and git clients older than v1.7.11.7 had trouble handling these challenges). Since detecting a push during the ref advertisement requires matching the query string, and this is non-trivial to do in Apache, we have traditionally punted and instructed users to just protect "/git-receive-pack$". This patch provides the mod_rewrite recipe to actually match the ref advertisement, which is preferred. While we're at it, let's add the recipe to our test scripts so that we can be sure that it works, and doesn't get broken (either by our changes or by changes in Apache). Signed-off-by: Jeff King <peff@peff.net> Acked-by: Jakub Narębski <jnareb@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-04-09http-backend: respect GIT_NAMESPACE with dumb clientsLibravatar John Koleszar1-0/+5
Filter the list of refs returned via the dumb HTTP protocol according to the active namespace, consistent with other clients of the upload-pack service. Signed-off-by: John Koleszar <jkoleszar@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-02-04Verify Content-Type from smart HTTP serversLibravatar Shawn Pearce2-0/+15
Before parsing a suspected smart-HTTP response verify the returned Content-Type matches the standard. This protects a client from attempting to process a payload that smells like a smart-HTTP server response. JGit has been doing this check on all responses since the dawn of time. I mistakenly failed to include it in git-core when smart HTTP was introduced. At the time I didn't know how to get the Content-Type from libcurl. I punted, meant to circle back and fix this, and just plain forgot about it. Signed-off-by: Shawn Pearce <spearce@spearce.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-11-20Merge branch 'jk/maint-http-half-auth-fetch'Libravatar Junio C Hamano1-0/+7
Fixes fetch from servers that ask for auth only during the actual packing phase. This is not really a recommended configuration, but it cleans up the code at the same time. * jk/maint-http-half-auth-fetch: remote-curl: retry failed requests for auth even with gzip remote-curl: hoist gzip buffer size to top of post_rpc
2012-10-31remote-curl: retry failed requests for auth even with gzipLibravatar Jeff King1-0/+7
Commit b81401c taught the post_rpc function to retry the http request after prompting for credentials. However, it did not handle two cases: 1. If we have a large request, we do not retry. That's OK, since we would have sent a probe (with retry) already. 2. If we are gzipping the request, we do not retry. That was considered OK, because the intended use was for push (e.g., listing refs is OK, but actually pushing objects is not), and we never gzip on push. This patch teaches post_rpc to retry even a gzipped request. This has two advantages: 1. It is possible to configure a "half-auth" state for fetching, where the set of refs and their sha1s are advertised, but one cannot actually fetch objects. This is not a recommended configuration, as it leaks some information about what is in the repository (e.g., an attacker can try brute-forcing possible content in your repository and checking whether it matches your branch sha1). However, it can be slightly more convenient, since a no-op fetch will not require a password at all. 2. It future-proofs us should we decide to ever gzip more requests. Signed-off-by: Jeff King <peff@peff.net>
2012-09-07Merge branch 'jk/maint-http-half-auth-push'Libravatar Junio C Hamano1-10/+15
Pushing to smart HTTP server with recent Git fails without having the username in the URL to force authentication, if the server is configured to allow GET anonymously, while requiring authentication for POST. * jk/maint-http-half-auth-push: http: prompt for credentials on failed POST http: factor out http error code handling t: test http access to "half-auth" repositories t: test basic smart-http authentication t/lib-httpd: recognize */smart/* repos as smart-http t/lib-httpd: only route auth/dumb to dumb repos t5550: factor out http auth setup t5550: put auth-required repo in auth/dumb
2012-08-27t: test http access to "half-auth" repositoriesLibravatar Jeff King1-0/+7
Some sites set up http access to repositories such that fetching is anonymous and unauthenticated, but pushing is authenticated. While there are multiple ways to do this, the technique advertised in the git-http-backend manpage is to block access to locations matching "/git-receive-pack$". Let's emulate that advice in our test setup, which makes it clear that this advice does not actually work. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-08-27t/lib-httpd: recognize */smart/* repos as smart-httpLibravatar Jeff King1-9/+7
We do not currently test authentication for smart-http repos at all. Part of the infrastructure to do this is recognizing that auth/smart is indeed a smart-http repo. The current apache config recognizes only "^/smart/*" as smart-http. Let's instead treat anything with /smart/ in the URL as smart-http. This is obviously a stupid thing to do for a real production site, but for our test suite we know that our repositories will not have this magic string in the name. Note that we will route /foo/smart/bar.git directly to git-http-backend/bar.git; in other words, everything before the "/smart/" is irrelevant to finding the repo on disk (but may impact apache config, for example by triggering auth checks). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-08-27t/lib-httpd: only route auth/dumb to dumb reposLibravatar Jeff King1-1/+1
Our test apache config points all of auth/ directly to the on-disk repositories via an Alias directive. This works fine because everything authenticated is currently in auth/dumb, which is a subset. However, this would conflict with a ScriptAlias for auth/smart (which will come in future patches), so let's narrow the Alias. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-07-24t/lib-httpd: handle running under --valgrindLibravatar Jeff King1-1/+4
Running the http tests with valgrind does not work for two reasons: 1. Apache complains about following the symbolic link from git-http-backend to valgrind.sh. 2. Apache does not pass through the GIT_VALGRIND variable to the backend CGI. This patch fixes both problems. Unfortunately, there is a slight hack we need to handle passing environment variables through Apache. If we just tell it: PassEnv GIT_VALGRIND then Apache will complain when GIT_VALGRIND is not set. If we try: SetEnv GIT_VALGRIND ${GIT_VALGRIND} then when GIT_VALGRIND is not set, it will pass through the literal "${GIT_VALGRIND}". Instead, we now unconditionally pass through GIT_VALGRIND from lib-httpd.sh into apache, even if it is empty. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-03-30http-backend: respect existing GIT_COMMITTER_* variablesLibravatar Jeff King1-0/+7
The http-backend program sets default GIT_COMMITTER_NAME and GIT_COMMITTER_EMAIL variables based on the REMOTE_USER and REMOTE_ADDR variables provided by the webserver. However, it unconditionally overwrites any existing GIT_COMMITTER variables, which may have been customized by site-specific code in the webserver (or in a script wrapping http-backend). Let's leave those variables intact if they already exist, assuming that any such configuration was intentional. There is a slight chance of a regression if somebody has set GIT_COMMITTER_* for the entire webserver, not intending it to leak through http-backend. We could protect against this by passing the information in alternate variables. However, it seems unlikely that anyone will care about that regression, and there is value in the simplicity of using the common variable names that are used elsewhere in git. While we're tweaking the environment-handling in http-backend, let's switch it to use argv_array to handle the list of variables. That makes the memory management much simpler. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-12-13t5540: test DAV push with authenticationLibravatar Jeff King1-0/+3
We don't currently test this case at all, and instead just test the DAV mechanism over an unauthenticated push. That isn't very realistic, as most people will want to authenticate pushes. Two of the tests expect_failure as they reveal bugs: 1. Pushing without a username in the URL fails to ask for credentials when we get an HTTP 401. This has always been the case, but it would be nice if it worked like smart-http. 2. Pushing with a username fails to ask for the password since 986bbc0 (http: don't always prompt for password, 2011-11-04). This is a severe regression in v1.7.8, as authenticated push-over-DAV is now totally unusable unless you have credentials in your .netrc. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-12-08Merge branch 'gc/http-with-non-ascii-username-url'Libravatar Junio C Hamano2-0/+30
* gc/http-with-non-ascii-username-url: Fix username and password extraction from HTTP URLs t5550: test HTTP authentication and userinfo decoding Conflicts: t/lib-httpd/apache.conf
2010-11-17t5550: test HTTP authentication and userinfo decodingLibravatar Gabriel Corona2-0/+30
Add a test for HTTP authentication and proper percent-decoding of the userinfo (username and password) part of the URL. Signed-off-by: Gabriel Corona <gabriel.corona@enst-bretagne.fr> Acked-by: Tay Ray Chuan <rctay89@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-09-27smart-http: Don't change POST to GET when following redirectLibravatar Tay Ray Chuan1-0/+7
For a long time (29508e1 "Isolate shared HTTP request functionality", Fri Nov 18 11:02:58 2005), we've followed HTTP redirects with CURLOPT_FOLLOWLOCATION. However, when the remote HTTP server returns a redirect the default libcurl action is to change a POST request into a GET request while following the redirect, but the remote http backend does not expect that. Fix this by telling libcurl to always keep the request as type POST with CURLOPT_POSTREDIR. For users of libcurl older than 7.19.1, use CURLOPT_POST301 instead, which only follows 301s instead of both 301s and 302s. Signed-off-by: Andreas Schwab <schwab@linux-m68k.org> Signed-off-by: Tay Ray Chuan <rctay89@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-01-06Smart-http: check if repository is OK to export before serving itLibravatar Tarmigan Casebolt1-0/+5
Similar to how git-daemon checks whether a repository is OK to be exported, smart-http should also check. This check can be satisfied in two different ways: the environmental variable GIT_HTTP_EXPORT_ALL may be set to export all repositories, or the individual repository may have the file git-daemon-export-ok. Acked-by: Shawn O. Pearce <spearce@spearce.org> Signed-off-by: Tarmigan Casebolt <tarmigan+git@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>