summaryrefslogtreecommitdiff
path: root/t/t5537-fetch-shallow.sh
AgeCommit message (Collapse)AuthorFilesLines
2019-01-17tests: define GIT_TEST_SIDEBAND_ALLLibravatar Jonathan Tan1-1/+2
Define a GIT_TEST_SIDEBAND_ALL environment variable meant to be used from tests. When set to true, this overrides uploadpack.allowsidebandall to true, allowing the entire test suite to be run as if this configuration is in place for all repositories. As of this patch, all tests pass whether GIT_TEST_SIDEBAND_ALL is unset or set to 1. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-25repack -ad: prune the list of shallow commitsLibravatar Johannes Schindelin1-1/+1
`git repack` can drop unreachable commits without further warning, making the corresponding entries in `.git/shallow` invalid, which causes serious problems when deepening the branches. One scenario where unreachable commits are dropped by `git repack` is when a `git fetch --prune` (or even a `git fetch` when a ref was force-pushed in the meantime) can make a commit unreachable that was reachable before. Therefore it is not safe to assume that a `git repack -adlf` will keep unreachable commits alone (under the assumption that they had not been packed in the first place, which is an assumption at least some of Git's code seems to make). This is particularly important to keep in mind when looking at the `.git/shallow` file: if any commits listed in that file become unreachable, it is not a problem, but if they go missing, it *is* a problem. One symptom of this problem is that a deepening fetch may now fail with fatal: error in object: unshallow <commit-hash> To avoid this problem, let's prune the shallow list in `git repack` when the `-d` option is passed, unless `-A` is passed, too (which would force the now-unreachable objects to be turned into loose objects instead of being deleted). Additionally, we also need to take `--keep-reachable` and `--unpack-unreachable=<date>` into account. Note: an alternative solution discussed during the review of this patch was to teach `git fetch` to simply ignore entries in .git/shallow if the corresponding commits do not exist locally. A quick test, however, revealed that the .git/shallow file is written during a shallow *clone*, in which case the commits do not exist, either, but the "shallow" line *does* need to be sent. Therefore, this approach would be a lot more finicky than the approach presented by the this patch. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-25repack: point out a bug handling stale shallow infoLibravatar Johannes Schindelin1-0/+27
A `git fetch --prune` can turn previously-reachable objects unreachable, even commits that are in the `shallow` list. A subsequent `git repack -ad` will then unceremoniously drop those unreachable commits, and the `shallow` list will become stale. This means that when we try to fetch with a larger `--depth` the next time, we may end up with: fatal: error in object: unshallow <commit-hash> Reported by Alejandro Pauly. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-24Merge branch 'jt/connectivity-check-after-unshallow'Libravatar Junio C Hamano1-0/+43
"git fetch" failed to correctly validate the set of objects it received when making a shallow history deeper, which has been corrected. * jt/connectivity-check-after-unshallow: fetch-pack: write shallow, then check connectivity fetch-pack: implement ref-in-want fetch-pack: put shallow info in output parameter fetch: refactor to make function args narrower fetch: refactor fetch_refs into two functions fetch: refactor the population of peer ref OIDs upload-pack: test negotiation with changing repository upload-pack: implement ref-in-want test-pkt-line: add unpack-sideband subcommand
2018-07-03fetch-pack: write shallow, then check connectivityLibravatar Jonathan Tan1-0/+43
When fetching, connectivity is checked after the shallow file is updated. There are 2 issues with this: (1) the connectivity check is only performed up to ancestors of existing refs (which is not thorough enough if we were deepening an existing ref in the first place), and (2) there is no rollback of the shallow file if the connectivity check fails. To solve (1), update the connectivity check to check the ancestry chain completely in the case of a deepening fetch by refraining from passing "--not --all" when invoking rev-list in connected.c. To solve (2), have fetch_pack() perform its own connectivity check before updating the shallow file. To support existing use cases in which "git fetch-pack" is used to download objects without much regard as to the connectivity of the resulting objects with respect to the existing repository, the connectivity check is only done if necessary (that is, the fetch is not a clone, and the fetch involves shallow/deepen functionality). "git fetch" still performs its own connectivity check, preserving correctness but sometimes performing redundant work. This redundancy is mitigated by the fact that fetch_pack() reports if it has performed a connectivity check itself, and if the transport supports connect or stateless-connect, it will bubble up that report so that "git fetch" knows not to perform the connectivity check in such a case. This was noticed when a user tried to deepen an existing repository by fetching with --no-shallow from a server that did not send all necessary objects - the connectivity check as run by "git fetch" succeeded, but a subsequent "git fsck" failed. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-06-15tests: clean after SANITY testsLibravatar Junio C Hamano1-1/+1
Some of our tests try to make sure Git behaves sensibly in a read-only directory, by dropping 'w' permission bit before doing a test and then restoring it after it is done. The latter is needed for the test framework to clean after itself without leaving a leftover directory that cannot be removed. Ancient parts of tests however arrange the above with chmod a-w . && ... do the test ... status=$? chmod 775 . (exit $status) which obviously would not work if the test somehow dies before it has the chance to do "chmod 775". Rewrite them by following a more robust pattern recently written tests use, which is test_when_finished "chmod 775 ." && chmod a-w . && ... do the test ... Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-01-04t/t5537-fetch-shallow.sh: use the $( ... ) construct for command substitutionLibravatar Elia Pinto1-2/+2
The Git CodingGuidelines prefer the $(...) construct for command substitution instead of using the backquotes `...`. The backquoted form is the traditional method for command substitution, and is supported by POSIX. However, all but the simplest uses become complicated quickly. In particular, embedded command substitutions and/or the use of double quotes require careful escaping with the backslash character. The patch was generated by: for _f in $(find . -name "*.sh") do perl -i -pe 'BEGIN{undef $/;} s/`(.+?)`/\$(\1)/smg' "${_f}" done and then carefully proof-read. Signed-off-by: Elia Pinto <gitter.spiros@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-05-30t5537: re-drop http testsLibravatar Jeff King1-28/+0
These were originally removed by 0232852 (t5537: move http tests out to t5539, 2014-02-13). However, they were accidentally re-added in 1ddb4d7 (Merge branch 'nd/upload-pack-shallow', 2014-03-21). This looks like an error in manual conflict resolution. Here's what happened: 1. v1.9.0 shipped with the http tests in t5537. 2. We realized that this caused problems, and built 0232852 on top to move the tests to their own file. This fix made it into v1.9.1. 3. We later had another fix in nd/upload-pack-shallow that also touched t5537. It was built directly on v1.9.0. When we merged nd/upload-pack-shallow to master, we got a conflict; it was built on a version with the http tests, but we had since removed them. The correct resolution was to drop the http tests and keep the new ones, but instead we kept everything. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-03-11upload-pack: send shallow info over stdin to pack-objectsLibravatar Nguyễn Thái Ngọc Duy1-0/+13
Before cdab485 (upload-pack: delegate rev walking in shallow fetch to pack-objects - 2013-08-16) upload-pack does not write to the source repository. cdab485 starts to write $GIT_DIR/shallow_XXXXXX if it's a shallow fetch, so the source repo must be writable. git:// servers do not need write access to repos and usually don't have it, which means cdab485 breaks shallow clone over git:// Instead of using a temporary file as the media for shallow points, we can send them over stdin to pack-objects as well. Prepend shallow SHA-1 with --shallow so pack-objects knows what is what. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-01-09t5537: fix incorrect expectation in test case 10Libravatar Nguyễn Thái Ngọc Duy1-1/+2
Commit 48d25ca adds a new commit "7" to the repo that the next test case in commit 1609488 clones from. But the next test case does not expect this commit. For these tests, it's the bottom that's important, not the top. Fix the expected commit list. While at it, fix the default http port number to 5537. Otherwise when t5536 learns to test httpd, running test in parallel may fail. References: 48d25ca fetch: add --update-shallow to accept... - 2013-12-05 1609488 smart-http: support shallow fetch/clone - 2013-12-05 Noticed-by: Jeff King <peff@peff.net> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-12-10smart-http: support shallow fetch/cloneLibravatar Nguyễn Thái Ngọc Duy1-0/+27
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-12-10fetch: add --update-shallow to accept refs that update .git/shallowLibravatar Nguyễn Thái Ngọc Duy1-0/+32
The same steps are done as in when --update-shallow is not given. The only difference is we now add all shallow commits in "ours" and "theirs" to .git/shallow (aka "step 8"). Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-12-10upload-pack: make sure deepening preserves shallow rootsLibravatar Nguyễn Thái Ngọc Duy1-0/+16
When "fetch --depth=N" where N exceeds the longest chain of history in the source repo, usually we just send an "unshallow" line to the client so full history is obtained. When the source repo is shallow we need to make sure to "unshallow" the current shallow point _and_ "shallow" again when the commit reaches its shallow bottom in the source repo. This should fix both cases: large <N> and --unshallow. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-12-10fetch: support fetching from a shallow repositoryLibravatar Nguyễn Thái Ngọc Duy1-0/+128
This patch just put together pieces from the 8 steps patch. We stop at step 7 and reject refs that require new shallow commits. Note that, by rejecting refs that require new shallow commits, we leave dangling objects in the repo, which become "object islands" by the next "git fetch" of the same source. If the first fetch our "ours" set is zero and we do practically nothing at step 7, "ours" is full at the next fetch and we may need to walk through commits for reachability test. Room for improvement. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>