summaryrefslogtreecommitdiff
path: root/t/t5504-fetch-receive-strict.sh
AgeCommit message (Collapse)AuthorFilesLines
2019-05-16fsck: rename and touch up init_skiplist()Libravatar Barret Rhoden1-7/+7
init_skiplist() took a file consisting of SHA-1s and comments and added the objects to an oidset. This functionality is useful for other commands and will be moved to oidset.c in a future commit. In preparation for that move, this commit renames it to oidset_parse_file() to reflect its more generic usage and cleans up a few of the names. Signed-off-by: Barret Rhoden <brho@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-12fsck: support comments & empty lines in skipListLibravatar Ævar Arnfjörð Bjarmason1-3/+16
It's annoying not to be able to put comments and empty lines in the skipList, when e.g. keeping a big central list of commits to skip in /etc/gitconfig, which was my motivation for 1362df0d41 ("fetch: implement fetch.fsck.*", 2018-07-27). Implement that, and document what version of Git this was changed in, since this on-disk format can be expected to be used by multiple versions of git. There is no notable performance impact from this change, using the test setup described a couple of commits back: Test HEAD~ HEAD ---------------------------------------------------------------------------------------- 1450.3: fsck with 0 skipped bad commits 7.69(7.27+0.42) 7.86(7.48+0.37) +2.2% 1450.5: fsck with 1 skipped bad commits 7.69(7.30+0.38) 7.83(7.47+0.36) +1.8% 1450.7: fsck with 10 skipped bad commits 7.76(7.38+0.38) 7.79(7.38+0.41) +0.4% 1450.9: fsck with 100 skipped bad commits 7.76(7.38+0.38) 7.74(7.36+0.38) -0.3% 1450.11: fsck with 1000 skipped bad commits 7.71(7.30+0.41) 7.72(7.34+0.38) +0.1% 1450.13: fsck with 10000 skipped bad commits 7.74(7.34+0.40) 7.72(7.34+0.38) -0.3% 1450.15: fsck with 100000 skipped bad commits 7.75(7.40+0.35) 7.70(7.29+0.40) -0.6% 1450.17: fsck with 1000000 skipped bad commits 7.12(6.86+0.26) 7.13(6.87+0.26) +0.1% Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-12fsck: use strbuf_getline() to read skiplist fileLibravatar René Scharfe1-1/+1
The buffer is unlikely to contain a NUL character, so printing its contents using %s in a die() format is unsafe (detected with ASan). Use an idiomatic strbuf_getline() loop instead, which ensures the buffer is always NUL-terminated, supports CRLF files as well, accepts files without a newline after the last line, supports any hash length automatically, and is shorter. This fixes a bug where emitting an error about an invalid line on say line 1 would continue printing subsequent lines, and usually continue into uninitialized memory. The performance impact of this, on a CentOS 7 box with RedHat GCC 4.8.5-28: $ GIT_PERF_REPEAT_COUNT=5 GIT_PERF_MAKE_OPTS='-j56 CFLAGS="-O3"' ./run HEAD~ HEAD p1451-fsck-skip-list.sh Test HEAD~ HEAD ---------------------------------------------------------------------------------------- 1450.3: fsck with 0 skipped bad commits 7.75(7.39+0.35) 7.68(7.29+0.39) -0.9% 1450.5: fsck with 1 skipped bad commits 7.70(7.30+0.40) 7.80(7.42+0.37) +1.3% 1450.7: fsck with 10 skipped bad commits 7.77(7.37+0.40) 7.87(7.47+0.40) +1.3% 1450.9: fsck with 100 skipped bad commits 7.82(7.41+0.40) 7.88(7.43+0.44) +0.8% 1450.11: fsck with 1000 skipped bad commits 7.88(7.49+0.39) 7.84(7.43+0.40) -0.5% 1450.13: fsck with 10000 skipped bad commits 8.02(7.63+0.39) 8.07(7.67+0.39) +0.6% 1450.15: fsck with 100000 skipped bad commits 8.01(7.60+0.41) 8.08(7.70+0.38) +0.9% 1450.17: fsck with 1000000 skipped bad commits 7.60(7.10+0.50) 7.37(7.18+0.19) -3.0% Helped-by: Jeff King <peff@peff.net> Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-12fsck: document that skipList input must be unabbreviatedLibravatar Ævar Arnfjörð Bjarmason1-0/+6
Abbreviating the SHA-1s in the skipList input has never worked, but the documentation hasn't unambiguously stated that this is an error, and there was no test for it. Let's fix both since it would be easy for some later refactoring e.g. switch to accidentally switch to a looser OID parsing function, causing the tests before this change to pass, but for older versions of git to be incompatible with the new skipList format. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-12fsck: document and test commented & empty line skipList inputLibravatar Ævar Arnfjörð Bjarmason1-0/+21
There is currently no comment syntax for the fsck.skipList, this isn't really by design, and it would be nice to have support for comments. Document that this doesn't work, and test for how this errors out. These tests reveal a current bug, if there's invalid input the output will emit some of the next line, and then go into uninitialized memory. This is fixed in a subsequent change. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-12fsck: document and test sorted skipList inputLibravatar Ævar Arnfjörð Bjarmason1-0/+19
Ever since the skipList support was first added in cd94c6f91 ("fsck: git receive-pack: support excluding objects from fsck'ing", 2015-06-22) the documentation for the format has that the file is a sorted list of object names. Thus, anyone using the feature would have thought the list needed to be sorted. E.g. I recently in conjunction with my fetch.fsck.* implementation in 1362df0d41 ("fetch: implement fetch.fsck.*", 2018-07-27) wrote some code to ship a skipList, and went out of my way to sort it. Doing so seems intuitive, since it contains fixed-width records, and has no support for comments, so one might expect it to be binary searched in-place on-disk. However, as documented here this was never a requirement, so let's change the documentation. Since this is a file format change let's also document what was said about this in the past, so e.g. someone like myself reading the new docs can see this never needed to be sorted ("why do I have all this code to sort this thing..."). Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-12fsck tests: add a test for no skipList inputLibravatar Ævar Arnfjörð Bjarmason1-0/+5
The recent 65a836fa6b ("fsck: add stress tests for fsck.skipList", 2018-07-27) added various stress tests for odd invocations of fsck.skipList, but didn't tests for some very simple ones, such as asserting that providing to skipList with a bad commit causes fsck to exit with a non-zero exit code. Add such a test. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-12fsck tests: setup of bogus commit objectLibravatar Ævar Arnfjörð Bjarmason1-4/+4
Several fsck tests used the exact same git-hash-object output, but had copy/pasted that part of the setup code. Let's instead do that setup once and use it in subsequent tests. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-27fsck: test and document unknown fsck.<msg-id> valuesLibravatar Ævar Arnfjörð Bjarmason1-0/+14
When fsck.<msg-id> is set to an unknown value it'll cause "fsck" to die, but the same is not true of the "fetch" and "receive" variants. Document this and test for it. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-27fsck: add stress tests for fsck.skipListLibravatar Ævar Arnfjörð Bjarmason1-0/+28
Stress test the parsing logic shared by fsck.skipList and {fetch,receive}.fsck.skipList added in cd94c6f91e ("fsck: git receive-pack: support excluding objects from fsck'ing", 2015-06-22). There were no tests for the work done by the init_skiplist() routine, e.g. how it dies on invalid input. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-27fsck: test & document {fetch,receive}.fsck.* config fallbackLibravatar Ævar Arnfjörð Bjarmason1-2/+24
Test and document that the {fetch,receive}.fsck.* family of variables doesn't fall back on the corresponding .fsck.* variables. This was alluded to in the existing documentation by saying that "receive" looks at receive.fsck.* and "fsck" looks at fsck.* etc., but it wasn't explicitly stated that there was no fallback, and if you'd e.g. like to configure the skipList you need to do that for all three. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-27fetch: implement fetch.fsck.*Libravatar Ævar Arnfjörð Bjarmason1-0/+46
Implement support for fetch.fsck.* corresponding with the existing receive.fsck.*. This allows for pedantically cloning repositories with specific issues without turning off fetch.fsckObjects. One such repository is https://github.com/robbyrussell/oh-my-zsh.git which before this change will emit this error when cloned with fetch.fsckObjects: error: object 2b7227859263b6aabcc28355b0b994995b7148b6: zeroPaddedFilemode: contains zero-padded file modes fatal: Error in object fatal: index-pack failed Now with fetch.fsck.zeroPaddedFilemode=warn we'll warn about that issue, but the clone will succeed: warning: object 2b7227859263b6aabcc28355b0b994995b7148b6: zeroPaddedFilemode: contains zero-padded file modes warning: object a18c4d13c2a5fa2d4ecd5346c50e119b999b807d: zeroPaddedFilemode: contains zero-padded file modes warning: object 84df066176c8da3fd59b13731a86d90f4f1e5c9d: zeroPaddedFilemode: contains zero-padded file modes The motivation for this is to be able to turn on fetch.fsckObjects globally across a fleet of computers but still be able to manually clone various legacy repositories by either white-listing specific issues, or better yet whitelist specific objects. The use of --git-dir=* instead of -C in the tests could be considered somewhat archaic, but the tests I'm adding here are duplicating the corresponding receive.* tests with as few changes as possible. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-27transfer.fsckObjects tests: untangle confusing setupLibravatar Ævar Arnfjörð Bjarmason1-1/+11
The tests for transfer.fsckObjects have grown organically over time to not make much sense. Initially when these were added in b10a53583f ("test: fetch/receive with fsckobjects", 2011-09-04) they were only testing the "corrupt or missing object" case, but later on in 70a4ae73d8 ("fsck: add a simple test for receive.fsck.<msg-id>", 2015-06-22) they were expanded to check for the fsck.<msg-id> feature. The problem was that we still kept the same corrupt test repo, making it harder to add new tests that check the entirety of the repository between operations via "git fsck" to see whether only known issues that can be ignored with fsck.<msg-id> have occurred. The tests only did the right thing because such a full "git fsck" was never done after a certain point, and instead we were only manipulating specific refs. This makes it harder to add new tests, and none of the fsck.<msg-id> tests relied on this. So let's not confuse the two and repair the corrupt repository before we run the fsck.<msg-id> tests. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-27receive.fsck.<msg-id> tests: remove dead codeLibravatar Ævar Arnfjörð Bjarmason1-2/+0
Remove the setting of a receive.fsck.badDate config variable to "ignore". This was added in efaba7cc77 ("fsck: optionally ignore specific fsck issues completely", 2015-06-22) but never did anything, presumably it was part of some work-in-progress code that never made it into git.git. None of these tests will emit the "invalid author/committer line - bad date" warning. The dates on the commit objects we're setting up are not invalid. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-01-07don't use test_must_fail with grepLibravatar Pranit Bauva1-1/+1
test_must_fail should only be used for testing git commands. To test the failure of other commands use `!`. Reported-by: Stefan Beller <sbeller@google.com> Signed-off-by: Pranit Bauva <pranit.bauva@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-18test-lib.sh: introduce and use $EMPTY_TREELibravatar Nguyễn Thái Ngọc Duy1-2/+2
This is a special SHA1. Let's keep it at one place, easier to replace later when the hash change comes, easier to recognize. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-04-20t5504: drop sigpipe=ok from push testsLibravatar Jeff King1-6/+4
These were added by 8bf4bec (add "ok=sigpipe" to test_must_fail and use it to fix flaky tests, 2015-11-27) because we would racily die via SIGPIPE when the pack was rejected by the other side. But since we have recently de-flaked send-pack, we should be able to tighten up these tests (including re-adding the expected output checks). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-02-25t5504: handle expected output from SIGPIPE deathLibravatar Jeff King1-1/+4
Commit 8bf4bec (add "ok=sigpipe" to test_must_fail and use it to fix flaky tests, 2015-11-27) taught t5504 to handle "git push" racily exiting with SIGPIPE rather than failing. However, one of the tests checks the output of the command, as well. In the SIGPIPE case, we will not have produced any output. If we want the test to be truly non-flaky, we have to accept either output. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-11-28add "ok=sigpipe" to test_must_fail and use it to fix flaky testsLibravatar Lars Schneider1-3/+2
t5516 "75 - deny fetch unreachable SHA1, allowtipsha1inwant=true" is flaky in the following case: 1. remote upload-pack finds out "not our ref" 2. remote sends a response and closes the pipe 3. fetch-pack still tries to write commands to the remote upload-pack 4. write call in wrapper.c dies with SIGPIPE The test is flaky because the sending fetch-pack may or may not have finished writing its output by step (3). If it did, then we see a closed pipe on the next read() call. If it didn't, then we get the SIGPIPE from step (4) above. Both are fine, but the latter fools test_must_fail. t5504 "9 - push with transfer.fsckobjects" is flaky, too, and returns SIGPIPE once in a while. I had to remove the final "To dst..." output check because there is no output if the process dies with SIGPIPE. Accept such a death-with-sigpipe also as OK when we are expecting a failure. Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Jeff King <peff@peff.net>
2015-06-23fsck: git receive-pack: support excluding objects from fsck'ingLibravatar Johannes Schindelin1-0/+12
The optional new config option `receive.fsck.skipList` specifies the path to a file listing the names, i.e. SHA-1s, one per line, of objects that are to be ignored by `git receive-pack` when `receive.fsckObjects = true`. This is extremely handy in case of legacy repositories where it would cause more pain to change incorrect objects than to live with them (e.g. a duplicate 'author' line in an early commit object). The intended use case is for server administrators to inspect objects that are reported by `git push` as being too problematic to enter the repository, and to add the objects' SHA-1 to a (preferably sorted) file when the objects are legitimate, i.e. when it is determined that those problematic objects should be allowed to enter the server. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-06-23fsck: optionally ignore specific fsck issues completelyLibravatar Johannes Schindelin1-1/+8
An fsck issue in a legacy repository might be so common that one would like not to bother the user with mentioning it at all. With this change, that is possible by setting the respective message type to "ignore". This change "abuses" the missingEmail=warn test to verify that "ignore" is also accepted and works correctly. And while at it, it makes sure that multiple options work, too (they are passed to unpack-objects or index-pack as a comma-separated list via the --strict=... command-line option). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-06-23fsck: disallow demoting grave fsck errors to warningsLibravatar Johannes Schindelin1-0/+11
Some kinds of errors are intrinsically unrecoverable (e.g. errors while uncompressing objects). It does not make sense to allow demoting them to mere warnings. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-06-23fsck: add a simple test for receive.fsck.<msg-id>Libravatar Johannes Schindelin1-0/+21
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-09-21receive-pack: drop "n/a" on unpacker errorsLibravatar Jeff King1-1/+1
The output from git push currently looks like this: $ git push dest HEAD fatal: [some message from index-pack] error: unpack failed: index-pack abnormal exit To dest ! [remote rejected] HEAD -> master (n/a (unpacker error)) That n/a is meant to be "the per-ref status is not available" but the nested parentheses just make it look ugly. Let's turn the final line into just: ! [remote rejected] HEAD -> master (unpacker error) Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-02-13do not override receive-pack errorsLibravatar Clemens Buchacher1-4/+18
Receive runs rev-list --verify-objects in order to detect missing objects. However, such errors are ignored and overridden later. Instead, consequently ignore all update commands for which an error has already been detected. Some tests in t5504 are obsoleted by this change, because invalid objects are detected even if fsck is not enabled. Instead, they now test for different error messages depending on whether or not fsck is turned on. A better fix would be to force a corruption that will be detected by fsck but not by rev-list. Signed-off-by: Clemens Buchacher <drizzd@aon.at> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-10-05Merge branch 'jc/fetch-verify'Libravatar Junio C Hamano1-2/+2
* jc/fetch-verify: fetch: verify we have everything we need before updating our ref rev-list --verify-object list-objects: pass callback data to show_objects()
2011-09-04test: fetch/receive with fsckobjectsLibravatar Junio C Hamano1-0/+104
Add tests for the new fetch.fsckobjects, and also tests for receive.fsckobjects we have had for quite some time. Signed-off-by: Junio C Hamano <gitster@pobox.com>