summaryrefslogtreecommitdiff
path: root/t/README
AgeCommit message (Collapse)AuthorFilesLines
2018-10-30test-lib: introduce the '-V' short option for '--verbose-log'Libravatar SZEDER Gábor1-0/+1
'--verbose-log' is one of the most useful and thus most frequently used test options, but due to its length it's a pain to type on the command line. Let's introduce the corresponding short option '-V' to save some keystrokes. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-23pack-objects: fix performance issues on packing large deltasLibravatar Nguyễn Thái Ngọc Duy1-0/+4
Let's start with some background about oe_delta_size() and oe_set_delta_size(). If you already know, skip the next paragraph. These two are added in 0aca34e826 (pack-objects: shrink delta_size field in struct object_entry - 2018-04-14) to help reduce 'struct object_entry' size. The delta size field in this struct is reduced to only contain max 1MB. So if any new delta is produced and larger than 1MB, it's dropped because we can't really save such a large size anywhere. Fallback is provided in case existing packfiles already have large deltas, then we can retrieve it from the pack. While this should help small machines repacking large repos without large deltas (i.e. less memory pressure), dropping large deltas during the delta selection process could end up with worse pack files. And if existing packfiles already have >1MB delta and pack-objects is instructed to not reuse deltas, all of them will be dropped on the floor, and the resulting pack would be definitely bigger. There is also a regression in terms of CPU/IO if we have large on-disk deltas because fallback code needs to parse the pack every time the delta size is needed and just access to the mmap'd pack data is enough for extra page faults when memory is under pressure. Both of these issues were reported on the mailing list. Here's some numbers for comparison. Version Pack (MB) MaxRSS(kB) Time (s) ------- --------- ---------- -------- 2.17.0 5498 43513628 2494.85 2.18.0 10531 40449596 4168.94 This patch provides a better fallback that is - cheaper in terms of cpu and io because we won't have to read existing pack files as much - better in terms of pack size because the pack heuristics is back to 2.17.0 time, we do not drop large deltas at all If we encounter any delta (on-disk or created during try_delta phase) that is larger than the 1MB limit, we stop using delta_size_ field for this because it can't contain such size anyway. A new array of delta size is dynamically allocated and can hold all the deltas that 2.17.0 can. This array only contains delta sizes that delta_size_ can't contain. With this, we do not have to drop deltas in try_delta() anymore. Of course the downside is we use slightly more memory, even compared to 2.17.0. But since this is considered an uncommon case, a bit more memory consumption should not be a problem. Delta size limit is also raised from 1MB to 16MB to better cover common case and avoid that extra memory consumption (99.999% deltas in this reported repo are under 12MB; Jeff noted binary artifacts topped out at about 3MB in some other private repos). Other fields are shuffled around to keep this struct packed tight. We don't use more memory in common case even with this limit update. A note about thread synchronization. Since this code can be run in parallel during delta searching phase, we need a mutex. The realloc part in packlist_alloc() is not protected because it only happens during the object counting phase, which is always single-threaded. Access to e->delta_size_ (and by extension pack->delta_size[e - pack->objects]) is unprotected as before, the thread scheduler in pack-objects must make sure "e" is never updated by two different threads. The area under the new lock is as small as possible, avoiding locking at all in common case, since lock contention with high thread count could be expensive (most blobs are small enough that delta compute time is short and we end up taking the lock very often). The previous attempt to always hold a lock in oe_delta_size() and oe_set_delta_size() increases execution time by 33% when repacking linux.git with with 40 threads. Reported-by: Elijah Newren <newren@gmail.com> Helped-by: Elijah Newren <newren@gmail.com> Helped-by: Jeff King <peff@peff.net> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-04-16pack-objects: shrink size field in struct object_entryLibravatar Nguyễn Thái Ngọc Duy1-0/+6
It's very very rare that an uncompressed object is larger than 4GB (partly because Git does not handle those large files very well to begin with). Let's optimize it for the common case where object size is smaller than this limit. Shrink size field down to 31 bits and one overflow bit. If the size is too large, we read it back from disk. As noted in the previous patch, we need to return the delta size instead of canonical size when the to-be-reused object entry type is a delta instead of a canonical one. Add two compare helpers that can take advantage of the overflow bit (e.g. if the file is 4GB+, chances are it's already larger than core.bigFileThreshold and there's no point in comparing the actual value). Another note about oe_get_size_slow(). This function MUST be thread safe because SIZE() macro is used inside try_delta() which may run in parallel. Outside parallel code, no-contention locking should be dirt cheap (or insignificant compared to i/o access anyway). To exercise this code, it's best to run the test suite with something like make test GIT_TEST_OE_SIZE=4 which forces this code on all objects larger than 3 bytes. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-04-16pack-objects: move in_pack out of struct object_entryLibravatar Nguyễn Thái Ngọc Duy1-0/+5
Instead of using 8 bytes (on 64 bit arch) to store a pointer to a pack. Use an index instead since the number of packs should be relatively small. This limits the number of packs we can handle to 1k. Since we can't be sure people can never run into the situation where they have more than 1k pack files. Provide a fall back route for it. If we find out they have too many packs, the new in_pack_by_idx[] array (which has at most 1k elements) will not be used. Instead we allocate in_pack[] array that holds nr_objects elements. This is similar to how the optional in_pack_pos field is handled. The new simple test is just to make sure the too-many-packs code path is at least executed. The true test is running make test GIT_TEST_FULL_IN_PACK_ARRAY=1 to take advantage of other special case tests. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-04-16read-cache.c: make $GIT_TEST_SPLIT_INDEX booleanLibravatar Nguyễn Thái Ngọc Duy1-0/+11
While at there, document about this special mode when running the test suite. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-03-14Merge branch 'sg/test-x'Libravatar Junio C Hamano1-3/+20
Running test scripts under -x option of the shell is often not a useful way to debug them, because the error messages from the commands tests try to capture and inspect are contaminated by the tracing output by the shell. An earlier work done to make it more pleasant to run tests under -x with recent versions of bash is extended to cover posix shells that do not support BASH_XTRACEFD. * sg/test-x: travis-ci: run tests with '-x' tracing t/README: add a note about don't saving stderr of compound commands t1510-repo-setup: mark as untraceable with '-x' t9903-bash-prompt: don't check the stderr of __git_ps1() t5570-git-daemon: don't check the stderr of a subshell t5526: use $TRASH_DIRECTORY to specify the path of GIT_TRACE log file t5500-fetch-pack: don't check the stderr of a subshell t3030-merge-recursive: don't check the stderr of a subshell t1507-rev-parse-upstream: don't check the stderr of a shell function t: add means to disable '-x' tracing for individual test scripts t: prevent '-x' tracing from interfering with test helpers' stderr
2018-02-28t/README: add a note about don't saving stderr of compound commandsLibravatar SZEDER Gábor1-3/+17
Explain in 't/README' why it is a bad idea to redirect and verify the stderr of compound commands, in the hope that future contributions will follow this advice and the test suite will keep working with '-x' tracing and /bin/sh. While at it, since we can now run the test suite with '-x' without needing a Bash version supporting BASH_XTRACEFD, remove the now outdated caution note about non-Bash shells from the description of the '-x' option. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-02-27t: add means to disable '-x' tracing for individual test scriptsLibravatar SZEDER Gábor1-0/+3
The previous patch resolved most of the test failures caused by running our test suite with '-x' tracing and /bin/sh, and the following patches in this series will resolve almost all of the remaining failures. Unfortunately, not yet all. Add means to disable '-x' tracing for individual test scripts by setting the $test_untraceable variable to a non-empty value in the test script before sourcing 'test-lib.sh'. However, since '-x' tracing is not an issue with recent Bash versions supporting BASH_XTRACEFD, i.e. v4.1 and later, don't disable tracing when the test script is run with such a Bash version even when $test_untraceable is set. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-02-27Merge branch 'sg/doc-test-must-fail-args'Libravatar Junio C Hamano1-2/+12
Devdoc update. * sg/doc-test-must-fail-args: t: document 'test_must_fail ok=<signal-name>'
2018-02-12t: document 'test_must_fail ok=<signal-name>'Libravatar SZEDER Gábor1-2/+12
Since 'test_might_fail' is implemented as a thin wrapper around 'test_must_fail', it also accepts the same options. Mention this in the docs as well. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-12-13Merge branch 'tg/t-readme-updates'Libravatar Junio C Hamano1-4/+6
Developer doc updates. * tg/t-readme-updates: t/README: document test_cmp_rev t/README: remove mention of adding copyright notices
2017-11-27t/README: document test_cmp_revLibravatar Thomas Gummerer1-0/+5
test_cmp_rev is a useful function that's used in quite a few test scripts. It is however not documented in t/README. Document it. Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-11-27t/README: remove mention of adding copyright noticesLibravatar Thomas Gummerer1-4/+1
We generally no longer include copyright notices in new test scripts. However t/README still mentions it as something to include at the top of every new script. Remove that mention as it's outdated. Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-11-24test-lib: add LIBPCRE1 & LIBPCRE2 prerequisitesLibravatar Ævar Arnfjörð Bjarmason1-0/+12
Add LIBPCRE1 and LIBPCRE2 prerequisites which are true when git is compiled with USE_LIBPCRE1=YesPlease or USE_LIBPCRE2=YesPlease, respectively. The syntax of PCRE1 and PCRE2 isn't the same in all cases (see pcresyntax(3) and pcre2syntax(3)). If test are added that test for those they'll need to be guarded by these new prerequisites. The subsequent patch will make use of LIBPCRE2, so LIBPCRE1 isn't strictly needed for now, but let's add it for consistency and so that checking for it doesn't have to be done with the less obvious "PCRE, !LIBPCRE2", which while semantically the same is more confusing, and would lead to bugs if PCRE v3 is ever released as the tests would mean v1, not any non-v2 version. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-09-19t/README: fix typo and grammatically improve a sentenceLibravatar Kaartic Sivaraam1-3/+3
Signed-off-by: Kaartic Sivaraam <kaarticsivaraam91196@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-26test-lib: add a PTHREADS prerequisiteLibravatar Ævar Arnfjörð Bjarmason1-0/+4
Add a PTHREADS prerequisite which is false when git is compiled with NO_PTHREADS=YesPlease. There's lots of custom code that runs when threading isn't available, but before this prerequisite there was no way to test it. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-21test-lib: rename the LIBPCRE prerequisite to PCRELibravatar Ævar Arnfjörð Bjarmason1-2/+2
Rename the LIBPCRE prerequisite to PCRE. This is for preparation for libpcre2 support, where having just "LIBPCRE" would be confusing as it implies v1 of the library. None of these tests are incompatible between versions 1 & 2 of libpcre, it's less confusing to give them a more general name to make it clear that they work on both library versions. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-26t/README: clarify the test_have_prereq documentationLibravatar Ævar Arnfjörð Bjarmason1-3/+5
Clarify the test_have_prereq documentation so that it's clear in the reader's mind when the text says "most common use of this directly" what the answer to "as opposed to what?" is. Usually this function isn't used in lieu of using the prerequisite support built into test_expect_*, mention that explicitly. This changes documentation that I added in commit 9a897893a7 ("t/README: Document the prereq functions, and 3-arg test_*", 2010-07-02). Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-22t/README: change "Inside <X> part" to "Inside the <X> part"Libravatar Ævar Arnfjörð Bjarmason1-1/+1
Change the mention of "Inside the <script> part, the standard output..." to use the definite article, which makes more sense in this context. This changes documentation I originally added back in commit 20873f45e7 ("t/README: Document the do's and don'ts of tests", 2010-07-02). Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-22t/README: link to metacpan.org, not search.cpan.orgLibravatar Ævar Arnfjörð Bjarmason1-1/+1
Change a link to the web version of the TAP::Parser::Grammar documentation to link to metacpan.org instead of search.cpan.org. This is something I added back in commit 20873f45e7 ("t/README: Document the do's and don'ts of tests", 2010-07-02), at the time search.cpan.org was the more actively maintained CPAN web-interface, nowadays that's metacpan.org. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-10-26Merge branch 'jk/tap-verbose-fix'Libravatar Junio C Hamano1-0/+6
The Travis CI configuration we ship ran the tests with --verbose option but this risks non-TAP output that happens to be "ok" to be misinterpreted as TAP signalling a test that passed. This resulted in unnecessary failure. This has been corrected by introducing a new mode to run our tests in the test harness to send the verbose output separately to the log file. * jk/tap-verbose-fix: test-lib: bail out when "-v" used under "prove" travis: use --verbose-log test option test-lib: add --verbose-log option test-lib: handle TEST_OUTPUT_DIRECTORY with spaces
2016-10-21test-lib: add --verbose-log optionLibravatar Jeff King1-0/+6
The "--verbose" option redirects output from arbitrary test commands to stdout. This is useful for examining the output manually, like: ./t5547-push-quarantine.sh -v | less But it also means that the output is intermingled with the TAP directives, which can confuse a TAP parser like "prove". This has always been a potential problem, but became an issue recently when one test happened to output the word "ok" on a line by itself, which prove interprets as a test success: $ prove t5547-push-quarantine.sh :: -v t5547-push-quarantine.sh .. 1/? To dest.git * [new branch] HEAD -> master To dest.git ! [remote rejected] reject -> reject (pre-receive hook declined) error: failed to push some refs to 'dest.git' fatal: git cat-file d08c8eba97f4e683ece08654c7c8d2ba0c03b129: bad file t5547-push-quarantine.sh .. Failed -1/4 subtests Test Summary Report ------------------- t5547-push-quarantine.sh (Wstat: 0 Tests: 5 Failed: 0) Parse errors: Tests out of sequence. Found (2) but expected (3) Tests out of sequence. Found (3) but expected (4) Tests out of sequence. Found (4) but expected (5) Bad plan. You planned 4 tests but ran 5. Files=1, Tests=5, 0 wallclock secs ( 0.01 usr + 0.01 sys = 0.02 CPU) Result: FAIL One answer is "if it hurts, don't do it", but that's not quite the whole story. The Travis tests use "--verbose --tee" so that they can get the benefit of prove's parallel options, along with a verbose log in case there is a failure. We just need the verbose output to go to the log, but keep stdout clean. Getting this right turns out to be surprisingly difficult. Here's the progression of alternatives I considered: 1. Add an option to write verbose output to stderr. This is hard to capture, though, because we want each test to have its own log (because they're all run in parallel and the jumbled output would be useless). 2. Add an option to write verbose output to a file in test-results. This works, but the log is missing all of the non-verbose output, which gives context. 3. Like (2), but teach say_color() to additionally output to the log. This mostly works, but misses any output that happens outside of the say() functions (which isn't a lot, but is a potential maintenance headache). 4. Like (2), but make the log file the same as the "--tee" file. That almost works, but now we have two processes opening the same file. That gives us two separate descriptors, each with their own idea of the current position. They'll each start writing at offset 0, and overwrite each other's data. 5. Like (4), but in each case open the file for appending. That atomically positions each write at the end of the file. It's possible we may still get sheared writes between the two processes, but this is already the case when writing to stdout. It's not a problem in practice because the test harness generally waits for snippets to finish before writing the TAP output. We can ignore buffering issues with tee, because POSIX mandates that it does not buffer. Likewise, POSIX specifies "tee -a", so it should be available everywhere. This patch implements option (5), which seems to work well in practice. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-11Spelling fixesLibravatar Ville Skyttä1-1/+1
<BAD> <CORRECTED> accidently accidentally commited committed dependancy dependency emtpy empty existance existence explicitely explicitly git-upload-achive git-upload-archive hierachy hierarchy indegee indegree intial initial mulitple multiple non-existant non-existent precendence. precedence. priviledged privileged programatically programmatically psuedo-binary pseudo-binary soemwhere somewhere successfull successful transfering transferring uncommited uncommitted unkown unknown usefull useful writting writing Signed-off-by: Ville Skyttä <ville.skytta@iki.fi> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-31Merge branch 'jk/test-send-sh-x-trace-elsewhere' into maintLibravatar Junio C Hamano1-3/+3
Running tests with '-x' option to trace the individual command executions is a useful way to debug test scripts, but some tests that capture the standard error stream and check what the command said can be broken with the trace output mixed in. When running our tests under "bash", however, we can redirect the trace output to another file descriptor to keep the standard error of programs being tested intact. * jk/test-send-sh-x-trace-elsewhere: test-lib: set BASH_XTRACEFD automatically
2016-05-17Merge branch 'jk/test-send-sh-x-trace-elsewhere'Libravatar Junio C Hamano1-3/+3
Running tests with '-x' option to trace the individual command executions is a useful way to debug test scripts, but some tests that capture the standard error stream and check what the command said can be broken with the trace output mixed in. When running our tests under "bash", however, we can redirect the trace output to another file descriptor to keep the standard error of programs being tested intact. * jk/test-send-sh-x-trace-elsewhere: test-lib: set BASH_XTRACEFD automatically
2016-05-11test-lib: set BASH_XTRACEFD automaticallyLibravatar Jeff King1-3/+3
Passing "-x" to a test script enables the shell's "set -x" tracing, which can help with tracking down the command that is causing a failure. Unfortunately, it can also _cause_ failures in some tests that redirect the stderr of a shell function. Inside the function the shell continues to respect "set -x", and the trace output is collected along with whatever stderr is generated normally by the function. You can see an example of this by running: ./t0040-parse-options.sh -x -i which will fail immediately in the first test, as it expects: test_must_fail some-cmd 2>output.err to leave output.err empty (but with "-x" it has our trace output). Unfortunately there isn't a portable or scalable solution to this. We could teach test_must_fail to disable "set -x", but that doesn't help any of the other functions or subshells. However, we can work around it by pointing the "set -x" output to our descriptor 4, which always points to the original stderr of the test script. Unfortunately this only works for bash, but it's better than nothing (and other shells will just ignore the BASH_XTRACEFD variable). The patch itself is a simple one-liner, but note the caveats in the accompanying comments. Automatic tests for our "-x" option may be a bit too meta (and a pain, because they are bash-specific), but I did confirm that it works correctly both with regular "-x" and with "--verbose-only=1". This works because the latter flips "set -x" off and on for particular tests (if it didn't, we would get tracing for all tests, as going to descriptor 4 effectively circumvents the verbose flag). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-10-30test: facilitate debugging Git executables in tests with gdbLibravatar Johannes Schindelin1-0/+5
When prefixing a Git call in the test suite with 'debug ', it will now be run with GDB, allowing the developer to debug test failures more conveniently. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-03-20t/test-lib: introduce --chain-lint optionLibravatar Jeff King1-0/+10
It's easy to miss an "&&"-chain in a test script, like: test_expect_success 'check something important' ' cmd1 && cmd2 cmd3 ' The test harness will notice if cmd3 fails, but a failure of cmd1 or cmd2 will go unnoticed, as their exit status is lost after cmd3 runs. The toy example above is easy to spot because the "cmds" are all the same length, but real code is much more complicated. It's also difficult to detect these situations by statically analyzing the shell code with regexps (like the check-non-portable-shell script does); there's too much context required to know whether a &&-chain is appropriate on a given line or not. This patch instead lets the shell check each test by sticking a command with a specific and unusual return code at the top of each test, like: (exit 117) && cmd1 && cmd2 cmd3 In a well-formed test, the non-zero exit from the first command prevents any of the rest from being run, and the test's exit code is 117. In a bad test (like the one above), the 117 is lost, and cmd3 is run. When we encounter a failure of this check, we abort the test script entirely. For one thing, we have no clue which subset of the commands in the test snippet were actually run. Running further tests would be pointless, because we're now in an unknown state. And two, this is not a "test failure" in the traditional sense. The test script is buggy, not the code it is testing. We should be able to fix these problems in the script once, and not have them come back later as a regression in git's code. After checking a test snippet for --chain-lint, we do still run the test itself. We could actually have a pure-lint mode which just checks each test, but there are a few reasons not to. One, because the tests are executing arbitrary code, which could impact the later environment (e.g., that could impact which set of tests we run at all). And two, because a pure-lint mode would still be expensive to run, because a significant amount of code runs outside of the test_expect_* blocks. Instead, this option is designed to be used as part of a normal test suite run, where it adds very little overhead. Turning on this option detects quite a few problems in existing tests, which will be fixed in subsequent patches. However, there are a number of places it cannot reach: - it cannot find a failure to break out of loops on error, like: cmd1 && for i in a b c; do cmd2 $i done && cmd3 which will not notice failures of "cmd2 a" or "cmd b" - it cannot find a missing &&-chain inside a block or subfunction, like: foo () { cmd1 cmd2 } foo && bar which will not notice a failure of cmd1. - it only checks tests that you run; every platform will have some tests skipped due to missing prequisites, so it's impossible to say from one run that the test suite is free of broken &&-chains. However, all tests get run by _somebody_, so eventually we will notice problems. - it does not operate on test_when_finished or prerequisite blocks. It could, but these tends to be much shorter and less of a problem, so I punted on them in this patch. This patch was inspired by an earlier patch by Jonathan Nieder: http://article.gmane.org/gmane.comp.version-control.git/235913 This implementation and all bugs are mine. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-12-22Merge branch 'jc/refer-to-t-readme-from-submitting-patches'Libravatar Junio C Hamano1-1/+2
* jc/refer-to-t-readme-from-submitting-patches: t/README: justify why "! grep foo" is sufficient SubmittingPatches: refer to t/README for tests
2014-11-24t/README: justify why "! grep foo" is sufficientLibravatar Junio C Hamano1-1/+2
We require use of test_must_fail to check expected non-zero exit by Git itself, but discourage test_must_fail to be used for checking exit status of non Git commands that are supplied by the system. The current text explains the reason for the former but not the latter. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-10-13test-lib.sh: support -x option for shell-tracingLibravatar Jeff King1-0/+6
Usually running a test under "-v" makes it clear which command is failing. However, sometimes it can be useful to also see a complete trace of the shell commands being run in the test. You can do so without any support from the test suite by running "sh -x tXXXX-foo.sh". However, this produces quite a large bit of output, as we see a trace of the entire test suite. This patch instead introduces a "-x" option to the test scripts (i.e., "./tXXXX-foo.sh -x"). When enabled, this turns on "set -x" only for the tests themselves. This can still be a bit verbose, but should keep things to a more manageable level. You can even use "--verbose-only" to see the trace only for a specific test. The implementation is a little invasive. We turn on the "set -x" inside the "eval" of the test code. This lets the eval itself avoid being reported in the trace (which would be long, and redundant with the verbose listing we already showed). And then after the eval runs, we do some trickery with stderr to avoid showing the "set +x" to the user. We also show traces for test_cleanup functions (since they can impact the test outcome, too). However, we do avoid running the noop ":" cleanup (the default if the test does not use test_cleanup at all), as it creates unnecessary noise in the "set -x" output. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-16Merge branch 'ib/test-selectively-run'Libravatar Junio C Hamano1-4/+81
Allow specifying only certain individual test pieces to be run using a range notation (e.g. "t1234-test.sh --run='1-4 6 8 9-'"). * ib/test-selectively-run: t0000-*.sh: fix the GIT_SKIP_TESTS sub-tests test-lib: '--run' to run only specific tests test-lib: tests skipped by GIT_SKIP_TESTS say so test-lib: document short options in t/README
2014-06-10test doc: test_write_lines does not split its argumentsLibravatar Jonathan Nieder1-4/+3
test_write_lines carefully quotes its arguments as "$@", so test_write_lines "a b" c writes two lines as requested, not three. Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-10test: add test_write_lines helperLibravatar Michael S. Tsirkin1-0/+22
API and implementation as suggested by Junio. Signed-off-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-06test-lib: '--run' to run only specific testsLibravatar Ilya Bobyr1-4/+77
Allow better control of the set of tests that will be executed for a single test suite. Mostly useful while debugging or developing as it allows to focus on a specific test. Signed-off-by: Ilya Bobyr <ilya.bobyr@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-06test-lib: document short options in t/READMELibravatar Ilya Bobyr1-0/+4
Most arguments that could be provided to a test have short forms. Unless documented, the only way to learn them is to read the code. Signed-off-by: Ilya Bobyr <ilya.bobyr@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-11-04Merge branch 'jk/wrap-perl-used-in-tests'Libravatar Junio C Hamano1-4/+8
* jk/wrap-perl-used-in-tests: t: use perl instead of "$PERL_PATH" where applicable t: provide a perl() function which uses $PERL_PATH
2013-10-29t: provide a perl() function which uses $PERL_PATHLibravatar Jeff King1-4/+8
Once upon a time, we assumed that calling a bare "perl" in the test scripts was OK, because we would find the perl from the user's PATH, and we were only asking that perl to do basic operations that work even on old versions of perl. Later, we found that some systems really prefer to use $PERL_PATH even for these basic cases, because the system perl misbehaves in some way (e.g., by handling line endings differently). We then switched "perl" invocations to "$PERL_PATH" to respect the user's choice. Having to use "$PERL_PATH" is ugly and cumbersome, though. Instead, let's provide a perl() shell function that tests can use, which will transparently do the right thing. Unfortunately, test writers still have to use $PERL_PATH in certain situations, so we still need to keep the advice in the README. Note that this may fix test failures in t5004, t5503, t6002, t6003, t6300, t8001, and t8002, depending on your system's perl setup. All of these can be detected by running: ln -s /bin/false bin-wrappers/perl make test which fails before this patch, and passes after. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-10-28t/README: tests can use perl even with NO_PERLLibravatar Jonathan Nieder1-4/+11
The git build system supports a NO_PERL switch to avoid installing perl bindings or other features (like "git add --patch") that rely on perl on runtime, but even with NO_PERL it has not been possible for a long time to run tests without perl. Helpers such as nul_to_q () { "$PERL_PATH" -pe 'y/\000/Q/' } use perl as a better tr or sed and are regularly used in tests without worrying to add a PERL prerequisite. Perl is portable enough that it seems fine to keep relying on it for this kind of thing in tests (and more readable than the alternative of trying to find POSIXy equivalents). Update the test documentation to clarify this. Reported-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-07-05Merge branch 'tr/test-v-and-v-subtest-only'Libravatar Junio C Hamano1-0/+10
Allows N instances of tests run in parallel, each running 1/N parts of the test suite under Valgrind, to speed things up. * tr/test-v-and-v-subtest-only: perf-lib: fix start/stop of perf tests test-lib: support running tests under valgrind in parallel test-lib: allow prefixing a custom string before "ok N" etc. test-lib: valgrind for only tests matching a pattern test-lib: verbose mode for only tests matching a pattern test-lib: self-test that --verbose works test-lib: rearrange start/end of test_expect_* and test_skip test-lib: refactor $GIT_SKIP_TESTS matching test-lib: enable MALLOC_* for the actual tests
2013-06-23test-lib: valgrind for only tests matching a patternLibravatar Thomas Rast1-0/+5
With the new --valgrind-only=<pattern> option, one can enable --valgrind at a per-test granularity, exactly analogous to --verbose-only from the previous commit. The options are wired such that --valgrind implies --verbose (as before), but --valgrind-only=<pattern> implies --verbose-only=<pattern> unless --verbose is also in effect. Signed-off-by: Thomas Rast <trast@inf.ethz.ch> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-06-23test-lib: verbose mode for only tests matching a patternLibravatar Thomas Rast1-0/+5
With the new --verbose-only=<pattern> option, one can enable --verbose at a per-test granularity. The pattern is matched against the test number, e.g. ./t0000-basic.sh --verbose-only='2[0-2]' to see only the full output of test 20-22, while showing the rest in the one-liner format. As suggested by Jeff King, this takes care to wrap the entire test_expect_* block, but nothing else, in the verbose toggling. We can use the test_start/end functions from the previous commit for the purpose. This is arguably not *too* useful on its own, but makes the next patch easier to follow. Helped-by: Jeff King <peff@peff.net> Signed-off-by: Thomas Rast <trast@inf.ethz.ch> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-06-20Merge branch 'js/test-ln-s-add'Libravatar Junio C Hamano1-0/+14
Many tests that check the behaviour of symbolic links stored in the index or the tree objects do not have to be skipped on a filesystem that lack symbolic link support. * js/test-ln-s-add: t4011: remove SYMLINKS prerequisite t6035: use test_ln_s_add to remove SYMLINKS prerequisite t3509, t4023, t4114: use test_ln_s_add to remove SYMLINKS prerequisite t3100: use test_ln_s_add to remove SYMLINKS prerequisite t3030: use test_ln_s_add to remove SYMLINKS prerequisite t0000: use test_ln_s_add to remove SYMLINKS prerequisite tests: use test_ln_s_add to remove SYMLINKS prerequisite (trivial cases) tests: introduce test_ln_s_add t3010: modernize style test-chmtime: Fix exit code on Windows
2013-06-07tests: introduce test_ln_s_addLibravatar Johannes Sixt1-0/+14
Add a new function that creates a symbolic link and adds it to the index to be used in cases where a symbolic link is not required on the file system. We will use it to remove many SYMLINKS prerequisites from test cases. Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-06-04t/README: test_must_fail is for testing GitLibravatar Junio C Hamano1-0/+3
When a test wants to make sure there is no <string> in an output file, we should just say "! grep string output". "test_must_fail" is there only to test Git command and catch unusual deaths we know about (e.g. segv) as an error, not as an expected failure. "test_must_fail grep string output" is unnecessary, as we are not making sure the system binaries do not dump core or anything like that. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-04-11Merge branch 'maint'Libravatar Junio C Hamano1-1/+9
* maint: Typo fix: replacing it's -> its t: make PIPE a standard test prerequisite archive: clarify explanation of --worktree-attributes t/README: --immediate skips cleanup commands for failed tests
2013-04-11t: make PIPE a standard test prerequisiteLibravatar Adam Spiers1-0/+5
The 'PIPE' test prerequisite was already defined identically by t9010 and t9300, therefore it makes sense to make it a predefined prerequisite. Signed-off-by: Adam Spiers <git@adamspiers.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-04-09t/README: --immediate skips cleanup commands for failed testsLibravatar Simon Ruderich1-1/+4
Signed-off-by: Simon Ruderich <simon@ruderich.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-04-01tests --valgrind: provide a mode without --track-originsLibravatar Thomas Rast1-0/+5
With --valgrind=memcheck-fast, the tests run under memcheck but without the autodetected --track-origins. If you just run valgrind to see *if* there is any memory issue with your program, the extra information is not needed, and it comes at a roughly 30% hit in runtime. While it is possible to achieve the same through GIT_VALGRIND_OPTIONS, this should be more discoverable and hopefully encourage more users to run their tests with valgrind. Signed-off-by: Thomas Rast <trast@inf.ethz.ch> Acked-by: Johannes Schindelin <Johannes.Schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-04-01tests: parameterize --valgrind optionLibravatar Thomas Rast1-5/+10
Running tests under helgrind and DRD recently proved useful in tracking down thread interaction issues. This can unfortunately not be done through GIT_VALGRIND_OPTIONS because any tool other than memcheck would complain about unknown options. Let --valgrind take an optional parameter that describes the valgrind tool to invoke. The default mode is to run memcheck as before. Signed-off-by: Thomas Rast <trast@inf.ethz.ch> Acked-by: Johannes Schindelin <Johannes.Schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>