summaryrefslogtreecommitdiff
path: root/t/test-lib.sh
AgeCommit message (Collapse)AuthorFilesLines
2010-11-24Merge branch 'en/and-cascade-tests'Libravatar Junio C Hamano1-18/+55
* en/and-cascade-tests: (25 commits) t4124 (apply --whitespace): use test_might_fail t3404: do not use 'describe' to implement test_cmp_rev t3404 (rebase -i): introduce helper to check position of HEAD t3404 (rebase -i): move comment to description t3404 (rebase -i): unroll test_commit loops t3301 (notes): use test_expect_code for clarity t1400 (update-ref): use test_must_fail t1502 (rev-parse --parseopt): test exit code from "-h" t6022 (renaming merge): chain test commands with && test-lib: introduce test_line_count to measure files tests: add missing &&, batch 2 tests: add missing && Introduce sane_unset and use it to ensure proper && chaining t7800 (difftool): add missing && t7601 (merge-pull-config): add missing && t7001 (mv): add missing && t6016 (rev-list-graph-simplify-history): add missing && t5602 (clone-remote-exec): add missing && t4026 (color): remove unneeded and unchained command t4019 (diff-wserror): add lots of missing && ... Conflicts: t/t7006-pager.sh
2010-11-17Merge branch 'kb/maint-diff-ws-check'Libravatar Junio C Hamano1-8/+41
* kb/maint-diff-ws-check: diff: handle lines containing only whitespace and tabs better test-lib: extend test_decode_color to handle more color codes
2010-11-17Merge branch 'jk/push-progress'Libravatar Junio C Hamano1-7/+19
* jk/push-progress: push: pass --progress down to git-pack-objects t5523-push-upstream: test progress messages t5523-push-upstream: add function to ensure fresh upstream repo test_terminal: ensure redirections work reliably test_terminal: catch use without TTY prerequisite test-lib: allow test code to check the list of declared prerequisites tests: test terminal output to both stdout and stderr tests: factor out terminal handling from t7006
2010-11-17Merge branch 'jk/maint-rev-list-nul'Libravatar Junio C Hamano1-0/+4
* jk/maint-rev-list-nul: rev-list: handle %x00 NUL in user format
2010-11-09test-lib: introduce test_line_count to measure filesLibravatar Jonathan Nieder1-0/+22
Some tests check their output with code like the following: test "$(git ls-files -u B | wc -l)" -eq 3 || { echo "BAD: should have left stages for B" return 1 } The verbose failure condition is used because test, unlike diff, does not print any useful information about the nature of the failure when it fails. Introduce a test_line_count function to help. If used like git ls-files -u B >output && test_line_count -eq 3 output it will produce output like test_line_count: line count for output !-eq 3 100644 b023018cabc396e7692c70bbf5784a93d3f738ab 2 hi.c 100644 45b983be36b73c0788dc9cbcb76cbb80fc7bb057 3 hi.c on failure. Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-10-20test-lib: extend test_decode_color to handle more color codesLibravatar Kevin Ballard1-8/+41
Enhance the test_decode_color function to handle all common color codes, including background colors and escapes that contain multiple codes. This change necessitates changing <WHITE> to <BOLD>, so update t4034 as well. This change is necessary for the next commit in order to test background colors properly. Signed-off-by: Kevin Ballard <kevin@sb.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-10-18test-lib: allow test code to check the list of declared prerequisitesLibravatar Jonathan Nieder1-7/+19
This is plumbing to prepare helpers like test_terminal to notice buggy test scripts that do not declare all of the necessary prerequisites. Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-10-13rev-list: handle %x00 NUL in user formatLibravatar Jeff King1-0/+4
The code paths for showing commits in "git log" and "git rev-list --graph" correctly handle embedded NULs by looking only at the resulting strbuf's length, and never treating it as a C string. The code path for regular rev-list, however, used printf("%s"), which resulted in truncated output. This patch uses fwrite instead, like the --graph code path. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-10-06Introduce sane_unset and use it to ensure proper && chainingLibravatar Elijah Newren1-0/+11
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-10-06test-lib: make test_expect_code a test commandLibravatar Ævar Arnfjörð Bjarmason1-18/+22
Change test_expect_code to be a normal test command instead of a top-level command. As a top-level command it would fail in cases like: test_expect_code 1 'phoney' ' foo && bar && (exit 1) ' Here the test might incorrectly succeed if "foo" or "bar" happened to fail with exit status 1. Instead we now do: test_expect_success 'phoney' ' foo && bar && test_expect_code 1 "(exit 1)" ' Which will only succeed if "foo" and "bar" return status 0, and "(exit 1)" returns status 1. Note that test_expect_code has been made slightly noisier, as it reports the exit code it receives even upon success. Some test code in t0000-basic.sh relied on the old semantics of test_expect_code to test the test_when_finished command. I've converted that code to use an external test similar to the TODO test I added in v1.7.3-rc0~2^2~3. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Acked-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-10-01Do not strip CR when grepping HTTP headers.Libravatar Pat Thoyts1-0/+1
By default, MSYS grep reads in text-mode and converts CRLF into LF line endings. For testing HTTP use binary mode (-U) as checking is done for CR in HTTP headers Signed-off-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Pat Thoyts <patthoyts@users.sourceforge.net>
2010-10-01Skip t1300.70 and 71 on msysGit.Libravatar Pat Thoyts1-0/+1
These two tests fail on msysGit because /dev/null is an alias for nul on Windows and when reading the value back from git config the alias does not match the real filename. Also the HOME environment variable has a unix-style path but git returns a native equivalent path for '~'. As these are platform-dependent equivalent results it seems simplest to skip the test entirely. Moves the NOT_MINGW prereq from t5503 into the test library. Signed-off-by: Pat Thoyts <patthoyts@users.sourceforge.net>
2010-09-08Merge branch 'jk/test-must-fail-missing'Libravatar Junio C Hamano1-2/+21
* jk/test-must-fail-missing: tests: make test_might_fail fail on missing commands tests: make test_might_fail more verbose tests: make test_must_fail fail on missing commands tests: make test_must_fail more verbose
2010-09-04Merge branch 'ab/test-2'Libravatar Junio C Hamano1-36/+83
* ab/test-2: (51 commits) tests: factor HOME=$(pwd) in test-lib.sh test-lib: use subshell instead of cd $new && .. && cd $old tests: simplify "missing PREREQ" message t/t0000-basic.sh: Run the passing TODO test inside its own test-lib test-lib: Allow overriding of TEST_DIRECTORY test-lib: Use "$GIT_BUILD_DIR" instead of "$TEST_DIRECTORY"/../ test-lib: Use $TEST_DIRECTORY or $GIT_BUILD_DIR instead of $(pwd) and ../ test: Introduce $GIT_BUILD_DIR cvs tests: do not touch test CVS repositories shipped with source t/t9602-cvsimport-branches-tags.sh: Add a PERL prerequisite t/t9601-cvsimport-vendor-branch.sh: Add a PERL prerequisite t/t7105-reset-patch.sh: Add a PERL prerequisite t/t9001-send-email.sh: convert setup code to tests t/t9001-send-email.sh: change from skip_all=* to prereq skip t/t9001-send-email.sh: Remove needless PROG=* assignment t/t9600-cvsimport.sh: change from skip_all=* to prereq skip lib-patch-mode tests: change from skip_all=* to prereq skip t/t3701-add-interactive.sh: change from skip_all=* to prereq skip tests: Move FILEMODE prerequisite to lib-prereq-FILEMODE.sh t/Makefile: Create test-results dir for smoke target ... Conflicts: t/t6035-merge-dir-to-symlink.sh
2010-08-31tests: factor HOME=$(pwd) in test-lib.shLibravatar Matthieu Moy1-0/+3
The same pattern is used in many tests, and makes it easy for new ones to rely on $HOME being a trashable, clean, directory. Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-08-31test-lib: use subshell instead of cd $new && .. && cd $oldLibravatar Ævar Arnfjörð Bjarmason1-6/+6
Change the test_create_repo code added in v1.2.2~6 to use a subshell instead of keeping track of the old working directory and cd-ing back when it's done. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-08-31tests: simplify "missing PREREQ" messageLibravatar Jonathan Nieder1-1/+7
When a test has no prerequisites satisfied (the usual case), instead of "missing THING of THING", just say "missing THING". This does not affect the output when a test is skipped due to a missing prerequisites if another prerequisite is satisfied. For example: instead of ok 8 # skip notes work (missing EXPENSIVE of EXPENSIVE) ok 9 # skip notes timing with /usr/bin/time (missing EXPENSIVE of USR_BIN_TIME,EXPENSIVE) write ok 8 # skip notes work (missing EXPENSIVE) ok 9 # skip notes timing with /usr/bin/time (missing EXPENSIVE of USR_BIN_TIME,EXPENSIVE) Cc: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-08-31test-lib: Allow overriding of TEST_DIRECTORYLibravatar Ævar Arnfjörð Bjarmason1-1/+7
Tests that test the test-lib.sh itself need to be executed in the dynamically created trash directory, so we can't assume $TEST_DIRECTORY is ../ for those. As a side benefit this change also makes it easy for us to move the t/*.sh tests into subdirectories if we ever want to do that. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-08-31test-lib: Use "$GIT_BUILD_DIR" instead of "$TEST_DIRECTORY"/../Libravatar Ævar Arnfjörð Bjarmason1-14/+15
Change code that used $TEST_DIRECTORY/.. to use $GIT_BUILD_DIR instead, the two are equivalent, but the latter is easier to read. This required moving the assignment od GIT_BUILD_DIR to earlier in the test-lib.sh file. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-08-31test-lib: Use $TEST_DIRECTORY or $GIT_BUILD_DIR instead of $(pwd) and ../Libravatar Ævar Arnfjörð Bjarmason1-8/+8
Change the redundant calls to $(pwd) to use $TEST_DIRECTORY instead. None of these were being executed after we cd'd somewhere else so they weren't actually needed. This also makes it easier to add support for overriding the test library location and run tests in a different directory than t/. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-08-31test: Introduce $GIT_BUILD_DIRLibravatar Thomas Rast1-0/+1
Introduce a new variable $GIT_BUILD_DIR which can be used to locate data that resides under the build directory, and use that instead. Signed-off-by: Thomas Rast <trast@student.ethz.ch> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-08-31tests: make test_might_fail fail on missing commandsLibravatar Jonathan Nieder1-0/+3
Detect and report hard-to-notice spelling mistakes like test_might_fail "git config --unset whatever" (the extra quotes prevent the shell from running git as intended; instead, the shell looks for a "git config --unset whatever" file). Cc: Jeff King <peff@peff.net> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-08-31tests: make test_might_fail more verboseLibravatar Jonathan Nieder1-1/+6
Let test_might_fail say something about its failures for consistency with test_must_fail. Cc: Jeff King <peff@peff.net> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-08-31tests: make test_must_fail fail on missing commandsLibravatar Jeff King1-0/+3
The point of it is to run a command that produces failure. A missing command is more likely an error in the test script (e.g., using 'test_must_fail "command with arguments"', or relying on a missing command). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-08-31tests: make test_must_fail more verboseLibravatar Jeff King1-1/+9
Because test_must_fail fails when a command succeeds, the command frequently does not produce any output (since, after all, it thought it was succeeding). So let's have test_must_fail itself report that a problem occurred. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-08-21Merge branch 'mm/rebase-i-exec'Libravatar Junio C Hamano1-0/+32
* mm/rebase-i-exec: git-rebase--interactive.sh: use printf instead of echo to print commit message git-rebase--interactive.sh: rework skip_unnecessary_picks test-lib: user-friendly alternatives to test [-d|-f|-e] rebase -i: add exec command to launch a shell command Conflicts: git-rebase--interactive.sh t/t3404-rebase-interactive.sh
2010-08-18test-lib: Multi-prereq support only checked the last prereqLibravatar Ævar Arnfjörð Bjarmason1-3/+18
The support for multiple test prerequisites added by me in "test-lib: Add support for multiple test prerequisites" was broken. The for iterated over each prerequisite and returned true/false within a case statement, but since it missed a return statement only the last prerequisite in the list of prerequisites was ever considered, the rest were ignored. Fix that by changing the test_have_prereq code to something less clever that keeps a count of the total prereqs and the ones we have and compares the count at the end. This comes with the added advantage that it's easy to list the missing prerequisites in the test output, implement that while I'm at it. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-08-18tests: A SANITY test prereq for testing if we're rootLibravatar Ævar Arnfjörð Bjarmason1-0/+4
Some tests depend on not being able to write to files after chmod -w. This doesn't work when running the tests as root. Change test-lib.sh to test if this works, and if so it sets a new SANITY test prerequisite. The tests that use this previously failed when run under root. There was already a test for this in t3600-rm.sh, added by Junio C Hamano in 2283645 in 2006. That check now uses the new SANITY prerequisite. Some of this was resurrected from the "Tests in Cygwin" thread in May 2009: http://thread.gmane.org/gmane.comp.version-control.git/116729/focus=118385 Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-08-18test-lib: Print missing prerequisites in test outputLibravatar Ævar Arnfjörð Bjarmason1-1/+1
Change the test output to print needed prerequisites as part of the TAP. This makes it easy to see at a glance why a test was skipped. Before: ok 7 # skip <message> ok 9 # skip <message> After: ok 7 # skip <message> (prereqs: DONTHAVEIT) ok 9 # skip <message> (prereqs: HAVEIT,DONTHAVEIT) This'll also be useful for smoke testing output, where the developer reading the output may not be familiar with the system where tests are being skipped. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-08-18test-lib: Add support for multiple test prerequisitesLibravatar Ævar Arnfjörð Bjarmason1-6/+14
Change the test_have_prereq function in test-lib.sh to support a comma-separated list of prerequisites. This is useful for tests that need e.g. both POSIXPERM and SANITY. The implementation was stolen from Junio C Hamano and Johannes Sixt, the tests and documentation were not. See the "Tests in Cygwin" thread in May 2009 for the originals: http://thread.gmane.org/gmane.comp.version-control.git/116729/focus=118385 http://thread.gmane.org/gmane.comp.version-control.git/116729/focus=118434 Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-08-18test-lib: Don't write test-results when HARNESS_ACTIVELibravatar Ævar Arnfjörð Bjarmason1-10/+13
TAP harnesses don't need to read test-results/*, since they keep track of the number of passing/failing tests internally. Skip the generation of these files when HARNESS_ACTIVE is set. It's now possible to run the Git test suite without writing anything to the t/ directory at all if you use a TAP harness and the --root switch: cd t sudo mount -t tmpfs none /tmp/memory -o size=300m prove -j9 ./t[0-9]*.sh :: --root=/tmp/memory The I/O that the ~500 test-results/* files contributed was very minimal, but I thought this was worth mentioning. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-08-18Merge branch 'jn/rebase-rename-am'Libravatar Junio C Hamano1-0/+4
* jn/rebase-rename-am: rebase: protect against diff.renames configuration t3400 (rebase): whitespace cleanup Teach "apply --index-info" to handle rename patches t4150 (am): futureproof against failing tests t4150 (am): style fix
2010-08-11test-lib: user-friendly alternatives to test [-d|-f|-e]Libravatar Matthieu Moy1-0/+32
The helper functions are implemented, documented, and used in a few places to validate them, but not everywhere to avoid useless code churn. Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-08-02Merge branch 'ab/tap' into maintLibravatar Junio C Hamano1-4/+3
* ab/tap: test-lib: Remove 3 year old no-op --no-python option test-lib: Ignore --quiet under a TAP harness
2010-08-02test-lib: Remove 3 year old no-op --no-python optionLibravatar Ævar Arnfjörð Bjarmason1-3/+0
The --no-python option was added to test-lib.sh by Johannes Schindelin in early 2006 in abb7c7b3. It was later turned into a no-op by Junio C Hamano in 7cdbff14 the same year. Over three years is long enough before removing this old wart which was retained for backwards compatibility. Our tests have been using NO_PYTHON and "test_have_prereq PYTHON" for a long time now. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-08-02test-lib: Ignore --quiet under a TAP harnessLibravatar Ævar Arnfjörð Bjarmason1-1/+3
Running the tests with --quiet under a TAP harness will always fail, since a TAP harness always needs actual test output to go along with the plan that's being emitted. Change the test-lib.sh to ignore the --quiet option under HARNESS_ACTIVE to work around this. Then users that have --quiet in their GIT_TEST_OPTS can run tests under prove(1) without everything breaking. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-07-23t4150 (am): style fixLibravatar Jonathan Nieder1-0/+4
Place setup commands in test_expect_success blocks. This makes the rare event of the setup commands breaking on some platform easier to diagnose, and more importantly, it visually distinguishes where each test begins and ends. Instead of running test -z against the result of "git diff" command substitution, use "git diff --exit-code", to improve output when running with the "-v" option. Use test_cmp in place of "test $(foo) = $(bar)" for similar reasons. Remove whitespace after the > and < redirection operators for consistency with other tests. The order of arguments to test_cmp is "test_cmp expected actual". Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-07-16Only run aggregate-results over actual countsLibravatar Thomas Rast1-1/+1
The current make target 'aggregate-results' scanned all files matching test-results/t*-*. Normally these are only the test counts (and the exit values, which are ignored), but with --tee the suite also dumps all output. Furthermore, with --verbose t1450 contains several lines starting with "broken link from ..." which matches the criteria used by aggregate-results.sh. Rename the counts output files to *.counts, and only scan those. Signed-off-by: Thomas Rast <trast@student.ethz.ch> Acked-by: Sverre Rabbelier <srabbelier@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-07-12test-lib: TAP compliance for skipping tests on requestLibravatar Michael J Gruber1-2/+2
Make the output TAP compliant for tests skipped on request (GIT_SKIP_TESTS). Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net> Acked-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-07-12test-lib: simplify GIT_SKIP_TESTS loopLibravatar Michael J Gruber1-10/+3
04ece59 (GIT_SKIP_TESTS: allow users to omit tests that are known to break, 2006-12-28) introduced GIT_SKIP_TESTS, and since then we have had two nested loops iterating over GIT_SKIP_TESTS with the same loop variable. Reduce this to one loop. Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-06-25test-lib: output a newline before "ok" under a TAP harnessLibravatar Ævar Arnfjörð Bjarmason1-0/+3
Some tests in the testsuite will emit a line that doesn't end with a newline, right before we're about to output "ok" or "not ok". This breaks the TAP output with "Tests out of sequence" errors since a TAP harness can't understand this: ok 1 - A test [some output here]ok 2 - Another test ok 3 - Yet another test Work around it by emitting an empty line before we're about to say "ok" or "not ok", but only if we're running under --verbose and HARNESS_ACTIVE=1 is set, which'll only be the case when running under a harnesses like prove(1). I think it's better to do this than fix each tests by adding `&& echo' everywhere. More tests might be added that break TAP in the future, and a human isn't going to look at the extra whitespace, since HARNESS_ACTIVE=1 always means a harness is reading it. The tests that had issues were: t1007, t3410, t3413, t3409, t3414, t3415, t3416, t3412, t3404, t5407, t7402, t7003, t9001 With this workaround the entire test suite runs without errors under: prove -j 10 ./t[0-9]*.sh :: --verbose Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-06-25test-lib: Make the test_external_* functions TAP-awareLibravatar Ævar Arnfjörð Bjarmason1-12/+39
Before TAP we just ran the Perl test and assumed that it failed if nothing was printed on STDERR. Continue doing that, but introduce a `test_external_has_tap' variable which tests can set to indicate that they're outputting TAP. If it's set we won't output a test plan, but trust the external test to do so. That way we can make external tests work with a TAP harness, but still maintain compatibility with test-lib's own way of tracking tests through the test-results directory. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-06-25test-lib: Adjust output to be valid TAP formatLibravatar Ævar Arnfjörð Bjarmason1-12/+18
TAP, the Test Anything Protocol, is a simple text-based interface between testing modules in a test harness. test-lib.sh's output was already very close to being valid TAP. This change brings it all the way there. Before: $ ./t0005-signals.sh * ok 1: sigchain works * passed all 1 test(s) And after: $ ./t0005-signals.sh ok 1 - sigchain works # passed all 1 test(s) 1..1 The advantage of using TAP is that any program that reads the format (a "test harness") can run the tests. The most popular of these is the prove(1) utility that comes with Perl. It can run tests in parallel, display colored output, format the output to console, file, HTML etc., and much more. An example: $ prove ./t0005-signals.sh ./t0005-signals.sh .. ok All tests successful. Files=1, Tests=1, 0 wallclock secs ( 0.03 usr 0.00 sys + 0.01 cusr 0.02 csys = 0.06 CPU) Result: PASS prove(1) gives you human readable output without being too verbose. Running the test suite in parallel with `make test -j15` produces a flood of text. Running them with `prove -j 15 ./t[0-9]*.sh` makes it easy to follow what's going on. All this patch does is re-arrange the output a bit so that it conforms with the TAP spec, everything that the test suite did before continues to work. That includes aggregating results in t/test-results/, the --verbose, --debug and other options for tests, and the test color output. TAP harnesses ignore everything that they don't know about, so running the tests with --verbose works: $ prove ./t0005-signals.sh :: --verbose --debug ./t0005-signals.sh .. Terminated ./t0005-signals.sh .. ok All tests successful. Files=1, Tests=1, 0 wallclock secs ( 0.02 usr 0.01 sys + 0.01 cusr 0.01 csys = 0.05 CPU) Result: PASS Just supply the -v option to prove itself to get all the verbose output that it suppresses: $ prove -v ./t0005-signals.sh :: --verbose --debug ./t0005-signals.sh .. Initialized empty Git repository in /home/avar/g/git/t/trash directory.t0005-signals/.git/ expecting success: test-sigchain >actual case "$?" in 143) true ;; # POSIX w/ SIGTERM=15 3) true ;; # Windows *) false ;; esac && test_cmp expect actual Terminated ok 1 - sigchain works # passed all 1 test(s) 1..1 ok All tests successful. Files=1, Tests=1, 0 wallclock secs ( 0.02 usr 0.00 sys + 0.01 cusr 0.01 csys = 0.04 CPU) Result: PASS As a further example, consider this test script that uses a lot of test-lib.sh features by Jakub Narebski: #!/bin/sh test_description='this is a sample test. This test is here to see various test outputs.' . ./test-lib.sh say 'diagnostic message' test_expect_success 'true test' 'true' test_expect_success 'false test' 'false' test_expect_failure 'true test (todo)' 'true' test_expect_failure 'false test (todo)' 'false' test_debug 'echo "debug message"' test_done The output of that was previously: * diagnostic message # yellow * ok 1: true test * FAIL 2: false test # bold red false * FIXED 3: true test (todo) * still broken 4: false test (todo) # bold green * fixed 1 known breakage(s) # green * still have 1 known breakage(s) # bold red * failed 1 among remaining 3 test(s) # bold red But is now: diagnostic message # yellow ok 1 - true test not ok - 2 false test # bold red # false ok 3 - true test (todo) # TODO known breakage not ok 4 - false test (todo) # TODO known breakage # bold green # fixed 1 known breakage(s) # green # still have 1 known breakage(s) # bold red # failed 1 among remaining 3 test(s) # bold red 1..4 All the coloring is preserved when the test is run manually. Under prove(1) the test performs as expected, even with --debug and --verbose options: $ prove ./example.sh :: --debug --verbose ./example.sh .. Dubious, test returned 1 (wstat 256, 0x100) Failed 1/4 subtests (1 TODO test unexpectedly succeeded) Test Summary Report ------------------- ./example.sh (Wstat: 256 Tests: 4 Failed: 1) Failed test: 2 TODO passed: 3 Non-zero exit status: 1 Files=1, Tests=4, 0 wallclock secs ( 0.02 usr 0.00 sys + 0.00 cusr 0.01 csys = 0.03 CPU) Result: FAIL The TAP harness itself doesn't get confused by the color output, they aren't used by test-lib.sh stdout isn't open to a terminal (test -t 1). Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-06-21Merge branch 'gv/portable'Libravatar Junio C Hamano1-1/+10
* gv/portable: test-lib: use DIFF definition from GIT-BUILD-OPTIONS build: propagate $DIFF to scripts Makefile: Tru64 portability fix Makefile: HP-UX 10.20 portability fixes Makefile: HPUX11 portability fixes Makefile: SunOS 5.6 portability fix inline declaration does not work on AIX Allow disabling "inline" Some platforms lack socklen_t type Make NO_{INET_NTOP,INET_PTON} configured independently Makefile: some platforms do not have hstrerror anywhere git-compat-util.h: some platforms with mmap() lack MAP_FAILED definition test_cmp: do not use "diff -u" on platforms that lack one fixup: do not unconditionally disable "diff -u" tests: use "test_cmp", not "diff", when verifying the result Do not use "diff" found on PATH while building and installing enums: omit trailing comma for portability Makefile: -lpthread may still be necessary when libc has only pthread stubs Rewrite dynamic structure initializations to runtime assignment Makefile: pass CPPFLAGS through to fllow customization Conflicts: Makefile wt-status.h
2010-06-11test-lib: use DIFF definition from GIT-BUILD-OPTIONSLibravatar Junio C Hamano1-10/+10
Otherwise running individual tests from t/ directory may lack the definition of $DIFF, $GIT_TEST_CMP and friends. Noticed and initial patch provided by Thomas Rast, alternative solution suggested by Brandon Casey, which this patch implements. Signed-off-by: Junio C Hamano <gitster@pobox.com> Acked-by: Thomas Rast <trast@student.ethz.ch>
2010-05-31fixup: do not unconditionally disable "diff -u"Libravatar Junio C Hamano1-1/+10
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-05-21Merge branch 'jn/gitweb-caching-prep'Libravatar Junio C Hamano1-0/+3
* jn/gitweb-caching-prep: gitweb: Move generating page title to separate subroutine gitweb: Add custom error handler using die_error gitweb: Use nonlocal jump instead of 'exit' in die_error gitweb: href(..., -path_info => 0|1) Export more test-related variables when running external tests
2010-05-21Merge branch 'jn/t7006-fixup'Libravatar Junio C Hamano1-0/+16
* jn/t7006-fixup: t7006: guard cleanup with test_expect_success
2010-05-06test-lib: some shells do not let $? propagate into an evalLibravatar Jonathan Nieder1-3/+4
In 3bf7886 (test-lib: Let tests specify commands to be run at end of test, 2010-05-02), the git test harness learned to run cleanup commands unconditionally at the end of a test. During each test, the intended cleanup actions are collected in the test_cleanup variable and evaluated. That variable looks something like this: eval_ret=$?; clean_something && (exit "$eval_ret") eval_ret=$?; clean_something_else && (exit "$eval_ret") eval_ret=$?; final_cleanup && (exit "$eval_ret") eval_ret=$? All cleanup actions are run unconditionally but if one of them fails it is properly reported through $eval_ret. On FreeBSD, unfortunately, $? is set at the beginning of an ‘eval’ to 0 instead of the exit status of the previous command. This results in tests using test_expect_code appearing to fail and all others appearing to pass, unless their cleanup fails. Avoid the problem by setting eval_ret before the ‘eval’ begins. Thanks to Jeff King for the explanation. Cc: Jeff King <peff@peff.net> Cc: Johannes Sixt <j6t@kdbg.org> Acked-by: Jeff King <peff@peff.net> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-05-04test-lib: Let tests specify commands to be run at end of testLibravatar Jonathan Nieder1-1/+27
Certain actions can imply that if the test fails early, recovery from within other tests is too much to expect: - creating unwritable directories, like the EACCESS test in t0001-init - setting unusual configuration, like user.signingkey in t7004-tag - crashing and leaving the index lock held, like t3600-rm once did Some test scripts work around this by running cleanup actions outside the supervision of the test harness, with the unfortunate consequence that those commands are not appropriately echoed and their output not suppressed. Others explicitly save exit status, clean up, and then reset the exit status within the tests, which has excellent behavior but makes the tests hard to read. Still others ignore the problem. Allow tests a fourth option: by calling this function, tests can stack up commands they would like to be run to clean up. Commands passed to test_when_finished during a test are unconditionally run in the test environment immediately before the test is completed, in last-in-first-out order. If some cleanup command fails, then the other cleanup commands are still run before the failure is reported and the test script allowed to continue. Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>