summaryrefslogtreecommitdiff
path: root/t/test-lib.sh
AgeCommit message (Collapse)AuthorFilesLines
2016-10-31Merge branch 'nd/test-helpers'Libravatar Junio C Hamano1-1/+8
Update to the test framework made in 2.9 timeframe broke running the tests under valgrind, which has been fixed. * nd/test-helpers: valgrind: support test helpers
2016-10-27valgrind: support test helpersLibravatar René Scharfe1-1/+8
Tests run with --valgrind call git commands through a wrapper script that invokes valgrind on them. This script (valgrind.sh) is in turn invoked through symlinks created for each command in t/valgrind/bin/. Since e6e7530d (test helpers: move test-* to t/helper/ subdirectory) these symlinks have been broken for test helpers -- they point to the old locations in the root of the build directory. Fix that by teaching the code for creating the links about the new location of the binaries, and do the same in the wrapper script to allow it to find its payload. Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-10-26Merge branch 'jk/tap-verbose-fix'Libravatar Junio C Hamano1-4/+30
The Travis CI configuration we ship ran the tests with --verbose option but this risks non-TAP output that happens to be "ok" to be misinterpreted as TAP signalling a test that passed. This resulted in unnecessary failure. This has been corrected by introducing a new mode to run our tests in the test harness to send the verbose output separately to the log file. * jk/tap-verbose-fix: test-lib: bail out when "-v" used under "prove" travis: use --verbose-log test option test-lib: add --verbose-log option test-lib: handle TEST_OUTPUT_DIRECTORY with spaces
2016-10-24test-lib: bail out when "-v" used under "prove"Libravatar Jeff King1-0/+10
When there is a TAP harness consuming the output of our test scripts, the "--verbose" breaks the output by mingling test command output with TAP. Because the TAP::Harness module used by "prove" is fairly lenient, this _usually_ works, but it violates the spec, and things get very confusing if the commands happen to output a line that looks like TAP (e.g., the word "ok" on its own line). Let's detect this situation and complain. Just calling error() isn't great, though; prove will tell us that the script failed, but the message doesn't make it through to the user. Instead, we can use the special TAP signal "Bail out!". This not only shows the message to the user, but instructs the harness to stop running the tests entirely. This is exactly what we want here, as the problem is in the command-line options, and every test script would produce the same error. The result looks like this (the first "Bailout called" line is in red if prove uses color on your terminal): $ make GIT_TEST_OPTS='--verbose --tee' rm -f -r 'test-results' *** prove *** Bailout called. Further testing stopped: verbose mode forbidden under TAP harness; try --verbose-log FAILED--Further testing stopped: verbose mode forbidden under TAP harness; try --verbose-log Makefile:39: recipe for target 'prove' failed make: *** [prove] Error 255 Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-10-21test-lib: add --verbose-log optionLibravatar Jeff King1-3/+19
The "--verbose" option redirects output from arbitrary test commands to stdout. This is useful for examining the output manually, like: ./t5547-push-quarantine.sh -v | less But it also means that the output is intermingled with the TAP directives, which can confuse a TAP parser like "prove". This has always been a potential problem, but became an issue recently when one test happened to output the word "ok" on a line by itself, which prove interprets as a test success: $ prove t5547-push-quarantine.sh :: -v t5547-push-quarantine.sh .. 1/? To dest.git * [new branch] HEAD -> master To dest.git ! [remote rejected] reject -> reject (pre-receive hook declined) error: failed to push some refs to 'dest.git' fatal: git cat-file d08c8eba97f4e683ece08654c7c8d2ba0c03b129: bad file t5547-push-quarantine.sh .. Failed -1/4 subtests Test Summary Report ------------------- t5547-push-quarantine.sh (Wstat: 0 Tests: 5 Failed: 0) Parse errors: Tests out of sequence. Found (2) but expected (3) Tests out of sequence. Found (3) but expected (4) Tests out of sequence. Found (4) but expected (5) Bad plan. You planned 4 tests but ran 5. Files=1, Tests=5, 0 wallclock secs ( 0.01 usr + 0.01 sys = 0.02 CPU) Result: FAIL One answer is "if it hurts, don't do it", but that's not quite the whole story. The Travis tests use "--verbose --tee" so that they can get the benefit of prove's parallel options, along with a verbose log in case there is a failure. We just need the verbose output to go to the log, but keep stdout clean. Getting this right turns out to be surprisingly difficult. Here's the progression of alternatives I considered: 1. Add an option to write verbose output to stderr. This is hard to capture, though, because we want each test to have its own log (because they're all run in parallel and the jumbled output would be useless). 2. Add an option to write verbose output to a file in test-results. This works, but the log is missing all of the non-verbose output, which gives context. 3. Like (2), but teach say_color() to additionally output to the log. This mostly works, but misses any output that happens outside of the say() functions (which isn't a lot, but is a potential maintenance headache). 4. Like (2), but make the log file the same as the "--tee" file. That almost works, but now we have two processes opening the same file. That gives us two separate descriptors, each with their own idea of the current position. They'll each start writing at offset 0, and overwrite each other's data. 5. Like (4), but in each case open the file for appending. That atomically positions each write at the end of the file. It's possible we may still get sheared writes between the two processes, but this is already the case when writing to stdout. It's not a problem in practice because the test harness generally waits for snippets to finish before writing the TAP output. We can ignore buffering issues with tee, because POSIX mandates that it does not buffer. Likewise, POSIX specifies "tee -a", so it should be available everywhere. This patch implements option (5), which seems to work well in practice. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-10-21test-lib: handle TEST_OUTPUT_DIRECTORY with spacesLibravatar Jeff King1-2/+2
We are careful in test_done to handle a results directory with a space in it, but the "--tee" code path does not. Doing: export TEST_OUTPUT_DIRECTORY='/tmp/path with spaces' ./t000-init.sh --tee results in errors. Let's consistently double-quote our path variables so that this works. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-09-21Merge branch 'jt/accept-capability-advertisement-when-fetching-from-void'Libravatar Junio C Hamano1-0/+4
JGit can show a fake ref "capabilities^{}" to "git fetch" when it does not advertise any refs, but "git fetch" was not prepared to see such an advertisement. When the other side disconnects without giving any ref advertisement, we used to say "there may not be a repository at that URL", but we may have seen other advertisement like "shallow" and ".have" in which case we definitely know that a repository is there. The code to detect this case has also been updated. * jt/accept-capability-advertisement-when-fetching-from-void: connect: advertized capability is not a ref connect: tighten check for unexpected early hang up tests: move test_lazy_prereq JGIT to test-lib.sh
2016-09-12Merge branch 'ep/use-git-trace-curl-in-tests'Libravatar Junio C Hamano1-0/+1
Update a few tests that used to use GIT_CURL_VERBOSE to use the newer GIT_TRACE_CURL. * ep/use-git-trace-curl-in-tests: t5551-http-fetch-smart.sh: use the GIT_TRACE_CURL environment var t5550-http-fetch-dumb.sh: use the GIT_TRACE_CURL environment var test-lib.sh: preserve GIT_TRACE_CURL from the environment t5541-http-push-smart.sh: use the GIT_TRACE_CURL environment var
2016-09-09tests: move test_lazy_prereq JGIT to test-lib.shLibravatar Jonathan Tan1-0/+4
This enables JGIT to be used as a prereq in invocations of test_expect_success (and other functions) in other test scripts. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-09-07test-lib.sh: preserve GIT_TRACE_CURL from the environmentLibravatar Elia Pinto1-0/+1
Turning on this variable can be useful when debugging http tests. It can break a few tests in t5541 if not set to an absolute path but it is not a variable that the user is likely to have enabled accidentally. Signed-off-by: Elia Pinto <gitter.spiros@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-30test-lib: drop PID from test-results/*.countLibravatar Jeff King1-2/+2
Each test run generates a "count" file in t/test-results that stores the number of successful, failed, etc tests. If you run "t1234-foo.sh", that file is named as "t/test-results/t1234-foo-$$.count" The addition of the PID there is serving no purpose, and makes analysis of the count files harder. The presence of the PID dates back to 2d84e9f (Modify test-lib.sh to output stats to t/test-results/*, 2008-06-08), but no reasoning is given there. Looking at the current code, we can see that other files we write to test-results (like *.exit and *.out) do _not_ have the PID included. So the presence of the PID does not meaningfully allow one to store the results from multiple runs anyway. Moreover, anybody wishing to read the *.count files to aggregate results has to deal with the presence of multiple files for a given test (and figure out which one is the most recent based on their timestamps!). The only consumer of these files is the aggregate.sh script, which arguably gets this wrong. If a test is run multiple times, its counts will appear multiple times in the total (I say arguably only because the desired semantics aren't documented anywhere, but I have trouble seeing how this behavior could be useful). So let's just drop the PID, which fixes aggregate.sh, and will make new features based around the count files easier to write. Note that since the count-file may already exist (when re-running a test), we also switch the "cat" from appending to truncating. The use of append here was pointless in the first place, as we expected to always write to a unique file. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-08Merge branch 'nd/test-helpers' into maintLibravatar Junio C Hamano1-1/+1
Build clean-up. * nd/test-helpers: t/test-lib.sh: fix running tests with --valgrind Makefile: use VCSSVN_LIB to refer to svn library Makefile: drop extra dependencies for test helpers
2016-08-08Merge branch 'nd/cache-tree-ita' into maintLibravatar Junio C Hamano1-1/+4
"git add -N dir/file && git write-tree" produced an incorrect tree when there are other paths in the same directory that sorts after "file". * nd/cache-tree-ita: cache-tree: do not generate empty trees as a result of all i-t-a subentries cache-tree.c: fix i-t-a entry skipping directory updates sometimes test-lib.sh: introduce and use $EMPTY_BLOB test-lib.sh: introduce and use $EMPTY_TREE
2016-07-25Merge branch 'nd/cache-tree-ita'Libravatar Junio C Hamano1-1/+4
"git add -N dir/file && git write-tree" produced an incorrect tree when there are other paths in the same directory that sorts after "file". * nd/cache-tree-ita: cache-tree: do not generate empty trees as a result of all i-t-a subentries cache-tree.c: fix i-t-a entry skipping directory updates sometimes test-lib.sh: introduce and use $EMPTY_BLOB test-lib.sh: introduce and use $EMPTY_TREE
2016-07-25Merge branch 'nd/test-helpers'Libravatar Junio C Hamano1-1/+1
Build clean-up. * nd/test-helpers: t/test-lib.sh: fix running tests with --valgrind Makefile: use VCSSVN_LIB to refer to svn library Makefile: drop extra dependencies for test helpers
2016-07-18test-lib.sh: introduce and use $EMPTY_BLOBLibravatar Nguyễn Thái Ngọc Duy1-1/+2
Similar to $EMPTY_TREE this makes it easier to recognize this special SHA-1 and change hash later. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-18test-lib.sh: introduce and use $EMPTY_TREELibravatar Nguyễn Thái Ngọc Duy1-1/+3
This is a special SHA1. Let's keep it at one place, easier to replace later when the hash change comes, easier to recognize. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-15Merge branch 'jk/tzoffset-fix' into maintLibravatar Junio C Hamano1-0/+9
Skip tests that are unrunnable on platforms without 64-bit long to avoid unnecessary test failures. * jk/tzoffset-fix: t0006: skip "far in the future" test when unsigned long is not long enough
2016-07-15t0006: skip "far in the future" test when unsigned long is not long enoughLibravatar Jeff King1-0/+9
Git's source code refers to timestamps as unsigned longs. On 32-bit platforms, as well as on Windows, unsigned long is not large enough to capture dates that are "absurdly far in the future". While we can fix this issue properly by replacing unsigned long with a larger type, we want to be a bit more conservative and just skip those tests on the maint track. Signed-off-by: Jeff King <peff@peff.net> Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-11t/test-lib.sh: fix running tests with --valgrindLibravatar Johannes Schindelin1-1/+1
We forgot to adjust this code path after moving the test helpers to t/helper/. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Acked-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-31Merge branch 'jk/test-send-sh-x-trace-elsewhere' into maintLibravatar Junio C Hamano1-0/+13
Running tests with '-x' option to trace the individual command executions is a useful way to debug test scripts, but some tests that capture the standard error stream and check what the command said can be broken with the trace output mixed in. When running our tests under "bash", however, we can redirect the trace output to another file descriptor to keep the standard error of programs being tested intact. * jk/test-send-sh-x-trace-elsewhere: test-lib: set BASH_XTRACEFD automatically
2016-05-17Merge branch 'jk/test-send-sh-x-trace-elsewhere'Libravatar Junio C Hamano1-0/+13
Running tests with '-x' option to trace the individual command executions is a useful way to debug test scripts, but some tests that capture the standard error stream and check what the command said can be broken with the trace output mixed in. When running our tests under "bash", however, we can redirect the trace output to another file descriptor to keep the standard error of programs being tested intact. * jk/test-send-sh-x-trace-elsewhere: test-lib: set BASH_XTRACEFD automatically
2016-05-11test-lib: set BASH_XTRACEFD automaticallyLibravatar Jeff King1-0/+13
Passing "-x" to a test script enables the shell's "set -x" tracing, which can help with tracking down the command that is causing a failure. Unfortunately, it can also _cause_ failures in some tests that redirect the stderr of a shell function. Inside the function the shell continues to respect "set -x", and the trace output is collected along with whatever stderr is generated normally by the function. You can see an example of this by running: ./t0040-parse-options.sh -x -i which will fail immediately in the first test, as it expects: test_must_fail some-cmd 2>output.err to leave output.err empty (but with "-x" it has our trace output). Unfortunately there isn't a portable or scalable solution to this. We could teach test_must_fail to disable "set -x", but that doesn't help any of the other functions or subshells. However, we can work around it by pointing the "set -x" output to our descriptor 4, which always points to the original stderr of the test script. Unfortunately this only works for bash, but it's better than nothing (and other shells will just ignore the BASH_XTRACEFD variable). The patch itself is a simple one-liner, but note the caveats in the accompanying comments. Automatic tests for our "-x" option may be a bit too meta (and a pain, because they are bash-specific), but I did confirm that it works correctly both with regular "-x" and with "--verbose-only=1". This works because the latter flips "set -x" off and on for particular tests (if it didn't, we would get tracing for all tests, as going to descriptor 4 effectively circumvents the verbose flag). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-03Merge branch 'sg/test-lib-simplify-expr-away'Libravatar Junio C Hamano1-5/+5
Code cleanup. * sg/test-lib-simplify-expr-away: test-lib: simplify '--option=value' parsing
2016-04-22test-lib: simplify '--option=value' parsingLibravatar SZEDER Gábor1-5/+5
To get the 'value' from '--option=value', test-lib.sh parses said option running 'expr' with a regexp. This involves a subshell, an external process, and a lot of non-alphanumeric characters in the regexp. Use a much simpler POSIX-defined shell parameter expansion instead to do the same. Signed-off-by: SZEDER Gábor <szeder@ira.uka.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-04-15test helpers: move test-* to t/helper/ subdirectoryLibravatar Nguyễn Thái Ngọc Duy1-2/+2
This keeps top dir a bit less crowded. And because these programs are for testing purposes, it makes sense that they stay somewhere in t/ Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-02-17Merge branch 'js/mingw-tests'Libravatar Junio C Hamano1-1/+1
Test scripts have been updated to remove assumptions that are not portable between Git for POSIX and Git for Windows, or to skip ones with expectations that are not satisfiable on Git for Windows. * js/mingw-tests: (21 commits) gitignore: ignore generated test-fake-ssh executable mingw: do not bother to test funny file names mingw: skip a test in t9130 that cannot pass on Windows mingw: handle the missing POSIXPERM prereq in t9124 mingw: avoid illegal filename in t9118 mingw: mark t9100's test cases with appropriate prereqs t0008: avoid absolute path mingw: work around pwd issues in the tests mingw: fix t9700's assumption about directory separators mingw: skip test in t1508 that fails due to path conversion tests: turn off git-daemon tests if FIFOs are not available mingw: disable mkfifo-based tests mingw: accomodate t0060-path-utils for MSYS2 mingw: fix t5601-clone.sh mingw: let lstat() fail with errno == ENOTDIR when appropriate mingw: try to delete target directory before renaming mingw: prepare the TMPDIR environment variable for shell scripts mingw: factor out Windows specific environment setup Git.pm: stop assuming that absolute paths start with a slash mingw: do not trust MSYS2's MinGW gettext.sh ...
2016-02-17Merge branch 'js/test-lib-windows-emulated-yes' into maintLibravatar Junio C Hamano1-2/+4
The emulated "yes" command used in our test scripts has been tweaked not to spend too much time generating unnecessary output that is not used, to help those who test on Windows where it would not stop until it fills the pipe buffer due to lack of SIGPIPE. * js/test-lib-windows-emulated-yes: test-lib: limit the output of the yes utility
2016-02-10Merge branch 'js/test-lib-windows-emulated-yes'Libravatar Junio C Hamano1-2/+4
The emulated "yes" command used in our test scripts has been tweaked not to spend too much time generating unnecessary output that is not used, to help those who test on Windows where it would not stop until it fills the pipe buffer due to lack of SIGPIPE. * js/test-lib-windows-emulated-yes: test-lib: limit the output of the yes utility
2016-02-05Merge branch 'jk/sanity' into maintLibravatar Junio C Hamano1-5/+13
The description for SANITY prerequisite the test suite uses has been clarified both in the comment and in the implementation. * jk/sanity: test-lib: clarify and tighten SANITY
2016-02-02test-lib: limit the output of the yes utilityLibravatar Johannes Schindelin1-2/+4
On Windows, there is no SIGPIPE. A consequence of this is that the upstream process of a pipe does not notice the death of the downstream process until the pipe buffer is full and writing more data returns an error. This behavior is the reason for an annoying delay during the execution of t7610-mergetool.sh: There are a number of test cases where 'yes' is invoked upstream. Since the utility is basically an endless loop it runs, on Windows, until the pipe buffer is full. This does take a few seconds. The test suite has its own implementation of 'yes'. Modify it to produce only a limited amount of output that is sufficient for the test suite. The amount chosen should be sufficiently high for any test case, assuming that future test cases will not exaggerate their demands of input from an upstream 'yes' invocation. [j6t: commit message] Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-01-28Merge branch 'jk/sanity'Libravatar Junio C Hamano1-5/+13
The description for SANITY prerequisite the test suite uses has been clarified both in the comment and in the implementation. * jk/sanity: test-lib: clarify and tighten SANITY
2016-01-27mingw: disable mkfifo-based testsLibravatar Johannes Schindelin1-1/+1
MSYS2 (the POSIX emulation layer used by Git for Windows' Bash) actually has a working mkfifo. The only problem is that it is only emulating named pipes through the MSYS2 runtime; The Win32 API has no idea about named pipes, hence the Git executable cannot access those pipes either. The symptom is that Git fails with a '<name>: No such file or directory' because MSYS2 emulates named pipes through special-crafted '.lnk' files. The solution is to tell the test suite explicitly that we cannot use named pipes when we want to test on Windows. This lets t4056-diff-order.sh, t9010-svn-fe.sh and t9300-fast-import.sh pass. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-01-19test-lib: clarify and tighten SANITYLibravatar Junio C Hamano1-5/+13
f400e51c (test-lib.sh: set prerequisite SANITY by testing what we really need, 2015-01-27) improved the way SANITY prerequisite was determined, but made the resulting code (incorrectly) imply that SANITY is all about effects of permission bits of the containing directory has on the files contained in it by the comment it added, its log message and the actual tests. State what SANITY is about more clearly in the comment, and test that a file whose permission bits says should be unreadble truly cannot be read. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-08-19Merge branch 'jk/test-with-x'Libravatar Junio C Hamano1-2/+11
Running tests with the "-x" option to make them verbose had some unpleasant interactions with other features of the test suite. * jk/test-with-x: test-lib: disable trace when test is not verbose test-lib: turn off "-x" tracing during chain-lint check
2015-08-07test-lib: disable trace when test is not verboseLibravatar Jeff King1-2/+6
The "-x" test-script option turns on the shell's "-x" tracing, which can help show why a particular test is failing. Unfortunately, this can create false negatives in some tests if they invoke a shell function with its stderr redirected. t5512.10 is such a test, as it does: test_must_fail git ls-remote refs*master >actual 2>&1 && test_cmp exp actual The "actual" file gets the "-x" trace for the test_must_fail function, which prevents it from matching the expected output. There's no way to avoid this without managing the trace flag inside each sub-function, which isn't really a workable solution. But unless you specifically care about t5512.10, we can work around it by enabling tracing only for the specific tests we want. You can already do: ./t5512-ls-remote.sh -x --verbose-only=16 to see the trace only for a specific test. But that doesn't _disable_ the tracing in the other tests; it just sends it to /dev/null. However, there's no point in generating a trace that the user won't see, so we can simply disable tracing whenever it doesn't have a matching verbose flag. The normal case of just "./t5512-ls-remote.sh -x" stays the same, as "-x" already implies "--verbose" (and "--verbose-only" overrides "--verbose", which is why this works at all). And for our test, we need only check $verbose, as maybe_setup_verbose will have already set that flag based on the $verbose_only list). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-08-07test-lib: turn off "-x" tracing during chain-lint checkLibravatar Jeff King1-0/+5
Now that GIT_TEST_CHAIN_LINT is on by default, running: ./t0000-basic.sh -x --verbose-only=1 starts with: expecting success: find .git/objects -type f -print >should-be-empty && test_line_count = 0 should-be-empty + exit 117 error: last command exited with $?=117 + find .git/objects -type f -print + test_line_count = 0 should-be-empty + test 3 != 3 + wc -l + test 0 = 0 ok 1 - .git/objects should be empty after git init in an empty repo This is confusing, as the "exit 117" line and the error line (which is printed in red, no less!) are not part of the test at all, but are rather in the separate chain-lint test_eval. Let's unset the "trace" variable when eval-ing the chain lint check, which avoids this. Note that we cannot just do a one-shot variable like: trace= test_eval ... as the behavior of one-shot variables for function calls is not portable. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-07-27Merge branch 'rh/test-color-avoid-terminfo-in-original-home' into maintLibravatar Junio C Hamano1-52/+47
An ancient test framework enhancement to allow color was not entirely correct; this makes it work even when tput needs to read from the ~/.terminfo under the user's real HOME directory. * rh/test-color-avoid-terminfo-in-original-home: test-lib.sh: fix color support when tput needs ~/.terminfo Revert "test-lib.sh: do tests for color support after changing HOME"
2015-07-10Merge branch 'rh/test-color-avoid-terminfo-in-original-home'Libravatar Junio C Hamano1-52/+47
An ancient test framework enhancement to allow color was not entirely correct; this makes it work even when tput needs to read from the ~/.terminfo under the user's real HOME directory. * rh/test-color-avoid-terminfo-in-original-home: test-lib.sh: fix color support when tput needs ~/.terminfo Revert "test-lib.sh: do tests for color support after changing HOME"
2015-06-17test-lib.sh: fix color support when tput needs ~/.terminfoLibravatar Richard Hansen1-29/+28
If tput needs ~/.terminfo for the current $TERM, then tput will succeed before HOME is changed to $TRASH_DIRECTORY (causing color to be set to 't') but fail afterward. One possible way to fix this is to treat HOME like TERM: back up the original value and temporarily restore it before say_color() runs tput. Instead, pre-compute and save the color control sequences before changing either TERM or HOME. Use the saved control sequences in say_color() rather than call tput each time. This avoids the need to back up and restore the TERM and HOME variables, and it avoids the overhead of a subshell and two invocations of tput per call to say_color(). Signed-off-by: Richard Hansen <rhansen@bbn.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-06-17Revert "test-lib.sh: do tests for color support after changing HOME"Libravatar Richard Hansen1-47/+43
This reverts commit 102fc80d32094ad6598b17ab9d607516ee8edc4a. There are two issues with that commit: * It is buggy. In pseudocode, it is doing: color is set || TERM != dumb && color works && color=t when it should be doing: color is set || { TERM != dumb && color works && color=t } * It unnecessarily disables color when tput needs to read ~/.terminfo to get the control sequences. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-05-19Merge branch 'jk/test-chain-lint'Libravatar Junio C Hamano1-1/+1
Developer support to automatically detect broken &&-chain in the test scripts is now turned on by default. * jk/test-chain-lint: test-lib: turn on GIT_TEST_CHAIN_LINT by default t7502-commit.sh: fix a broken and-chain
2015-04-28test-lib: turn on GIT_TEST_CHAIN_LINT by defaultLibravatar Jeff King1-1/+1
Now that the feature has had time to prove itself, and any topics in flight have had a chance to clean up any broken &&-chains, we can flip this feature on by default. This makes one less thing submitters need to configure or check before sending their patches. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-03-26Merge branch 'jk/test-chain-lint'Libravatar Junio C Hamano1-0/+16
People often forget to chain the commands in their test together with &&, leaving a failure from an earlier command in the test go unnoticed. The new GIT_TEST_CHAIN_LINT mechanism allows you to catch such a mistake more easily. * jk/test-chain-lint: (36 commits) t9001: drop save_confirm helper t0020: use test_* helpers instead of hand-rolled messages t: simplify loop exit-code status variables t: fix some trivial cases of ignored exit codes in loops t7701: fix ignored exit code inside loop t3305: fix ignored exit code inside loop t0020: fix ignored exit code inside loops perf-lib: fix ignored exit code inside loop t6039: fix broken && chain t9158, t9161: fix broken &&-chain in git-svn tests t9104: fix test for following larger parents t4104: drop hand-rolled error reporting t0005: fix broken &&-chains t7004: fix embedded single-quotes t0050: appease --chain-lint t9001: use test_when_finished t4117: use modern test_* helpers t6034: use modern test_* helpers t1301: use modern test_* helpers t0020: use modern test_* helpers ...
2015-03-20t/test-lib: introduce --chain-lint optionLibravatar Jeff King1-0/+16
It's easy to miss an "&&"-chain in a test script, like: test_expect_success 'check something important' ' cmd1 && cmd2 cmd3 ' The test harness will notice if cmd3 fails, but a failure of cmd1 or cmd2 will go unnoticed, as their exit status is lost after cmd3 runs. The toy example above is easy to spot because the "cmds" are all the same length, but real code is much more complicated. It's also difficult to detect these situations by statically analyzing the shell code with regexps (like the check-non-portable-shell script does); there's too much context required to know whether a &&-chain is appropriate on a given line or not. This patch instead lets the shell check each test by sticking a command with a specific and unusual return code at the top of each test, like: (exit 117) && cmd1 && cmd2 cmd3 In a well-formed test, the non-zero exit from the first command prevents any of the rest from being run, and the test's exit code is 117. In a bad test (like the one above), the 117 is lost, and cmd3 is run. When we encounter a failure of this check, we abort the test script entirely. For one thing, we have no clue which subset of the commands in the test snippet were actually run. Running further tests would be pointless, because we're now in an unknown state. And two, this is not a "test failure" in the traditional sense. The test script is buggy, not the code it is testing. We should be able to fix these problems in the script once, and not have them come back later as a regression in git's code. After checking a test snippet for --chain-lint, we do still run the test itself. We could actually have a pure-lint mode which just checks each test, but there are a few reasons not to. One, because the tests are executing arbitrary code, which could impact the later environment (e.g., that could impact which set of tests we run at all). And two, because a pure-lint mode would still be expensive to run, because a significant amount of code runs outside of the test_expect_* blocks. Instead, this option is designed to be used as part of a normal test suite run, where it adds very little overhead. Turning on this option detects quite a few problems in existing tests, which will be fixed in subsequent patches. However, there are a number of places it cannot reach: - it cannot find a failure to break out of loops on error, like: cmd1 && for i in a b c; do cmd2 $i done && cmd3 which will not notice failures of "cmd2 a" or "cmd b" - it cannot find a missing &&-chain inside a block or subfunction, like: foo () { cmd1 cmd2 } foo && bar which will not notice a failure of cmd1. - it only checks tests that you run; every platform will have some tests skipped due to missing prequisites, so it's impossible to say from one run that the test suite is free of broken &&-chains. However, all tests get run by _somebody_, so eventually we will notice problems. - it does not operate on test_when_finished or prerequisite blocks. It could, but these tends to be much shorter and less of a problem, so I punted on them in this patch. This patch was inspired by an earlier patch by Jonathan Nieder: http://article.gmane.org/gmane.comp.version-control.git/235913 This implementation and all bugs are mine. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-03-12t5541: move run_with_cmdline_limit to test-lib.shLibravatar Jeff King1-0/+6
We use this to test http pushing with a restricted commandline. Other scripts (like t5551, which does http fetching) will want to use it, too. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-03-12t: redirect stderr GIT_TRACE to descriptor 4Libravatar Jeff King1-4/+1
If you run a test script like: GIT_TRACE=1 ./t0061-run-command.sh you may get test failures, because some tests capture and check the stderr output from git commands (and with GIT_TRACE set to 1, the trace output will be included there). When we see GIT_TRACE set like this, we print a warning to the user. However, we can do even better than that by just pointing it to descriptor 4, which all tests leave connected to the test script's stderr. That's likely what the user intended (and any scripts that do want to see GIT_TRACE output will set GIT_TRACE themselves). Not only does this avoid false negatives in the tests, but it means the user will actually see trace output for git calls that redirect their stderr (whereas before, it was sometimes confusingly buried in a file). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-03-12t: translate SIGINT to an exitLibravatar Jeff King1-0/+1
Right now if a test script receives SIGINT (e.g., because a test was hanging and the user hit ^C), the shell exits immediately. This can be annoying if the test script did any global setup, like starting apache or git-daemon, as it will not have an opportunity to clean up after itself. A subsequent run of the test won't be able to start its own daemon, and will either fail or skip the tests. Instead, let's trap SIGINT to make sure we do a clean shutdown, and just chain it to a normal exit (which will trigger any cleanup). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-03-05Merge branch 'jk/sanity' into maintLibravatar Junio C Hamano1-3/+27
The tests that wanted to see that file becomes unreadable after running "chmod a-r file", and the tests that wanted to make sure it is not run as root, we used "can we write into the / directory?" as a cheap substitute, but on some platforms that is not a good heuristics. The tests and their prerequisites have been updated to check what they really require. * jk/sanity: test-lib.sh: set prerequisite SANITY by testing what we really need tests: correct misuses of POSIXPERM t/lib-httpd: switch SANITY check for NOT_ROOT
2015-02-25Merge branch 'jk/sanity'Libravatar Junio C Hamano1-3/+27
The tests that wanted to see that file becomes unreadable after running "chmod a-r file", and the tests that wanted to make sure it is not run as root, we used "can we write into the / directory?" as a cheap substitute, but on some platforms that is not a good heuristics. The tests and their prerequisites have been updated to check what they really require. * jk/sanity: test-lib.sh: set prerequisite SANITY by testing what we really need tests: correct misuses of POSIXPERM t/lib-httpd: switch SANITY check for NOT_ROOT