summaryrefslogtreecommitdiff
path: root/t/test-lib-functions.sh
AgeCommit message (Collapse)AuthorFilesLines
2018-08-08t7406: prefer test_* helper functions to test -[feds]Libravatar Elijah Newren1-0/+8
test -e, test -s, etc. do not provide nice error messages when we hit test failures, so use the test_* helper functions from test-lib-functions.sh. Also, add test_path_exists() to test-lib-function.sh while at it, so that we don't need to worry whether submodule/.git is a file or a directory. It currently is a file with contents of the form gitdir: ../.git/modules/submodule but it could be changed in the future to be a directory; this test only really cares that it exists. Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-05-23Merge branch 'js/test-unset-prereq'Libravatar Junio C Hamano1-1/+13
Test debugging aid. * js/test-unset-prereq: tests: introduce test_unset_prereq, for debugging
2018-04-30tests: introduce test_unset_prereq, for debuggingLibravatar Johannes Schindelin1-1/+13
While working on the --convert-graft-file test, I missed that I was relying on the GPG prereq, by using output of test cases that were only run under that prereq. For debugging, it was really convenient to force that prereq to be unmet, but there was no easy way to do that. So I came up with a way, and this patch reflects the cleaned-up version of that way. For convenience, the following two methods are now supported ways to pretend that a prereq is not met: test_set_prereq !GPG and test_unset_prereq GPG Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-04-25Make running git under other debugger-like programs easyLibravatar Elijah Newren1-4/+20
This allows us to run git, when using the script from bin-wrappers, under other programs. A few examples for usage within testsuite scripts: debug git checkout master debug --debugger=nemiver git $ARGS debug -d "valgrind --tool-memcheck --track-origins=yes" git $ARGS Or, if someone has bin-wrappers/ in their $PATH and is executing git outside the testsuite: GIT_DEBUGGER="gdb --args" git $ARGS GIT_DEBUGGER=nemiver git $ARGS GIT_DEBUGGER="valgrind --tool=memcheck --track-origins=yes" git $ARGS There is also a handy shortcut of GIT_DEBUGGER=1 meaning the same as GIT_DEBUGGER="gdb --args" Original-patch-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-04-11Merge branch 'jc/test-must-be-empty'Libravatar Junio C Hamano1-5/+2
Test helper update. * jc/test-must-be-empty: test_must_be_empty: simplify file existence check
2018-03-27test_must_be_empty: simplify file existence checkLibravatar SZEDER Gábor1-5/+2
Commit 11395a3b4b (test_must_be_empty: make sure the file exists, not just empty, 2018-02-27) basically duplicated the 'test_path_is_file' helper function in 'test_must_be_empty'. Just call 'test_path_is_file' to avoid this code duplication. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-03-14Merge branch 'sg/test-x'Libravatar Junio C Hamano1-12/+12
Running test scripts under -x option of the shell is often not a useful way to debug them, because the error messages from the commands tests try to capture and inspect are contaminated by the tracing output by the shell. An earlier work done to make it more pleasant to run tests under -x with recent versions of bash is extended to cover posix shells that do not support BASH_XTRACEFD. * sg/test-x: travis-ci: run tests with '-x' tracing t/README: add a note about don't saving stderr of compound commands t1510-repo-setup: mark as untraceable with '-x' t9903-bash-prompt: don't check the stderr of __git_ps1() t5570-git-daemon: don't check the stderr of a subshell t5526: use $TRASH_DIRECTORY to specify the path of GIT_TRACE log file t5500-fetch-pack: don't check the stderr of a subshell t3030-merge-recursive: don't check the stderr of a subshell t1507-rev-parse-upstream: don't check the stderr of a shell function t: add means to disable '-x' tracing for individual test scripts t: prevent '-x' tracing from interfering with test helpers' stderr
2018-03-08Merge branch 'jc/test-must-be-empty'Libravatar Junio C Hamano1-1/+5
Test framework tweak to catch developer thinko. * jc/test-must-be-empty: test_must_be_empty: make sure the file exists, not just empty
2018-03-06Merge branch 'jk/test-helper-v-output-fix'Libravatar Junio C Hamano1-10/+10
Test framework update. * jk/test-helper-v-output-fix: t: send verbose test-helper output to fd 4
2018-02-27test_must_be_empty: make sure the file exists, not just emptyLibravatar Junio C Hamano1-1/+5
The helper function test_must_be_empty is meant to make sure the given file is empty, but its implementation is: if test -s "$1" then ... not empty, we detected a failure ... fi Surely, the file having non-zero size is a sign that the condition "the file must be empty" is violated, but it misses the case where the file does not even exist. It is an accident waiting to happen with a buggy test like this: git frotz 2>error-message && test_must_be_empty errro-message that won't get caught until you deliberately break 'git frotz' and notice why the test does not fail. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-02-27t: prevent '-x' tracing from interfering with test helpers' stderrLibravatar SZEDER Gábor1-12/+12
Running a test script with '-x' turns on 'set -x' tracing, the output of which is normally sent to stderr. This causes a lot of test failures, because many tests redirect and verify the stderr of shell functions, most frequently that of 'test_must_fail'. These issues were worked around somewhat in d88785e424 (test-lib: set BASH_XTRACEFD automatically, 2016-05-11), so at least we could reliably run tests with '-x' tracing under a Bash version supporting BASH_XTRACEFD, i.e. v4.1 and later. Futhermore, redirecting the stderr of test helper functions like 'test_must_fail' or 'test_expect_code' is the cause of a different issue as well. If these functions detect something unexpected, they will write their error messages intended to the user to thier stderr. However, if their stderr is redirected in order to save and verify the stderr of the tested git command invoked in the function, then the function's error messages will be redirected as well. Consequently, those messages won't reach the user, making the test's verbose output less useful. This patch makes it safe to redirect and verify the stderr of those test helper functions which are meant to run the tested command given as argument, even when running tests with '-x' and /bin/sh. This is achieved through a couple of file descriptor redirections: - Duplicate stderr of the tested command executed in the test helper function from the function's fd 7 (see next point), to ensure that the tested command's error messages go to a different fd than the '-x' trace of the commands executed in the function or the function's error messages. - Duplicate the test helper function's fd 7 from the function's original stderr, meaning that, after taking a detour through fd 7, the error messages of the tested command do end up on the function's original stderr. - Duplicate stderr of the test helper function from fd 4, i.e. the fd connected to the test script's original stderr and the fd used for BASH_XTRACEFD. This ensures that the '-x' trace of the commands executed in the function - doesn't go to the function's original stderr, so it won't mess with callers who want to save and verify the tested command's stderr. - does go to the same fd independently from the shell running the test script, be it /bin/sh, an older Bash without BASH_XTRACEFD, or a more recent Bash already supporting BASH_XTRACEFD. Furthermore, this also makes sure that the function's error messages go to this fd 4, meaning that the user will be able to see them even if the function's stderr is redirected in the test. - Specify the latter two redirections above in the test helper function's definition, so they are performed every time the function is invoked, without the need to modify the callsites of the function. Perform these redirections in those test helper functions which can be expected to have their stderr redirected, i.e. in the functions 'test_must_fail', 'test_might_fail', 'test_expect_code', 'test_env', 'nongit', 'test_terminal' and 'perl'. Note that 'test_might_fail', 'test_env', and 'nongit' are not involved in any test failures when running tests with '-x' and /bin/sh. The other test helper functions are left unchanged, because they either don't run commands specified as their arguments, or redirecting their stderr wouldn't make sense, or both. With this change the number of failures when running the test suite with '-x' tracing and /bin/sh goes down from 340 failed tests in 43 test scripts to 22 failed tests in 6 scripts (or 23 in 7, if the system (OSX) uses an older Bash version without BASH_XTRACEFD to run 't9903-bash-prompt.sh'). Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-02-27Merge branch 'sg/doc-test-must-fail-args'Libravatar Junio C Hamano1-0/+10
Devdoc update. * sg/doc-test-must-fail-args: t: document 'test_must_fail ok=<signal-name>'
2018-02-22t: send verbose test-helper output to fd 4Libravatar Jeff King1-10/+10
Test helper functions like test_must_fail may produce messages to stderr when they see a problem. When the tests are run with "--verbose", this ends up on the test script's stderr, and the user can read it. But there's a problem. Some tests record stderr as part of the test, like: test_must_fail git foo 2>output && test_i18ngrep expected.message output In this case the error text goes into "output". This makes the --verbose output less useful (it also means we might accidentally match it in the second, though in practice we tend to produce these messages only on error, so we'd abort the test when the first command fails). Let's instead send this user-facing output directly to descriptor 4, which always points to the original stderr (or /dev/null in non-verbose mode). And it's already forbidden to redirect descriptor 4, since we use it for BASH_XTRACEFD, as explained in 9be795fbce (t5615: avoid re-using descriptor 4, 2017-12-08). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-02-21Merge branch 'sg/test-i18ngrep'Libravatar Junio C Hamano1-0/+54
Test fixes. * sg/test-i18ngrep: t: make 'test_i18ngrep' more informative on failure t: validate 'test_i18ngrep's parameters t: move 'test_i18ncmp' and 'test_i18ngrep' to 'test-lib-functions.sh' t5536: let 'test_i18ngrep' read the file without redirection t5510: consolidate 'grep' and 'test_i18ngrep' patterns t4001: don't run 'git status' upstream of a pipe t6022: don't run 'git merge' upstream of a pipe t5812: add 'test_i18ngrep's missing filename parameter t5541: add 'test_i18ngrep's missing filename parameter
2018-02-12t: document 'test_must_fail ok=<signal-name>'Libravatar SZEDER Gábor1-0/+10
Since 'test_might_fail' is implemented as a thin wrapper around 'test_must_fail', it also accepts the same options. Mention this in the docs as well. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-02-08t: make 'test_i18ngrep' more informative on failureLibravatar SZEDER Gábor1-4/+20
When 'test_i18ngrep' can't find the expected pattern, it exits completely silently; when its negated form does find the pattern that shouldn't be there, it prints the matching line(s) but otherwise exits without any error message. This leaves the developer puzzled about what could have gone wrong. Make 'test_i18ngrep' more informative on failure by printing an error message including the invoked 'grep' command and the contents of the file it had to scan through. Note that this "dump the scanned file" part is not quite perfect, as it dumps only the file specified as the function's last positional parameter, thus assuming that there is only a single file parameter. I think that's a reasonable assumption to make, one that holds true in the current code base. And even if someone were to scan multiple files at once in the future, the worst thing that could happen is that the verbose error message won't include the contents of all those files, only the last one. Alas, we can't really do any better than this, because checking whether the other positional parameters match a filename can result in false positives: 't3400-rebase.sh' and 't3404-rebase-interactive.sh' contain one test each, where the 'test_i18ngrep's pattern verbatimly matches a file in the trash directory. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Reviewed-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-02-08t: validate 'test_i18ngrep's parametersLibravatar SZEDER Gábor1-0/+12
Some of the previous patches in this series fixed bogus 'test_i18ngrep' invocations: - Two invocations where the tested git command's standard output is directly piped into 'test_i18ngrep'. While convenient, this is an antipattern, because the pipe hides the git command's exit code, and the test could continue even if the command exited with error. - Two invocations that had neither a filename parameter nor anything piped into their standard input, yet both managed to remain unnoticed for years. A third similarly bogus invocation is currently lurking in 'pu' for a couple of weeks now. Prevent similar mistakes in the future by validating 'test_i18ngrep's parameters requiring that - The last parameter names an existing file to be read, effectively forbidding piping into 'test_i18ngrep'. Note that this change will also forbid cases where 'test_i18ngrep' would legitimately read its standard input, e.g. when its standard input is redirected from a file, or when a git command's standard output is first written to an intermediate file, which is then preprocessed by a non-git command before the results are piped into 'test_i18ngrep'. See two of the previous patches for the only such cases we had in our test suite. However, reliably preventing the piping antipattern is arguably more important than supporting these cases, which can be easily worked around by opening the file directly or using an intermediate file anyway. - There are at least two parameters, not including the optional '!' to negate the pattern. This ought to catch corner cases when 'test_i18ngrep' looks for the name of an existing file on its standard input; the above check would miss this case becase the filename as pattern would be the last parameter. Note that this is not quite perfect, as it doesn't account for any 'grep --options' given as parameters. However, doing so would be far too complicated, considering that patterns can start with dashes as well, and in the majority of the cases we don't use any such options anyway. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Reviewed-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-02-08t: move 'test_i18ncmp' and 'test_i18ngrep' to 'test-lib-functions.sh'Libravatar SZEDER Gábor1-0/+26
Both 'test_i18ncmp' and 'test_i18ngrep' helper functions are supposed to be called from our test scripts, so they should be in 'test-lib-functions.sh'. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Reviewed-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-01-25t/lib-git-daemon: add network-protocol helpersLibravatar Jeff King1-0/+34
All of our git-protocol tests rely on invoking the client and having it make a request of a server. That gives a nice real-world test of how the two behave together, but it doesn't leave any room for testing how a server might react to _other_ clients. Let's add a few test helper functions which can be used to manually conduct a git-protocol conversation with a remote git-daemon: 1. To connect to a remote git-daemon, we need something like "netcat". But not everybody will have netcat. And even if they do, the behavior with respect to half-duplex shutdowns is not portable (openbsd netcat has "-N", with others you must rely on "-q 1", which is racy). Here we provide a "fake_nc" that is capable of doing a client-side netcat, with sane half-duplex semantics. It relies on perl's IO::Socket::INET. That's been in the base distribution since 5.6.0, so it's probably available everywhere. But just to be on the safe side, we'll add a prereq. 2. To help tests speak and read pktline, this patch adds packetize() and depacketize() functions. I've put fake_nc() into lib-git-daemon.sh, since that's really the only server where we'd need to use a network socket. Whereas the pktline helpers may be of more general use, so I've added them to test-lib-functions.sh. Programs like upload-pack speak pktline, but can talk directly over stdio without a network socket. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-08-23Merge branch 'jk/ref-filter-colors' into maintLibravatar Junio C Hamano1-0/+1
"%C(color name)" in the pretty print format always produced ANSI color escape codes, which was an early design mistake. They now honor the configuration (e.g. "color.ui = never") and also tty-ness of the output medium. * jk/ref-filter-colors: ref-filter: consult want_color() before emitting colors pretty: respect color settings for %C placeholders rev-list: pass diffopt->use_colors through to pretty-print for-each-ref: load config earlier color: check color.ui in git_default_config() ref-filter: pass ref_format struct to atom parsers ref-filter: factor out the parsing of sorting atoms ref-filter: make parse_ref_filter_atom a private function ref-filter: provide a function for parsing sort options ref-filter: move need_color_reset_at_eol into ref_format ref-filter: abstract ref format into its own struct ref-filter: simplify automatic color reset t: use test_decode_color rather than literal ANSI codes docs/for-each-ref: update pointer to color syntax check return value of verify_ref_format()
2017-08-11Merge branch 'jk/ref-filter-colors'Libravatar Junio C Hamano1-0/+1
"%C(color name)" in the pretty print format always produced ANSI color escape codes, which was an early design mistake. They now honor the configuration (e.g. "color.ui = never") and also tty-ness of the output medium. * jk/ref-filter-colors: ref-filter: consult want_color() before emitting colors pretty: respect color settings for %C placeholders rev-list: pass diffopt->use_colors through to pretty-print for-each-ref: load config earlier color: check color.ui in git_default_config() ref-filter: pass ref_format struct to atom parsers ref-filter: factor out the parsing of sorting atoms ref-filter: make parse_ref_filter_atom a private function ref-filter: provide a function for parsing sort options ref-filter: move need_color_reset_at_eol into ref_format ref-filter: abstract ref format into its own struct ref-filter: simplify automatic color reset t: use test_decode_color rather than literal ANSI codes docs/for-each-ref: update pointer to color syntax check return value of verify_ref_format()
2017-07-17t: handle EOF in test_copy_bytes()Libravatar Jeff King1-0/+1
The test_copy_bytes() function claims to read up to N bytes, or until it gets EOF. But we never handle EOF in our loop, and a short input will cause perl to go into an infinite loop of read() getting zero bytes. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-07-13t: use test_decode_color rather than literal ANSI codesLibravatar Jeff King1-0/+1
When we put literal ANSI terminal codes into our test scripts, it makes diffs on those scripts hard to read (the colors may be indistinguishable from diff coloring, or in the case of a reset, may not be visible at all). Some scripts get around this by including human-readable names and converting to literal codes with a git-config hack. This makes the actual code diffs look OK, but test_cmp output suffers from the same problem. Let's use test_decode_color instead, which turns the codes into obvious text tags. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-25t1301: move modebits() to test-lib-functions.shLibravatar Christian Couder1-0/+5
As the modebits() function can be useful outside t1301, let's move it into test-lib-functions.sh, and while at it let's rename it test_modebits(). Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-18tests: make the 'test_pause' helper work in non-verbose modeLibravatar SZEDER Gábor1-7/+2
When the 'test_pause' helper function invokes the shell mid-test, it explicitly redirects the shell's stdout and stderr to file descriptors 3 and 4, which are the stdout and stderr of the tests (i.e. where they would be connected anyway without those redirections). These file descriptors are only attached to the terminal in verbose mode, hence the restriction of 'test_pause' to work only with '-v'. Redirect the shell's stdout and stderr to the test environment's original stdout and stderr, allowing it to work properly even in non-verbose mode, and the restriction can be lifted. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-18tests: create an interactive gdb session with the 'debug' helperLibravatar SZEDER Gábor1-1/+1
The 'debug' test helper is supposed to facilitate debugging by running a command of the test suite under gdb. Unfortunately, its usefulness is severely limited, because that gdb session is not interactive, since the test's, and thus gdb's standard input is redirected from /dev/null (for a good reason, see 781f76b15 (test-lib: redirect stdin of tests, 2011-12-15)). Redirect gdb's standard file descriptors from/to the test environment's stdin, stdout and stderr in the 'debug' helper, thus creating an interactive gdb session (even in non-verbose mode), which is much, much more useful. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-01-10Merge branch 'sb/submodule-embed-gitdir'Libravatar Junio C Hamano1-5/+15
A new submodule helper "git submodule embedgitdirs" to make it easier to move embedded .git/ directory for submodules in a superproject to .git/modules/ (and point the latter with the former that is turned into a "gitdir:" file) has been added. * sb/submodule-embed-gitdir: worktree: initialize return value for submodule_uses_worktrees submodule: add absorb-git-dir function move connect_work_tree_and_git_dir to dir.h worktree: check if a submodule uses worktrees test-lib-functions.sh: teach test_commit -C <dir> submodule helper: support super prefix submodule: use absolute path for computing relative path connecting
2016-12-16t5000: extract nongit function to test-lib-functions.shLibravatar Jeff King1-0/+14
This function abstracts the idea of running a command outside of any repository (which is slightly awkward to do because even if you make a non-repo directory, git may keep walking up outside of the trash directory). There are several scripts that use the same technique, so let's make the function available for everyone. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-09test-lib-functions.sh: teach test_commit -C <dir>Libravatar Stefan Beller1-5/+15
Specifically when setting up submodule tests, it comes in handy if we can create commits in repositories that are not at the root of the tested trash dir. Add "-C <dir>" similar to gits -C parameter that will perform the operation in the given directory. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-11test-lib-functions.sh: add lf_to_nul helperLibravatar Jeff Hostetler1-0/+4
Add lf_to_nul helper function to test-lib-functions. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-19Merge branch 'jk/test-match-signal'Libravatar Junio C Hamano1-1/+16
The test framework learned a new helper test_match_signal to check an exit code from getting killed by an expected signal. * jk/test-match-signal: t/lib-git-daemon: use test_match_signal test_must_fail: use test_match_signal t0005: use test_match_signal as appropriate tests: factor portable signal check out of t0005
2016-07-06test_must_fail: use test_match_signalLibravatar Jeff King1-1/+1
In 8bf4bec (add "ok=sigpipe" to test_must_fail and use it to fix flaky tests, 2015-11-27), test_must_fail learned to recognize "141" as a sigpipe failure. However, testing for a signal is more complicated than that; we should use test_match_signal to implement more portable checking. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-06tests: factor portable signal check out of t0005Libravatar Jeff King1-0/+15
In POSIX shells, a program which exits due to a signal generally has an exit code of 128 plus the signal number. However, ksh uses 256 plus the signal number. We've accounted for that in t0005, but not in other tests. Let's pull out the logic so we can use it elsewhere. It would be nice for debugging if this additionally printed errors to stderr, like our other test_* helpers. But we're going to need to use it in other places besides the innards of a test_expect block. So let's leave it as generic as possible. Note that we also leave the magic "3" for Windows out of the generic helper. This is an artifact of the way we use raise() to kill ourselves in test-sigchain.c, and will not necessarily apply to all programs. So it's better to keep it out of the helper, to reduce the chance of confusing it with a real call to exit(3). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-01t9300: factor out portable "head -c" replacementLibravatar Jeff King1-0/+14
It is sometimes useful to be able to read exactly N bytes from a pipe. Doing this portably turns out to be surprisingly difficult in shell scripts. We want a solution that: - is portable - never reads more than N bytes due to buffering (which would mean those bytes are not available to the next program to read from the same pipe) - handles partial reads by looping until N bytes are read (or we see EOF) - is resilient to stray signals giving us EINTR while trying to read (even though we don't send them, things like SIGWINCH could cause apparently-random failures) Some possible solutions are: - "head -c" is not portable, and implementations may buffer (though GNU head does not) - "read -N" is a bash-ism, and thus not portable - "dd bs=$n count=1" does not handle partial reads. GNU dd has iflags=fullblock, but that is not portable - "dd bs=1 count=$n" fixes the partial read problem (all reads are 1-byte, so there can be no partial response). It does make a lot of write() calls, but for our tests that's unlikely to matter. It's fairly portable. We already use it in our tests, and it's unlikely that implementations would screw up any of our criteria. The most unknown one would be signal handling. - perl can do a sysread() loop pretty easily. On my Linux system, at least, it seems to restart the read() call automatically. If that turns out not to be portable, though, it would be easy for us to handle it. That makes the perl solution the least bad (because we conveniently omitted "length of code" as a criterion). It's also what t9300 is currently using, so we can just pull the implementation from there. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-01test-lib: add in-shell "env" replacementLibravatar Jeff King1-0/+22
The one-shot environment variable syntax: FOO=BAR some-program is unportable when some-program is actually a shell function, like test_must_fail (on some shells FOO remains set after the function returns, and on others it does not). We sometimes get around this by using env, like: test_must_fail env FOO=BAR some-program But that only works because test_must_fail's arguments are themselves a command which can be run. You can't run: env FOO=BAR test_must_fail some-program because env does not know about our shell functions. So there is no equivalent for test_commit, for example, and one must resort to: ( FOO=BAR export FOO test_commit ) which is a bit verbose. Let's add a version of "env" that works _inside_ the shell, by creating a subshell, exporting variables from its argument list, and running the command. Its use is demonstrated on a currently-unportable case in t4014. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-26Merge branch 'jc/test-seq' into maintLibravatar Junio C Hamano1-15/+13
Test fix. * jc/test-seq: test-lib-functions.sh: rewrite test_seq without Perl test-lib-functions.sh: remove misleading comment on test_seq
2016-05-09test-lib-functions.sh: rewrite test_seq without PerlLibravatar Junio C Hamano1-1/+6
Rewrite the 'seq' imitation using only commands and features that are typically found built into modern POSIX shells, instead of relying on Perl to run a single-liner script. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-09test-lib-functions.sh: remove misleading comment on test_seqLibravatar Junio C Hamano1-14/+7
We never used the "letters" form since we came up with "test_seq" to replace use of non-portable "seq" in our test script, which we introduced it at d17cf5f3 (tests: Introduce test_seq, 2012-08-04). We use this helper to either iterate for N times (i.e. the values on the lines do not even matter), or just to get N distinct strings (i.e. the values on the lines themselves do not really matter, but we care that they are different from each other and reproducible). Stop promising that we may allow using "letters"; this would open an easier reimplementation that does not rely on $PERL, if somebody later wants to. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-02-25test_must_fail: report number of unexpected signalLibravatar Jeff King1-1/+1
If a command is marked as test_must_fail but dies with a signal, we consider that a problem and report the error to stderr. However, we don't say _which_ signal; knowing that can make debugging easier. Let's share as much as we know. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-11-28add "ok=sigpipe" to test_must_fail and use it to fix flaky testsLibravatar Lars Schneider1-0/+3
t5516 "75 - deny fetch unreachable SHA1, allowtipsha1inwant=true" is flaky in the following case: 1. remote upload-pack finds out "not our ref" 2. remote sends a response and closes the pipe 3. fetch-pack still tries to write commands to the remote upload-pack 4. write call in wrapper.c dies with SIGPIPE The test is flaky because the sending fetch-pack may or may not have finished writing its output by step (3). If it did, then we see a closed pipe on the next read() call. If it didn't, then we get the SIGPIPE from step (4) above. Both are fine, but the latter fools test_must_fail. t5504 "9 - push with transfer.fsckobjects" is flaky, too, and returns SIGPIPE once in a while. I had to remove the final "To dst..." output check because there is no output if the process dies with SIGPIPE. Accept such a death-with-sigpipe also as OK when we are expecting a failure. Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Jeff King <peff@peff.net>
2015-11-28implement test_might_fail using a refactored test_must_failLibravatar Lars Schneider1-14/+33
Add an (optional) first parameter "ok=<special case>" to test_must_fail and return success for "<special case>". Add "success" as "<special case>" and use it to implement "test_might_fail". This removes redundancies in test-lib-function.sh. You can pass multiple <special case> arguments divided by comma (e.g. "test_must_fail ok=success,something") Signed-off-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Ramsay Jones <ramsay@ramsayjones.plus.com> Signed-off-by: Jeff King <peff@peff.net>
2015-10-30test: facilitate debugging Git executables in tests with gdbLibravatar Johannes Schindelin1-0/+8
When prefixing a Git call in the test suite with 'debug ', it will now be run with GDB, allowing the developer to debug test failures more conveniently. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-09-08test-lib-functions: detect test_when_finished in subshellLibravatar John Keeping1-0/+5
test_when_finished does nothing in a subshell because the change to test_cleanup does not affect the parent. There is no POSIX way to detect that we are in a subshell ($$ and $PPID are specified to remain unchanged), but we can detect it on Bash and fall back to ignoring the bug on other shells. Signed-off-by: John Keeping <john@keeping.me.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-09-08test-lib-functions: support "test_config -C <dir> ..."Libravatar John Keeping1-3/+17
If used in a subshell, test_config cannot unset variables at the end of a test. This is a problem when testing submodules because we do not want to "cd" at to top level of a test script in order to run the command inside the submodule. Add a "-C" option to test_config (and test_unconfig) so that test_config can be kept outside subshells and still affect subrepositories. Signed-off-by: John Keeping <john@keeping.me.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-05-19Merge branch 'jc/test-prereq-validate'Libravatar Junio C Hamano1-0/+9
Help us to find broken test script that splits the body part of the test by mistaken use of wrong kind of quotes. * jc/test-prereq-validate: test: validate prerequistes syntax
2015-05-13Merge branch 'ep/fix-test-lib-functions-report' into maintLibravatar Junio C Hamano1-2/+2
* ep/fix-test-lib-functions-report: test-lib-functions.sh: fix the second argument to some helper functions
2015-05-05Merge branch 'ep/fix-test-lib-functions-report'Libravatar Junio C Hamano1-2/+2
* ep/fix-test-lib-functions-report: test-lib-functions.sh: fix the second argument to some helper functions
2015-04-28test: validate prerequistes syntaxLibravatar Junio C Hamano1-0/+9
Brian Carson noticed that a test piece in t5601 had a pair of single quotes in the body, which made it into 4 parameter call to test_expect_success, as if its test title were a prerequisite. As the prerequisites have a specific syntax (i.e. comma separated tokens spelled in capital letters, possibly prefixed with ! for negation), validate them to catch such a mistake in the future. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-04-16test-lib-functions.sh: fix the second argument to some helper functionsLibravatar Elia Pinto1-2/+2
The second argument to test_path_is_file and test_path_is_dir must be $2 and not $*, which instead would repeat the file name in the error message. Signed-off-by: Elia Pinto <gitter.spiros@gmail.com> Reviewed-by: Matthieu Moy <Matthieu.Moy@grenoble-inp.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-03-13Merge branch 'jc/diff-test-updates' into maintLibravatar Junio C Hamano1-1/+3
Test clean-up. * jc/diff-test-updates: test_ln_s_add: refresh stat info of fake symbolic links t4008: modernise style t/diff-lib: check exact object names in compare_diff_raw tests: do not borrow from COPYING and README from the real source t4010: correct expected object names t9300: correct expected object names t4008: correct stale comments