summaryrefslogtreecommitdiff
path: root/t/t0000-basic.sh
AgeCommit message (Collapse)AuthorFilesLines
2019-03-14test-lib: introduce 'test_atexit'Libravatar Johannes Schindelin1-0/+18
When running Apache, 'git daemon', or p4d, we want to kill them at the end of the test script, otherwise a leftover daemon process will keep its port open indefinitely, and thus will interfere with subsequent executions of the same test script. So far, we stop these daemon processes "manually", i.e.: - by registering functions or commands in the trap on EXIT to stop the daemon while preserving the last seen exit code before the trap (to deal with a failure when run with '--immediate' or with interrupts by ctrl-C), - and by invoking these functions/commands last thing before 'test_done' (and sometimes restoring the test framework's default trap on EXIT, to prevent the daemons from being killed twice). On one hand, we do this inconsistently, e.g. 'git p4' tests invoke different functions in the trap on EXIT and in the last test before 'test_done', and they neither restore the test framework's default trap on EXIT nor preserve the last seen exit code. On the other hand, this is error prone, because, as shown in a previous patch in this series, any output from the cleanup commands in the trap on EXIT can prevent a proper cleanup when a test script run with '--verbose-log' and certain shells, notably 'dash', is interrupted. Let's introduce 'test_atexit', which is loosely modeled after 'test_when_finished', but has a broader scope: rather than running the commands after the current test case, run them when the test script finishes, and also run them when the test is interrupted, or exits early in case of a failure while the '--immediate' option is in effect. When running the cleanup commands at the end of a successful test, then they will be run in 'test_done' before it removes the trash directory, i.e. the cleanup commands will still be able to access any pidfiles or socket files in there. When running the cleanup commands after an interrupt or failure with '--immediate', then they will be run in the trap on EXIT. In both cases they will be run in 'test_eval_', i.e. both standard error and output of all cleanup commands will go where they should according to the '-v' or '--verbose-log' options, and thus won't cause any troubles when interrupting a test script run with '--verbose-log'. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-11-09i18n: make GETTEXT_POISON a runtime optionLibravatar Ævar Arnfjörð Bjarmason1-1/+1
Change the GETTEXT_POISON compile-time + runtime GIT_GETTEXT_POISON test parameter to only be a GIT_TEST_GETTEXT_POISON=<non-empty?> runtime parameter, to be consistent with other parameters documented in "Running tests with special setups" in t/README. When I added GETTEXT_POISON in bb946bba76 ("i18n: add GETTEXT_POISON to simulate unfriendly translator", 2011-02-22) I was concerned with ensuring that the _() function would get constant folded if NO_GETTEXT was defined, and likewise that GETTEXT_POISON would be compiled out unless it was defined. But as the benchmark in my [1] shows doing a one-off runtime getenv("GIT_TEST_[...]") is trivial, and since GETTEXT_POISON was originally added the GIT_TEST_* env variables have become the common idiom for turning on special test setups. So change GETTEXT_POISON to work the same way. Now the GETTEXT_POISON=YesPlease compile-time option is gone, and running the tests with GIT_TEST_GETTEXT_POISON=[YesPlease|] can be toggled on/off without recompiling. This allows for conditionally amending tests to test with/without poison, similar to what 859fdc0c3c ("commit-graph: define GIT_TEST_COMMIT_GRAPH", 2018-08-29) did for GIT_TEST_COMMIT_GRAPH. Do some of that, now we e.g. always run the t0205-gettext-poison.sh test. I did enough there to remove the GETTEXT_POISON prerequisite, but its inverse C_LOCALE_OUTPUT is still around, and surely some tests using it can be converted to e.g. always set GIT_TEST_GETTEXT_POISON=. Notes on the implementation: * We still compile a dedicated GETTEXT_POISON build in Travis CI. Perhaps this should be revisited and integrated into the "linux-gcc" build, see ae59a4e44f ("travis: run tests with GIT_TEST_SPLIT_INDEX", 2018-01-07) for prior art in that area. Then again maybe not, see [2]. * We now skip a test in t0000-basic.sh under GIT_TEST_GETTEXT_POISON=YesPlease that wasn't skipped before. This test relies on C locale output, but due to an edge case in how the previous implementation of GETTEXT_POISON worked (reading it from GIT-BUILD-OPTIONS) wasn't enabling poison correctly. Now it does, and needs to be skipped. * The getenv() function is not reentrant, so out of paranoia about code of the form: printf(_("%s"), getenv("some-env")); call use_gettext_poison() in our early setup in git_setup_gettext() so we populate the "poison_requested" variable in a codepath that's won't suffer from that race condition. * We error out in the Makefile if you're still saying GETTEXT_POISON=YesPlease to prompt users to change their invocation. * We should not print out poisoned messages during the test initialization itself to keep it more readable, so the test library hides the variable if set in $GIT_TEST_GETTEXT_POISON_ORIG during setup. See [3]. See also [4] for more on the motivation behind this patch, and the history of the GETTEXT_POISON facility. 1. https://public-inbox.org/git/871s8gd32p.fsf@evledraar.gmail.com/ 2. https://public-inbox.org/git/20181102163725.GY30222@szeder.dev/ 3. https://public-inbox.org/git/20181022202241.18629-2-szeder.dev@gmail.com/ 4. https://public-inbox.org/git/878t2pd6yu.fsf@evledraar.gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-19Merge branch 'bp/rename-test-env-var'Libravatar Junio C Hamano1-0/+4
Some environment variables that control the runtime options of Git used during tests are getting renamed for consistency. * bp/rename-test-env-var: t0000: do not get self-test disrupted by environment warnings preload-index: update GIT_FORCE_PRELOAD_TEST support read-cache: update TEST_GIT_INDEX_VERSION support fsmonitor: update GIT_TEST_FSMONITOR support preload-index: use git_env_bool() not getenv() for customization t/README: correct spelling of "uncommon"
2018-10-16Merge branch 'md/test-cleanup'Libravatar Junio C Hamano1-1/+1
Various test scripts have been updated for style and also correct handling of exit status of various commands. * md/test-cleanup: tests: order arguments to git-rev-list properly t9109: don't swallow Git errors upstream of pipes tests: don't swallow Git errors upstream of pipes t/*: fix ordering of expected/observed arguments tests: standardize pipe placement Documentation: add shell guidelines t/README: reformat Do, Don't, Keep in mind lists
2018-10-07t/*: fix ordering of expected/observed argumentsLibravatar Matthew DeVore1-1/+1
Fix various places where the ordering was obviously wrong, meaning it was easy to find with grep. Signed-off-by: Matthew DeVore <matvore@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-28t0000: do not get self-test disrupted by environment warningsLibravatar Junio C Hamano1-0/+4
The test framework test-lib.sh itself would want to give warnings and hints, e.g. when it sees a deprecated environment variable is in use that we want to encourage users to migrate to another variable. The self-test of test framework done in t0000 however do not expect to see these warnings and hints, so depending on the settings of environment variables, a running test may or may not produce these messages to the standard error output, breaking the expectations of self-test test framework does on itself. Here is what we see: $ TEST_GIT_INDEX_VERSION=4 sh t0000-basic.sh -i -v ... 'err' is not empty, it contains: warning: TEST_GIT_INDEX_VERSION is now GIT_TEST_INDEX_VERSION hint: set GIT_TEST_INDEX_VERSION too during the transition period not ok 5 - pretend we have a fully passing test suite The following quick attempt to work it around does not work, because some tests in t0000 do want to see expected errors from the test framework itself. t/t0000-basic.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/t/t0000-basic.sh b/t/t0000-basic.sh index 850f651e4e..88c6ed4696 100755 --- a/t/t0000-basic.sh +++ b/t/t0000-basic.sh @@ -88,7 +88,7 @@ _run_sub_test_lib_test_common () { ' # Point to the t/test-lib.sh, which isn't in ../ as usual - . "\$TEST_DIRECTORY"/test-lib.sh + . "\$TEST_DIRECTORY"/test-lib.sh >/dev/null 2>&1 EOF cat >>"$name.sh" && chmod +x "$name.sh" && There are a few possible ways to work this around: * We could strip the warning: and hint: unconditionally from the error output before the error messages are checked in the self-test (helper functions check_sub_test_lib_test_err and check_sub_test_lib_test); the problem with this approach is that it will make it impossible to write self-tests to ensure that right warnings and hints are given. * We could force a sane environment settings before the test helper _run_sub_test_lib_test_common dot-sources test-lib.sh; the problem with this approach is that _run_sub_test_lib_test_common now needs to be aware of what pairs of environment variables are checked in test-lib.sh using check_var_migration helper. The final patch I came up with is probably the solution that is least bad. Set a variable to tell test-lib.sh that we are running a self-test, so that various pieces in test-lib.sh can react to keep the output stable. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-13t0000: update tests for SHA-256Libravatar brian m. carlson1-61/+102
Test t0000 tests the "basics of the basics" and as such, checks that we have various fixed hard-coded object IDs. The tests relying on these assertions have been marked with the SHA1 prerequisite, as they will obviously not function in their current form with SHA-256. Use the test_oid helper to update these assertions and provide values for both SHA-1 and SHA-256. These object IDs were synthesized using a set of scripts that created the objects for both SHA-1 and SHA-256 using the same method to ensure that they are indeed the correct values. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-13t0000: use hash translation tableLibravatar brian m. carlson1-6/+7
If the hash we're using is 32 bytes in size, attempting to insert a 20-byte object name won't work. Since these are synthesized objects that are almost all zeros, look them up in a translation table. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-13t: add test functions to translate hash-related valuesLibravatar brian m. carlson1-0/+37
Add several test functions to make working with various hash-related values easier. Add test_oid_init, which loads common hash-related constants and placeholder object IDs from the newly added files in t/oid-info. Provide values for these constants for both SHA-1 and SHA-256. Add test_oid_cache, which accepts data on standard input in the form of hash-specific key-value pairs that can be looked up later, using the same format as the files in t/oid-info. Document this format in a t/oid-info/README directory so that it's easier to use in the future. Add test_oid, which is used to specify look up a per-hash value (produced on standard output) based on the key specified as its argument. Usually the data to be looked up will be a hash-related constant (such as the size of the hash in binary or hexadecimal), a well-known or placeholder object ID (such as the all-zeros object ID or one consisting of "deadbeef" repeated), or something similar. For these reasons, test_oid will usually be used within a command substitution. Consequently, redirect the error output to standard error, since otherwise it will not be displayed. Add test_detect_hash, which currently only detects SHA-1, and test_set_hash, which can be used to set a different hash algorithm for test purposes. In the future, test_detect_hash will learn to actually detect the hash depending on how the testsuite is to be run. Use the local keyword within these functions to avoid overwriting other shell variables. We have had a test balloon in place for a couple of releases to catch shells that don't have this keyword and have not received any reports of failure. Note that the varying usages of local used here are supported by all common open-source shells supporting the local keyword. Test these new functions as part of t0000, which also serves to demonstrate basic usage of them. In addition, add documentation on how to format the lookup data and how to use the test functions. Implement two basic lookup charts, one for common invalid or synthesized object IDs, and one for various facts about the hash function in use. Provide versions of the data for both SHA-1 and SHA-256. Since we use shell variables for storage, names used for lookup can currently consist only of shell identifier characters. If this is a problem in the future, we can hash the names before use. Improved-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-27Merge branch 'sg/test-must-be-empty'Libravatar Junio C Hamano1-1/+1
Test fixes. * sg/test-must-be-empty: tests: use 'test_must_be_empty' instead of 'test_cmp <empty> <out>' tests: use 'test_must_be_empty' instead of 'test_cmp /dev/null <out>' tests: use 'test_must_be_empty' instead of 'test ! -s' tests: use 'test_must_be_empty' instead of '! test -s'
2018-08-21tests: use 'test_must_be_empty' instead of '! test -s'Libravatar SZEDER Gábor1-1/+1
Using 'test_must_be_empty' is preferable to '! test -s', because it gives a helpful error message if the given file is unexpectedly not empty, while the latter remains completely silent. Furthermore, it also catches cases when the given file unexpectedly does not exist at all. This patch was basically created by: sed -i -e 's/! test -s/test_must_be_empty/' t[0-9]*.sh with the following notable exceptions: - The '! test -s' check in '.gitmodules ignore=dirty suppresses submodules with untracked content' in 't7508-status.sh' is left as-is, because it's bogus and, therefore, it's subject of a dedicated patch. - The '! test -s' checks in 't9131-git-svn-empty-symlink.sh' and 't9135-git-svn-moved-branch-empty-file.sh' are immediately preceeded by a 'test -f' to ensure that the files exist in the first place. 'test_must_be_empty' ensures that as well, so those 'test -f' commands are removed as well. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-16t0000-t0999: fix broken &&-chainsLibravatar Eric Sunshine1-1/+1
Signed-off-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-05-14t0000: annotate with SHA1 prerequisiteLibravatar brian m. carlson1-12/+12
Since this is a core test that tests basic functionality, annotate the assertions that have dependencies on SHA-1 with the appropriate prerequisite. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-10-31t0000: check whether the shell supports the "local" keywordLibravatar Michael Haggerty1-0/+25
Add a test balloon to see if we get complaints from anybody who is using a shell that doesn't support the "local" keyword. If so, this test can be reverted. If not, we might want to consider using "local" in shell code throughout the git code base. Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-18test-lib.sh: introduce and use $EMPTY_TREELibravatar Nguyễn Thái Ngọc Duy1-1/+1
This is a special SHA1. Let's keep it at one place, easier to replace later when the hash change comes, easier to recognize. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-06typofix: assorted typofixes in comments, documentation and messagesLibravatar Li Peng1-1/+1
Many instances of duplicate words (e.g. "the the path") and a few typoes are fixed, originally in multiple patches. wildmatch: fix duplicate words of "the" t: fix duplicate words of "output" transport-helper: fix duplicate words of "read" Git.pm: fix duplicate words of "return" path: fix duplicate words of "look" pack-protocol.txt: fix duplicate words of "the" precompose-utf8: fix typo of "sequences" split-index: fix typo worktree.c: fix typo remote-ext: fix typo utf8: fix duplicate words of "the" git-cvsserver: fix duplicate words Signed-off-by: Li Peng <lip@dtdream.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-03-20t: fix trivial &&-chain breakageLibravatar Jeff King1-2/+2
These are tests which are missing a link in their &&-chain, but during a setup phase. We may fail to notice failure in commands that build the test environment, but these are typically not expected to fail at all (but it's still good to double-check that our test environment is what we expect). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-06t0000-*.sh: fix the GIT_SKIP_TESTS sub-testsLibravatar Ramsay Jones1-51/+57
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-06test-lib: '--run' to run only specific testsLibravatar Ilya Bobyr1-4/+352
Allow better control of the set of tests that will be executed for a single test suite. Mostly useful while debugging or developing as it allows to focus on a specific test. Signed-off-by: Ilya Bobyr <ilya.bobyr@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-06test-lib: tests skipped by GIT_SKIP_TESTS say soLibravatar Ilya Bobyr1-0/+63
We used to show "(missing )" next to tests skipped because they are specified in GIT_SKIP_TESTS. Use "(GIT_SKIP_TESTS)" instead. Plus tests that check basic GIT_SKIP_TESTS functions. Signed-off-by: Ilya Bobyr <ilya.bobyr@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-01-02t0000: drop "known breakage" testLibravatar Jeff King1-3/+0
Having a simulated "known breakage" test means that the test suite will always tell us there is a bug to be fixed, even though it is only simulated. The right way to test this is in a sub-test, that can also check that we provide the correct exit status and output. Fortunately, we already have such a test (added much later by 5ebf89e). We could arguably get rid of the simulated success test immediately above, as well, as it is also redundant with the tests added in 5ebf89e. However, it does not have the annoying behavior of the "known breakage" test. It may also be easier to debug if the test suite is truly broken, since it is not a test-within-a-test, as the later tests are. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-01-02t0000: simplify HARNESS_ACTIVE hackLibravatar Jeff King1-9/+5
Commit 517cd55 set HARNESS_ACTIVE unconditionally in sub-tests, because that value affects the output of "--verbose". t0000 needs stable output from its sub-tests, and we may or may not be running under a TAP harness. That commit made the decision to always set the variable, since it has another useful side effect, which is suppressing writes to t/test-results by the sub-tests (which would just pollute the real results). Since the last commit, though, the sub-tests have their own test-results directories, so this is no longer an issue. We can now update a few comments that are no longer accurate nor necessary. We can also revisit the choice of HARNESS_ACTIVE. Since we must choose one value for stability, it's probably saner to have it off. This means that future patches could test things like the test-results writing, or the "--quiet" option, which is currently ignored when run under a harness. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-01-02t0000: set TEST_OUTPUT_DIRECTORY for sub-testsLibravatar Jeff King1-0/+2
Running t0000 produces more trash directories than expected and does not clean up after itself: $ ./t0000-basic.sh [...] $ ls -d trash\ directory.* trash directory.failing-cleanup trash directory.mixed-results1 trash directory.mixed-results2 trash directory.partial-pass trash directory.test-verbose trash directory.test-verbose-only-2 These scratch areas for sub-tests should be under the t0000 trash directory, but because TEST_OUTPUT_DIRECTORY defaults to TEST_DIRECTORY, which is exported to help sub-tests find test-lib.sh, the sub-test trash directories are created under the toplevel t/ directory instead. Because some of the sub-tests simulate failures, their trash directories are kept around. Fix it by explicitly setting TEST_OUTPUT_DIRECTORY appropriately for sub-tests. An alternative fix would be to pass the --root parameter that only specifies where to put the trash directories, which would also work. However, using TEST_OUTPUT_DIRECTORY is more futureproof in case tests want to write more output in addition to the test-results/ (which are already suppressed in sub-tests using the HARNESS_ACTIVE setting) and trash directories. This fixes a regression introduced by 38b074d (t/test-lib.sh: fix TRASH_DIRECTORY handling, 2013-04-14). Before that commit, the TEST_OUTPUT_DIRECTORY setting was not respected consistently so most tests did their work in a "trash" subdirectory of the current directory instead of the output dir. Signed-off-by: Jeff King <peff@peff.net> Clarified-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-07-11Merge branch 'tr/test-v-and-v-subtest-only'Libravatar Junio C Hamano1-1/+2
Finishing touches to a topic that is already in master for the upcoming release. * tr/test-v-and-v-subtest-only: t0000: do not use export X=Y
2013-07-08t0000: do not use export X=YLibravatar Torsten Bögershausen1-1/+2
The shell syntax "export X=Y A=B" is not understood by all shells. Signed-off-by: Torsten Bögershausen <tboegi@web.de> Acked-by: Thomas Rast <trast@inf.ethz.ch> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-07-05Merge branch 'tr/test-v-and-v-subtest-only'Libravatar Junio C Hamano1-1/+60
Allows N instances of tests run in parallel, each running 1/N parts of the test suite under Valgrind, to speed things up. * tr/test-v-and-v-subtest-only: perf-lib: fix start/stop of perf tests test-lib: support running tests under valgrind in parallel test-lib: allow prefixing a custom string before "ok N" etc. test-lib: valgrind for only tests matching a pattern test-lib: verbose mode for only tests matching a pattern test-lib: self-test that --verbose works test-lib: rearrange start/end of test_expect_* and test_skip test-lib: refactor $GIT_SKIP_TESTS matching test-lib: enable MALLOC_* for the actual tests
2013-06-23test-lib: verbose mode for only tests matching a patternLibravatar Thomas Rast1-0/+24
With the new --verbose-only=<pattern> option, one can enable --verbose at a per-test granularity. The pattern is matched against the test number, e.g. ./t0000-basic.sh --verbose-only='2[0-2]' to see only the full output of test 20-22, while showing the rest in the one-liner format. As suggested by Jeff King, this takes care to wrap the entire test_expect_* block, but nothing else, in the verbose toggling. We can use the test_start/end functions from the previous commit for the purpose. This is arguably not *too* useful on its own, but makes the next patch easier to follow. Helped-by: Jeff King <peff@peff.net> Signed-off-by: Thomas Rast <trast@inf.ethz.ch> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-06-23test-lib: self-test that --verbose worksLibravatar Thomas Rast1-1/+36
t0000 contains some light self-tests of test-lib.sh, but --verbose was not covered. Add a test. The only catch is that the presence of a test harness influences the output (specifically, the presence of some empty lines). So we need to unset TEST_HARNESS or set it to a known value. Leaving it unset leads to spurious test failures in the final summary, which come from the subtest. So we always set it. Signed-off-by: Thomas Rast <trast@inf.ethz.ch> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-06-07t0000: use test_ln_s_add to remove SYMLINKS prerequisiteLibravatar Johannes Sixt1-29/+10
t0000-basic hard-codes many object IDs. To cater to file systems that do not support symbolic links, different IDs are used depending on the SYMLINKS prerequisite. But we can observe the symbolic links are only needed to generate index entries. Use test_ln_s_add to generate the index entries and get rid of explicit SYMLINKS checks. This undoes the special casing introduced in this test by 704a3143 (Use prerequisite tags to skip tests that depend on symbolic links, 2009-03-04). Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-01-03Merge branch 'as/test-tweaks'Libravatar Junio C Hamano1-48/+166
Output from the tests is coloured using "green is okay, yellow is questionable, red is bad and blue is informative" scheme. * as/test-tweaks: tests: paint unexpectedly fixed known breakages in bold red tests: test the test framework more thoroughly tests: refactor mechanics of testing in a sub test-lib tests: change info messages from yellow/brown to cyan tests: paint skipped tests in blue tests: paint known breakages in yellow tests: test number comes first in 'not ok $count - $message'
2012-12-20tests: paint unexpectedly fixed known breakages in bold redLibravatar Adam Spiers1-6/+24
Change color of unexpectedly fixed known breakages to bold red. An unexpectedly passing test indicates that the test code is somehow broken or out of sync with the code it is testing. Either way this is an error which is potentially as bad as a failing test, and as such is no longer portrayed as a pass in the output. Signed-off-by: Adam Spiers <git@adamspiers.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-12-20tests: test the test framework more thoroughlyLibravatar Adam Spiers1-0/+105
Add 5 new full test suite runs each with a different number of passing/failing/broken/fixed tests, in order to ensure that the correct exit code and output are generated in each case. As before, these are run in a subdirectory to avoid disrupting the metrics for the parent tests. Signed-off-by: Adam Spiers <git@adamspiers.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-12-20tests: refactor mechanics of testing in a sub test-libLibravatar Adam Spiers1-45/+40
This will allow us to test the test framework more thoroughly without disrupting the top-level test metrics. Signed-off-by: Adam Spiers <git@adamspiers.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-12-16tests: test number comes first in 'not ok $count - $message'Libravatar Adam Spiers1-2/+2
The old output to say "not ok - 1 messsage" was working by accident only because the test numbers are optional in TAP. Signed-off-by: Adam Spiers <git@adamspiers.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-11-15test-lib: allow negation of prerequisitesLibravatar Jeff King1-0/+32
You can set and test a prerequisite like this: test_set_prereq FOO test_have_prereq FOO && echo yes You can negate the test in the shell like this: ! test_have_prereq && echo no However, when you are using the automatic prerequisite checking in test_expect_*, there is no opportunity to use the shell negation. This patch introduces the syntax "!FOO" to indicate that the test should only run if a prerequisite is not meant. One alternative is to set an explicit negative prerequisite, like: if system_has_foo; then test_set_prereq FOO else test_set_prereq NO_FOO fi However, this doesn't work for lazy prerequisites, which associate a single test with a single name. We could teach the lazy prereq evaluator to set both forms, but the code change ends up quite similar to this one (because we still need to convert NO_FOO into FOO to find the correct lazy script). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-09-25Merge branch 'rr/test-make-sure-we-have-git'Libravatar Junio C Hamano1-10/+0
Only the first test t0000 in the test suite made sure we have built Git to be tested; move the check to test-lib so that it applies to all tests equally. * rr/test-make-sure-we-have-git: t/test-lib: make sure Git has already been built
2012-09-18t/test-lib: make sure Git has already been builtLibravatar Ramkumar Ramachandra1-10/+0
When tests were run without building git, they stopped with: .: 54: Can't open /path/to/git/source/t/../GIT-BUILD-OPTIONS Move the check that makes sure that git has already been built from t0000 to test-lib, so that any test will do so before it runs. Signed-off-by: Ramkumar Ramachandra <artagnon@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-09-06t0060: move tests of real_path() from t0000 to hereLibravatar Michael Haggerty1-18/+0
Suggested by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-03-02t0000: modernise styleLibravatar Stefano Lattarini1-274/+289
Match the style to more modern test scripts, namely: - Prefer tabs for indentation. - The first line of each test has prereq, title and opening sq for the script body. - Move cleanup or initialization of data used by a test inside the test itself. - Put a newline before the closing sq for each test. - Don't conclude the test descriptions with a full stop. - Prefer 'test_line_count = COUNT FILE' over 'test $(wc -l <FILE) = COUNT' - Prefer 'test_line_count = 0 FILE' over 'cmp -s /dev/null FILE' - Use '<<-EOF' style for here documents, so that they can be indented as well. Bot don't do that in case the resulting lines would be too long. Also when there is no $variable_substitution in the body of a here document, quote \EOF. - Don't redirect the output of commands to /dev/null unconditionally, the git testing framework should already take care of handling test verbosity transparently and uniformly. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-03-17Name make_*_path functions more accuratelyLibravatar Carlos Martín Nieto1-5/+5
Rename the make_*_path functions so it's clearer what they do, in particlar make clear what the differnce between make_absolute_path and make_nonrelative_path is by renaming them real_path and absolute_path respectively. make_relative_path has an understandable name and is renamed to relative_path to maintain the name convention. The function calls have been replaced 1-to-1 in their usage. Signed-off-by: Carlos Martín Nieto <cmn@elego.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-01-13t0000: quote TAP snippets in test codeLibravatar Thomas Rast1-18/+18
t0000 contains two snippets of actual test output. This causes problems when passing -v to the test[*]: the test infrastructure echoes the tests before running them, and the TAP parser then sees this test output and concludes that two tests failed and that the TAP output was badly formatted. Guard against this by quoting the output in the source. [*] either by running 'make smoke' with GIT_TEST_OPTS=-v, or with prove ./t0000-basic.sh :: -v Signed-off-by: Thomas Rast <trast@student.ethz.ch> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-10-06test-lib: make test_expect_code a test commandLibravatar Ævar Arnfjörð Bjarmason1-10/+45
Change test_expect_code to be a normal test command instead of a top-level command. As a top-level command it would fail in cases like: test_expect_code 1 'phoney' ' foo && bar && (exit 1) ' Here the test might incorrectly succeed if "foo" or "bar" happened to fail with exit status 1. Instead we now do: test_expect_success 'phoney' ' foo && bar && test_expect_code 1 "(exit 1)" ' Which will only succeed if "foo" and "bar" return status 0, and "(exit 1)" returns status 1. Note that test_expect_code has been made slightly noisier, as it reports the exit code it receives even upon success. Some test code in t0000-basic.sh relied on the old semantics of test_expect_code to test the test_when_finished command. I've converted that code to use an external test similar to the TODO test I added in v1.7.3-rc0~2^2~3. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Acked-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-08-31t/t0000-basic.sh: Run the passing TODO test inside its own test-libLibravatar Ævar Arnfjörð Bjarmason1-0/+31
Change the passing TODO test in t0000-basic.sh to run inside its own test-lib.sh. The motivation is to have nothing out of the ordinary on a normal test run for test smoking purposes. If every normal test run has a passing TODO you're more likely to turn a blind eye to it and not to investigate cases where things really are passing unexpectedly. It also makes the prove(1) output less noisy. Before: All tests successful. Test Summary Report ------------------- ./t0000-basic.sh (Wstat: 0 Tests: 46 Failed: 0) TODO passed: 5 Files=484, Tests=6229, 143 wallclock secs ( 4.00 usr 4.15 sys + 104.77 cusr 351.57 csys = 464.49 CPU) Result: PASS And after: All tests successful. Files=484, Tests=6228, 139 wallclock secs ( 4.07 usr 4.25 sys + 104.54 cusr 350.85 csys = 463.71 CPU) Result: PASS Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-08-18test-lib: Multi-prereq support only checked the last prereqLibravatar Ævar Arnfjörð Bjarmason1-1/+5
The support for multiple test prerequisites added by me in "test-lib: Add support for multiple test prerequisites" was broken. The for iterated over each prerequisite and returned true/false within a case statement, but since it missed a return statement only the last prerequisite in the list of prerequisites was ever considered, the rest were ignored. Fix that by changing the test_have_prereq code to something less clever that keeps a count of the total prereqs and the ones we have and compares the count at the end. This comes with the added advantage that it's easy to list the missing prerequisites in the test output, implement that while I'm at it. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-08-18test-lib: Add support for multiple test prerequisitesLibravatar Ævar Arnfjörð Bjarmason1-0/+17
Change the test_have_prereq function in test-lib.sh to support a comma-separated list of prerequisites. This is useful for tests that need e.g. both POSIXPERM and SANITY. The implementation was stolen from Junio C Hamano and Johannes Sixt, the tests and documentation were not. See the "Tests in Cygwin" thread in May 2009 for the originals: http://thread.gmane.org/gmane.comp.version-control.git/116729/focus=118385 http://thread.gmane.org/gmane.comp.version-control.git/116729/focus=118434 Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-06-21Merge branch 'gv/portable'Libravatar Junio C Hamano1-1/+1
* gv/portable: test-lib: use DIFF definition from GIT-BUILD-OPTIONS build: propagate $DIFF to scripts Makefile: Tru64 portability fix Makefile: HP-UX 10.20 portability fixes Makefile: HPUX11 portability fixes Makefile: SunOS 5.6 portability fix inline declaration does not work on AIX Allow disabling "inline" Some platforms lack socklen_t type Make NO_{INET_NTOP,INET_PTON} configured independently Makefile: some platforms do not have hstrerror anywhere git-compat-util.h: some platforms with mmap() lack MAP_FAILED definition test_cmp: do not use "diff -u" on platforms that lack one fixup: do not unconditionally disable "diff -u" tests: use "test_cmp", not "diff", when verifying the result Do not use "diff" found on PATH while building and installing enums: omit trailing comma for portability Makefile: -lpthread may still be necessary when libc has only pthread stubs Rewrite dynamic structure initializations to runtime assignment Makefile: pass CPPFLAGS through to fllow customization Conflicts: Makefile wt-status.h
2010-05-31tests: use "test_cmp", not "diff", when verifying the resultLibravatar Gary V. Vaughan1-1/+1
In tests, call test_cmp rather than raw diff where possible (i.e. if the output does not go to a pipe), to allow the use of, say, 'cmp' when the default 'diff -u' is not compatible with a vendor diff. When that is not possible, use $DIFF, as set in GIT-BUILD-OPTIONS. Signed-off-by: Gary V. Vaughan <gary@thewrittenword.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-05-06test-lib: some shells do not let $? propagate into an evalLibravatar Jonathan Nieder1-0/+21
In 3bf7886 (test-lib: Let tests specify commands to be run at end of test, 2010-05-02), the git test harness learned to run cleanup commands unconditionally at the end of a test. During each test, the intended cleanup actions are collected in the test_cleanup variable and evaluated. That variable looks something like this: eval_ret=$?; clean_something && (exit "$eval_ret") eval_ret=$?; clean_something_else && (exit "$eval_ret") eval_ret=$?; final_cleanup && (exit "$eval_ret") eval_ret=$? All cleanup actions are run unconditionally but if one of them fails it is properly reported through $eval_ret. On FreeBSD, unfortunately, $? is set at the beginning of an ‘eval’ to 0 instead of the exit status of the previous command. This results in tests using test_expect_code appearing to fail and all others appearing to pass, unless their cleanup fails. Avoid the problem by setting eval_ret before the ‘eval’ begins. Thanks to Jeff King for the explanation. Cc: Jeff King <peff@peff.net> Cc: Johannes Sixt <j6t@kdbg.org> Acked-by: Jeff King <peff@peff.net> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-03-22Use prerequisite tags to skip tests that depend on symbolic linksLibravatar Johannes Sixt1-12/+31
Many tests depend on that symbolic links work. This introduces a check that sets the prerequisite tag SYMLINKS if the file system supports symbolic links. Since so many tests have to check for this prerequisite, we do the check in test-lib.sh, so that we don't need to repeat the test in many scripts. To check for 'ln -s' failures, you can use a FAT partition on Linux: $ mkdosfs -C git-on-fat 1000000 $ sudo mount -o loop,uid=j6t,gid=users,shortname=winnt git-on-fat /mnt Clone git to /mnt and $ GIT_SKIP_TESTS='t0001.1[34] t0010 t1301 t403[34] t4129.[47] t5701.7 t7701.3 t9100 t9101.26 t9119 t9124.[67] t9200.10 t9600.6' \ make test (These additionally skipped tests depend on POSIX permissions that FAT on Linux does not provide.) Signed-off-by: Johannes Sixt <j6t@kdbg.org>
2009-03-21test-lib: Infrastructure to test and check for prerequisitesLibravatar Johannes Sixt1-0/+15
Some tests can be run only if a particular prerequisite is available. For example, some tests require that an UTF-8 locale is available. Here we introduce functions that are used in this way: 1. Insert code that checks whether the prerequisite is available. If it is, call test_set_prereq with an arbitrary tag name that subsequently can be used to check for the prerequisite: case $LANG in *.utf-8) test_set_prereq UTF8 ;; esac 2. In the calls to test_expect_success pass the tag name: test_expect_success UTF8 '...description...' '...tests...' 3. There is an auxiliary predicate that can be used anywhere to test for a prerequisite explicitly: if test_have_prereq UTF8 then ...code to be skipped if prerequisite is not available... fi Signed-off-by: Johannes Sixt <j6t@kdbg.org>