diff options
Diffstat (limited to 't/README')
| -rw-r--r-- | t/README | 155 |
1 files changed, 146 insertions, 9 deletions
@@ -69,7 +69,8 @@ You can also run each test individually from command line, like this: You can pass --verbose (or -v), --debug (or -d), and --immediate (or -i) command line argument to the test, or by setting GIT_TEST_OPTS -appropriately before running "make". +appropriately before running "make". Short options can be bundled, i.e. +'-d -v' is the same as '-dv'. -v:: --verbose:: @@ -170,6 +171,15 @@ appropriately before running "make". implied by other options like --valgrind and GIT_TEST_INSTALLED. +--no-bin-wrappers:: + By default, the test suite uses the wrappers in + `../bin-wrappers/` to execute `git` and friends. With this option, + `../git` and friends are run directly. This is not recommended + in general, as the wrappers contain safeguards to ensure that no + files from an installed Git are used, but can speed up test runs + especially on platforms where running shell scripts is expensive + (most notably, Windows). + --root=<directory>:: Create "trash" directories used to store all temporary data during testing under <directory>, instead of the t/ directory. @@ -186,6 +196,29 @@ appropriately before running "make". this feature by setting the GIT_TEST_CHAIN_LINT environment variable to "1" or "0", respectively. +--stress:: + Run the test script repeatedly in multiple parallel jobs until + one of them fails. Useful for reproducing rare failures in + flaky tests. The number of parallel jobs is, in order of + precedence: the value of the GIT_TEST_STRESS_LOAD + environment variable, or twice the number of available + processors (as shown by the 'getconf' utility), or 8. + Implies `--verbose -x --immediate` to get the most information + about the failure. Note that the verbose output of each test + job is saved to 't/test-results/$TEST_NAME.stress-<nr>.out', + and only the output of the failed test job is shown on the + terminal. The names of the trash directories get a + '.stress-<nr>' suffix, and the trash directory of the failed + test job is renamed to end with a '.stress-failed' suffix. + +--stress-jobs=<N>:: + Override the number of parallel jobs. Implies `--stress`. + +--stress-limit=<N>:: + When combined with --stress run the test script repeatedly + this many times in each of the parallel jobs or until one of + them fails, whichever comes first. Implies `--stress`. + You can also set the GIT_TEST_INSTALLED environment variable to the bindir of an existing git installation to test that installation. You still need to have built this git sandbox, from which various @@ -302,15 +335,27 @@ that cannot be easily covered by a few specific test cases. These could be enabled by running the test suite with correct GIT_TEST_ environment set. -GIT_TEST_GETTEXT_POISON=<non-empty?> turns all strings marked for -translation into gibberish if non-empty (think "test -n"). Used for -spotting those tests that need to be marked with a C_LOCALE_OUTPUT -prerequisite when adding more strings for translation. See "Testing -marked strings" in po/README for details. +GIT_TEST_FAIL_PREREQS=<boolean> fails all prerequisites. This is +useful for discovering issues with the tests where say a later test +implicitly depends on an optional earlier test. + +There's a "FAIL_PREREQS" prerequisite that can be used to test for +whether this mode is active, and e.g. skip some tests that are hard to +refactor to deal with it. The "SYMLINKS" prerequisite is currently +excluded as so much relies on it, but this might change in the future. + +GIT_TEST_GETTEXT_POISON=<boolean> turns all strings marked for +translation into gibberish if true. Used for spotting those tests that +need to be marked with a C_LOCALE_OUTPUT prerequisite when adding more +strings for translation. See "Testing marked strings" in po/README for +details. GIT_TEST_SPLIT_INDEX=<boolean> forces split-index mode on the whole test suite. Accept any boolean values that are accepted by git-config. +GIT_TEST_PROTOCOL_VERSION=<n>, when set, makes 'protocol.version' +default to n. + GIT_TEST_FULL_IN_PACK_ARRAY=<boolean> exercises the uncommon pack-objects code path where there are more than 1024 packs even if the actual number of packs in repository is below this limit. Accept @@ -334,6 +379,11 @@ GIT_TEST_COMMIT_GRAPH=<boolean>, when true, forces the commit-graph to be written after every 'git commit' command, and overrides the 'core.commitGraph' setting to true. +GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS=<boolean>, when true, forces +commit-graph write to compute and write changed path Bloom filters for +every 'git commit-graph write', as if the `--changed-paths` option was +passed in. + GIT_TEST_FSMONITOR=$PWD/t7519/fsmonitor-all exercises the fsmonitor code path for utilizing a file system monitor to speed up detecting new or changed files. @@ -342,11 +392,15 @@ GIT_TEST_INDEX_VERSION=<n> exercises the index read/write code path for the index version specified. Can be set to any valid version (currently 2, 3, or 4). +GIT_TEST_PACK_SPARSE=<boolean> if disabled will default the pack-objects +builtin to use the non-sparse object walk. This can still be overridden by +the --sparse command-line argument. + GIT_TEST_PRELOAD_INDEX=<boolean> exercises the preload-index code path by overriding the minimum number of cache entries required per thread. -GIT_TEST_REBASE_USE_BUILTIN=<boolean>, when false, disables the -builtin version of git-rebase. See 'rebase.useBuiltin' in +GIT_TEST_ADD_I_USE_BUILTIN=<boolean>, when true, enables the +built-in version of git add -i. See 'add.interactive.useBuiltin' in git-config(1). GIT_TEST_INDEX_THREADS=<n> enables exercising the multi-threaded loading @@ -358,6 +412,15 @@ GIT_TEST_MULTI_PACK_INDEX=<boolean>, when true, forces the multi-pack- index to be written after every 'git repack' command, and overrides the 'core.multiPackIndex' setting to true. +GIT_TEST_SIDEBAND_ALL=<boolean>, when true, overrides the +'uploadpack.allowSidebandAll' setting to true, and when false, forces +fetch-pack to not request sideband-all (even if the server advertises +sideband-all). + +GIT_TEST_DISALLOW_ABBREVIATED_OPTIONS=<boolean>, when true (which is +the default when running tests), errors out when an abbreviated option +is used. + Naming Tests ------------ @@ -425,7 +488,8 @@ This test harness library does the following things: - Creates an empty test directory with an empty .git/objects database and chdir(2) into it. This directory is 't/trash directory.$test_name_without_dotsh', with t/ subject to change by - the --root option documented above. + the --root option documented above, and a '.stress-<N>' suffix + appended by the --stress option. - Defines standard test helper functions for your scripts to use. These functions are designed to make all scripts behave @@ -488,6 +552,41 @@ Here are the "do's:" reports "ok" or "not ok" to the end user running the tests. Under --verbose, they are shown to help debug the tests. + - Be careful when you loop + + You may need to verify multiple things in a loop, but the + following does not work correctly: + + test_expect_success 'test three things' ' + for i in one two three + do + test_something "$i" + done && + test_something_else + ' + + Because the status of the loop itself is the exit status of the + test_something in the last round, the loop does not fail when + "test_something" for "one" or "two" fails. This is not what you + want. + + Instead, you can break out of the loop immediately when you see a + failure. Because all test_expect_* snippets are executed inside + a function, "return 1" can be used to fail the test immediately + upon a failure: + + test_expect_success 'test three things' ' + for i in one two three + do + test_something "$i" || return 1 + done && + test_something_else + ' + + Note that we still &&-chain the loop to propagate failures from + earlier commands. + + And here are the "don'ts:" - Don't exit() within a <script> part. @@ -822,6 +921,26 @@ library for your script to use. ... ' + - test_atexit <script> + + Prepend <script> to a list of commands to run unconditionally to + clean up before the test script exits, e.g. to stop a daemon: + + test_expect_success 'test git daemon' ' + git daemon & + daemon_pid=$! && + test_atexit 'kill $daemon_pid' && + hello world + ' + + The commands will be executed before the trash directory is removed, + i.e. the atexit commands will still be able to access any pidfiles or + socket files. + + Note that these commands will be run even when a test script run + with '--immediate' fails. Be careful with your atexit commands to + minimize any changes to the failed state. + - test_write_lines <lines> Write <lines> on standard output, one line per argument. @@ -892,6 +1011,24 @@ library for your script to use. test_oid_init or test_oid_cache. Providing an unknown key is an error. + - yes [<string>] + + This is often seen in modern UNIX but some platforms lack it, so + the test harness overrides the platform implementation with a + more limited one. Use this only when feeding a handful lines of + output to the downstream---unlike the real version, it generates + only up to 99 lines. + + - test_bool_env <env-variable-name> <default-value> + + Given the name of an environment variable with a bool value, + normalize its value to a 0 (true) or 1 (false or empty string) + return code. Return with code corresponding to the given default + value if the variable is unset. + Abort the test script if either the value of the variable or the + default are not valid bool values. + + Prerequisites ------------- |