summaryrefslogtreecommitdiff
path: root/t/test-lib.sh
AgeCommit message (Collapse)AuthorFilesLines
2020-07-06Merge branch 'bc/sha-256-part-2'Libravatar Junio C Hamano1-0/+1
SHA-256 migration work continues. * bc/sha-256-part-2: (44 commits) remote-testgit: adapt for object-format bundle: detect hash algorithm when reading refs t5300: pass --object-format to git index-pack t5704: send object-format capability with SHA-256 t5703: use object-format serve option t5702: offer an object-format capability in the test t/helper: initialize the repository for test-sha1-array remote-curl: avoid truncating refs with ls-remote t1050: pass algorithm to index-pack when outside repo builtin/index-pack: add option to specify hash algorithm remote-curl: detect algorithm for dumb HTTP by size builtin/ls-remote: initialize repository based on fetch t5500: make hash independent serve: advertise object-format capability for protocol v2 connect: parse v2 refs with correct hash algorithm connect: pass full packet reader when parsing v2 refs Documentation/technical: document object-format for protocol v2 t1302: expect repo format version 1 for SHA-256 builtin/show-index: provide options to determine hash algo t5302: modernize test formatting ...
2020-06-08Merge branch 'cb/t4210-illseq-auto-detect'Libravatar Junio C Hamano1-6/+0
As FreeBSD is not the only platform whose regexp library reports a REG_ILLSEQ error when fed invalid UTF-8, add logic to detect that automatically and skip the affected tests. * cb/t4210-illseq-auto-detect: t4210: detect REG_ILLSEQ dynamically and skip affected tests t/helper: teach test-regex to report pattern errors (like REG_ILLSEQ)
2020-05-24Merge branch 'gp/hppa-stack-test-fix'Libravatar Junio C Hamano1-2/+10
Platform dependent tweak to a test for HP-PA. * gp/hppa-stack-test-fix: tests: skip small-stack tests on hppa architecture
2020-05-18t4210: detect REG_ILLSEQ dynamically and skip affected testsLibravatar Carlo Marcelo Arenas Belón1-6/+0
7187c7bbb8 (t4210: skip i18n tests that don't work on FreeBSD, 2019-11-27) adds a REG_ILLSEQ prerequisite, and to do that copies the common branch in test-lib and expands it to include it in a special case for FreeBSD. Instead; test for it using a previously added extension to test-tool and use that, together with a function that identifies when regcomp/regexec will be called with broken patterns to avoid any test that would otherwise rely on undefined behaviour. The description of the first test which wasn't accurate has been corrected, and the test rearranged for clarity, including a helper function that avoids overly long lines. Only the affected engines will have their tests suppressed, also including "fixed" if the PCRE optimization that uses LIBPCRE2 since b65abcafc7 (grep: use PCRE v2 for optimized fixed-string search, 2019-07-01) is not available. Helped-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-05-18tests: skip small-stack tests on hppa architectureLibravatar Greg Price1-2/+10
On hppa these tests crash because the allocated stack space is too small, even after it was doubled in b9a190789 (and the data size doubled to match) to make it work on powerpc. For this arch just skip these tests, which is enough to make the whole suite pass. Fixes: https://bugs.debian.org/757402 Based-on-patch-by: John David Anglin <dave.anglin@bell.net> Signed-off-by: Greg Price <gnprice@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-05-15Revert "tests: when run in Bash, annotate test failures with file name/line ↵Libravatar Junio C Hamano1-13/+1
number" This reverts commit 662f9cf1548cf069cb819e9e95f224657015fcf9, to fix the TAP output broken for bash.
2020-05-15Revert "t/test_lib: avoid naked bash arrays in file_lineno"Libravatar Junio C Hamano1-10/+8
This reverts commit 303775a25f0b4ac5d6ad2e96eb4404c24209cad8; instead of trying to salvage the tap-breaking change, let's revert the whole thing for now.
2020-05-12t1050: match object ID paths in a hash-insensitive wayLibravatar brian m. carlson1-0/+1
The pattern here looking for failures is specific to SHA-1. Let's create a variable that matches the regex or glob pattern for a path within the objects directory. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-05-07t/test_lib: avoid naked bash arrays in file_linenoLibravatar Carlo Marcelo Arenas Belón1-8/+10
662f9cf154 (tests: when run in Bash, annotate test failures with file name/line number, 2020-04-11), introduces a way to report the location (file:lineno) of a failed test case by traversing the bash callstack. The implementation requires bash and uses shell arrays and is therefore protected by a guard but NetBSD sh will still have to parse the function and therefore will result in: ** t0000-basic.sh *** ./test-lib.sh: 681: Syntax error: Bad substitution Enclose the bash specific code inside an eval to avoid parsing errors in the same way than 5826b7b595 (test-lib: check Bash version for '-x' without using shell arrays, 2019-01-03) Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-05-01Merge branch 'tb/commit-graph-fd-exhaustion-fix'Libravatar Junio C Hamano1-0/+9
The commit-graph code exhausted file descriptors easily when it does not have to. * tb/commit-graph-fd-exhaustion-fix: commit-graph: close descriptors after mmap commit-graph.c: gracefully handle file descriptor exhaustion t/test-lib.sh: make ULIMIT_FILE_DESCRIPTORS available to tests commit-graph.c: don't use discarded graph_name in error
2020-04-29Merge branch 'dd/ci-swap-azure-pipelines-with-github-actions'Libravatar Junio C Hamano1-1/+13
Update the CI configuration to use GitHub Actions, retiring the one based on Azure Pipelines. * dd/ci-swap-azure-pipelines-with-github-actions: ci: let GitHub Actions upload failed tests' directories ci: add a problem matcher for GitHub Actions tests: when run in Bash, annotate test failures with file name/line number ci: retire the Azure Pipelines definition README: add a build badge for the GitHub Actions runs ci: configure GitHub Actions for CI/PR ci: run gem with sudo to install asciidoctor ci: explicit install all required packages ci: fix the `jobname` of the `GETTEXT_POISON` job ci/lib: set TERM environment variable if not exist ci/lib: allow running in GitHub Actions ci/lib: if CI type is unknown, show the environment variables
2020-04-23t/test-lib.sh: make ULIMIT_FILE_DESCRIPTORS available to testsLibravatar Taylor Blau1-0/+9
In t1400 the prerequisite 'ULIMIT_FILE_DESCRIPTORS' is defined and used to effectively guard the helper function 'run_with_limited_open_files' from being used on systems that do not satisfy this prerequisite. In the subsequent patch, we will introduce another test outside of t1400 that would benefit from using this prerequisite. So, move it to 'test-lib.sh' instead so that it can be used by multiple tests. Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-22Merge branch 'mt/test-lib-bundled-short-options'Libravatar Junio C Hamano1-14/+47
Minor test usability improvement. * mt/test-lib-bundled-short-options: test-lib: allow short options to be bundled
2020-04-22Merge branch 'js/test-junit-finalization-fix'Libravatar Junio C Hamano1-0/+1
Test fix. * js/test-junit-finalization-fix: tests(junit-xml): avoid invalid XML
2020-04-22Merge branch 'js/tests-gpg-integration-on-windows'Libravatar Junio C Hamano1-2/+4
Enable tests that require GnuPG on Windows. * js/tests-gpg-integration-on-windows: tests: increase the verbosity of the GPG-related prereqs tests: turn GPG, GPGSM and RFC1991 into lazy prereqs tests: do not let lazy prereqs inside `test_expect_*` turn off tracing t/lib-gpg.sh: stop pretending to be a stand-alone script tests(gpg): allow the gpg-agent to start on Windows
2020-04-10tests: when run in Bash, annotate test failures with file name/line numberLibravatar Johannes Schindelin1-1/+13
When a test fails, it is nice to see where the corresponding code lives in the worktree. Sadly, it seems that only Bash allows us to infer this information. Let's do it when we detect that we're running in a Bash. This will come in handy in the next commit, where we teach the GitHub Actions workflow to annotate failed test runs with this information. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-03-26tests: do not let lazy prereqs inside `test_expect_*` turn off tracingLibravatar Johannes Schindelin1-2/+4
The `test_expect_*` functions use `test_eval_` and so does `test_run_lazy_prereq_`. If tracing is enabled via the `-x` option, `test_eval_` turns on tracing while evaluating the code block, and turns it off directly after it. This is unwanted for nested invocations. One somewhat surprising example of this is when running a test that calls `test_i18ngrep`: that function requires the `C_LOCALE_OUTPUT` prereq, and that prereq is a lazy one, so it is evaluated via `test_eval_`, the command tracing is turned off, and the test case continues to run _without tracing the commands_. Another somewhat surprising example is when one lazy prereq depends on another lazy prereq: the former will call `test_have_prereq` with the latter one, which in turn calls `test_eval_` and -- you guessed it -- tracing (if enabled) will be turned off _before_ returning to evaluating the other lazy prereq. As we will introduce just such a scenario with the GPG, GPGSM and RFC1991 prereqs, let's fix that by introducing a variable that keeps track of the current trace level: nested `test_eval_` calls will increment and then decrement the level, and only when it reaches 0, the tracing will _actually_ be turned off. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-03-25test-lib: allow short options to be bundledLibravatar Matheus Tavares1-14/+47
When debugging a test (or a set of tests), it's common to execute it with some combination of short options, such as: $ ./txxx-testname.sh -d -x -i In cases like this, CLIs usually allow the short options to be bundled in a single argument, for convenience and agility. Let's add this feature to test-lib, allowing the above command to be run as: $ ./txxx-testname.sh -dxi (or any other permutation, e.g. '-ixd') Note: Short options that require an argument can also be used in a bundle, in any position. So, for example, '-r 5 -x', '-xr 5' and '-rx 5' are all valid and equivalent. A special case would be having a bundle with more than one of such options. To keep things simple, this case is not allowed for now. This shouldn't be a major limitation, though, as the only short option that requires an argument today is '-r'. And concatenating '-r's as in '-rr 5 6' would probably not be very practical: its unbundled format would be '-r 5 -r 6', for which test-lib currently considers only the last argument. Therefore, if '-rr 5 6' were to be allowed, it would have the same effect as just typing '-r 6'. Note: the test-lib currently doesn't support '-r5', as an alternative for '-r 5', so the former is not supported in bundles as well. Helped-by: Jeff King <peff@peff.net> Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-03-23tests(junit-xml): avoid invalid XMLLibravatar Johannes Schindelin1-0/+1
When a test case is run in a subshell, we finalize the JUnit-style XML when said subshell exits. But then we continue to write into that XML as if nothing had happened. This leads to Azure Pipelines' Publish Test Results task complaining: Failed to read /home/vsts/work/1/s/t/out/TEST-t0000-basic.xml. Error : Unexpected end tag. Line 110, position 5. And indeed, the resulting XML is incorrect. Let's "re-open" the XML in such a case, i.e. remove the previously added closing tags. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-02-24t: use hash-specific lookup tables to define test constantsLibravatar brian m. carlson1-15/+14
In the future, we'll allow developers to run the testsuite with a hash algorithm of their choice. To make this easier, compute the fixed constants using test_oid. Move the constant initialization down below the point where test-lib-functions.sh is loaded so the functions are defined. Note that we don't provide a value for the OID_REGEX value directly because writing a large number of instances of "[0-9a-f]" in the oid-info files is unwieldy and there isn't a way to compute it based on those values. Instead, compute it based on ZERO_OID. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-02-17Merge branch 'js/test-write-junit-xml-fix'Libravatar Junio C Hamano1-1/+2
Testfix. * js/test-write-junit-xml-fix: tests: fix --write-junit-xml with subshells
2020-02-12tests: fix --write-junit-xml with subshellsLibravatar Johannes Schindelin1-1/+2
In t0000, more precisely in its `test_bool_env` test case, there are two subshells that are supposed to fail. To be even _more_ precise, they fail by calling the `error` function, and that is okay, because it is in a subshell, and it is expected that those two subshell invocations fail. However, the `error` function also tries to finalize the JUnit XML (if that XML was asked for, via `--write-junit-xml`. As a consequence, the XML is edited to add a `time` attribute for the `testsuite` tag. And since there are two expected `error` calls in addition to the final `test_done`, the `finalize_junit_xml` function is called three times and naturally the `time` attribute is added _three times_. Azure Pipelines is not happy with that, complaining thusly: ##[warning]Failed to read D:\a\1\s\t\out\TEST-t0000-basic.xml. Error : 'time' is a duplicate attribute name. Line 2, position 82.. One possible way to address this would be to unset `write_junit_xml` in the `test_bool_env` test case. But that would be fragile, as other `error` calls in subshells could be introduced. So let's just modify `finalize_junit_xml` to remove any `time` attribute before adding the authoritative one. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-10Merge branch 'em/test-skip-regex-illseq'Libravatar Junio C Hamano1-1/+7
Test portability fix. * em/test-skip-regex-illseq: t4210: skip i18n tests that don't work on FreeBSD
2019-12-05Merge branch 'sg/test-bool-env'Libravatar Junio C Hamano1-5/+5
Recently we have declared that GIT_TEST_* variables take the usual boolean values (it used to be that some used "non-empty means true" and taking GIT_TEST_VAR=YesPlease as true); make sure we notice and fail when non-bool strings are given to these variables. * sg/test-bool-env: t5608-clone-2gb.sh: turn GIT_TEST_CLONE_2GB into a bool tests: add 'test_bool_env' to catch non-bool GIT_TEST_* values
2019-12-01Merge branch 'sg/skip-skipped-prereq'Libravatar Junio C Hamano1-6/+6
Test update to avoid wasted cycles. * sg/skip-skipped-prereq: test-lib: don't check prereqs of test cases that won't be run anyway
2019-11-30t4210: skip i18n tests that don't work on FreeBSDLibravatar Ed Maste1-1/+7
A number of t4210-log-i18n tests added in 4e2443b181 set LC_ALL to a UTF-8 locale (is_IS.UTF-8) but then pass an invalid UTF-8 string to --grep. FreeBSD's regcomp() fails in this case with REG_ILLSEQ, "illegal byte sequence," which git then passes to die(): fatal: command line: '�': illegal byte sequence When these tests were added the commit message stated: | It's possible that this | test breaks the "basic" and "extended" backends on some systems that | are more anal than glibc about the encoding of locale issues with | POSIX functions that I can remember which seems to be the case here. Extend test-lib.sh to add a REGEX_ILLSEQ prereq, set it on FreeBSD, and add !REGEX_ILLSEQ to the two affected tests. Signed-off-by: Ed Maste <emaste@freebsd.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-23tests: add 'test_bool_env' to catch non-bool GIT_TEST_* valuesLibravatar SZEDER Gábor1-5/+5
Since 3b072c577b (tests: replace test_tristate with "git env--helper", 2019-06-21) we get the normalized bool values of various GIT_TEST_* environment variables via 'git env--helper'. Now, while the 'git env--helper' command itself does catch invalid values in the environment variable or in the given --default and exits with error (exit code 128 or 129, respectively), it's invoked in conditions like 'if ! git env--helper ...', which means that all invalid bool values are interpreted the same as the ordinary 'false' (exit code 1). This has led to inadvertently skipped httpd tests in our CI builds for a couple of weeks, see 3960290675 (ci: restore running httpd tests, 2019-09-06). Let's be more careful about what the test suite accepts as bool values in GIT_TEST_* environment variables, and error out loud and clear on invalid values instead of simply skipping tests. Add the 'test_bool_env' helper function to encapsulate the invocation of 'git env--helper' and the verification of its exit code, and replace all invocations of that command in our test framework and test suite with a call to this new helper (except in 't0017-env-helper.sh', of course). $ GIT_TEST_GIT_DAEMON=YesPlease ./t5570-git-daemon.sh fatal: bad numeric config value 'YesPlease' for 'GIT_TEST_GIT_DAEMON': invalid unit error: test_bool_env requires bool values both for $GIT_TEST_GIT_DAEMON and for the default fallback Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-13test-lib: don't check prereqs of test cases that won't be run anywayLibravatar SZEDER Gábor1-6/+6
With './t1234-foo.sh -r 5,6' we can run only specific test cases in a test script, but our test framwork still evaluates all lazy prereqs that the excluded test cases might depend on. This is unnecessary and produces verbose and trace output that can be distracting. This has been an issue ever since the '-r|--run=' options were introduced in 0445e6f0a1 (test-lib: '--run' to run only specific tests, 2014-04-30), because that commit added the check of the list of test cases specified with '-r' after evaluating the prereqs. Avoid this unnecessary prereq evaluation by checking the list of test cases specified with '-r' before looking at the prereqs. Note that GIT_SKIP_TESTS has always been checked before the prereqs, so prereqs necessary for tests skipped that way were not evaluated. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-30pretty: add "%aL" etc. to show local-part of email addressesLibravatar Prarit Bhargava1-2/+6
In many projects the number of contributors is low enough that users know each other and the full email address doesn't need to be displayed. Displaying only the author's username saves a lot of columns on the screen. Existing 'e/E' (as in "%ae" and "%aE") placeholders would show the author's address as "prarit@redhat.com", which would waste columns to show the same domain-part for all contributors when used in a project internal to redhat. Introduce 'l/L' placeholders that strip '@' and domain part from the e-mail address. Signed-off-by: Prarit Bhargava <prarit@redhat.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-15Merge branch 'js/azure-pipelines-msvc'Libravatar Junio C Hamano1-16/+22
CI updates. * js/azure-pipelines-msvc: ci: also build and test with MS Visual Studio on Azure Pipelines ci: really use shallow clones on Azure Pipelines tests: let --immediate and --write-junit-xml play well together test-tool run-command: learn to run (parts of) the testsuite vcxproj: include more generated files vcxproj: only copy `git-remote-http.exe` once it was built msvc: work around a bug in GetEnvironmentVariable() msvc: handle DEVELOPER=1 msvc: ignore some libraries when linking compat/win32/path-utils.h: add #include guards winansi: use FLEX_ARRAY to avoid compiler warning msvc: avoid using minus operator on unsigned types push: do not pretend to return `int` from `die_push_simple()`
2019-10-06tests: let --immediate and --write-junit-xml play well togetherLibravatar Johannes Schindelin1-16/+22
When the `--immediate` option is in effect, any test failure will immediately exit the test script. Together with `--write-junit-xml`, we will want the JUnit-style `.xml` file to be finalized (and not leave the XML incomplete). Let's make it so. This comes in particularly handy when trying to debug via Azure Pipelines, where the JUnit-style XML is consumed to present the test results in an informative and helpful way. While at it, also handle the `error()` code path. The only remaining code path that sets `GIT_EXIT_OK` happens whenever the trash directory could not be set up, i.e. long before the JUnit XML was written, therefore we should _not_ try to finalize that XML in that case. It is tempting to change the `immediate` code path to just hand off to `error`, simplifying the code in the process. That would, however, result in a change of behavior (an additional error message) in the test suite, which is outside of the purview of the current patch series: its goal is to allow building Git with Visual Studio and testing it with a portable version of Git for Windows. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-09-06t: use common $SQ variableLibravatar Denton Liu1-0/+3
In many test scripts, there are bespoke definitions of the single quote that are some variation of this: SQ="'" Define a common $SQ variable in test-lib.sh and replace all usages of these bespoke variables with the common one. This change was done by running `git grep =\"\'\" t/` and `git grep =\\\\\'` and manually changing the resulting definitions and corresponding usages. Signed-off-by: Denton Liu <liu.denton@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-22Merge branch 'sg/show-failed-test-names'Libravatar Junio C Hamano1-0/+2
The first line of verbose output from each test piece now carries the test name and number to help scanning with eyeballs. * sg/show-failed-test-names: tests: show the test name and number at the start of verbose output t0000-basic: use realistic test script names in the verbose tests
2019-08-05tests: show the test name and number at the start of verbose outputLibravatar SZEDER Gábor1-0/+2
The verbose output of every test looks something like this: expecting success: echo content >file && git add file && git commit -m "add file" [master (root-commit) d1fbfbd] add file Author: A U Thor <author@example.com> 1 file changed, 1 insertion(+) create mode 100644 file ok 1 - commit works i.e. first an "expecting success" (or "checking known breakage") line followed by the commands to be executed, then the output of those comamnds, and finally an "ok"/"not ok" line containing the test name. Note that the test's name is only shown at the very end. With '-x' tracing enabled and/or in longer tests the verbose output might be several screenfulls long, making it harder than necessary to find where the output of the test with a given name starts (especially when the outputs to different file descriptors are racing, and the "expecting success"/command block arrives earlier than the "ok" line of the previous test). Print the test name at the start of the test's verbose output, i.e. at the end of the "expecting success" and "checking known breakage" lines, to make the start of a particular test a bit easier to recognize. Also print the test script and test case numbers, to help those poor souls who regularly have to scan through the combined verbose output of several test scripts. So the dummy test above would start like this: expecting success of 9999.1 'commit works': echo content >file && [...] Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-07-25Merge branch 'ab/test-env'Libravatar Junio C Hamano1-9/+22
Many GIT_TEST_* environment variables control various aspects of how our tests are run, but a few followed "non-empty is true, empty or unset is false" while others followed the usual "there are a few ways to spell true, like yes, on, etc., and also ways to spell false, like no, off, etc." convention. * ab/test-env: env--helper: mark a file-local symbol as static tests: make GIT_TEST_FAIL_PREREQS a boolean tests: replace test_tristate with "git env--helper" tests README: re-flow a previously changed paragraph tests: make GIT_TEST_GETTEXT_POISON a boolean t6040 test: stop using global "script" variable config.c: refactor die_bad_number() to not call gettext() early env--helper: new undocumented builtin wrapping git_env_*() config tests: simplify include cycle test
2019-07-09Merge branch 'sg/trace2-rename'Libravatar Junio C Hamano1-1/+0
Dev support update to help tracing out tests. * sg/trace2-rename: trace2: correct typo in technical documentation Revert "test-lib: whitelist GIT_TR2_* in the environment"
2019-06-21tests: make GIT_TEST_FAIL_PREREQS a booleanLibravatar Ævar Arnfjörð Bjarmason1-4/+19
Change the GIT_TEST_FAIL_PREREQS variable from being "non-empty?" to being a more standard boolean variable. I recently added the variable in dfe1a17df9 ("tests: add a special setup where prerequisites fail", 2019-05-13), having to add another "non-empty?" special-case is what prompted me to write the "git env--helper" utility being used here. Converting this one is a bit tricky since we use it so early and frequently in the guts of the test code itself, so let's set a GIT_TEST_FAIL_PREREQS_INTERNAL which can be tested with the old "test -n" for the purposes of the shell code, and change the user-exposed and documented GIT_TEST_FAIL_PREREQS variable to a boolean. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-06-21tests: make GIT_TEST_GETTEXT_POISON a booleanLibravatar Ævar Arnfjörð Bjarmason1-5/+3
Change the GIT_TEST_GETTEXT_POISON variable from being "non-empty?" to being a more standard boolean variable. Since it needed to be checked in both C code and shellscript (via test -n) it was one of the remaining shellscript-like variables. Now that we have "env--helper" we can change that. There's a couple of tricky edge cases that arise because we're using git_env_bool() early, and the config-reading "env--helper". If GIT_TEST_GETTEXT_POISON is set to an invalid value die_bad_number() will die, but to do so it would usually call gettext(). Let's detect the special case of GIT_TEST_GETTEXT_POISON and always emit that message in the C locale, lest we infinitely loop. As seen in the updated tests in t0017-env-helper.sh there's also a caveat related to "env--helper" needing to read the config for trace2 purposes. Since the C_LOCALE_OUTPUT prerequisite is lazy and relies on "env--helper" we could get invalid results if we failed to read the config (e.g. because we'd loop on includes) when combined with e.g. "test_i18ngrep" wanting to check with "env--helper" if GIT_TEST_GETTEXT_POISON was true or not. I'm crossing my fingers and hoping that a test similar to the one I removed in the earlier "config tests: simplify include cycle test" change in this series won't happen again, and testing for this explicitly in "env--helper"'s own tests. This change breaks existing uses of e.g. GIT_TEST_GETTEXT_POISON=YesPlease, which we've documented in po/README and other places. As noted in [1] we might want to consider also accepting "YesPlease" in "env--helper" as a special-case. But as the lack of uproar over 6cdccfce1e ("i18n: make GETTEXT_POISON a runtime option", 2018-11-08) demonstrates the audience for this option is a really narrow set of git developers, who shouldn't have much trouble modifying their test scripts, so I think it's better to deal with that minor headache now and make all the relevant GIT_TEST_* variables boolean in the same way than carry the "YesPlease" special-case forward. 1. https://public-inbox.org/git/xmqqtvckm3h8.fsf@gitster-ct.c.googlers.com/ Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-06-13Merge branch 'ab/fail-prereqs-in-test'Libravatar Junio C Hamano1-0/+4
Developer support to emulate unsatisfied prerequisites in tests to ensure that the remainer of the tests still succeeds when tests with prerequisites are skipped. * ab/fail-prereqs-in-test: tests: add a special setup where prerequisites fail
2019-06-12Revert "test-lib: whitelist GIT_TR2_* in the environment"Libravatar Ævar Arnfjörð Bjarmason1-1/+0
This reverts my commit c1ee5796dc ("test-lib: whitelist GIT_TR2_* in the environment", 2019-03-30), which is now redundant. Since e4b75d6a1d ("trace2: rename environment variables to GIT_TRACE2*", 2019-05-19) the GIT_TRACE2* variables match the existing GIT_TRACE* pattern added in 95a1d12e9b ("tests: scrub environment of GIT_* variables", 2011-03-15), so we no longer need to list TR2 here. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-05-19Merge branch 'tz/test-lib-check-working-jgit'Libravatar Junio C Hamano1-1/+1
A prerequiste check in the test suite to see if a working jgit is available was made more robust. * tz/test-lib-check-working-jgit: test-lib: try harder to ensure a working jgit
2019-05-15test-lib: try harder to ensure a working jgitLibravatar Todd Zullinger1-1/+1
The JGIT prereq uses `type jgit` to determine whether jgit is present. While this is usually sufficient, it won't help if the jgit found is badly broken. This wastes time running tests which fail due to no fault of our own. Use `jgit --version` instead, to guard against cases where jgit is present on the system, but will fail to run, e.g. because of some JRE issue, or missing Java dependencies. Checking that it gets far enough to process the '--version' argument isn't perfect, but seems to be good enough in practice. It's also consistent with how we detect some other dependencies, see e.g. the CURL and UNZIP prerequisites. Signed-off-by: Todd Zullinger <tmz@pobox.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-05-14tests: add a special setup where prerequisites failLibravatar Ævar Arnfjörð Bjarmason1-0/+4
As discussed in [1] there's a regression in the "pu" branch now because a new test implicitly assumed that a previous test guarded by a prerequisite had been run. Add a "GIT_TEST_FAIL_PREREQS" special test setup where we'll skip (nearly) all tests guarded by prerequisites, allowing us to easily emulate those platform where we don't run these tests. As noted in the documentation I'm adding I'm whitelisting the SYMLINKS prerequisite for now. A lot of tests started failing if we lied about not supporting symlinks. It's also unlikely that we'll have a failing test due to a hard dependency on symlinks without that being the obvious cause, so for now it's not worth the effort to make it work. 1. https://public-inbox.org/git/nycvar.QRO.7.76.6.1905131531000.44@tvgsbejvaqbjf.bet/ Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-25Merge branch 'jc/gettext-test-fix'Libravatar Junio C Hamano1-0/+1
The GETTEXT_POISON test option has been quite broken ever since it was made runtime-tunable, which has been fixed. * jc/gettext-test-fix: gettext tests: export the restored GIT_TEST_GETTEXT_POISON
2019-04-25Merge branch 'ab/test-lib-pass-trace2-env'Libravatar Junio C Hamano1-0/+1
Allow tracing of Git executable while running the testsuite. * ab/test-lib-pass-trace2-env: test-lib: whitelist GIT_TR2_* in the environment
2019-04-25Merge branch 'sg/test-atexit'Libravatar Junio C Hamano1-1/+27
Test framework update to more robustly clean up leftover files and processes after tests are done. * sg/test-atexit: t9811-git-p4-label-import: fix pipeline negation git p4 test: disable '-x' tracing in the p4d watchdog loop git p4 test: simplify timeout handling git p4 test: clean up the p4d cleanup functions git p4 test: use 'test_atexit' to kill p4d and the watchdog process t0301-credential-cache: use 'test_atexit' to stop the credentials helper tests: use 'test_atexit' to stop httpd git-daemon: use 'test_atexit` to stop 'git-daemon' test-lib: introduce 'test_atexit' t/lib-git-daemon: make sure to kill the 'git-daemon' process test-lib: fix interrupt handling with 'dash' and '--verbose-log -x'
2019-04-15gettext tests: export the restored GIT_TEST_GETTEXT_POISONLibravatar Junio C Hamano1-0/+1
6cdccfce ("i18n: make GETTEXT_POISON a runtime option", 2018-11-08) made the gettext-poison test a runtime option (which was a good move) and adjusted the test framework so that Git commands we run as part of the framework, as opposed to the ones that are part of the test proper, are not affected by the setting. The original value for the GIT_TEST_GETTEXT_POISON environment variable is saved away in another variable and gets unset, and then later the saved value is restored to the environment variable. But the code forgot to export the variable again, which is necessary to restore the "export" bit that was lost when the variable was unset. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-15tests: disallow the use of abbreviated options (by default)Libravatar Johannes Schindelin1-0/+7
Git's command-line parsers support uniquely abbreviated options, e.g. `git init --ba` would automatically expand `--ba` to `--bare`. This is a very convenient feature in every day life for Git users, in particular when tab completion is not available. However, it is not a good idea to rely on that in Git's test suite, as something that is a unique abbreviation of a command line option today might no longer be a unique abbreviation tomorrow. For example, if a future contribution added a new mode `git init --babyproofing` and a previously-introduced test case used the fact that `git init --ba` expanded to `git init --bare`, that future contribution would now have to touch seemingly unrelated tests just to keep the test suite from failing. So let's disallow abbreviated options in the test suite by default. Note: for ease of implementation, this patch really only touches the `parse-options` machinery: more and more hand-rolled option parsers are converted to use that internal API, and more and more scripts are converted to built-ins (naturally using the parse-options API, too), so in practice this catches most issues, and is definitely the biggest bang for the buck. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-01test-lib: whitelist GIT_TR2_* in the environmentLibravatar Ævar Arnfjörð Bjarmason1-0/+1
Add GIT_TR2_* to the whitelist of environment variables that we don't clear when running the test suite. This allows us to use the test suite to produce trace2 test data, which is handy to e.g. write consumers that collate the trace data itself. One caveat here is that we produce trace output for not *just* the tests, but also e.g. from this line in test-lib.sh: # It appears that people try to run tests without building... "${GIT_TEST_INSTALLED:-$GIT_BUILD_DIR}/git$X" >/dev/null [...] I consider this not just OK but a feature. Let's log *all* the git commands we're going to execute, not just those within test_expect_*(). Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-20Merge branch 'js/stress-test-ui-tweak'Libravatar Junio C Hamano1-1/+8
Dev support. * js/stress-test-ui-tweak: tests: introduce --stress-jobs=<N> tests: let --stress-limit=<N> imply --stress