summaryrefslogtreecommitdiff
path: root/t/test-lib.sh
AgeCommit message (Collapse)AuthorFilesLines
2021-04-16Merge branch 'en/ort-readiness'Libravatar Junio C Hamano1-0/+2
Plug the ort merge backend throughout the rest of the system, and start testing it as a replacement for the recursive backend. * en/ort-readiness: Add testing with merge-ort merge strategy t6423: mark remaining expected failure under merge-ort as such Revert "merge-ort: ignore the directory rename split conflict for now" merge-recursive: add a bunch of FIXME comments documenting known bugs merge-ort: write $GIT_DIR/AUTO_MERGE whenever we hit a conflict t: mark several submodule merging tests as fixed under merge-ort merge-ort: implement CE_SKIP_WORKTREE handling with conflicted entries t6428: new test for SKIP_WORKTREE handling and conflicts merge-ort: support subtree shifting merge-ort: let renormalization change modify/delete into clean delete merge-ort: have ll_merge() use a special attr_index for renormalization merge-ort: add a special minimal index just for renormalization merge-ort: use STABLE_QSORT instead of QSORT where required
2021-03-20Add testing with merge-ort merge strategyLibravatar Elijah Newren1-0/+2
In preparation for switching from merge-recursive to merge-ort as the default strategy, have the testsuite default to running with merge-ort. Keep coverage of the recursive backend by having the linux-gcc job run with it. Signed-off-by: Elijah Newren <newren@gmail.com> Reviewed-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-02-25Merge branch 'ab/detox-gettext-tests'Libravatar Junio C Hamano1-5/+0
Removal of GIT_TEST_GETTEXT_POISON continues. * ab/detox-gettext-tests: tests: remove most uses of test_i18ncmp tests: remove last uses of C_LOCALE_OUTPUT tests: remove most uses of C_LOCALE_OUTPUT tests: remove last uses of GIT_TEST_GETTEXT_POISON=false
2021-02-22Merge branch 'ab/test-lib'Libravatar Junio C Hamano1-30/+0
Test framework clean-up. * ab/test-lib: test-lib-functions: assert correct parameter count test-lib-functions: remove bug-inducing "diagnostics" helper param test libs: rename "diff-lib" to "lib-diff" t/.gitattributes: sort lines test-lib-functions: move function to lib-bitmap.sh test libs: rename gitweb-lib.sh to lib-gitweb.sh test libs: rename bundle helper to "lib-bundle.sh" test-lib-functions: remove generate_zero_bytes() wrapper test-lib-functions: move test_set_index_version() to its user test lib: change "error" to "BUG" as appropriate test-lib: remove check_var_migration
2021-02-10tests: remove last uses of C_LOCALE_OUTPUTLibravatar Ævar Arnfjörð Bjarmason1-5/+0
Remove the last uses of the C_LOCALE_OUTPUT prerequisite as well as the prerequisite itself. This is a follow-up to d162b25f956 (tests: remove support for GIT_TEST_GETTEXT_POISON, 2021-01-20), as well as the preceding commit where we removed the simpler uses of C_LOCALE_OUTPUT. Here I'm slightly refactoring a test added in 21e5ad50fc5 (safecrlf: Add mechanism to warn about irreversible crlf conversions, 2008-02-06), as well as getting rid of another "test_have_prereq C_LOCALE_OUTPUT" use. I'm not leaving the prerequisite itself in place for in-flight changes as there currently are none that introduce new tests that rely on it, and because C_LOCALE_OUTPUT is currently a noop on the master branch we likely won't have any new submissions that use it. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-02-10Merge branch 'ab/detox-gettext-tests'Libravatar Junio C Hamano1-19/+4
Get rid of "GETTEXT_POISON" support altogether, which may or may not be controversial. * ab/detox-gettext-tests: tests: remove uses of GIT_TEST_GETTEXT_POISON=false tests: remove support for GIT_TEST_GETTEXT_POISON ci: remove GETTEXT_POISON jobs
2021-02-10Merge branch 'ab/retire-pcre1'Libravatar Junio C Hamano1-2/+1
The support for deprecated PCRE1 library has been dropped. * ab/retire-pcre1: Remove support for v1 of the PCRE library config.mak.uname: remove redundant NO_LIBPCRE1_JIT flag
2021-02-10test-lib: remove check_var_migrationLibravatar Ævar Arnfjörð Bjarmason1-30/+0
Remove the check_var_migration() migration helper. This was added back in [1], [2] and [3] to warn users to migrate from e.g. the "GIT_FSMONITOR_TEST" name to "GIT_TEST_FSMONITOR". I daresay that having been warning about this since late 2018 (or v2.20.0) was sufficient time to give everyone interested a heads-up about moving to the new names. I don't see the need for going through the "do this later" codepath anticipated in [1], let's just remove this instead. 1. 4cb54d0aa8e (fsmonitor: update GIT_TEST_FSMONITOR support, 2018-09-18) 2. 1f357b045b5 (read-cache: update TEST_GIT_INDEX_VERSION support, 2018-09-18) 3. 5765d97b71d (preload-index: update GIT_FORCE_PRELOAD_TEST support, 2018-09-18) Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-01-26test-lib: prevent '--stress-jobs=X' from being ignoredLibravatar SZEDER Gábor1-4/+4
'./t1234-foo.sh --stress-jobs=X ...' is supposed to run that test script in X parallel jobs, but the number of jobs specified on the command line is entirely ignored if other '--stress'-related options follow. I.e. both './t1234-foo.sh --stress-jobs=X --stress-limit=Y' and './t1234-foo.sh --stress-jobs=X --stress' fall back to using twice the number of CPUs parallel jobs instead. The former has been broken since commit de69e6f6c9 (tests: let --stress-limit=<N> imply --stress, 2019-03-03) [1], which started to unconditionally overwrite the $stress variable holding the specified number of jobs in its effort to imply '--stress'. The latter has been broken since f545737144 (tests: introduce --stress-jobs=<N>, 2019-03-03), because it didn't consider that handling '--stress' will overwrite that variable as well. We could fix this by being more careful about (over)writing that $stress variable and checking first whether it has already been set. But I think it's cleaner to use a dedicated variable to hold the number of specified parallel jobs, so let's do that instead. [1] In de69e6f6c9 there was no '--stress-jobs=X' option yet, the number of parallel jobs had to be specified via '--stress=X', so, strictly speaking, de69e6f6c9 broke './t1234-foo.sh --stress=X --stress-limit=Y'. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-01-23Remove support for v1 of the PCRE libraryLibravatar Ævar Arnfjörð Bjarmason1-2/+1
Remove support for using version 1 of the PCRE library. Its use has been discouraged by upstream for a long time, and it's in a bugfix-only state. Anyone who was relying on v1 in particular got a nudge to move to v2 in e6c531b808 (Makefile: make USE_LIBPCRE=YesPlease mean v2, not v1, 2018-03-11), which was first released as part of v2.18.0. With this the LIBPCRE2 test prerequisites is redundant to PCRE. But I'm keeping it for self-documentation purposes, and to avoid conflict with other in-flight PCRE patches. I'm also not changing all of our own "pcre2" names to "pcre", i.e. the inverse of 6d4b5747f0 (grep: change internal *pcre* variable & function names to be *pcre1*, 2017-05-25). I don't see the point, and it makes the history/blame harder to read. Maybe if there's ever a PCRE v3... Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-01-21tests: remove support for GIT_TEST_GETTEXT_POISONLibravatar Ævar Arnfjörð Bjarmason1-19/+4
This removes the ability to inject "poison" gettext() messages via the GIT_TEST_GETTEXT_POISON special test setup. I initially added this as a compile-time option in bb946bba761 (i18n: add GETTEXT_POISON to simulate unfriendly translator, 2011-02-22), and most recently modified to be toggleable at runtime in 6cdccfce1e0 (i18n: make GETTEXT_POISON a runtime option, 2018-11-08).. The reason for its removal is that the trade-off of maintaining it v.s. what it's getting us has long since flipped. When gettext was integrated in 5e9637c6297 (i18n: add infrastructure for translating Git with gettext, 2011-11-18) there was understandable concern on the Git ML that in marking messages for translation en-masse we'd inadvertently mark plumbing messages. The GETTEXT_POISON facility was a way to smoke those out via our test suite. Nowadays however we're done (or almost entirely done) with any marking of messages for translation. New messages are usually marked by their authors, who'll know whether it makes sense to translate them or not. If not any errors in marking the messages are much more likely to be spotted in review than in the the initial deluge of i18n patches in the 2011-2012 era. So let's just remove this. This leaves the test suite in a state where we still have a lot of test_i18n, C_LOCALE_OUTPUT etc. uses. Subsequent commits will remove those too. The change to t/lib-rebase.sh is a selective revert of the relevant part of f2d17068fd (i18n: rebase-interactive: mark comments of squash for translation, 2016-06-17), and the comment in t/t3406-rebase-message.sh is from c7108bf9ed (i18n: rebase: mark messages for translation, 2012-07-25). Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-01-15Merge branch 'ds/maintenance-part-4'Libravatar Junio C Hamano1-3/+4
Follow-up on the "maintenance part-3" which introduced scheduled maintenance tasks to support platforms whose native scheduling methods are not 'cron'. * ds/maintenance-part-4: maintenance: use Windows scheduled tasks maintenance: use launchctl on macOS maintenance: include 'cron' details in docs maintenance: extract platform-specific scheduling
2020-12-16tests: drop the `PREPARE_FOR_MAIN_BRANCH` prereqLibravatar Johannes Schindelin1-8/+0
We no longer use it. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-11-24maintenance: extract platform-specific schedulingLibravatar Derrick Stolee1-3/+4
The existing schedule mechanism using 'cron' is supported by POSIX platforms, but not Windows. It also works slightly differently on macOS to significant detriment of the user experience. To allow for new implementations on these platforms, extract a method that performs the platform-specific scheduling mechanism. This will be swapped at compile time with new implementations on specialized platforms. As we add this generality, rename GIT_TEST_CRONTAB to GIT_TEST_MAINT_SCHEDULER. Further, this variable is now parsed as "<scheduler>:<command>" so we can test platform-specific scheduling logic even when not on the correct platform. By specifying the <scheduler> in this string, we will be able to test all three sets of Git logic from a Linux machine. Co-authored-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-11-18Merge branch 'ds/maintenance-part-3'Libravatar Junio C Hamano1-0/+7
Parts of "git maintenance" to ease writing crontab entries (and other scheduling system configuration) for it. * ds/maintenance-part-3: maintenance: add troubleshooting guide to docs maintenance: use 'incremental' strategy by default maintenance: create maintenance.strategy config maintenance: add start/stop subcommands maintenance: add [un]register subcommands for-each-repo: run subcommands on configured repos maintenance: add --schedule option and config maintenance: optionally skip --auto process
2020-11-09Merge branch 'jk/perl-warning'Libravatar Junio C Hamano1-0/+6
Dev support. * jk/perl-warning: perl: check for perl warnings while running tests
2020-11-02Merge branch 'js/default-branch-name-part-4-minus-1'Libravatar Junio C Hamano1-0/+7
Adjust tests so that they won't scream when the default initial branch name is changed to 'main'. * js/default-branch-name-part-4-minus-1: t1400: prepare for `main` being default branch name tests: prepare aligned mentions of the default branch name t9902: prepare a test for the upcoming default branch name t3200: prepare for `main` being shorter than `master` t5703: adjust a test case for the upcoming default branch name t6200: adjust suppression pattern to also match "main" tests: start moving to a different default main branch name t9801: use `--` in preparation for default branch rename fmt-merge-msg: also suppress "into main" by default
2020-10-23tests: start moving to a different default main branch nameLibravatar Johannes Schindelin1-0/+7
To allow for an incremental conversion to a new default main branch name, let's introduce `GIT_TEST_DEFAULT_MAIN_BRANCH_NAME`. This environment variable can be set at the top of each converted test script, overriding the default main branch name to use when initializing new repositories (or cloning empty repositories). Note: the `GIT_TEST_DEFAULT_MAIN_BRANCH_NAME` is _not_ intended to be used manually; many tests require a specific main branch name and cannot simply work with another one. This `GIT_TEST_*` variable is meant purely for the transitional period while the entire test suite is converted to use `main` as the initial branch name by default. We also introduce the `PREPARE_FOR_MAIN_BRANCH` prereq that determines whether the default main branch name is `main`, and adjust a couple of test functions to use it. This prereq will be used to temporarily disable a couple test cases to allow for adjusting the test script incrementally. Once an entire test is adjusted, we will adjust the test so that it is run with `GIT_TEST_DEFAULT_MAIN_BRANCH_NAME=main`. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-10-21perl: check for perl warnings while running testsLibravatar Jeff King1-0/+6
We set "use warnings" in most of our perl code to catch problems. But as the name implies, warnings just emit a message to stderr and don't otherwise affect the program. So our tests are quite likely to miss that warnings are being spewed, as most of them do not look at stderr. We could ask perl to make all warnings fatal, but this is likely annoying for non-developers, who would rather have a running program with a warning than something that refuses to work at all. So instead, let's teach the perl code to respect an environment variable (GIT_PERL_FATAL_WARNINGS) to increase the severity of the warnings. This can be set for day-to-day running if people want to be really pedantic, but the primary use is to trigger it within the test suite. We could also trigger that for every test run, but likewise even the tests failing may be annoying to distro builders, etc (just as -Werror would be for compiling C code). So we'll tie it to a special test-mode variable (GIT_TEST_PERL_FATAL_WARNINGS) that can be set in the environment or as a Makefile knob, and we'll automatically turn the knob when DEVELOPER=1 is set. That should give developers and CI the more careful view without disrupting normal users or packagers. Note that the mapping from the GIT_TEST_* form to the GIT_* form in test-lib.sh is necessary even if they had the same name: the perl scripts need it to be normalized to a perl truth value, and we also have to make sure it's exported (we might have gotten it from the environment, but we might also have gotten it from GIT-BUILD-OPTIONS directly). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-10-18test-lib: reduce verbosity of skipped testsLibravatar Elijah Newren1-1/+0
When using the --run flag to run just two or three tests from a test file which contains several dozen tests, having every skipped test print out dozens of lines of output for the test code for that skipped test (in addition to the TAP output line) adds up to hundreds or thousands of lines of irrelevant output that make it very hard to fish out the relevant results you were looking for. Simplify the output for skipped tests to remove this extra output, leaving only the TAP output line (i.e. the line reading "ok <number> # skip <test-description>", which already mentions that the test was "skip"ped). Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-10-18test-lib: allow selecting tests by substring/glob with --runLibravatar Elijah Newren1-8/+12
Many of our test scripts have several "setup" tests. It's a lot easier to say ./t0050-filesystem.sh --run=setup,9 in order to run all the setup tests as well as test #9, than it is to track down what all the setup tests are and enter all their numbers in the list. Also, I often find myself wanting to run just one or a couple tests from the test file, but I don't know the numbering of any of the tests -- to get it I either have to first run the whole test file (or start counting by hand or figure out some other clever but non-obvious tricks). It's really convenient to be able to just look at the test description(s) and then run ./t6416-recursive-corner-cases.sh --run=symlink or ./t6402-merge-rename.sh --run='setup,unnecessary update' Add such an ability to test selection which relies on merely matching against the test description. Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-25maintenance: add start/stop subcommandsLibravatar Derrick Stolee1-0/+6
Add new subcommands to 'git maintenance' that start or stop background maintenance using 'cron', when available. This integration is as simple as I could make it, barring some implementation complications. The schedule is laid out as follows: 0 1-23 * * * $cmd maintenance run --schedule=hourly 0 0 * * 1-6 $cmd maintenance run --schedule=daily 0 0 * * 0 $cmd maintenance run --schedule=weekly where $cmd is a properly-qualified 'git for-each-repo' execution: $cmd=$path/git --exec-path=$path for-each-repo --config=maintenance.repo where $path points to the location of the Git executable running 'git maintenance start'. This is critical for systems with multiple versions of Git. Specifically, macOS has a system version at '/usr/bin/git' while the version that users can install resides at '/usr/local/bin/git' (symlinked to '/usr/local/libexec/git-core/git'). This will also use your locally-built version if you build and run this in your development environment without installing first. This conditional schedule avoids having cron launch multiple 'git for-each-repo' commands in parallel. Such parallel commands would likely lead to the 'hourly' and 'daily' tasks competing over the object database lock. This could lead to to some tasks never being run! Since the --schedule=<frequency> argument will run all tasks with _at least_ the given frequency, the daily runs will also run the hourly tasks. Similarly, the weekly runs will also run the daily and hourly tasks. The GIT_TEST_CRONTAB environment variable is not intended for users to edit, but instead as a way to mock the 'crontab [-l]' command. This variable is set in test-lib.sh to avoid a future test from accidentally running anything with the cron integration from modifying the user's schedule. We use GIT_TEST_CRONTAB='test-tool crontab <file>' in our tests to check how the schedule is modified in 'git maintenance (start|stop)' commands. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-08-11Merge branch 'bc/sha-256-part-3'Libravatar Junio C Hamano1-1/+8
The final leg of SHA-256 transition. * bc/sha-256-part-3: (39 commits) t: remove test_oid_init in tests docs: add documentation for extensions.objectFormat ci: run tests with SHA-256 t: make SHA1 prerequisite depend on default hash t: allow testing different hash algorithms via environment t: add test_oid option to select hash algorithm repository: enable SHA-256 support by default setup: add support for reading extensions.objectformat bundle: add new version for use with SHA-256 builtin/verify-pack: implement an --object-format option http-fetch: set up git directory before parsing pack hashes t0410: mark test with SHA1 prerequisite t5308: make test work with SHA-256 t9700: make hash size independent t9500: ensure that algorithm info is preserved in config t9350: make hash size independent t9301: make hash size independent t9300: use $ZERO_OID instead of hard-coded object ID t9300: abstract away SHA-1-specific constants t8011: make hash size independent ...
2020-07-30Merge branch 'jk/tests-timestamp-fix' into masterLibravatar Junio C Hamano1-0/+3
The test framework has been updated so that most tests will run with predictable (artificial) timestamps. * jk/tests-timestamp-fix: t9100: stop depending on commit timestamps test-lib: set deterministic default author/committer date t9100: explicitly unset GIT_COMMITTER_DATE t5539: make timestamp requirements more explicit t9700: loosen ident timezone regex t6000: use test_tick consistently
2020-07-30t: make SHA1 prerequisite depend on default hashLibravatar brian m. carlson1-1/+5
Currently, the SHA1 prerequisite depends on the output of git hash-object. However, in order for that to produce sane behavior, we must be in a repository. If we are not, the default will remain SHA-1, and we'll produce wrong results if we're using SHA-256 for the testsuite but the test assertion starts when we're not in a repository. Check the environment variable we use for this purpose, leaving it to default to SHA-1 if none is specified. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Reviewed-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-30t: allow testing different hash algorithms via environmentLibravatar brian m. carlson1-0/+3
To allow developers to run the testsuite with a different algorithm than the default, provide an environment variable, GIT_TEST_DEFAULT_HASH, to specify the algorithm to use. Compute the fixed constants using test_oid. Move the constant initialization down below the point where test-lib-functions.sh is loaded so the functions are defined. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Reviewed-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-14test-lib: set deterministic default author/committer dateLibravatar Jeff King1-0/+3
We always set the name and email for committer and author idents to make the test suite more deterministic, but not timestamps. Many scripts use test_tick to get consistent and sensibly incrementing timestamps as they create commits. But other scripts don't particularly care about the timestamp, and are happy to use whatever the current system time is. This non-determinism can be annoying: - when debugging a test, comparing results between two runs can be difficult, because the commit ids change - this can sometimes cause tests to be racy. E.g., traversal order depends on timestamp order. Even in a well-ordered set of commands, because our timestamp granularity is one second, two commits might sometimes have the same timestamp and sometimes differ. Let's set a default timestamp for all scripts to use. Any that use test_tick already will be unaffected (because their first test_tick call will overwrite our default), but it will make things a bit more deterministic for those that don't. We should be able to choose any time we want here. I picked this one because: - it differs from the initial test_tick default, which may make it easier to distinguish when debugging tests. I picked "April 1st 13:14:15" in the hope that it might stand out. - it's slightly before the test_tick default. Some tests create some commits before the first call to test_tick, so using an older timestamps for those makes sense chronologically. Note that this isn't how things currently work (where system times are usually more recent than test_tick), but that also allows us to flush out a few hidden timestamp dependencies (like the one recently fixed in t5539). - we could likewise pick any timezone we want. Choosing +0000 would have required fixing up fewer tests, but we're more likely to turn up interesting cases by not matching $TZ exactly. And since test_tick already checks "-0700", let's try something in the "+" zone range for variety. It's possible that the non-deterministic times could help flush out bugs (e.g., if something broke when the clock flipped over to 2021, our test suite would let us know). But historically that hasn't been the case; all time-dependent outcomes we've seen turned out to be accidentally flaky tests (which we fixed by using test_tick). If we do want to cover handling the current time, we should dedicate one script to doing so, and have it unset GIT_COMMITTER_DATE explicitly. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-06Merge branch 'bc/sha-256-part-2'Libravatar Junio C Hamano1-0/+1
SHA-256 migration work continues. * bc/sha-256-part-2: (44 commits) remote-testgit: adapt for object-format bundle: detect hash algorithm when reading refs t5300: pass --object-format to git index-pack t5704: send object-format capability with SHA-256 t5703: use object-format serve option t5702: offer an object-format capability in the test t/helper: initialize the repository for test-sha1-array remote-curl: avoid truncating refs with ls-remote t1050: pass algorithm to index-pack when outside repo builtin/index-pack: add option to specify hash algorithm remote-curl: detect algorithm for dumb HTTP by size builtin/ls-remote: initialize repository based on fetch t5500: make hash independent serve: advertise object-format capability for protocol v2 connect: parse v2 refs with correct hash algorithm connect: pass full packet reader when parsing v2 refs Documentation/technical: document object-format for protocol v2 t1302: expect repo format version 1 for SHA-256 builtin/show-index: provide options to determine hash algo t5302: modernize test formatting ...
2020-06-08Merge branch 'cb/t4210-illseq-auto-detect'Libravatar Junio C Hamano1-6/+0
As FreeBSD is not the only platform whose regexp library reports a REG_ILLSEQ error when fed invalid UTF-8, add logic to detect that automatically and skip the affected tests. * cb/t4210-illseq-auto-detect: t4210: detect REG_ILLSEQ dynamically and skip affected tests t/helper: teach test-regex to report pattern errors (like REG_ILLSEQ)
2020-05-24Merge branch 'gp/hppa-stack-test-fix'Libravatar Junio C Hamano1-2/+10
Platform dependent tweak to a test for HP-PA. * gp/hppa-stack-test-fix: tests: skip small-stack tests on hppa architecture
2020-05-18t4210: detect REG_ILLSEQ dynamically and skip affected testsLibravatar Carlo Marcelo Arenas Belón1-6/+0
7187c7bbb8 (t4210: skip i18n tests that don't work on FreeBSD, 2019-11-27) adds a REG_ILLSEQ prerequisite, and to do that copies the common branch in test-lib and expands it to include it in a special case for FreeBSD. Instead; test for it using a previously added extension to test-tool and use that, together with a function that identifies when regcomp/regexec will be called with broken patterns to avoid any test that would otherwise rely on undefined behaviour. The description of the first test which wasn't accurate has been corrected, and the test rearranged for clarity, including a helper function that avoids overly long lines. Only the affected engines will have their tests suppressed, also including "fixed" if the PCRE optimization that uses LIBPCRE2 since b65abcafc7 (grep: use PCRE v2 for optimized fixed-string search, 2019-07-01) is not available. Helped-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-05-18tests: skip small-stack tests on hppa architectureLibravatar Greg Price1-2/+10
On hppa these tests crash because the allocated stack space is too small, even after it was doubled in b9a190789 (and the data size doubled to match) to make it work on powerpc. For this arch just skip these tests, which is enough to make the whole suite pass. Fixes: https://bugs.debian.org/757402 Based-on-patch-by: John David Anglin <dave.anglin@bell.net> Signed-off-by: Greg Price <gnprice@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-05-15Revert "tests: when run in Bash, annotate test failures with file name/line ↵Libravatar Junio C Hamano1-13/+1
number" This reverts commit 662f9cf1548cf069cb819e9e95f224657015fcf9, to fix the TAP output broken for bash.
2020-05-15Revert "t/test_lib: avoid naked bash arrays in file_lineno"Libravatar Junio C Hamano1-10/+8
This reverts commit 303775a25f0b4ac5d6ad2e96eb4404c24209cad8; instead of trying to salvage the tap-breaking change, let's revert the whole thing for now.
2020-05-12t1050: match object ID paths in a hash-insensitive wayLibravatar brian m. carlson1-0/+1
The pattern here looking for failures is specific to SHA-1. Let's create a variable that matches the regex or glob pattern for a path within the objects directory. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-05-07t/test_lib: avoid naked bash arrays in file_linenoLibravatar Carlo Marcelo Arenas Belón1-8/+10
662f9cf154 (tests: when run in Bash, annotate test failures with file name/line number, 2020-04-11), introduces a way to report the location (file:lineno) of a failed test case by traversing the bash callstack. The implementation requires bash and uses shell arrays and is therefore protected by a guard but NetBSD sh will still have to parse the function and therefore will result in: ** t0000-basic.sh *** ./test-lib.sh: 681: Syntax error: Bad substitution Enclose the bash specific code inside an eval to avoid parsing errors in the same way than 5826b7b595 (test-lib: check Bash version for '-x' without using shell arrays, 2019-01-03) Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-05-01Merge branch 'tb/commit-graph-fd-exhaustion-fix'Libravatar Junio C Hamano1-0/+9
The commit-graph code exhausted file descriptors easily when it does not have to. * tb/commit-graph-fd-exhaustion-fix: commit-graph: close descriptors after mmap commit-graph.c: gracefully handle file descriptor exhaustion t/test-lib.sh: make ULIMIT_FILE_DESCRIPTORS available to tests commit-graph.c: don't use discarded graph_name in error
2020-04-29Merge branch 'dd/ci-swap-azure-pipelines-with-github-actions'Libravatar Junio C Hamano1-1/+13
Update the CI configuration to use GitHub Actions, retiring the one based on Azure Pipelines. * dd/ci-swap-azure-pipelines-with-github-actions: ci: let GitHub Actions upload failed tests' directories ci: add a problem matcher for GitHub Actions tests: when run in Bash, annotate test failures with file name/line number ci: retire the Azure Pipelines definition README: add a build badge for the GitHub Actions runs ci: configure GitHub Actions for CI/PR ci: run gem with sudo to install asciidoctor ci: explicit install all required packages ci: fix the `jobname` of the `GETTEXT_POISON` job ci/lib: set TERM environment variable if not exist ci/lib: allow running in GitHub Actions ci/lib: if CI type is unknown, show the environment variables
2020-04-23t/test-lib.sh: make ULIMIT_FILE_DESCRIPTORS available to testsLibravatar Taylor Blau1-0/+9
In t1400 the prerequisite 'ULIMIT_FILE_DESCRIPTORS' is defined and used to effectively guard the helper function 'run_with_limited_open_files' from being used on systems that do not satisfy this prerequisite. In the subsequent patch, we will introduce another test outside of t1400 that would benefit from using this prerequisite. So, move it to 'test-lib.sh' instead so that it can be used by multiple tests. Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-22Merge branch 'mt/test-lib-bundled-short-options'Libravatar Junio C Hamano1-14/+47
Minor test usability improvement. * mt/test-lib-bundled-short-options: test-lib: allow short options to be bundled
2020-04-22Merge branch 'js/test-junit-finalization-fix'Libravatar Junio C Hamano1-0/+1
Test fix. * js/test-junit-finalization-fix: tests(junit-xml): avoid invalid XML
2020-04-22Merge branch 'js/tests-gpg-integration-on-windows'Libravatar Junio C Hamano1-2/+4
Enable tests that require GnuPG on Windows. * js/tests-gpg-integration-on-windows: tests: increase the verbosity of the GPG-related prereqs tests: turn GPG, GPGSM and RFC1991 into lazy prereqs tests: do not let lazy prereqs inside `test_expect_*` turn off tracing t/lib-gpg.sh: stop pretending to be a stand-alone script tests(gpg): allow the gpg-agent to start on Windows
2020-04-10tests: when run in Bash, annotate test failures with file name/line numberLibravatar Johannes Schindelin1-1/+13
When a test fails, it is nice to see where the corresponding code lives in the worktree. Sadly, it seems that only Bash allows us to infer this information. Let's do it when we detect that we're running in a Bash. This will come in handy in the next commit, where we teach the GitHub Actions workflow to annotate failed test runs with this information. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-03-26tests: do not let lazy prereqs inside `test_expect_*` turn off tracingLibravatar Johannes Schindelin1-2/+4
The `test_expect_*` functions use `test_eval_` and so does `test_run_lazy_prereq_`. If tracing is enabled via the `-x` option, `test_eval_` turns on tracing while evaluating the code block, and turns it off directly after it. This is unwanted for nested invocations. One somewhat surprising example of this is when running a test that calls `test_i18ngrep`: that function requires the `C_LOCALE_OUTPUT` prereq, and that prereq is a lazy one, so it is evaluated via `test_eval_`, the command tracing is turned off, and the test case continues to run _without tracing the commands_. Another somewhat surprising example is when one lazy prereq depends on another lazy prereq: the former will call `test_have_prereq` with the latter one, which in turn calls `test_eval_` and -- you guessed it -- tracing (if enabled) will be turned off _before_ returning to evaluating the other lazy prereq. As we will introduce just such a scenario with the GPG, GPGSM and RFC1991 prereqs, let's fix that by introducing a variable that keeps track of the current trace level: nested `test_eval_` calls will increment and then decrement the level, and only when it reaches 0, the tracing will _actually_ be turned off. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-03-25test-lib: allow short options to be bundledLibravatar Matheus Tavares1-14/+47
When debugging a test (or a set of tests), it's common to execute it with some combination of short options, such as: $ ./txxx-testname.sh -d -x -i In cases like this, CLIs usually allow the short options to be bundled in a single argument, for convenience and agility. Let's add this feature to test-lib, allowing the above command to be run as: $ ./txxx-testname.sh -dxi (or any other permutation, e.g. '-ixd') Note: Short options that require an argument can also be used in a bundle, in any position. So, for example, '-r 5 -x', '-xr 5' and '-rx 5' are all valid and equivalent. A special case would be having a bundle with more than one of such options. To keep things simple, this case is not allowed for now. This shouldn't be a major limitation, though, as the only short option that requires an argument today is '-r'. And concatenating '-r's as in '-rr 5 6' would probably not be very practical: its unbundled format would be '-r 5 -r 6', for which test-lib currently considers only the last argument. Therefore, if '-rr 5 6' were to be allowed, it would have the same effect as just typing '-r 6'. Note: the test-lib currently doesn't support '-r5', as an alternative for '-r 5', so the former is not supported in bundles as well. Helped-by: Jeff King <peff@peff.net> Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-03-23tests(junit-xml): avoid invalid XMLLibravatar Johannes Schindelin1-0/+1
When a test case is run in a subshell, we finalize the JUnit-style XML when said subshell exits. But then we continue to write into that XML as if nothing had happened. This leads to Azure Pipelines' Publish Test Results task complaining: Failed to read /home/vsts/work/1/s/t/out/TEST-t0000-basic.xml. Error : Unexpected end tag. Line 110, position 5. And indeed, the resulting XML is incorrect. Let's "re-open" the XML in such a case, i.e. remove the previously added closing tags. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-02-24t: use hash-specific lookup tables to define test constantsLibravatar brian m. carlson1-15/+14
In the future, we'll allow developers to run the testsuite with a hash algorithm of their choice. To make this easier, compute the fixed constants using test_oid. Move the constant initialization down below the point where test-lib-functions.sh is loaded so the functions are defined. Note that we don't provide a value for the OID_REGEX value directly because writing a large number of instances of "[0-9a-f]" in the oid-info files is unwieldy and there isn't a way to compute it based on those values. Instead, compute it based on ZERO_OID. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-02-17Merge branch 'js/test-write-junit-xml-fix'Libravatar Junio C Hamano1-1/+2
Testfix. * js/test-write-junit-xml-fix: tests: fix --write-junit-xml with subshells
2020-02-12tests: fix --write-junit-xml with subshellsLibravatar Johannes Schindelin1-1/+2
In t0000, more precisely in its `test_bool_env` test case, there are two subshells that are supposed to fail. To be even _more_ precise, they fail by calling the `error` function, and that is okay, because it is in a subshell, and it is expected that those two subshell invocations fail. However, the `error` function also tries to finalize the JUnit XML (if that XML was asked for, via `--write-junit-xml`. As a consequence, the XML is edited to add a `time` attribute for the `testsuite` tag. And since there are two expected `error` calls in addition to the final `test_done`, the `finalize_junit_xml` function is called three times and naturally the `time` attribute is added _three times_. Azure Pipelines is not happy with that, complaining thusly: ##[warning]Failed to read D:\a\1\s\t\out\TEST-t0000-basic.xml. Error : 'time' is a duplicate attribute name. Line 2, position 82.. One possible way to address this would be to unset `write_junit_xml` in the `test_bool_env` test case. But that would be fragile, as other `error` calls in subshells could be introduced. So let's just modify `finalize_junit_xml` to remove any `time` attribute before adding the authoritative one. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-10Merge branch 'em/test-skip-regex-illseq'Libravatar Junio C Hamano1-1/+7
Test portability fix. * em/test-skip-regex-illseq: t4210: skip i18n tests that don't work on FreeBSD