summaryrefslogtreecommitdiff
path: root/t/test-lib.sh
AgeCommit message (Collapse)AuthorFilesLines
2021-01-15Merge branch 'ds/maintenance-part-4'Libravatar Junio C Hamano1-3/+4
Follow-up on the "maintenance part-3" which introduced scheduled maintenance tasks to support platforms whose native scheduling methods are not 'cron'. * ds/maintenance-part-4: maintenance: use Windows scheduled tasks maintenance: use launchctl on macOS maintenance: include 'cron' details in docs maintenance: extract platform-specific scheduling
2020-12-16tests: drop the `PREPARE_FOR_MAIN_BRANCH` prereqLibravatar Johannes Schindelin1-8/+0
We no longer use it. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-11-24maintenance: extract platform-specific schedulingLibravatar Derrick Stolee1-3/+4
The existing schedule mechanism using 'cron' is supported by POSIX platforms, but not Windows. It also works slightly differently on macOS to significant detriment of the user experience. To allow for new implementations on these platforms, extract a method that performs the platform-specific scheduling mechanism. This will be swapped at compile time with new implementations on specialized platforms. As we add this generality, rename GIT_TEST_CRONTAB to GIT_TEST_MAINT_SCHEDULER. Further, this variable is now parsed as "<scheduler>:<command>" so we can test platform-specific scheduling logic even when not on the correct platform. By specifying the <scheduler> in this string, we will be able to test all three sets of Git logic from a Linux machine. Co-authored-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-11-18Merge branch 'ds/maintenance-part-3'Libravatar Junio C Hamano1-0/+7
Parts of "git maintenance" to ease writing crontab entries (and other scheduling system configuration) for it. * ds/maintenance-part-3: maintenance: add troubleshooting guide to docs maintenance: use 'incremental' strategy by default maintenance: create maintenance.strategy config maintenance: add start/stop subcommands maintenance: add [un]register subcommands for-each-repo: run subcommands on configured repos maintenance: add --schedule option and config maintenance: optionally skip --auto process
2020-11-09Merge branch 'jk/perl-warning'Libravatar Junio C Hamano1-0/+6
Dev support. * jk/perl-warning: perl: check for perl warnings while running tests
2020-11-02Merge branch 'js/default-branch-name-part-4-minus-1'Libravatar Junio C Hamano1-0/+7
Adjust tests so that they won't scream when the default initial branch name is changed to 'main'. * js/default-branch-name-part-4-minus-1: t1400: prepare for `main` being default branch name tests: prepare aligned mentions of the default branch name t9902: prepare a test for the upcoming default branch name t3200: prepare for `main` being shorter than `master` t5703: adjust a test case for the upcoming default branch name t6200: adjust suppression pattern to also match "main" tests: start moving to a different default main branch name t9801: use `--` in preparation for default branch rename fmt-merge-msg: also suppress "into main" by default
2020-10-23tests: start moving to a different default main branch nameLibravatar Johannes Schindelin1-0/+7
To allow for an incremental conversion to a new default main branch name, let's introduce `GIT_TEST_DEFAULT_MAIN_BRANCH_NAME`. This environment variable can be set at the top of each converted test script, overriding the default main branch name to use when initializing new repositories (or cloning empty repositories). Note: the `GIT_TEST_DEFAULT_MAIN_BRANCH_NAME` is _not_ intended to be used manually; many tests require a specific main branch name and cannot simply work with another one. This `GIT_TEST_*` variable is meant purely for the transitional period while the entire test suite is converted to use `main` as the initial branch name by default. We also introduce the `PREPARE_FOR_MAIN_BRANCH` prereq that determines whether the default main branch name is `main`, and adjust a couple of test functions to use it. This prereq will be used to temporarily disable a couple test cases to allow for adjusting the test script incrementally. Once an entire test is adjusted, we will adjust the test so that it is run with `GIT_TEST_DEFAULT_MAIN_BRANCH_NAME=main`. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-10-21perl: check for perl warnings while running testsLibravatar Jeff King1-0/+6
We set "use warnings" in most of our perl code to catch problems. But as the name implies, warnings just emit a message to stderr and don't otherwise affect the program. So our tests are quite likely to miss that warnings are being spewed, as most of them do not look at stderr. We could ask perl to make all warnings fatal, but this is likely annoying for non-developers, who would rather have a running program with a warning than something that refuses to work at all. So instead, let's teach the perl code to respect an environment variable (GIT_PERL_FATAL_WARNINGS) to increase the severity of the warnings. This can be set for day-to-day running if people want to be really pedantic, but the primary use is to trigger it within the test suite. We could also trigger that for every test run, but likewise even the tests failing may be annoying to distro builders, etc (just as -Werror would be for compiling C code). So we'll tie it to a special test-mode variable (GIT_TEST_PERL_FATAL_WARNINGS) that can be set in the environment or as a Makefile knob, and we'll automatically turn the knob when DEVELOPER=1 is set. That should give developers and CI the more careful view without disrupting normal users or packagers. Note that the mapping from the GIT_TEST_* form to the GIT_* form in test-lib.sh is necessary even if they had the same name: the perl scripts need it to be normalized to a perl truth value, and we also have to make sure it's exported (we might have gotten it from the environment, but we might also have gotten it from GIT-BUILD-OPTIONS directly). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-10-18test-lib: reduce verbosity of skipped testsLibravatar Elijah Newren1-1/+0
When using the --run flag to run just two or three tests from a test file which contains several dozen tests, having every skipped test print out dozens of lines of output for the test code for that skipped test (in addition to the TAP output line) adds up to hundreds or thousands of lines of irrelevant output that make it very hard to fish out the relevant results you were looking for. Simplify the output for skipped tests to remove this extra output, leaving only the TAP output line (i.e. the line reading "ok <number> # skip <test-description>", which already mentions that the test was "skip"ped). Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-10-18test-lib: allow selecting tests by substring/glob with --runLibravatar Elijah Newren1-8/+12
Many of our test scripts have several "setup" tests. It's a lot easier to say ./t0050-filesystem.sh --run=setup,9 in order to run all the setup tests as well as test #9, than it is to track down what all the setup tests are and enter all their numbers in the list. Also, I often find myself wanting to run just one or a couple tests from the test file, but I don't know the numbering of any of the tests -- to get it I either have to first run the whole test file (or start counting by hand or figure out some other clever but non-obvious tricks). It's really convenient to be able to just look at the test description(s) and then run ./t6416-recursive-corner-cases.sh --run=symlink or ./t6402-merge-rename.sh --run='setup,unnecessary update' Add such an ability to test selection which relies on merely matching against the test description. Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-25maintenance: add start/stop subcommandsLibravatar Derrick Stolee1-0/+6
Add new subcommands to 'git maintenance' that start or stop background maintenance using 'cron', when available. This integration is as simple as I could make it, barring some implementation complications. The schedule is laid out as follows: 0 1-23 * * * $cmd maintenance run --schedule=hourly 0 0 * * 1-6 $cmd maintenance run --schedule=daily 0 0 * * 0 $cmd maintenance run --schedule=weekly where $cmd is a properly-qualified 'git for-each-repo' execution: $cmd=$path/git --exec-path=$path for-each-repo --config=maintenance.repo where $path points to the location of the Git executable running 'git maintenance start'. This is critical for systems with multiple versions of Git. Specifically, macOS has a system version at '/usr/bin/git' while the version that users can install resides at '/usr/local/bin/git' (symlinked to '/usr/local/libexec/git-core/git'). This will also use your locally-built version if you build and run this in your development environment without installing first. This conditional schedule avoids having cron launch multiple 'git for-each-repo' commands in parallel. Such parallel commands would likely lead to the 'hourly' and 'daily' tasks competing over the object database lock. This could lead to to some tasks never being run! Since the --schedule=<frequency> argument will run all tasks with _at least_ the given frequency, the daily runs will also run the hourly tasks. Similarly, the weekly runs will also run the daily and hourly tasks. The GIT_TEST_CRONTAB environment variable is not intended for users to edit, but instead as a way to mock the 'crontab [-l]' command. This variable is set in test-lib.sh to avoid a future test from accidentally running anything with the cron integration from modifying the user's schedule. We use GIT_TEST_CRONTAB='test-tool crontab <file>' in our tests to check how the schedule is modified in 'git maintenance (start|stop)' commands. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-08-11Merge branch 'bc/sha-256-part-3'Libravatar Junio C Hamano1-1/+8
The final leg of SHA-256 transition. * bc/sha-256-part-3: (39 commits) t: remove test_oid_init in tests docs: add documentation for extensions.objectFormat ci: run tests with SHA-256 t: make SHA1 prerequisite depend on default hash t: allow testing different hash algorithms via environment t: add test_oid option to select hash algorithm repository: enable SHA-256 support by default setup: add support for reading extensions.objectformat bundle: add new version for use with SHA-256 builtin/verify-pack: implement an --object-format option http-fetch: set up git directory before parsing pack hashes t0410: mark test with SHA1 prerequisite t5308: make test work with SHA-256 t9700: make hash size independent t9500: ensure that algorithm info is preserved in config t9350: make hash size independent t9301: make hash size independent t9300: use $ZERO_OID instead of hard-coded object ID t9300: abstract away SHA-1-specific constants t8011: make hash size independent ...
2020-07-30Merge branch 'jk/tests-timestamp-fix' into masterLibravatar Junio C Hamano1-0/+3
The test framework has been updated so that most tests will run with predictable (artificial) timestamps. * jk/tests-timestamp-fix: t9100: stop depending on commit timestamps test-lib: set deterministic default author/committer date t9100: explicitly unset GIT_COMMITTER_DATE t5539: make timestamp requirements more explicit t9700: loosen ident timezone regex t6000: use test_tick consistently
2020-07-30t: make SHA1 prerequisite depend on default hashLibravatar brian m. carlson1-1/+5
Currently, the SHA1 prerequisite depends on the output of git hash-object. However, in order for that to produce sane behavior, we must be in a repository. If we are not, the default will remain SHA-1, and we'll produce wrong results if we're using SHA-256 for the testsuite but the test assertion starts when we're not in a repository. Check the environment variable we use for this purpose, leaving it to default to SHA-1 if none is specified. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Reviewed-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-30t: allow testing different hash algorithms via environmentLibravatar brian m. carlson1-0/+3
To allow developers to run the testsuite with a different algorithm than the default, provide an environment variable, GIT_TEST_DEFAULT_HASH, to specify the algorithm to use. Compute the fixed constants using test_oid. Move the constant initialization down below the point where test-lib-functions.sh is loaded so the functions are defined. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Reviewed-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-14test-lib: set deterministic default author/committer dateLibravatar Jeff King1-0/+3
We always set the name and email for committer and author idents to make the test suite more deterministic, but not timestamps. Many scripts use test_tick to get consistent and sensibly incrementing timestamps as they create commits. But other scripts don't particularly care about the timestamp, and are happy to use whatever the current system time is. This non-determinism can be annoying: - when debugging a test, comparing results between two runs can be difficult, because the commit ids change - this can sometimes cause tests to be racy. E.g., traversal order depends on timestamp order. Even in a well-ordered set of commands, because our timestamp granularity is one second, two commits might sometimes have the same timestamp and sometimes differ. Let's set a default timestamp for all scripts to use. Any that use test_tick already will be unaffected (because their first test_tick call will overwrite our default), but it will make things a bit more deterministic for those that don't. We should be able to choose any time we want here. I picked this one because: - it differs from the initial test_tick default, which may make it easier to distinguish when debugging tests. I picked "April 1st 13:14:15" in the hope that it might stand out. - it's slightly before the test_tick default. Some tests create some commits before the first call to test_tick, so using an older timestamps for those makes sense chronologically. Note that this isn't how things currently work (where system times are usually more recent than test_tick), but that also allows us to flush out a few hidden timestamp dependencies (like the one recently fixed in t5539). - we could likewise pick any timezone we want. Choosing +0000 would have required fixing up fewer tests, but we're more likely to turn up interesting cases by not matching $TZ exactly. And since test_tick already checks "-0700", let's try something in the "+" zone range for variety. It's possible that the non-deterministic times could help flush out bugs (e.g., if something broke when the clock flipped over to 2021, our test suite would let us know). But historically that hasn't been the case; all time-dependent outcomes we've seen turned out to be accidentally flaky tests (which we fixed by using test_tick). If we do want to cover handling the current time, we should dedicate one script to doing so, and have it unset GIT_COMMITTER_DATE explicitly. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-06Merge branch 'bc/sha-256-part-2'Libravatar Junio C Hamano1-0/+1
SHA-256 migration work continues. * bc/sha-256-part-2: (44 commits) remote-testgit: adapt for object-format bundle: detect hash algorithm when reading refs t5300: pass --object-format to git index-pack t5704: send object-format capability with SHA-256 t5703: use object-format serve option t5702: offer an object-format capability in the test t/helper: initialize the repository for test-sha1-array remote-curl: avoid truncating refs with ls-remote t1050: pass algorithm to index-pack when outside repo builtin/index-pack: add option to specify hash algorithm remote-curl: detect algorithm for dumb HTTP by size builtin/ls-remote: initialize repository based on fetch t5500: make hash independent serve: advertise object-format capability for protocol v2 connect: parse v2 refs with correct hash algorithm connect: pass full packet reader when parsing v2 refs Documentation/technical: document object-format for protocol v2 t1302: expect repo format version 1 for SHA-256 builtin/show-index: provide options to determine hash algo t5302: modernize test formatting ...
2020-06-08Merge branch 'cb/t4210-illseq-auto-detect'Libravatar Junio C Hamano1-6/+0
As FreeBSD is not the only platform whose regexp library reports a REG_ILLSEQ error when fed invalid UTF-8, add logic to detect that automatically and skip the affected tests. * cb/t4210-illseq-auto-detect: t4210: detect REG_ILLSEQ dynamically and skip affected tests t/helper: teach test-regex to report pattern errors (like REG_ILLSEQ)
2020-05-24Merge branch 'gp/hppa-stack-test-fix'Libravatar Junio C Hamano1-2/+10
Platform dependent tweak to a test for HP-PA. * gp/hppa-stack-test-fix: tests: skip small-stack tests on hppa architecture
2020-05-18t4210: detect REG_ILLSEQ dynamically and skip affected testsLibravatar Carlo Marcelo Arenas Belón1-6/+0
7187c7bbb8 (t4210: skip i18n tests that don't work on FreeBSD, 2019-11-27) adds a REG_ILLSEQ prerequisite, and to do that copies the common branch in test-lib and expands it to include it in a special case for FreeBSD. Instead; test for it using a previously added extension to test-tool and use that, together with a function that identifies when regcomp/regexec will be called with broken patterns to avoid any test that would otherwise rely on undefined behaviour. The description of the first test which wasn't accurate has been corrected, and the test rearranged for clarity, including a helper function that avoids overly long lines. Only the affected engines will have their tests suppressed, also including "fixed" if the PCRE optimization that uses LIBPCRE2 since b65abcafc7 (grep: use PCRE v2 for optimized fixed-string search, 2019-07-01) is not available. Helped-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-05-18tests: skip small-stack tests on hppa architectureLibravatar Greg Price1-2/+10
On hppa these tests crash because the allocated stack space is too small, even after it was doubled in b9a190789 (and the data size doubled to match) to make it work on powerpc. For this arch just skip these tests, which is enough to make the whole suite pass. Fixes: https://bugs.debian.org/757402 Based-on-patch-by: John David Anglin <dave.anglin@bell.net> Signed-off-by: Greg Price <gnprice@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-05-15Revert "tests: when run in Bash, annotate test failures with file name/line ↵Libravatar Junio C Hamano1-13/+1
number" This reverts commit 662f9cf1548cf069cb819e9e95f224657015fcf9, to fix the TAP output broken for bash.
2020-05-15Revert "t/test_lib: avoid naked bash arrays in file_lineno"Libravatar Junio C Hamano1-10/+8
This reverts commit 303775a25f0b4ac5d6ad2e96eb4404c24209cad8; instead of trying to salvage the tap-breaking change, let's revert the whole thing for now.
2020-05-12t1050: match object ID paths in a hash-insensitive wayLibravatar brian m. carlson1-0/+1
The pattern here looking for failures is specific to SHA-1. Let's create a variable that matches the regex or glob pattern for a path within the objects directory. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-05-07t/test_lib: avoid naked bash arrays in file_linenoLibravatar Carlo Marcelo Arenas Belón1-8/+10
662f9cf154 (tests: when run in Bash, annotate test failures with file name/line number, 2020-04-11), introduces a way to report the location (file:lineno) of a failed test case by traversing the bash callstack. The implementation requires bash and uses shell arrays and is therefore protected by a guard but NetBSD sh will still have to parse the function and therefore will result in: ** t0000-basic.sh *** ./test-lib.sh: 681: Syntax error: Bad substitution Enclose the bash specific code inside an eval to avoid parsing errors in the same way than 5826b7b595 (test-lib: check Bash version for '-x' without using shell arrays, 2019-01-03) Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-05-01Merge branch 'tb/commit-graph-fd-exhaustion-fix'Libravatar Junio C Hamano1-0/+9
The commit-graph code exhausted file descriptors easily when it does not have to. * tb/commit-graph-fd-exhaustion-fix: commit-graph: close descriptors after mmap commit-graph.c: gracefully handle file descriptor exhaustion t/test-lib.sh: make ULIMIT_FILE_DESCRIPTORS available to tests commit-graph.c: don't use discarded graph_name in error
2020-04-29Merge branch 'dd/ci-swap-azure-pipelines-with-github-actions'Libravatar Junio C Hamano1-1/+13
Update the CI configuration to use GitHub Actions, retiring the one based on Azure Pipelines. * dd/ci-swap-azure-pipelines-with-github-actions: ci: let GitHub Actions upload failed tests' directories ci: add a problem matcher for GitHub Actions tests: when run in Bash, annotate test failures with file name/line number ci: retire the Azure Pipelines definition README: add a build badge for the GitHub Actions runs ci: configure GitHub Actions for CI/PR ci: run gem with sudo to install asciidoctor ci: explicit install all required packages ci: fix the `jobname` of the `GETTEXT_POISON` job ci/lib: set TERM environment variable if not exist ci/lib: allow running in GitHub Actions ci/lib: if CI type is unknown, show the environment variables
2020-04-23t/test-lib.sh: make ULIMIT_FILE_DESCRIPTORS available to testsLibravatar Taylor Blau1-0/+9
In t1400 the prerequisite 'ULIMIT_FILE_DESCRIPTORS' is defined and used to effectively guard the helper function 'run_with_limited_open_files' from being used on systems that do not satisfy this prerequisite. In the subsequent patch, we will introduce another test outside of t1400 that would benefit from using this prerequisite. So, move it to 'test-lib.sh' instead so that it can be used by multiple tests. Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-22Merge branch 'mt/test-lib-bundled-short-options'Libravatar Junio C Hamano1-14/+47
Minor test usability improvement. * mt/test-lib-bundled-short-options: test-lib: allow short options to be bundled
2020-04-22Merge branch 'js/test-junit-finalization-fix'Libravatar Junio C Hamano1-0/+1
Test fix. * js/test-junit-finalization-fix: tests(junit-xml): avoid invalid XML
2020-04-22Merge branch 'js/tests-gpg-integration-on-windows'Libravatar Junio C Hamano1-2/+4
Enable tests that require GnuPG on Windows. * js/tests-gpg-integration-on-windows: tests: increase the verbosity of the GPG-related prereqs tests: turn GPG, GPGSM and RFC1991 into lazy prereqs tests: do not let lazy prereqs inside `test_expect_*` turn off tracing t/lib-gpg.sh: stop pretending to be a stand-alone script tests(gpg): allow the gpg-agent to start on Windows
2020-04-10tests: when run in Bash, annotate test failures with file name/line numberLibravatar Johannes Schindelin1-1/+13
When a test fails, it is nice to see where the corresponding code lives in the worktree. Sadly, it seems that only Bash allows us to infer this information. Let's do it when we detect that we're running in a Bash. This will come in handy in the next commit, where we teach the GitHub Actions workflow to annotate failed test runs with this information. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-03-26tests: do not let lazy prereqs inside `test_expect_*` turn off tracingLibravatar Johannes Schindelin1-2/+4
The `test_expect_*` functions use `test_eval_` and so does `test_run_lazy_prereq_`. If tracing is enabled via the `-x` option, `test_eval_` turns on tracing while evaluating the code block, and turns it off directly after it. This is unwanted for nested invocations. One somewhat surprising example of this is when running a test that calls `test_i18ngrep`: that function requires the `C_LOCALE_OUTPUT` prereq, and that prereq is a lazy one, so it is evaluated via `test_eval_`, the command tracing is turned off, and the test case continues to run _without tracing the commands_. Another somewhat surprising example is when one lazy prereq depends on another lazy prereq: the former will call `test_have_prereq` with the latter one, which in turn calls `test_eval_` and -- you guessed it -- tracing (if enabled) will be turned off _before_ returning to evaluating the other lazy prereq. As we will introduce just such a scenario with the GPG, GPGSM and RFC1991 prereqs, let's fix that by introducing a variable that keeps track of the current trace level: nested `test_eval_` calls will increment and then decrement the level, and only when it reaches 0, the tracing will _actually_ be turned off. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-03-25test-lib: allow short options to be bundledLibravatar Matheus Tavares1-14/+47
When debugging a test (or a set of tests), it's common to execute it with some combination of short options, such as: $ ./txxx-testname.sh -d -x -i In cases like this, CLIs usually allow the short options to be bundled in a single argument, for convenience and agility. Let's add this feature to test-lib, allowing the above command to be run as: $ ./txxx-testname.sh -dxi (or any other permutation, e.g. '-ixd') Note: Short options that require an argument can also be used in a bundle, in any position. So, for example, '-r 5 -x', '-xr 5' and '-rx 5' are all valid and equivalent. A special case would be having a bundle with more than one of such options. To keep things simple, this case is not allowed for now. This shouldn't be a major limitation, though, as the only short option that requires an argument today is '-r'. And concatenating '-r's as in '-rr 5 6' would probably not be very practical: its unbundled format would be '-r 5 -r 6', for which test-lib currently considers only the last argument. Therefore, if '-rr 5 6' were to be allowed, it would have the same effect as just typing '-r 6'. Note: the test-lib currently doesn't support '-r5', as an alternative for '-r 5', so the former is not supported in bundles as well. Helped-by: Jeff King <peff@peff.net> Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-03-23tests(junit-xml): avoid invalid XMLLibravatar Johannes Schindelin1-0/+1
When a test case is run in a subshell, we finalize the JUnit-style XML when said subshell exits. But then we continue to write into that XML as if nothing had happened. This leads to Azure Pipelines' Publish Test Results task complaining: Failed to read /home/vsts/work/1/s/t/out/TEST-t0000-basic.xml. Error : Unexpected end tag. Line 110, position 5. And indeed, the resulting XML is incorrect. Let's "re-open" the XML in such a case, i.e. remove the previously added closing tags. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-02-24t: use hash-specific lookup tables to define test constantsLibravatar brian m. carlson1-15/+14
In the future, we'll allow developers to run the testsuite with a hash algorithm of their choice. To make this easier, compute the fixed constants using test_oid. Move the constant initialization down below the point where test-lib-functions.sh is loaded so the functions are defined. Note that we don't provide a value for the OID_REGEX value directly because writing a large number of instances of "[0-9a-f]" in the oid-info files is unwieldy and there isn't a way to compute it based on those values. Instead, compute it based on ZERO_OID. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-02-17Merge branch 'js/test-write-junit-xml-fix'Libravatar Junio C Hamano1-1/+2
Testfix. * js/test-write-junit-xml-fix: tests: fix --write-junit-xml with subshells
2020-02-12tests: fix --write-junit-xml with subshellsLibravatar Johannes Schindelin1-1/+2
In t0000, more precisely in its `test_bool_env` test case, there are two subshells that are supposed to fail. To be even _more_ precise, they fail by calling the `error` function, and that is okay, because it is in a subshell, and it is expected that those two subshell invocations fail. However, the `error` function also tries to finalize the JUnit XML (if that XML was asked for, via `--write-junit-xml`. As a consequence, the XML is edited to add a `time` attribute for the `testsuite` tag. And since there are two expected `error` calls in addition to the final `test_done`, the `finalize_junit_xml` function is called three times and naturally the `time` attribute is added _three times_. Azure Pipelines is not happy with that, complaining thusly: ##[warning]Failed to read D:\a\1\s\t\out\TEST-t0000-basic.xml. Error : 'time' is a duplicate attribute name. Line 2, position 82.. One possible way to address this would be to unset `write_junit_xml` in the `test_bool_env` test case. But that would be fragile, as other `error` calls in subshells could be introduced. So let's just modify `finalize_junit_xml` to remove any `time` attribute before adding the authoritative one. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-10Merge branch 'em/test-skip-regex-illseq'Libravatar Junio C Hamano1-1/+7
Test portability fix. * em/test-skip-regex-illseq: t4210: skip i18n tests that don't work on FreeBSD
2019-12-05Merge branch 'sg/test-bool-env'Libravatar Junio C Hamano1-5/+5
Recently we have declared that GIT_TEST_* variables take the usual boolean values (it used to be that some used "non-empty means true" and taking GIT_TEST_VAR=YesPlease as true); make sure we notice and fail when non-bool strings are given to these variables. * sg/test-bool-env: t5608-clone-2gb.sh: turn GIT_TEST_CLONE_2GB into a bool tests: add 'test_bool_env' to catch non-bool GIT_TEST_* values
2019-12-01Merge branch 'sg/skip-skipped-prereq'Libravatar Junio C Hamano1-6/+6
Test update to avoid wasted cycles. * sg/skip-skipped-prereq: test-lib: don't check prereqs of test cases that won't be run anyway
2019-11-30t4210: skip i18n tests that don't work on FreeBSDLibravatar Ed Maste1-1/+7
A number of t4210-log-i18n tests added in 4e2443b181 set LC_ALL to a UTF-8 locale (is_IS.UTF-8) but then pass an invalid UTF-8 string to --grep. FreeBSD's regcomp() fails in this case with REG_ILLSEQ, "illegal byte sequence," which git then passes to die(): fatal: command line: '�': illegal byte sequence When these tests were added the commit message stated: | It's possible that this | test breaks the "basic" and "extended" backends on some systems that | are more anal than glibc about the encoding of locale issues with | POSIX functions that I can remember which seems to be the case here. Extend test-lib.sh to add a REGEX_ILLSEQ prereq, set it on FreeBSD, and add !REGEX_ILLSEQ to the two affected tests. Signed-off-by: Ed Maste <emaste@freebsd.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-23tests: add 'test_bool_env' to catch non-bool GIT_TEST_* valuesLibravatar SZEDER Gábor1-5/+5
Since 3b072c577b (tests: replace test_tristate with "git env--helper", 2019-06-21) we get the normalized bool values of various GIT_TEST_* environment variables via 'git env--helper'. Now, while the 'git env--helper' command itself does catch invalid values in the environment variable or in the given --default and exits with error (exit code 128 or 129, respectively), it's invoked in conditions like 'if ! git env--helper ...', which means that all invalid bool values are interpreted the same as the ordinary 'false' (exit code 1). This has led to inadvertently skipped httpd tests in our CI builds for a couple of weeks, see 3960290675 (ci: restore running httpd tests, 2019-09-06). Let's be more careful about what the test suite accepts as bool values in GIT_TEST_* environment variables, and error out loud and clear on invalid values instead of simply skipping tests. Add the 'test_bool_env' helper function to encapsulate the invocation of 'git env--helper' and the verification of its exit code, and replace all invocations of that command in our test framework and test suite with a call to this new helper (except in 't0017-env-helper.sh', of course). $ GIT_TEST_GIT_DAEMON=YesPlease ./t5570-git-daemon.sh fatal: bad numeric config value 'YesPlease' for 'GIT_TEST_GIT_DAEMON': invalid unit error: test_bool_env requires bool values both for $GIT_TEST_GIT_DAEMON and for the default fallback Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-13test-lib: don't check prereqs of test cases that won't be run anywayLibravatar SZEDER Gábor1-6/+6
With './t1234-foo.sh -r 5,6' we can run only specific test cases in a test script, but our test framwork still evaluates all lazy prereqs that the excluded test cases might depend on. This is unnecessary and produces verbose and trace output that can be distracting. This has been an issue ever since the '-r|--run=' options were introduced in 0445e6f0a1 (test-lib: '--run' to run only specific tests, 2014-04-30), because that commit added the check of the list of test cases specified with '-r' after evaluating the prereqs. Avoid this unnecessary prereq evaluation by checking the list of test cases specified with '-r' before looking at the prereqs. Note that GIT_SKIP_TESTS has always been checked before the prereqs, so prereqs necessary for tests skipped that way were not evaluated. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-30pretty: add "%aL" etc. to show local-part of email addressesLibravatar Prarit Bhargava1-2/+6
In many projects the number of contributors is low enough that users know each other and the full email address doesn't need to be displayed. Displaying only the author's username saves a lot of columns on the screen. Existing 'e/E' (as in "%ae" and "%aE") placeholders would show the author's address as "prarit@redhat.com", which would waste columns to show the same domain-part for all contributors when used in a project internal to redhat. Introduce 'l/L' placeholders that strip '@' and domain part from the e-mail address. Signed-off-by: Prarit Bhargava <prarit@redhat.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-15Merge branch 'js/azure-pipelines-msvc'Libravatar Junio C Hamano1-16/+22
CI updates. * js/azure-pipelines-msvc: ci: also build and test with MS Visual Studio on Azure Pipelines ci: really use shallow clones on Azure Pipelines tests: let --immediate and --write-junit-xml play well together test-tool run-command: learn to run (parts of) the testsuite vcxproj: include more generated files vcxproj: only copy `git-remote-http.exe` once it was built msvc: work around a bug in GetEnvironmentVariable() msvc: handle DEVELOPER=1 msvc: ignore some libraries when linking compat/win32/path-utils.h: add #include guards winansi: use FLEX_ARRAY to avoid compiler warning msvc: avoid using minus operator on unsigned types push: do not pretend to return `int` from `die_push_simple()`
2019-10-06tests: let --immediate and --write-junit-xml play well togetherLibravatar Johannes Schindelin1-16/+22
When the `--immediate` option is in effect, any test failure will immediately exit the test script. Together with `--write-junit-xml`, we will want the JUnit-style `.xml` file to be finalized (and not leave the XML incomplete). Let's make it so. This comes in particularly handy when trying to debug via Azure Pipelines, where the JUnit-style XML is consumed to present the test results in an informative and helpful way. While at it, also handle the `error()` code path. The only remaining code path that sets `GIT_EXIT_OK` happens whenever the trash directory could not be set up, i.e. long before the JUnit XML was written, therefore we should _not_ try to finalize that XML in that case. It is tempting to change the `immediate` code path to just hand off to `error`, simplifying the code in the process. That would, however, result in a change of behavior (an additional error message) in the test suite, which is outside of the purview of the current patch series: its goal is to allow building Git with Visual Studio and testing it with a portable version of Git for Windows. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-09-06t: use common $SQ variableLibravatar Denton Liu1-0/+3
In many test scripts, there are bespoke definitions of the single quote that are some variation of this: SQ="'" Define a common $SQ variable in test-lib.sh and replace all usages of these bespoke variables with the common one. This change was done by running `git grep =\"\'\" t/` and `git grep =\\\\\'` and manually changing the resulting definitions and corresponding usages. Signed-off-by: Denton Liu <liu.denton@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-22Merge branch 'sg/show-failed-test-names'Libravatar Junio C Hamano1-0/+2
The first line of verbose output from each test piece now carries the test name and number to help scanning with eyeballs. * sg/show-failed-test-names: tests: show the test name and number at the start of verbose output t0000-basic: use realistic test script names in the verbose tests
2019-08-05tests: show the test name and number at the start of verbose outputLibravatar SZEDER Gábor1-0/+2
The verbose output of every test looks something like this: expecting success: echo content >file && git add file && git commit -m "add file" [master (root-commit) d1fbfbd] add file Author: A U Thor <author@example.com> 1 file changed, 1 insertion(+) create mode 100644 file ok 1 - commit works i.e. first an "expecting success" (or "checking known breakage") line followed by the commands to be executed, then the output of those comamnds, and finally an "ok"/"not ok" line containing the test name. Note that the test's name is only shown at the very end. With '-x' tracing enabled and/or in longer tests the verbose output might be several screenfulls long, making it harder than necessary to find where the output of the test with a given name starts (especially when the outputs to different file descriptors are racing, and the "expecting success"/command block arrives earlier than the "ok" line of the previous test). Print the test name at the start of the test's verbose output, i.e. at the end of the "expecting success" and "checking known breakage" lines, to make the start of a particular test a bit easier to recognize. Also print the test script and test case numbers, to help those poor souls who regularly have to scan through the combined verbose output of several test scripts. So the dummy test above would start like this: expecting success of 9999.1 'commit works': echo content >file && [...] Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>