summaryrefslogtreecommitdiff
path: root/ci/lib.sh
AgeCommit message (Collapse)AuthorFilesLines
2021-12-15Merge branch 'ab/ci-updates'Libravatar Junio C Hamano1-45/+15
Drop support for TravisCI and update test workflows at GitHub. * ab/ci-updates: CI: don't run "make test" twice in one job CI: use "$runs_on_pool", not "$jobname" to select packages & config CI: rename the "Linux32" job to lower-case "linux32" CI: use shorter names that fit in UX tooltips CI: remove Travis CI support
2021-12-15Merge branch 'hn/reftable'Libravatar Junio C Hamano1-0/+1
The "reftable" backend for the refs API, without integrating into the refs subsystem, has been added. * hn/reftable: Add "test-tool dump-reftable" command. reftable: add dump utility reftable: implement stack, a mutable database of reftable files. reftable: implement refname validation reftable: add merged table view reftable: add a heap-based priority queue for reftable records reftable: reftable file level tests reftable: read reftable files reftable: generic interface to tables reftable: write reftable files reftable: a generic binary tree implementation reftable: reading/writing blocks Provide zlib's uncompress2 from compat/zlib-compat.c reftable: (de)serialization for the polymorphic record type. reftable: add blocksource, an abstraction for random access reads reftable: utility functions reftable: add error related functionality reftable: add LICENSE hash.h: provide constants for the hash IDs
2021-11-23CI: use "$runs_on_pool", not "$jobname" to select packages & configLibravatar Ævar Arnfjörð Bjarmason1-10/+11
Change the setup hooks for the CI to use "$runs_on_pool" for the "$regular" job. Now we won't need as much boilerplate when adding new jobs to the "regular" matrix, see 956d2e4639b (tests: add a test mode for SANITIZE=leak, run it in CI, 2021-09-23) for the last such commit. I.e. now instead of needing to enumerate each jobname when we select packages we can install things depending on the pool we're running in. That we didn't do this dates back to the now gone dependency on Travis CI, but even if we add a new CI target in the future this'll be easier to port over, since we can probably treat "ubuntu-latest" as a stand-in for some recent Linux that can run "apt" commands. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-11-23CI: rename the "Linux32" job to lower-case "linux32"Libravatar Ævar Arnfjörð Bjarmason1-1/+1
As a follow-up to the preceding commit's shortening of CI job names, rename the only job that starts with an upper-case letter to be consistent with the rest. It was added in 88dedd5e72c (Travis: also test on 32-bit Linux, 2017-03-05). Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-11-23CI: remove Travis CI supportLibravatar Ævar Arnfjörð Bjarmason1-34/+3
Remove support for running the CI in travis. The last builds in it are from 5 months ago[1] (as of 2021-11-19), and our documentation has referred to GitHub CI instead since f003a91f5c5 (SubmittingPatches: replace discussion of Travis with GitHub Actions, 2021-07-22). We'll now run the "t9810 t9816" and tests on OSX. We didn't before, as we'd carried the Travis exclusion of them forward from 522354d70f4 (Add Travis CI support, 2015-11-27). Let's hope whatever issue there was with them was either Travis specific, or fixed since then (I'm not sure). The "apt-add-repository" invocation (which we were doing in GitHub CI) isn't needed, it was another Travis-only case that was carried forward into more general code. See 0f0c51181df (travis-ci: install packages in 'ci/install-dependencies.sh', 2018-11-01). Remove the "linux-gcc-4.8" job added in fb9d7431cf4 (travis-ci: build with GCC 4.8 as well, 2019-07-18), it only ran in Travis CI. 1. https://travis-ci.org/github/git/git/builds Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-08Provide zlib's uncompress2 from compat/zlib-compat.cLibravatar Han-Wen Nienhuys1-0/+1
This will be needed for reading reflog blocks in reftable. Helped-by: Carlo Marcelo Arenas Belón <carenas@gmail.com> Signed-off-by: Han-Wen Nienhuys <hanwen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-23tests: add a test mode for SANITIZE=leak, run it in CILibravatar Ævar Arnfjörð Bjarmason1-1/+8
While git can be compiled with SANITIZE=leak, we have not run regression tests under that mode. Memory leaks have only been fixed as one-offs without structured regression testing. This change adds CI testing for it. We'll now build and small set of whitelisted t00*.sh tests under Linux with a new job called "linux-leaks". The CI target uses a new GIT_TEST_PASSING_SANITIZE_LEAK=true test mode. When running in that mode, we'll assert that we were compiled with SANITIZE=leak. We'll then skip all tests, except those that we've opted-in by setting "TEST_PASSES_SANITIZE_LEAK=true". A test setting "TEST_PASSES_SANITIZE_LEAK=true" setting can in turn make use of the "SANITIZE_LEAK" prerequisite, should they wish to selectively skip tests even under "GIT_TEST_PASSING_SANITIZE_LEAK=true". In the preceding commit we started doing this in "t0004-unwritable.sh" under SANITIZE=leak, now it'll combine nicely with "GIT_TEST_PASSING_SANITIZE_LEAK=true". This is how tests that don't set "TEST_PASSES_SANITIZE_LEAK=true" will be skipped under GIT_TEST_PASSING_SANITIZE_LEAK=true: $ GIT_TEST_PASSING_SANITIZE_LEAK=true ./t0001-init.sh 1..0 # SKIP skip all tests in t0001 under SANITIZE=leak, TEST_PASSES_SANITIZE_LEAK not set The intent is to add more TEST_PASSES_SANITIZE_LEAK=true annotations as follow-up change, but let's start small to begin with. In ci/run-build-and-tests.sh we make use of the default "*" case to run "make test" without any GIT_TEST_* modes. SANITIZE=leak is known to fail in combination with GIT_TEST_SPLIT_INDEX=true in t0016-oidmap.sh, and we're likely to have other such failures in various GIT_TEST_* modes. Let's focus on getting the base tests passing, we can expand coverage to GIT_TEST_* modes later. It would also be possible to implement a more lightweight version of this by only relying on setting "LSAN_OPTIONS". See <YS9OT/pn5rRK9cGB@coredump.intra.peff.net>[1] and <YS9ZIDpANfsh7N+S@coredump.intra.peff.net>[2] for a discussion of that. I've opted for this approach of adding a GIT_TEST_* mode instead because it's consistent with how we handle other special test modes. Being able to add a "!SANITIZE_LEAK" prerequisite and calling "test_done" early if it isn't satisfied also means that we can more incrementally add regression tests without being forced to fix widespread and hard-to-fix leaks at the same time. We have tests that do simple checking of some tool we're interested in, but later on in the script might be stressing trace2, or common sources of leaks like "git log" in combination with the tool (e.g. the commit-graph tests). To be clear having a prerequisite could also be accomplished by using "LSAN_OPTIONS" directly. On the topic of "LSAN_OPTIONS": It would be nice to have a mode to aggregate all failures in our various scripts, see [2] for a start at doing that which sets "log_path" in "LSAN_OPTIONS". I've punted on that for now, it can be added later. As of writing this we've got major regressions between master..seen, i.e. the t000*.sh tests and more fixed since 31f9acf9ce2 (Merge branch 'ah/plugleaks', 2021-08-04) have regressed recently. See the discussion at <87czsv2idy.fsf@evledraar.gmail.com>[3] about the lack of this sort of test mode, and 0e5bba53af (add UNLEAK annotation for reducing leak false positives, 2017-09-08) for the initial addition of SANITIZE=leak. See also 09595ab381 (Merge branch 'jk/leak-checkers', 2017-09-19), 7782066f67 (Merge branch 'jk/apache-lsan', 2019-05-19) and the recent 936e58851a (Merge branch 'ah/plugleaks', 2021-05-07) for some of the past history of "one-off" SANITIZE=leak (and more) fixes. As noted in [5] we can't support this on OSX yet until Clang 14 is released, at that point we'll probably want to resurrect that "osx-leaks" job. 1. https://github.com/google/sanitizers/wiki/AddressSanitizerLeakSanitizer 2. https://lore.kernel.org/git/YS9OT%2Fpn5rRK9cGB@coredump.intra.peff.net/ 3. https://lore.kernel.org/git/87czsv2idy.fsf@evledraar.gmail.com/ 4. https://lore.kernel.org/git/YS9ZIDpANfsh7N+S@coredump.intra.peff.net/ 5. https://lore.kernel.org/git/20210916035603.76369-1-carenas@gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-08t: use user-specified utf-8 locale for testing svnLibravatar Đoàn Trần Công Danh1-0/+1
In some test-cases, UTF-8 locale is required. To find such locale, we're using the first available UTF-8 locale that returned by "locale -a". However, the locale(1) utility is unavailable on some systems, e.g. Linux with musl libc. However, without "locale -a", we can't guess provided UTF-8 locale. Add a Makefile knob GIT_TEST_UTF8_LOCALE and activate it for linux-musl in our CI system. Rename t/lib-git-svn.sh:prepare_a_utf8_locale to prepare_utf8_locale, since we no longer prepare the variable named "a_utf8_locale", but set up a fallback value for GIT_TEST_UTF8_LOCALE instead. The fallback will be LC_ALL, LANG environment variable, or the first UTF-8 locale from output of "locale -a", in that order. Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-01-21ci: remove GETTEXT_POISON jobsLibravatar Ævar Arnfjörð Bjarmason1-2/+1
A subsequent commit will remove GETTEXT_POISON entirely, let's start by removing the CI jobs that enable the option. We cannot just remove the job because the CI is implicitly depending on the "poison" job being a sort of "default" job in the sense that it's the job that was otherwise run with the default compiler, no other GIT_TEST_* options etc. So let's keep it under the name "linux-gcc-default". This means we can remove the initial "make test" from the "linux-gcc" job (it does another one after setting a bunch of GIT_TEST_* variables). I'm not doing that because it would conflict with the in-flight 334afbc76fb (tests: mark tests relying on the current default for `init.defaultBranch`, 2020-11-18) (currently on the "seen" branch, so the SHA-1 will almost definitely change). It's going to use that "make test" again for different reasons, so let's preserve it for now. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-10-08ci: do not skip tagged revisions in GitHub workflowsLibravatar Johannes Schindelin1-0/+2
When `master` is tagged, and then both `master` and the tag are pushed, Travis CI will happily build both. That is a waste of energy, which is why we skip the build for `master` in that case. Our GitHub workflow is also triggered by tags. However, the run would fail because the `windows-test` jobs are _not_ skipped on tags, but the `windows-build` job _is skipped (and therefore fails to upload the build artifacts needed by the test jobs). In addition, we just added logic to our GitHub workflow that will skip runs altogether if there is already a successful run for the same commit or at least for the same tree. Let's just change the GitHub workflow to no longer specifically skip tagged revisions. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-21ci: stop linking built-ins to the dashed versionsLibravatar Johannes Schindelin1-0/+1
Since e4597aae6590 (run test suite without dashed git-commands in PATH, 2009-12-02), we stopped running our tests with `git-foo` binaries found at the top-level directory of a freshly built source tree; instead we have placed only `git` and selected `git-foo` commands that must be on `$PATH` in `bin-wrappers/` and prepended that `bin-wrappers/` to the `PATH` used in the test suite. We did that to catch the tests and scripted Git commands that still try to use the dashed form. Since CI jobs will not install the built Git to anywhere, and the hardlinks we make at the top-level of the source tree for `git-add` and friends are not even used during tests, they are pure waste of resources these days. Thanks to the newly invented `SKIP_DASHED_BUILT_INS` knob, we can now skip creating these links in the source tree. So let's do that. Note that this change introduces a subtle change of behavior: when Git's `cmd_main()` calls `setup_path()`, it inserts the value of `GIT_EXEC_PATH` (defaulting to `<prefix>/libexec/git-core`) at the beginning of the environment variable `PATH`. This is necessary to find e.g. scripted commands that are installed in that location. For the purposes of Git's test suite, the `bin-wrappers/` scripts override `GIT_EXEC_PATH` to point to the top-level directory of the source code. In other words, if a scripted command had used a dashed invocation of a built-in Git command, it would not have been caught previously, which is fixed by this change. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-23ci: use absolute PYTHON_PATH in the Linux jobsLibravatar SZEDER Gábor1-2/+2
In our test suite, when 'git p4' invokes a Git command as a subprocesses, then it should run the 'git' binary we are testing. Unfortunately, this is not the case in the 'linux-clang' and 'linux-gcc' jobs on Travis CI, where 'git p4' runs the system '/usr/bin/git' instead. Travis CI's default Linux image includes 'pyenv', and all Python invocations that involve PATH lookup go through 'pyenv', e.g. our 'PYTHON_PATH=$(which python3)' sets '/opt/pyenv/shims/python3' as PYTHON_PATH, which in turn will invoke '/usr/bin/python3'. Alas, the 'pyenv' version included in this image is buggy, and prepends the directory containing the Python binary to PATH even if that is a system directory already in PATH near the end. Consequently, 'git p4' in those jobs ends up with its PATH starting with '/usr/bin', and then runs '/usr/bin/git'. So use the absolute paths '/usr/bin/python{2,3}' explicitly when setting PYTHON_PATH in those Linux jobs to avoid the PATH lookup and thus the bogus 'pyenv' from interfering with our 'git p4' tests. Don't bother with special-casing Travis CI: while this issue doesn't affect the corresponding Linux jobs on GitHub Actions, both CI systems use Ubuntu LTS-based images, so we can safely rely on these Python paths. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-05-15Revert "ci: add a problem matcher for GitHub Actions"Libravatar Junio C Hamano1-5/+0
This reverts commit 676eb0c1ce0d380478eb16bdc5a3f2a7bc01c1d2; as we will be reverting the change to show these extra output tokens under bash, the pattern would not match anything. Helped-by: Carlo Marcelo Arenas Belón <carenas@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-10ci: add a problem matcher for GitHub ActionsLibravatar Johannes Schindelin1-0/+5
With this patch, test failures will be annotated with a helpful, clickable message in GitHub Actions. For details, see https://github.com/actions/toolkit/blob/master/docs/problem-matchers.md Note: we need to set `TEST_SHELL_PATH` to Bash so that the problem matcher is fed a file and line number for each test failure. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-07ci: fix the `jobname` of the `GETTEXT_POISON` jobLibravatar Johannes Schindelin1-1/+1
In 6cdccfce1e0f (i18n: make GETTEXT_POISON a runtime option, 2018-11-08), the `jobname` was adjusted to have the `GIT_TEST_` prefix, but that prefix makes no sense in this context. Co-authored-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-07ci/lib: set TERM environment variable if not existLibravatar Đoàn Trần Công Danh1-0/+3
GitHub Action doesn't set TERM environment variable, which is required by "tput". Fallback to dumb if it's not set. Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-07ci/lib: allow running in GitHub ActionsLibravatar Johannes Schindelin1-1/+19
For each CI system we support, we need a specific arm in that if/else construct in ci/lib.sh. Let's add one for GitHub Actions. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-07ci/lib: if CI type is unknown, show the environment variablesLibravatar Johannes Schindelin1-0/+1
This should help with adding new CI-specific if-else arms. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-07Merge branch 'dd/ci-musl-libc' into HEADLibravatar Junio C Hamano1-0/+8
* dd/ci-musl-libc: travis: build and test on Linux with musl libc and busybox ci/linux32: libify install-dependencies step ci: refactor docker runner script ci/linux32: parameterise command to switch arch ci/lib-docker: preserve required environment variables ci: make MAKEFLAGS available inside the Docker container in the Linux32 job
2020-04-06travis: build and test on Linux with musl libc and busyboxLibravatar Đoàn Trần Công Danh1-0/+5
Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-02ci: make MAKEFLAGS available inside the Docker container in the Linux32 jobLibravatar SZEDER Gábor1-0/+3
Once upon a time we ran 'make --jobs=2 ...' to build Git, its documentation, or to apply Coccinelle semantic patches. Then commit eaa62291ff (ci: inherit --jobs via MAKEFLAGS in run-build-and-tests, 2019-01-27) came along, and started using the MAKEFLAGS environment variable to centralize setting the number of parallel jobs in 'ci/libs.sh'. Alas, it forgot to update 'ci/run-linux32-docker.sh' to make MAKEFLAGS available inside the Docker container running the 32 bit Linux job, and, consequently, since then that job builds Git sequentially, and it ignores any Makefile knobs that we might set in MAKEFLAGS (though we don't set any for the 32 bit Linux job at the moment). So update the 'docker run' invocation in 'ci/run-linux32-docker.sh' to make MAKEFLAGS available inside the Docker container as well. Set CC=gcc for the 32 bit Linux job, because that's the compiler installed in the 32 bit Linux Docker image that we use (Travis CI nowadays sets CC=clang by default, but clang is not installed in this image). Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-03-10ci: use python3 in linux-gcc and osx-gcc and python2 elsewhereLibravatar SZEDER Gábor1-0/+6
Python2 reached end of life, and we have been preparing our Python scripts to work with Python3. 'git p4', the main in-tree user of Python, has just received a number of compatibility updates. Our other notable Python script 'contrib/svn-fe/svnrdump_sim.py' is only used in 't9020-remote-svn.sh', and is apparently already compatible with both Python2 and 3. Our CI jobs currently only use Python2. We want to make sure that these Python scripts do indeed work with Python3, and we also want to make sure that these scripts keep working with Python2 as well, for the sake of some older LTS/Enterprise setups. Therefore, pick two jobs and use Python3 there, while leaving other jobs to still stick to Python2 for now. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-06Merge branch 'sg/osx-force-gcc-9'Libravatar Junio C Hamano1-2/+1
TravisCI update. * sg/osx-force-gcc-9: ci: build Git with GCC 9 in the 'osx-gcc' build job
2019-11-29ci: build Git with GCC 9 in the 'osx-gcc' build jobLibravatar SZEDER Gábor1-2/+1
Our 'osx-gcc' build job on Travis CI relied on GCC 8 being installed (but not linked) in the image we use [1]. Alas, since the last update of this image a few days ago this is not the case anymore, and now it contains GCC 9 (installed and linked) instead of GCC 8. The results are failed 'osx-gcc' jobs, because they can't find the 'gcc-8' command [2]. Let's move on to use GCC 9, with hopefully better error reporting and improved -Wfoo flags and what not. On Travis CI this has the benefit that we can spare a few seconds while installing dependencies, because it already comes pre-installed, at least for now. The Azure Pipelines OSX image doesn't include GCC, so we have to install it ourselves anyway, and then we might as well install the newer version. In a vain attempt I tried to future-proof this a bit: - Install 'gcc@9' specifically, so we'll still get what we want even after GCC 10 comes out, and the "plain" 'gcc' package starts to refer to 'gcc@10'. - Run both 'brew install gcc@9' and 'brew link gcc@9'. If 'gcc@9' is already installed and linked, then both commands are noop and exit with success. But as we saw in the past, sometimes the image contains the expected GCC package installed but not linked, so maybe it will happen again in the future as well. In that case 'brew install' is still a noop, and instructs the user to run 'brew link' instead, so that's what we'll do. And if 'gcc@9' is not installed, then 'brew install' will install it, and the subsequent 'brew link' becomes a noop. An additional benefit of this patch is that from now on we won't unnecessarily install GCC and its dependencies in the 'osx-clang' jobs on Azure Pipelines. [1] 7d4733c501 (ci: fix GCC install in the Travis CI GCC OSX job, 2019-10-24) [2] https://travis-ci.org/git/git/jobs/615442297#L333 Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-23t5608-clone-2gb.sh: turn GIT_TEST_CLONE_2GB into a boolLibravatar SZEDER Gábor1-1/+1
The GIT_TEST_CLONE_2GB environment variable is only ever checked with 'test -z' in 't5608-clone-2gb.sh', so any non-empty value is interpreted as "yes, run these expensive tests", even 'GIT_TEST_CLONE_2GB=NoThanks'. Similar GIT_TEST_* environment variables have already been turned into bools in 3b072c577b (tests: replace test_tristate with "git env--helper", 2019-06-21), so let's turn GIT_TEST_CLONE_2GB into a bool as well, to follow suit. Our CI builds set GIT_TEST_CLONE_2GB=YesPlease, so adjust them accordingly, thus removing the last 'YesPlease' from our CI scripts. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-07Merge branch 'sg/travis-help-debug'Libravatar Junio C Hamano1-0/+5
Dev support update. * sg/travis-help-debug: travis-ci: do not skip successfully tested trees in debug mode
2019-09-28travis-ci: do not skip successfully tested trees in debug modeLibravatar SZEDER Gábor1-0/+5
Travis CI offers shell access to its virtual machine environment running the build jobs, called "debug mode" [1]. After restarting a build job in debug mode and logging in, the first thing I usually do is to install dependencies, i.e. run './ci/install-dependencies.sh'. This works just fine when I restarted a failed build job in debug mode. However, after restarting a successful build job in debug mode our CI scripts get all clever, and exit without doing anything useful, claiming that "This commit's tree has already been built and tested successfully" [2]. Our CI scripts are right, and we do want to skip building and testing already known good trees in "regular" CI builds. In debug mode, however, this is a nuisiance, because one has to delete the cache (or at least the 'good-trees' file in the cache) to proceed. Let's update our CI scripts, in particular the common 'ci/lib.sh', to not skip previously successfully built and tested trees in debug mode, so all those scripts will do what there were supposed to do even when a successful build job was restarted in debug mode. [1] https://docs.travis-ci.com/user/running-build-in-debug-mode/ [2] 9cc2c76f5e (travis-ci: record and skip successfully built trees, 2017-12-31) Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-09-06ci: restore running httpd testsLibravatar SZEDER Gábor1-1/+1
Once upon a time GIT_TEST_HTTPD was a tristate variable and we exported 'GIT_TEST_HTTPD=YesPlease' in our CI scripts to make sure that we run the httpd tests in the Linux Clang and GCC build jobs, or error out if they can't be run for any reason [1]. Then 3b072c577b (tests: replace test_tristate with "git env--helper", 2019-06-21) came along, turned GIT_TEST_HTTPD into a bool, but forgot to update our CI scripts accordingly. So, since GIT_TEST_HTTPD is set explicitly, but its value is not one of the standardized true values, our CI jobs have been simply skipping the httpd tests in the last couple of weeks. Set 'GIT_TEST_HTTPD=true' to restore running httpd tests in our CI jobs. [1] a1157b76eb (travis-ci: set GIT_TEST_HTTPD in 'ci/lib-travisci.sh', 2017-12-12) Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-07-25Merge branch 'ab/test-env'Libravatar Junio C Hamano1-1/+1
Many GIT_TEST_* environment variables control various aspects of how our tests are run, but a few followed "non-empty is true, empty or unset is false" while others followed the usual "there are a few ways to spell true, like yes, on, etc., and also ways to spell false, like no, off, etc." convention. * ab/test-env: env--helper: mark a file-local symbol as static tests: make GIT_TEST_FAIL_PREREQS a boolean tests: replace test_tristate with "git env--helper" tests README: re-flow a previously changed paragraph tests: make GIT_TEST_GETTEXT_POISON a boolean t6040 test: stop using global "script" variable config.c: refactor die_bad_number() to not call gettext() early env--helper: new undocumented builtin wrapping git_env_*() config tests: simplify include cycle test
2019-07-08ci/lib.sh: update a comment about installed P4 and Git-LFS versionsLibravatar SZEDER Gábor1-2/+4
A comment in 'ci/lib.sh' claims that the "OS X build installs the latest available versions" of P4 and Git-LFS, but since f2f47150 ("ci: don't update Homebrew", 2019-07-03) that's no longer the case, as it will install the versions which were recorded in the image's Homebrew database when the image was created. Update this comment accordingly. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Acked-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-06-21tests: make GIT_TEST_GETTEXT_POISON a booleanLibravatar Ævar Arnfjörð Bjarmason1-1/+1
Change the GIT_TEST_GETTEXT_POISON variable from being "non-empty?" to being a more standard boolean variable. Since it needed to be checked in both C code and shellscript (via test -n) it was one of the remaining shellscript-like variables. Now that we have "env--helper" we can change that. There's a couple of tricky edge cases that arise because we're using git_env_bool() early, and the config-reading "env--helper". If GIT_TEST_GETTEXT_POISON is set to an invalid value die_bad_number() will die, but to do so it would usually call gettext(). Let's detect the special case of GIT_TEST_GETTEXT_POISON and always emit that message in the C locale, lest we infinitely loop. As seen in the updated tests in t0017-env-helper.sh there's also a caveat related to "env--helper" needing to read the config for trace2 purposes. Since the C_LOCALE_OUTPUT prerequisite is lazy and relies on "env--helper" we could get invalid results if we failed to read the config (e.g. because we'd loop on includes) when combined with e.g. "test_i18ngrep" wanting to check with "env--helper" if GIT_TEST_GETTEXT_POISON was true or not. I'm crossing my fingers and hoping that a test similar to the one I removed in the earlier "config tests: simplify include cycle test" change in this series won't happen again, and testing for this explicitly in "env--helper"'s own tests. This change breaks existing uses of e.g. GIT_TEST_GETTEXT_POISON=YesPlease, which we've documented in po/README and other places. As noted in [1] we might want to consider also accepting "YesPlease" in "env--helper" as a special-case. But as the lack of uproar over 6cdccfce1e ("i18n: make GETTEXT_POISON a runtime option", 2018-11-08) demonstrates the audience for this option is a really narrow set of git developers, who shouldn't have much trouble modifying their test scripts, so I think it's better to deal with that minor headache now and make all the relevant GIT_TEST_* variables boolean in the same way than carry the "YesPlease" special-case forward. 1. https://public-inbox.org/git/xmqqtvckm3h8.fsf@gitster-ct.c.googlers.com/ Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-07ci: clear and mark MAKEFLAGS exported just onceLibravatar Junio C Hamano1-3/+6
Clearing it once upfront, and turning all the assignment into appending, would future-proof the code even more, to prevent mistakes the previous one fixed from happening again. Also, mark the variable exported just once at the beginning. There is no point in marking it exported repeatedly. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-07ci: make sure we build Git parallelLibravatar SZEDER Gábor1-1/+1
Commit 2c8921db2b (travis-ci: build with the right compiler, 2019-01-17) started to use MAKEFLAGS to specify which compiler to use to build Git. A bit later, and in a different topic branch commit eaa62291ff (ci: inherit --jobs via MAKEFLAGS in run-build-and-tests, 2019-01-27) started to use MAKEFLAGS as well. Unfortunately, there is a semantic conflict between these two commits: both of them set MAKEFLAGS, and since the line adding CC from 2c8921db2b comes later in 'ci/lib.sh', it overwrites the number of parallel jobs added in eaa62291ff. Consequently, since both commits have been merged all our CI jobs have been building Git, building its documentation, and applying semantic patches sequentially, making all build jobs a bit slower. Running the test suite is unaffected, because the number of test jobs comes from GIT_PROVE_OPTS. Append to MAKEFLAGS when setting the compiler to use, to ensure that the number of parallel jobs to use is preserved. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-06Merge branch 'js/vsts-ci'Libravatar Junio C Hamano1-0/+188
Prepare to run test suite on Azure Pipeline. * js/vsts-ci: (22 commits) test-date: drop unused parameter to getnanos() ci: parallelize testing on Windows ci: speed up Windows phase tests: optionally skip bin-wrappers/ t0061: workaround issues with --with-dashes and RUNTIME_PREFIX tests: add t/helper/ to the PATH with --with-dashes mingw: try to work around issues with the test cleanup tests: include detailed trace logs with --write-junit-xml upon failure tests: avoid calling Perl just to determine file sizes README: add a build badge (status of the Azure Pipelines build) mingw: be more generous when wrapping up the setitimer() emulation ci: use git-sdk-64-minimal build artifact ci: add a Windows job to the Azure Pipelines definition Add a build definition for Azure DevOps ci/lib.sh: add support for Azure Pipelines tests: optionally write results as JUnit-style .xml test-date: add a subcommand to measure times in shell scripts ci: use a junction on Windows instead of a symlink ci: inherit --jobs via MAKEFLAGS in run-build-and-tests ci/lib.sh: encapsulate Travis-specific things ...
2019-01-29ci: speed up Windows phaseLibravatar Johannes Schindelin1-0/+2
As Unix shell scripting comes at a hefty price on Windows, we have to see where we can save some time to run the test suite. Let's skip the chain linting and the bin-wrappers/ redirection on Windows; this seems to shave of anywhere between 10-30% from the overall runtime. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29ci/lib.sh: add support for Azure PipelinesLibravatar Johannes Schindelin1-0/+25
This patch introduces a conditional arm that defines some environment variables and a function that displays the URL given the job id (to identify previous runs for known-good trees). Because Azure Pipeline's macOS agents already have git-lfs and gettext installed, we can leave `BREW_INSTALL_PACKAGES` empty (unlike in Travis' case). Note: this patch does not introduce an Azure Pipelines definition yet; That is left for the next patch. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-28ci: inherit --jobs via MAKEFLAGS in run-build-and-testsLibravatar Johannes Schindelin1-0/+1
Let's not decide in the generic ci/ part how many jobs to run in parallel; different CI configurations would favor a different number of parallel jobs, and it is easy enough to hand that information down via the `MAKEFLAGS` variable. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-28ci/lib.sh: encapsulate Travis-specific thingsLibravatar Johannes Schindelin1-13/+32
The upcoming patches will allow building git.git via Azure Pipelines (i.e. Azure DevOps' Continuous Integration), where variable names and URLs look a bit different than in Travis CI. Also, the configurations of the available agents are different. For example, Travis' and Azure Pipelines' macOS agents are set up differently, so that on Travis, we have to install the git-lfs and gettext Homebrew packages, and on Azure Pipelines we do not need to. Likewise, Azure Pipelines' Ubuntu agents already have asciidoctor installed. Finally, on Azure Pipelines the natural way is not to base64-encode tar files of the trash directories of failed tests, but to publish build artifacts instead. Therefore, that code to log those base64-encoded tar files is guarded to be Travis-specific. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-28ci: rename the library of common functionsLibravatar Johannes Schindelin1-0/+132
The name is hard-coded to reflect that we use Travis CI for continuous testing. In the next commits, we will extend this to be able use Azure DevOps, too. So let's adjust the name to make it more generic. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>