summaryrefslogtreecommitdiff
path: root/t
AgeCommit message (Collapse)AuthorFilesLines
2021-03-10merge-ort: precompute subset of sources for which we need rename detectionLibravatar Elijah Newren1-0/+71
rename detection works by trying to pair all file deletions (or "sources") with all file additions (or "destinations"), checking similarity, and then marking the sufficiently similar ones as renames. This can be expensive if there are many sources and destinations on a given side of history as it results in an N x M comparison matrix. However, there are many cases where we can compute in advance that detecting renames for some of the sources provides no useful information and thus that we can exclude those sources from the matrix. To see why, first note that the merge machinery uses detected renames in two ways: * directory rename detection: when one side of history renames a directory, and the other side of history adds new files to that directory, we want to be able to warn the user about the need to chose whether those new files stay in the old directory or move to the new one. * three-way content merging: in order to do three-way content merging of files, we need three different file versions. If one side of history renamed a file, then some of the content for the file is found under a different path than in the merge base or on the other side of history. Add a simple testcase showing the two kinds of reasons renames are relevant; it's a testcase that will only pass if we detect both kinds of needed renames. Other than the testcase added above, this commit concentrates just on the three-way content merging; it will punt and mark all sources as needed for directory rename detection, and leave it to future commits to narrow that down more. The point of three-way content merging is to reconcile changes made on *both* sides of history. What if the file wasn't modified on both sides? There are two possibilities: * If it wasn't modified on the renamed side: -> then we get to do exact rename detection, which is cheap. * If it wasn't modified on the unrenamed side: -> then detection of a rename for that source file is irrelevant That latter claim might be surprising at first, so let's walk through a case to show why rename detection for that source file is irrelevant. Let's use two filenames, old.c & new.c, with the following abbreviated object ids (and where the value '000000' is used to denote that the file is missing in that commit): old.c new.c MERGE_BASE: 01d01d 000000 MERGE_SIDE1: 01d01d 000000 MERGE_SIDE2: 000000 5e1ec7 If the rename *isn't* detected: then old.c looks like it was unmodified on one side and deleted on the other and should thus be removed. new.c looks like a new file we should keep as-is. If the rename *is* detected: then a three-way content merge is done. Since the version of the file in MERGE_BASE and MERGE_SIDE1 are identical, the three-way merge will produce exactly the version of the file whose abbreviated object id is 5e1ec7. It will record that file at the path new.c, while removing old.c from the directory. Note that these two results are identical -- a single file named 'new.c' with object id 5e1ec7. In other words, it doesn't matter if the rename is detected in the case where the file is unmodified on the unrenamed side. Use this information to compute whether we need rename detection for each source created in add_pair(). It's probably worth noting that there used to be a few other edge or corner cases besides three-way content merges and directory rename detection where lack of rename detection could have affected the result, but those cases actually highlighted where conflict resolution methods were not consistent with each other. Fixing those inconsistencies were thus critically important to enabling this optimization. That work involved the following: * bringing consistency to add/add, rename/add, and rename/rename conflict types, as done back in the topic merged at commit ac193e0e0a ("Merge branch 'en/merge-path-collision'", 2019-01-04), and further extended in commits 2a7c16c980 ("t6422, t6426: be more flexible for add/add conflicts involving renames", 2020-08-10) and e8eb99d4a6 ("t642[23]: be more flexible for add/add conflicts involving pair renames", 2020-08-10) * making rename/delete more consistent with modify/delete as done in commits 1f3c9ba707 ("t6425: be more flexible with rename/delete conflict messages", 2020-08-10) and 727c75b23f ("t6404, t6423: expect improved rename/delete handling in ort backend", 2020-10-26) Since the set of relevant_sources we compute has not yet been narrowed down for directory rename detection, we do not pass it to diffcore_rename_extended() yet. That will be done after subsequent commits narrow down the list of relevant_sources needed for directory rename detection reasons. Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-02-15diffcore-rename: guide inexact rename detection based on basenamesLibravatar Elijah Newren1-3/+4
Make use of the new find_basename_matches() function added in the last two patches, to find renames more rapidly in cases where we can match up files based on basenames. As a quick reminder (see the last two commit messages for more details), this means for example that docs/extensions.txt and docs/config/extensions.txt are considered likely renames if there are no remaining 'extensions.txt' files elsewhere among the added and deleted files, and if a similarity check confirms they are similar, then they are marked as a rename without looking for a better similarity match among other files. This is a behavioral change, as covered in more detail in the previous commit message. We do not use this heuristic together with either break or copy detection. The point of break detection is to say that filename similarity does not imply file content similarity, and we only want to know about file content similarity. The point of copy detection is to use more resources to check for additional similarities, while this is an optimization that uses far less resources but which might also result in finding slightly fewer similarities. So the idea behind this optimization goes against both of those features, and will be turned off for both. For the testcases mentioned in commit 557ac0350d ("merge-ort: begin performance work; instrument with trace2_region_* calls", 2020-10-28), this change improves the performance as follows: Before After no-renames: 13.815 s ± 0.062 s 13.294 s ± 0.103 s mega-renames: 1799.937 s ± 0.493 s 187.248 s ± 0.882 s just-one-mega: 51.289 s ± 0.019 s 5.557 s ± 0.017 s Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-02-15t4001: add a test comparing basename similarity and content similarityLibravatar Elijah Newren1-0/+23
Add a simple test where a removed file is similar to two different added files; one of them has the same basename, and the other has a slightly higher content similarity. In the current test, content similarity is weighted higher than filename similarity. Subsequent commits will add a new rule that weighs a mixture of filename similarity and content similarity in a manner that will change the outcome of this testcase. Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-02-10Merge branch 'ab/detox-gettext-tests'Libravatar Junio C Hamano12-101/+20
Get rid of "GETTEXT_POISON" support altogether, which may or may not be controversial. * ab/detox-gettext-tests: tests: remove uses of GIT_TEST_GETTEXT_POISON=false tests: remove support for GIT_TEST_GETTEXT_POISON ci: remove GETTEXT_POISON jobs
2021-02-10Merge branch 'ab/grep-pcre-invalid-utf8'Libravatar Junio C Hamano4-7/+60
Update support for invalid UTF-8 in PCRE2. * ab/grep-pcre-invalid-utf8: grep/pcre2: better support invalid UTF-8 haystacks grep/pcre2 tests: don't rely on invalid UTF-8 data test
2021-02-10Merge branch 'ab/retire-pcre1'Libravatar Junio C Hamano2-14/+1
The support for deprecated PCRE1 library has been dropped. * ab/retire-pcre1: Remove support for v1 of the PCRE library config.mak.uname: remove redundant NO_LIBPCRE1_JIT flag
2021-02-10Merge branch 'jk/pretty-lazy-load-commit'Libravatar Junio C Hamano1-1/+1
Some pretty-format specifiers do not need the data in commit object (e.g. "%H"), but we were over-eager to load and parse it, which has been made even lazier. * jk/pretty-lazy-load-commit: pretty: lazy-load commit data when expanding user-format
2021-02-10Merge branch 'ds/more-index-cleanups'Libravatar Junio C Hamano3-2/+344
Cleaning various codepaths up. * ds/more-index-cleanups: t1092: test interesting sparse-checkout scenarios test-lib: test_region looks for trace2 regions sparse-checkout: load sparse-checkout patterns name-hash: use trace2 regions for init repository: add repo reference to index_state fsmonitor: de-duplicate BUG()s around dirty bits cache-tree: extract subtree_pos() cache-tree: simplify verify_cache() prototype cache-tree: clean up cache_tree_update()
2021-02-10Merge branch 'rs/worktree-list-verbose'Libravatar Junio C Hamano1-0/+96
`git worktree list` now annotates worktrees as prunable, shows locked and prunable attributes in --porcelain mode, and gained a --verbose option. * rs/worktree-list-verbose: worktree: teach `list` verbose mode worktree: teach `list` to annotate prunable worktree worktree: teach `list --porcelain` to annotate locked worktree t2402: ensure locked worktree is properly cleaned up worktree: teach worktree_lock_reason() to gently handle main worktree worktree: teach worktree to lazy-load "prunable" reason worktree: libify should_prune_worktree()
2021-02-10Merge branch 'js/rebase-i-commit-cleanup-fix'Libravatar Junio C Hamano1-0/+8
When "git rebase -i" processes "fixup" insn, there is no reason to clean up the commit log message, but we did the usual stripspace processing. This has been corrected. * js/rebase-i-commit-cleanup-fix: rebase -i: do leave commit message intact in fixup! chains
2021-02-10Merge branch 'jk/t0000-cleanups'Libravatar Junio C Hamano1-286/+284
Code clean-up. * jk/t0000-cleanups: t0000: consistently use single quotes for outer tests t0000: run cleaning test inside sub-test t0000: run prereq tests inside sub-test t0000: keep clean-up tests together
2021-02-10Merge branch 'sg/t7800-difftool-robustify'Libravatar Junio C Hamano1-19/+19
Test fix. * sg/t7800-difftool-robustify: t7800-difftool: don't accidentally match tmp dirs
2021-02-05Merge branch 'sg/test-stress-jobs'Libravatar Junio C Hamano1-4/+4
Test framework fix. * sg/test-stress-jobs: test-lib: prevent '--stress-jobs=X' from being ignored
2021-02-05Merge branch 'pb/blame-funcname-range-userdiff'Libravatar Junio C Hamano1-4/+4
Test fix. * pb/blame-funcname-range-userdiff: annotate-tests: quote variable expansions containing path names
2021-02-05Merge branch 'jk/p5303-sed-portability-fix'Libravatar Junio C Hamano1-4/+8
A perf script was made more portable. * jk/p5303-sed-portability-fix: p5303: avoid sed GNU-ism
2021-02-05Merge branch 'zh/ls-files-deduplicate'Libravatar Junio C Hamano1-0/+66
"git ls-files" can and does show multiple entries when the index is unmerged, which is a source for confusion unless -s/-u option is in use. A new option --deduplicate has been introduced. * zh/ls-files-deduplicate: ls-files.c: add --deduplicate option ls_files.c: consolidate two for loops into one ls_files.c: bugfix for --deleted and --modified
2021-02-05Merge branch 'ds/cache-tree-basics'Libravatar Junio C Hamano1-1/+1
Document, clean-up and optimize the code around the cache-tree extension in the index. * ds/cache-tree-basics: cache-tree: speed up consecutive path comparisons cache-tree: use ce_namelen() instead of strlen() index-format: discuss recursion of cache-tree better index-format: update preamble to cache tree extension index-format: use 'cache tree' over 'cached tree' cache-tree: trace regions for prime_cache_tree cache-tree: trace regions for I/O cache-tree: use trace2 in cache_tree_update() unpack-trees: add trace2 regions tree-walk: report recursion counts
2021-02-05Merge branch 'so/log-diff-merge'Libravatar Junio C Hamano5-1/+603
"git log" learned a new "--diff-merges=<how>" option. * so/log-diff-merge: (32 commits) t4013: add tests for --diff-merges=first-parent doc/git-show: include --diff-merges description doc/rev-list-options: document --first-parent changes merges format doc/diff-generate-patch: mention new --diff-merges option doc/git-log: describe new --diff-merges options diff-merges: add '--diff-merges=1' as synonym for 'first-parent' diff-merges: add old mnemonic counterparts to --diff-merges diff-merges: let new options enable diff without -p diff-merges: do not imply -p for new options diff-merges: implement new values for --diff-merges diff-merges: make -m/-c/--cc explicitly mutually exclusive diff-merges: refactor opt settings into separate functions diff-merges: get rid of now empty diff_merges_init_revs() diff-merges: group diff-merge flags next to each other inside 'rev_info' diff-merges: split 'ignore_merges' field diff-merges: fix -m to properly override -c/--cc t4013: add tests for -m failing to override -c/--cc t4013: support test_expect_failure through ':failure' magic diff-merges: revise revs->diff flag handling diff-merges: handle imply -p on -c/--cc logic for log.c ...
2021-02-03Merge branch 'jk/peel-iterated-oid'Libravatar Junio C Hamano3-28/+3
The peel_ref() API has been replaced with peel_iterated_oid(). * jk/peel-iterated-oid: refs: switch peel_ref() to peel_iterated_oid()
2021-02-03Merge branch 'ds/maintenance-prefetch-cleanup'Libravatar Junio C Hamano1-1/+32
Test clean-up plus UI improvement by hiding extra refs that the prefetch task uses from "log --decorate" output. * ds/maintenance-prefetch-cleanup: t7900: clean up some broken refs maintenance: set log.excludeDecoration durin prefetch
2021-01-30annotate-tests: quote variable expansions containing path namesLibravatar Johannes Sixt1-4/+4
The test case added by 9466e3809d ("blame: enable funcname blaming with userdiff driver", 2020-11-01) forgot to quote variable expansions. This causes failures when the current directory contains blanks. One variable that the test case introduces will not have IFS characters and could remain without quotes, but let's quote all expansions for consistency, not just the one that has the path name. Signed-off-by: Johannes Sixt <j6t@kdbg.org> Acked-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-01-30worktree: teach `list` verbose modeLibravatar Rafael Silva1-0/+31
"git worktree list" annotates each worktree according to its state such as "prunable" or "locked", however it is not immediately obvious why these worktrees are being annotated. For prunable worktrees a reason is available that is returned by should_prune_worktree() and for locked worktrees a reason might be available provided by the user via `lock` command. Let's teach "git worktree list" a --verbose mode that outputs the reason why the worktrees are being annotated. The reason is a text that can take virtually any size and appending the text on the default columned format will make it difficult to extend the command with other annotations and not fit nicely on the screen. In order to address this shortcoming the annotation is then moved to the next line indented followed by the reason If the reason is not available the annotation stays on the same line as the worktree itself. The output of "git worktree list" with verbose becomes like so: $ git worktree list --verbose ... /path/to/locked-no-reason acb124 [branch-a] locked /path/to/locked-with-reason acc125 [branch-b] locked: worktree with a locked reason /path/to/prunable-reason ace127 [branch-d] prunable: gitdir file points to non-existent location ... Helped-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Rafael Silva <rafaeloliveira.cs@gmail.com> Reviewed-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-01-30worktree: teach `list` to annotate prunable worktreeLibravatar Rafael Silva1-0/+32
The "git worktree list" command shows the absolute path to the worktree, the commit that is checked out, the name of the branch, and a "locked" annotation if the worktree is locked, however, it does not indicate whether the worktree is prunable. The "prune" command will remove a worktree if it is prunable unless `--dry-run` option is specified. This could lead to a worktree being removed without the user realizing before it is too late, in case the user forgets to pass --dry-run for instance. If the "list" command shows which worktree is prunable, the user could verify before running "git worktree prune" and hopefully prevents the working tree to be removed "accidentally" on the worse case scenario. Let's teach "git worktree list" to show when a worktree is a prunable candidate for both default and porcelain format. In the default format a "prunable" text is appended: $ git worktree list /path/to/main aba123 [main] /path/to/linked 123abc [branch-a] /path/to/prunable ace127 (detached HEAD) prunable In the --porcelain format a prunable label is added followed by its reason: $ git worktree list --porcelain ... worktree /path/to/prunable HEAD abc1234abc1234abc1234abc1234abc1234abc12 detached prunable gitdir file points to non-existent location ... Helped-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Rafael Silva <rafaeloliveira.cs@gmail.com> Reviewed-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-01-30worktree: teach `list --porcelain` to annotate locked worktreeLibravatar Rafael Silva1-0/+32
Commit c57b3367be (worktree: teach `list` to annotate locked worktree, 2020-10-11) taught "git worktree list" to annotate locked worktrees by appending "locked" text to its output, however, this is not listed in the --porcelain format. Teach "list --porcelain" to do the same and add a "locked" attribute followed by its reason, thus making both default and porcelain format consistent. If the locked reason is not available then only "locked" is shown. The output of the "git worktree list --porcelain" becomes like so: $ git worktree list --porcelain ... worktree /path/to/locked HEAD 123abcdea123abcd123acbd123acbda123abcd12 detached locked worktree /path/to/locked-with-reason HEAD abc123abc123abc123abc123abc123abc123abc1 detached locked reason why it is locked ... In porcelain mode, if the lock reason contains special characters such as newlines, they are escaped with backslashes and the entire reason is enclosed in double quotes. For example: $ git worktree list --porcelain ... locked "worktree's path mounted in\nremovable device" ... Furthermore, let's update the documentation to state that some attributes in the porcelain format might be listed alone or together with its value depending whether the value is available or not. Thus documenting the case of the new "locked" attribute. Helped-by: Phillip Wood <phillip.wood@dunelm.org.uk> Helped-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Rafael Silva <rafaeloliveira.cs@gmail.com> Reviewed-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-01-30t2402: ensure locked worktree is properly cleaned upLibravatar Rafael Silva1-0/+1
c57b3367be (worktree: teach `list` to annotate locked worktree, 2020-10-11) introduced a new test to ensure locked worktrees are listed with "locked" annotation. However, the test does not clean up after itself as "git worktree prune" is not going to remove the locked worktree in the first place. This not only leaves the test in an unclean state it also potentially breaks following tests that rely on the "git worktree list" output. Let's fix that by unlocking the worktree before the "prune" command. Helped-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Rafael Silva <rafaeloliveira.cs@gmail.com> Reviewed-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-01-29p5303: avoid sed GNU-ismLibravatar Jeff King1-4/+8
Using "1~5" isn't portable. Nobody seems to have noticed, since perhaps people don't tend to run the perf suite on more exotic platforms. Still, it's better to set a good example. We can use: perl -ne 'print if $. % 5 == 1' instead. But we can further observe that perl does a good job of the other parts of this pipeline, and fold the whole thing together. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-01-28pretty: lazy-load commit data when expanding user-formatLibravatar Jeff King1-1/+1
When we expand a user-format, we try to avoid work that isn't necessary for the output. For instance, we don't bother parsing the commit header until we know we need the author, subject, etc. But we do always load the commit object's contents from disk, even if the format doesn't require it (e.g., just "%H"). Traditionally this didn't matter much, because we'd have loaded it as part of the traversal anyway, and we'd typically have those bytes attached to the commit struct (or these days, cached in a commit-slab). But when we have a commit-graph, we might easily get to the point of pretty-printing a commit without ever having looked at the actual object contents. We should push off that load (and reencoding) until we're certain that it's needed. I think the results of p4205 show the advantage pretty clearly (we serve parent and tree oids out of the commit struct itself, so they benefit as well): # using git.git as the test repo Test HEAD^ HEAD ---------------------------------------------------------------------- 4205.1: log with %H 0.40(0.39+0.01) 0.03(0.02+0.01) -92.5% 4205.2: log with %h 0.45(0.44+0.01) 0.09(0.09+0.00) -80.0% 4205.3: log with %T 0.40(0.39+0.00) 0.04(0.04+0.00) -90.0% 4205.4: log with %t 0.46(0.46+0.00) 0.09(0.08+0.01) -80.4% 4205.5: log with %P 0.39(0.39+0.00) 0.03(0.03+0.00) -92.3% 4205.6: log with %p 0.46(0.46+0.00) 0.10(0.09+0.00) -78.3% 4205.7: log with %h-%h-%h 0.52(0.51+0.01) 0.15(0.14+0.00) -71.2% 4205.8: log with %an-%ae-%s 0.42(0.41+0.00) 0.42(0.41+0.01) +0.0% # using linux.git as the test repo Test HEAD^ HEAD ---------------------------------------------------------------------- 4205.1: log with %H 7.12(6.97+0.14) 0.76(0.65+0.11) -89.3% 4205.2: log with %h 7.35(7.19+0.16) 1.30(1.19+0.11) -82.3% 4205.3: log with %T 7.58(7.42+0.15) 1.02(0.94+0.08) -86.5% 4205.4: log with %t 8.05(7.89+0.15) 1.55(1.41+0.13) -80.7% 4205.5: log with %P 7.12(7.01+0.10) 0.76(0.69+0.07) -89.3% 4205.6: log with %p 7.38(7.27+0.10) 1.32(1.20+0.12) -82.1% 4205.7: log with %h-%h-%h 7.81(7.67+0.13) 1.79(1.67+0.12) -77.1% 4205.8: log with %an-%ae-%s 7.90(7.74+0.15) 7.81(7.66+0.15) -1.1% I added the final test to show where we don't improve (the 1% there is just lucky noise), but also as a regression test to make sure we're not doing anything stupid like loading the commit multiple times when there are several placeholders that need it. Reported-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-01-28rebase -i: do leave commit message intact in fixup! chainsLibravatar Johannes Schindelin1-0/+8
In 6e98de72c03 (sequencer (rebase -i): add support for the 'fixup' and 'squash' commands, 2017-01-02), this developer introduced a change of behavior by mistake: when encountering a `fixup!` commit (or multiple `fixup!` commits) without any `squash!` commit thrown in, the final `git commit` was invoked with `--cleanup=strip`. Prior to that commit, the commit command had been called without that `--cleanup` option. Since we explicitly read the original commit message from a file in that case, there is really no sense in forcing that clean-up. We actually need to actively suppress that clean-up lest a configured `commit.cleanup` may interfere with what we want to do: leave the commit message unchanged. Reported-by: Vojtěch Knyttl <vojtech@knyt.tl> Helped-by: Martin Ågren <martin.agren@gmail.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-01-28t0000: consistently use single quotes for outer testsLibravatar Jeff King1-190/+190
When we use the sub-test helpers, we end up defining one shell snippet inside another shell snippet. So if we use single-quotes for the outer snippet, we have to use double-quotes within the inner snippet (it's included as here-doc within the outer snippet, but using a single quote would end the outer snippet early). Or vice versa we can use double quotes for the outer snippet, but then single quotes in the inner. We have some of each in the script, and neither is wrong. But it would be nice to be consistent unless there is a good reason not to. Using single quotes for the outer script is preferable, because it requires less metacharacter quoting overall. For example, in: test_expect_success 'outer' ' run_sub_test_lib_test ... <<-\EOF echo $foo && test_expect_success "inner" " echo \$bar " EOF ' we need only quote inside "inner", but not inside "outer" or the here-doc. Whereas if we flip them, we have to quote in both places: test_expect_success 'outer' " run_sub_test_lib_test ... <<-\EOF echo \$foo && test_expect_success 'inner' ' echo \$bar ' EOF " The exception is when we need a literal single-quote in an expected output here-doc. There we can either use outer double-quotes, or just use ${SQ} within the doc. I chose the latter for consistency (within this test, but also with other test scripts that face the same problem). There is one other interesting case, which is some tests that do: test_expect_success ... " do_something --run='"'!3'"' " This is rather confusing to read, but is correct. The outer script sees '!3' in single-quotes, as does the eval'd snippet. This is perhaps being overly cautious. In many interactive shells, an exclamation triggers history expansion even inside double quotes, but that is not generally true in non-interactive shells. There's some conflicting information here. Commit 784ce03d55 (t4216: avoid unnecessary subshell in test_bloom_filters_not_used, 2020-05-19) reports it as a problem with OpenBSD 6.7's /bin/sh. However, we have many instances in this script of prereqs like !LAZY_TRUE, which haven't been a problem. I left them un-escaped here to test out this theory. It's much nicer if we can not worry about this as a portability issue, so it's worth knowing. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-01-28t0000: run cleaning test inside sub-testLibravatar Jeff King1-8/+17
Our check of test_when_finished is done directly in the main script, and if we failed to clean, we complain and exit immediately. It's nicer to signal a test failure here, for a few reasons: - this gives better output to the user when run under a TAP harness like "prove" - constency; it's the only test left in the file that behaves this way - half of its "if" conditional is nonsense anyway; it picked up a reference to GIT_TEST_FAIL_PREREQS_INTERNAL in dfe1a17df9 (tests: add a special setup where prerequisites fail, 2019-05-13) along with its neighbors, even though it has nothing to do with that flag We could actually do this without a sub-test at all, and just put our two tests (one to do cleanup, and one to check that it happened) in the main script. But doing it in a subtest is conceptually cleaner (from the perspective of the main test script, we are checking only one thing), and it remains consistent with the "cleanup when failing" test directly after it, which has to happen in a sub-test (to avoid the main script complaining of the failed test). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-01-28t0000: run prereq tests inside sub-testLibravatar Jeff King1-80/+69
We test the behavior of prerequisites in t0000 by setting up fake ones in the main test script, trying to run some tests, and then seeing if those tests impacted the environment correctly. If they didn't, then we write a message and manually call exit. Instead, let's push these down into a sub-test, like many of the other tests covering the framework itself. This has a few advantages: - it does not pollute the test output with mention of skipped tests (that we know are uninteresting -- the point of the test was to see that these are skipped). - when running in a TAP harness, we get a useful test failure message (whereas when the script exits early, a tool like "prove" simply says "Dubious, test returned 1"). - we do not have to worry about different test environments, such as when GIT_TEST_FAIL_PREREQS_INTERNAL is set. Our sub-test helpers already give us a known environment. - the tests themselves are a bit easier to read, as we can just check the test-framework output to see what happened (and get the usual test_cmp diff if it failed) A few notes on the implementation: - we could do one sub-test per each individual test_expect_success. I broke it up here into a few logical groups, as I think this makes it more readable - the original tests modified environment variables inside the test bodies. Instead, I've used "true" as the body of a test we expect to run and "false" otherwise. Technically this does not confirm that the body of the "true" test actually ran. We are trusting the framework output to believe that it truly ran, which is sufficient for these tests. And I think the end result is much simpler to follow. - the nested_prereq test uses a few bare "test -f" calls; I converted these to our usual test_path_is_* helpers while moving the code around. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-01-28t0000: keep clean-up tests togetherLibravatar Jeff King1-11/+11
We check that test_when_finished cleans up after a test, and that it runs even after a failure. Those two were originally adjacent, but got split apart by the new test added in 477dcaddb6 (tests: do not let lazy prereqs inside `test_expect_*` turn off tracing, 2020-03-26), and then further by more lazy-prereq tests. Let's move them back together. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-01-26test-lib: prevent '--stress-jobs=X' from being ignoredLibravatar SZEDER Gábor1-4/+4
'./t1234-foo.sh --stress-jobs=X ...' is supposed to run that test script in X parallel jobs, but the number of jobs specified on the command line is entirely ignored if other '--stress'-related options follow. I.e. both './t1234-foo.sh --stress-jobs=X --stress-limit=Y' and './t1234-foo.sh --stress-jobs=X --stress' fall back to using twice the number of CPUs parallel jobs instead. The former has been broken since commit de69e6f6c9 (tests: let --stress-limit=<N> imply --stress, 2019-03-03) [1], which started to unconditionally overwrite the $stress variable holding the specified number of jobs in its effort to imply '--stress'. The latter has been broken since f545737144 (tests: introduce --stress-jobs=<N>, 2019-03-03), because it didn't consider that handling '--stress' will overwrite that variable as well. We could fix this by being more careful about (over)writing that $stress variable and checking first whether it has already been set. But I think it's cleaner to use a dedicated variable to hold the number of specified parallel jobs, so let's do that instead. [1] In de69e6f6c9 there was no '--stress-jobs=X' option yet, the number of parallel jobs had to be specified via '--stress=X', so, strictly speaking, de69e6f6c9 broke './t1234-foo.sh --stress=X --stress-limit=Y'. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-01-25Merge branch 'ab/mailmap-fixup'Libravatar Junio C Hamano2-14/+21
Follow-up fixes and improvements to ab/mailmap topic. * ab/mailmap-fixup: t4203: make blame output massaging more robust mailmap doc: use correct environment variable 'GIT_WORK_TREE' t4203: stop losing return codes of git commands test-lib-functions.sh: fix usage for test_commit()
2021-01-25Merge branch 'ps/config-env-pairs'Libravatar Junio C Hamano1-1/+222
Introduce two new ways to feed configuration variable-value pairs via environment variables, and tweak the way GIT_CONFIG_PARAMETERS encodes variable/value pairs to make it more robust. * ps/config-env-pairs: config: allow specifying config entries via envvar pairs environment: make `getenv_safe()` a public function config: store "git -c" variables using more robust format config: parse more robust format in GIT_CONFIG_PARAMETERS config: extract function to parse config pairs quote: make sq_dequote_step() a public function config: add new way to pass config via `--config-env` git: add `--super-prefix` to usage string
2021-01-25Merge branch 'jx/bundle'Libravatar Junio C Hamano4-24/+511
"git bundle" learns "--stdin" option to read its refs from the standard input. Also, it now does not lose refs whey they point at the same object. * jx/bundle: bundle: arguments can be read from stdin bundle: lost objects when removing duplicate pendings test: add helper functions for git-bundle
2021-01-25Merge branch 'ab/mailmap'Libravatar Junio C Hamano6-291/+615
Clean-up docs, codepaths and tests around mailmap. * ab/mailmap: (22 commits) shortlog: remove unused(?) "repo-abbrev" feature mailmap doc + tests: document and test for case-insensitivity mailmap tests: add tests for empty "<>" syntax mailmap tests: add tests for whitespace syntax mailmap tests: add a test for comment syntax mailmap doc + tests: add better examples & test them tests: refactor a few tests to use "test_commit --append" test-lib functions: add an --append option to test_commit test-lib functions: add --author support to test_commit test-lib functions: document arguments to test_commit test-lib functions: expand "test_commit" comment template mailmap: test for silent exiting on missing file/blob mailmap tests: get rid of overly complex blame fuzzing mailmap tests: add a test for "not a blob" error mailmap tests: remove redundant entry in test mailmap tests: improve --stdin tests mailmap tests: modernize syntax & test idioms mailmap tests: use our preferred whitespace syntax mailmap doc: start by mentioning the comment syntax check-mailmap doc: note config options ...
2021-01-25Merge branch 'ps/fetch-atomic'Libravatar Junio C Hamano1-0/+168
"git fetch" learns to treat ref updates atomically in all-or-none fashion, just like "git push" does, with the new "--atomic" option. * ps/fetch-atomic: fetch: implement support for atomic reference updates fetch: allow passing a transaction to `s_update_ref()` fetch: refactor `s_update_ref` to use common exit path fetch: use strbuf to format FETCH_HEAD updates fetch: extract writing to FETCH_HEAD
2021-01-25Merge branch 'jk/log-cherry-pick-duplicate-patches'Libravatar Junio C Hamano1-0/+12
When more than one commit with the same patch ID appears on one side, "git log --cherry-pick A...B" did not exclude them all when a commit with the same patch ID appears on the other side. Now it does. * jk/log-cherry-pick-duplicate-patches: patch-ids: handle duplicate hashmap entries
2021-01-25Merge branch 'js/default-branch-name-tests-final-stretch'Libravatar Junio C Hamano354-3436/+4401
Prepare tests not to be affected by the name of the default branch "git init" creates. * js/default-branch-name-tests-final-stretch: (28 commits) tests: drop prereq `PREPARE_FOR_MAIN_BRANCH` where no longer needed t99*: adjust the references to the default branch name "main" tests(git-p4): transition to the default branch name `main` t9[5-7]*: adjust the references to the default branch name "main" t9[0-4]*: adjust the references to the default branch name "main" t8*: adjust the references to the default branch name "main" t7[5-9]*: adjust the references to the default branch name "main" t7[0-4]*: adjust the references to the default branch name "main" t6[4-9]*: adjust the references to the default branch name "main" t64*: preemptively adjust alignment to prepare for `master` -> `main` t6[0-3]*: adjust the references to the default branch name "main" t5[6-9]*: adjust the references to the default branch name "main" t55[4-9]*: adjust the references to the default branch name "main" t55[23]*: adjust the references to the default branch name "main" t551*: adjust the references to the default branch name "main" t550*: adjust the references to the default branch name "main" t5503: prepare aligned comment for replacing `master` with `main` t5[0-4]*: adjust the references to the default branch name "main" t5323: prepare centered comment for `master` -> `main` t4*: adjust the references to the default branch name "main" ...
2021-01-25Merge branch 'dl/reflog-with-single-entry'Libravatar Junio C Hamano2-2/+15
After expiring a reflog and making a single commit, the reflog for the branch would record a single entry that knows both @{0} and @{1}, but we failed to answer "what commit were we on?", i.e. @{1} * dl/reflog-with-single-entry: refs: allow @{n} to work with n-sized reflog refs: factor out set_read_ref_cutoffs()
2021-01-25Merge branch 'sj/untracked-files-in-submodule-directory-is-not-dirty'Libravatar Junio C Hamano4-18/+32
"git diff" showed a submodule working tree with untracked cruft as "Submodule commit <objectname>-dirty", but a natural expectation is that the "-dirty" indicator would align with "git describe --dirty", which does not consider having untracked files in the working tree as source of dirtiness. The inconsistency has been fixed. * sj/untracked-files-in-submodule-directory-is-not-dirty: diff: do not show submodule with untracked files as "-dirty"
2021-01-25Merge branch 'jc/deprecate-pack-redundant'Libravatar Junio C Hamano1-15/+17
Warn loudly when the "pack-redundant" command, which has been left stale with almost unusable performance issues, gets used, as we no longer want to recommend its use (instead just "repack -d" instead). * jc/deprecate-pack-redundant: pack-redundant: gauge the usage before proposing its removal
2021-01-25Merge branch 'jk/forbid-lf-in-git-url'Libravatar Junio C Hamano2-0/+20
Newline characters in the host and path part of git:// URL are now forbidden. * jk/forbid-lf-in-git-url: fsck: reject .gitmodules git:// urls with newlines git_connect_git(): forbid newlines in host and path
2021-01-25Merge branch 'ab/branch-sort'Libravatar Junio C Hamano1-1/+50
The implementation of "git branch --sort" wrt the detached HEAD display has always been hacky, which has been cleaned up. * ab/branch-sort: branch: show "HEAD detached" first under reverse sort branch: sort detached HEAD based on a flag ref-filter: move ref_sorting flags to a bitfield ref-filter: move "cmp_fn" assignment into "else if" arm ref-filter: add braces to if/else if/else chain branch tests: add to --sort tests branch: change "--local" to "--list" in comment
2021-01-25Merge branch 'en/diffcore-rename'Libravatar Junio C Hamano1-2/+112
File-level rename detection updates. * en/diffcore-rename: diffcore-rename: remove unnecessary duplicate entry checks diffcore-rename: accelerate rename_dst setup diffcore-rename: simplify and accelerate register_rename_src() t4058: explore duplicate tree entry handling in a bit more detail t4058: add more tests and documentation for duplicate tree entry handling diffcore-rename: reduce jumpiness in progress counters diffcore-rename: simplify limit check diffcore-rename: avoid usage of global in too_many_rename_candidates() diffcore-rename: rename num_create to num_destinations
2021-01-25Merge branch 'ab/mktag'Libravatar Junio C Hamano3-54/+186
"git mktag" validates its input using its own rules before writing a tag object---it has been updated to share the logic with "git fsck". * ab/mktag: (23 commits) mktag: add a --[no-]strict option mktag: mark strings for translation mktag: convert to parse-options mktag: allow omitting the header/body \n separator mktag: allow turning off fsck.extraHeaderEntry fsck: make fsck_config() re-usable mktag: use fsck instead of custom verify_tag() mktag: use puts(str) instead of printf("%s\n", str) mktag: remove redundant braces in one-line body "if" mktag: use default strbuf_read() hint mktag tests: test verify_object() with replaced objects mktag tests: improve verify_object() test coverage mktag tests: test "hash-object" compatibility mktag tests: stress test whitespace handling mktag tests: run "fsck" after creating "mytag" mktag tests: don't create "mytag" twice mktag tests: don't redirect stderr to a file needlessly mktag tests: remove needless SHA-1 hardcoding mktag tests: use "test_commit" helper mktag tests: don't needlessly use a subshell ...
2021-01-24grep/pcre2: better support invalid UTF-8 haystacksLibravatar Ævar Arnfjörð Bjarmason4-1/+59
Improve the support for invalid UTF-8 haystacks given a non-ASCII needle when using the PCREv2 backend. This is a more complete fix for a bug I started to fix in 870eea8166 (grep: do not enter PCRE2_UTF mode on fixed matching, 2019-07-26), now that PCREv2 has the PCRE2_MATCH_INVALID_UTF mode we can make use of it. This fixes the sort of case described in 8a5999838e (grep: stess test PCRE v2 on invalid UTF-8 data, 2019-07-26), i.e.: - The subject string is non-ASCII (e.g. "ævar") - We're under a is_utf8_locale(), e.g. "en_US.UTF-8", not "C" - We are using --ignore-case, or we're a non-fixed pattern If those conditions were satisfied and we matched found non-valid UTF-8 data PCREv2 might bark on it, in practice this only happened under the JIT backend (turned on by default on most platforms). Ultimately this fixes a "regression" in b65abcafc7 ("grep: use PCRE v2 for optimized fixed-string search", 2019-07-01), I'm putting that in scare-quotes because before then we wouldn't properly support these complex case-folding, locale etc. cases either, it just broke in different ways. There was a bug related to this the PCRE2_NO_START_OPTIMIZE flag fixed in PCREv2 10.36. It can be worked around by setting the PCRE2_NO_START_OPTIMIZE flag. Let's do that in those cases, and add tests for the bug. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-01-24grep/pcre2 tests: don't rely on invalid UTF-8 data testLibravatar Ævar Arnfjörð Bjarmason1-6/+1
As noted in [1] when I originally added this test in [2] the test was completely broken as it lacked a redirect[3]. I now think this whole thing is overly fragile. Let's only test if we have a segfault here. Before this the first test's "test_cmp" was pretty meaningless. We were only testing if PCREv2 was so broken that it would spew out something completely unrelated on stdout, which isn't very plausible. In the second test we're relying on PCREv2 forever holding to the current behavior of the PCRE_UTF8 flag, as opposed to learning some optimistic graceful fallback to PCRE2_MATCH_INVALID_UTF in the future. If that happens having this test broken under bisecting would suck. A follow-up commit will actually test this case in a meaningful way under the PCRE2_MATCH_INVALID_UTF flag. Let's run this one unconditionally, and just make sure we don't segfault. 1. e714b898c6 (t7812: expect failure for grep -i with invalid UTF-8 data, 2019-11-29) 2. 8a5999838e (grep: stess test PCRE v2 on invalid UTF-8 data, 2019-07-26) 3. c74b3cbb83 (t7812: add missing redirects, 2019-11-26) Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-01-23Remove support for v1 of the PCRE libraryLibravatar Ævar Arnfjörð Bjarmason2-14/+1
Remove support for using version 1 of the PCRE library. Its use has been discouraged by upstream for a long time, and it's in a bugfix-only state. Anyone who was relying on v1 in particular got a nudge to move to v2 in e6c531b808 (Makefile: make USE_LIBPCRE=YesPlease mean v2, not v1, 2018-03-11), which was first released as part of v2.18.0. With this the LIBPCRE2 test prerequisites is redundant to PCRE. But I'm keeping it for self-documentation purposes, and to avoid conflict with other in-flight PCRE patches. I'm also not changing all of our own "pcre2" names to "pcre", i.e. the inverse of 6d4b5747f0 (grep: change internal *pcre* variable & function names to be *pcre1*, 2017-05-25). I don't see the point, and it makes the history/blame harder to read. Maybe if there's ever a PCRE v3... Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>