summaryrefslogtreecommitdiff
AgeCommit message (Collapse)AuthorFilesLines
2021-02-17grep/pcre2: move definitions of pcre2_{malloc,free}Libravatar Ævar Arnfjörð Bjarmason1-24/+22
Move the definitions of the pcre2_{malloc,free} functions above the compile_pcre2_pattern() function they're used in. Before the preceding commit they used to be needed earlier, but now we can move them to be adjacent to the other PCREv2 functions. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-02-17grep/pcre2: move back to thread-only PCREv2 structuresLibravatar Ævar Arnfjörð Bjarmason3-28/+17
Change the setup of the "pcre2_general_context" to happen per-thread in compile_pcre2_pattern() instead of in grep_init(). This change brings it in line with how the rest of the pcre2_* members in the grep_pat structure are set up. As noted in the preceding commit the approach 513f2b0bbd4 (grep: make PCRE2 aware of custom allocator, 2019-10-16) took to allocate the pcre2_general_context seems to have been initially based on a misunderstanding of how PCREv2 memory allocation works. The approach of creating a global context in grep_init() is just added complexity for almost zero gain. On my system it's 24 bytes saved per-thread. For comparison PCREv2 will then go on to allocate at least a kilobyte for its own thread-local state. As noted in 6d423dd542f (grep: don't redundantly compile throwaway patterns under threading, 2017-05-25) the grep code is intentionally not trying to micro-optimize allocations by e.g. sharing some PCREv2 structures globally, while making others thread-local. So let's remove this special case and make all of them thread-local again for simplicity. With this change we could move the pcre2_{malloc,free} functions around to live closer to their current use. I'm not doing that here to keep this change small, that cleanup will be done in a follow-up commit. See also the discussion in 94da9193a6 (grep: add support for PCRE v2, 2017-06-01) about thread safety, and Johannes's comments[1] to the effect that we should be doing what this patch is doing. 1. https://lore.kernel.org/git/nycvar.QRO.7.76.6.1908052120302.46@tvgsbejvaqbjf.bet/ Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-02-17grep/pcre2: actually make pcre2 use custom allocatorLibravatar Ævar Arnfjörð Bjarmason1-2/+2
Continue work started in 513f2b0bbd4 (grep: make PCRE2 aware of custom allocator, 2019-10-16) and make PCREv2 use our pcre2_{malloc,free}(). functions for allocation. We'll now use it for all PCREv2 allocations. The reason 513f2b0bbd4 worked as a bugfix for the USE_NED_ALLOCATOR issue is because it targeted the allocation freed via free(), as opposed to by a pcre2_*free() function. I.e. the pcre2_maketables() and pcre2_maketables_free() pair. For most of the rest we continued allocating with stock malloc() inside PCREv2 itself, but didn't segfault because we'd use its corresponding free(). In a preceding commit of mine I changed the free() to pcre2_maketables_free() on versions of PCREv2 10.34 and newer. So as far as fixing the segfault goes we could revert 513f2b0bbd4. But then we wouldn't use the desired allocator, let's just use it instead. Before this patch we'd on e.g.: grep --threads=1 -iP æ.*var.*xyz Only use pcre2_{malloc,free}() for 2 malloc() calls and 2 corresponding free() calls. Now it's 12 calls to each. This can be observed with the GREP_PCRE2_DEBUG_MALLOC debug mode. Reading the history of how this bug got introduced it wasn't present in Johannes's original patch[1] to fix the issue. My reading of that thread is that the approach the follow-up patches to Johannes's original pursued were based on misunderstanding of how the PCREv2 API works. In particular this part of [2]: "most of the time (like when using UTF-8) the chartable (and therefore the global context) is not needed (even when using alternate allocators)" That's simply not how PCREv2 memory allocation works. It's easy to see how the misunderstanding came about. It's because (as noted above) the issue was noticed because of our use of free() in our own grep.c for freeing the memory allocated by pcre2_maketables(). Thus the misunderstanding that PCREv2's compile context is something only needed for pcre2_maketables(), and e.g. an aborted earlier attempt[3] to only set it up when we ourselves called pcre2_maketables(). That's not what PCREv2's compile context is. To quote PCREv2's documentation: "This context just contains pointers to (and data for) external memory management functions that are called from several places in the PCRE2 library." Thus the failed attempts to go down the route of only creating the general context in cases where we ourselves call pcre2_maketables(), before finally settling on the approach 513f2b0bbd4 took of always creating it, but then mostly not using it. Instead we should always create it, and then pass the general context to those functions that accept it, so that they'll consistently use our preferred memory allocation functions. 1. https://lore.kernel.org/git/3397e6797f872aedd18c6d795f4976e1c579514b.1565005867.git.gitgitgadget@gmail.com/ 2. https://lore.kernel.org/git/CAPUEsphMh_ZqcH3M7PXC9jHTfEdQN3mhTAK2JDkdvKBp53YBoA@mail.gmail.com/ 3. https://lore.kernel.org/git/20190806085014.47776-3-carenas@gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-02-17grep/pcre2: use pcre2_maketables_free() functionLibravatar Ævar Arnfjörð Bjarmason2-0/+7
Make use of the pcre2_maketables_free() function to free the memory allocated by pcre2_maketables(). At first sight it's strange that 10da030ab75 (grep: avoid leak of chartables in PCRE2, 2019-10-16) which added the free() call here doesn't make use of the pcre2_free() the author introduced in the preceding commit in 513f2b0bbd4 (grep: make PCRE2 aware of custom allocator, 2019-10-16). The reason is that at the time the function didn't exist. It was first introduced in PCREv2 version 10.34, released on 2019-11-21. Let's make use of it behind a macro. I don't think this matters for anything to do with custom allocators, but it makes our use of PCREv2 more discoverable. At some distant point in the future we'll be able to drop the version guard, as nobody will be running a version older than 10.34. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-02-17grep/pcre2: use compile-time PCREv2 version testLibravatar Ævar Arnfjörð Bjarmason2-14/+7
Replace a use of pcre2_config(PCRE2_CONFIG_VERSION, ...) which I added in 95ca1f987ed (grep/pcre2: better support invalid UTF-8 haystacks, 2021-01-24) with the same test done at compile-time. It might be cuter to do this at runtime since we don't have to do the "major >= 11 || (major >= 10 && ...)" test. But in the next commit we'll add another version comparison that absolutely needs to be done at compile-time, so we're better of being consistent across the board. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-02-17grep/pcre2: add GREP_PCRE2_DEBUG_MALLOC debug modeLibravatar Ævar Arnfjörð Bjarmason1-0/+10
Add optional printing of PCREv2 allocations to stderr for a developer who manually changes the GREP_PCRE2_DEBUG_MALLOC definition to "1". You need to manually change the definition in the source file similar to the DEBUG_MAILMAP, there's no Makefile knob for this. This will be referenced a subsequent commit, and is generally useful to manually see what's going on with PCREv2 allocations while working on that code. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-02-17grep/pcre2: prepare to add debugging to pcre2_malloc()Libravatar Ævar Arnfjörð Bjarmason1-1/+2
Change pcre2_malloc() in a way that'll make it easier for a debugging fprintf() to spew out the allocated pointer. This doesn't introduce any functional change, it just makes a subsequent commit's diff easier to read. Changes code added in 513f2b0bbd4 (grep: make PCRE2 aware of custom allocator, 2019-10-16). Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-02-17grep/pcre2: correct reference to grep_init() in commentLibravatar Ævar Arnfjörð Bjarmason1-1/+1
Correct a comment added in 513f2b0bbd4 (grep: make PCRE2 aware of custom allocator, 2019-10-16). This comment was never correct in git.git, but was consistent with an older version of the patch[1]. 1. https://lore.kernel.org/git/20190806163658.66932-3-carenas@gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-02-17grep/pcre2: drop needless assignment to NULLLibravatar Ævar Arnfjörð Bjarmason1-2/+0
Remove a redundant assignment of pcre2_compile_context dating back to my 94da9193a6e (grep: add support for PCRE v2, 2017-06-01). In create_grep_pat() we xcalloc() the "grep_pat" struct, so there's no need to NULL out individual members here. I think this was probably something left over from an earlier development version of mine. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-02-17grep/pcre2: drop needless assignment + assert() on opt->pcre2Libravatar Ævar Arnfjörð Bjarmason1-3/+0
Drop an assignment added in b65abcafc7a (grep: use PCRE v2 for optimized fixed-string search, 2019-07-01) and the overly cautious assert() I added in 94da9193a6e (grep: add support for PCRE v2, 2017-06-01). There was never a good reason for this, it's just a relic from when I initially wrote the PCREv2 support. We're not going to have confusion about compile_pcre2_pattern() being called when it shouldn't just because we forgot to cargo-cult this opt->pcre2 option. Furthermore the "struct grep_opt" is (mostly) used for the options the user supplied, let's avoid the pattern of needlessly assigning to it. With my recent removal of the PCREv1 backend in 7599730b7e2 (Remove support for v1 of the PCRE library, 2021-01-24) there's even less confusion around what we call where in these codepaths, which is one more reason to remove this. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-02-12The eighth batchLibravatar Junio C Hamano1-0/+21
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-02-12Merge branch 'tb/precompose-prefix-too'Libravatar Junio C Hamano11-29/+59
When commands are started from a subdirectory, they may have to compare the path to the subdirectory (called prefix and found out from $(pwd)) with the tracked paths. On macOS, $(pwd) and readdir() yield decomposed path, while the tracked paths are usually normalized to the precomposed form, causing mismatch. This has been fixed by taking the same approach used to normalize the command line arguments. * tb/precompose-prefix-too: MacOS: precompose_argv_prefix()
2021-02-12Merge branch 'jk/complete-branch-force-delete'Libravatar Junio C Hamano2-4/+6
The command line completion (in contrib/) completed "git branch -d" with branch names, but "git branch -D" offered tagnames in addition, which has been corrected. "git branch -M" had the same problem. * jk/complete-branch-force-delete: doc/git-branch: fix awkward wording for "-c" completion: handle other variants of "branch -m" completion: treat "branch -D" the same way as "branch -d"
2021-02-12Merge branch 'jv/upload-pack-filter-spec-quotefix'Libravatar Junio C Hamano2-8/+11
Fix in passing custom args from "git clone" to "upload-pack" on the other side. * jv/upload-pack-filter-spec-quotefix: t5544: clarify 'hook works with partial clone' test upload-pack.c: fix filter spec quoting bug
2021-02-12Merge branch 'tb/pack-revindex-on-disk'Libravatar Junio C Hamano22-42/+545
Introduce an on-disk file to record revindex for packdata, which traditionally was always created on the fly and only in-core. * tb/pack-revindex-on-disk: t5325: check both on-disk and in-memory reverse index pack-revindex: ensure that on-disk reverse indexes are given precedence t: support GIT_TEST_WRITE_REV_INDEX t: prepare for GIT_TEST_WRITE_REV_INDEX Documentation/config/pack.txt: advertise 'pack.writeReverseIndex' builtin/pack-objects.c: respect 'pack.writeReverseIndex' builtin/index-pack.c: write reverse indexes builtin/index-pack.c: allow stripping arbitrary extensions pack-write.c: prepare to write 'pack-*.rev' files packfile: prepare for the existence of '*.rev' files
2021-02-12Merge branch 'ab/tests-various-fixup'Libravatar Junio C Hamano5-67/+80
Various test updates. * ab/tests-various-fixup: rm tests: actually test for SIGPIPE in SIGPIPE test archive tests: use a cheaper "zipinfo -h" invocation to get header upload-pack tests: avoid a non-zero "grep" exit status git-svn tests: rewrite brittle tests to use "--[no-]merges". git svn mergeinfo tests: refactor "test -z" to use test_must_be_empty git svn mergeinfo tests: modernize redirection & quoting style cache-tree tests: explicitly test HEAD and index differences cache-tree tests: use a sub-shell with less indirection cache-tree tests: remove unused $2 parameter cache-tree tests: refactor for modern test style
2021-02-11Sync with maintLibravatar Junio C Hamano0-0/+0
2021-02-11Merge branch 'en/merge-ort-perf'Libravatar Junio C Hamano2-1/+94
The "ort" merge strategy. * en/merge-ort-perf: merge-ort: begin performance work; instrument with trace2_region_* calls merge-ort: ignore the directory rename split conflict for now merge-ort: fix massive leak
2021-02-11Merge branch 'en/ort-directory-rename'Libravatar Junio C Hamano1-19/+811
ORT merge strategy learns to infer "renamed directory" while merging. * en/ort-directory-rename: merge-ort: fix a directory rename detection bug merge-ort: process_renames() now needs more defensiveness merge-ort: implement apply_directory_rename_modifications() merge-ort: add a new toplevel_dir field merge-ort: implement handle_path_level_conflicts() merge-ort: implement check_for_directory_rename() merge-ort: implement apply_dir_rename() and check_dir_renamed() merge-ort: implement compute_collisions() merge-ort: modify collect_renames() for directory rename handling merge-ort: implement handle_directory_level_conflicts() merge-ort: implement compute_rename_counts() merge-ort: copy get_renamed_dir_portion() from merge-recursive.c merge-ort: add outline of get_provisional_directory_renames() merge-ort: add outline for computing directory renames merge-ort: collect which directories are removed in dirs_removed merge-ort: initialize and free new directory rename data structures merge-ort: add new data structures for directory rename detection
2021-02-11Merge branch 'tb/ci-run-cocci-with-18.04' into maintLibravatar Junio C Hamano1-1/+1
* tb/ci-run-cocci-with-18.04: .github/workflows/main.yml: run static-analysis on bionic
2021-02-10Merge branch 'tb/ci-run-cocci-with-18.04'Libravatar Junio C Hamano1-1/+1
The version of Ubuntu Linux used by default at GitHub Actions CI has been updated to one that lack coccinelle; until it gets fixed, work it around by sticking to the previous release (18.04). * tb/ci-run-cocci-with-18.04: .github/workflows/main.yml: run static-analysis on bionic
2021-02-10The seventh batchLibravatar Junio C Hamano1-34/+25
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-02-10Merge branch 'ab/detox-gettext-tests'Libravatar Junio C Hamano22-173/+29
Get rid of "GETTEXT_POISON" support altogether, which may or may not be controversial. * ab/detox-gettext-tests: tests: remove uses of GIT_TEST_GETTEXT_POISON=false tests: remove support for GIT_TEST_GETTEXT_POISON ci: remove GETTEXT_POISON jobs
2021-02-10Merge branch 'ab/grep-pcre-invalid-utf8'Libravatar Junio C Hamano7-8/+82
Update support for invalid UTF-8 in PCRE2. * ab/grep-pcre-invalid-utf8: grep/pcre2: better support invalid UTF-8 haystacks grep/pcre2 tests: don't rely on invalid UTF-8 data test
2021-02-10Merge branch 'ab/retire-pcre1'Libravatar Junio C Hamano8-216/+18
The support for deprecated PCRE1 library has been dropped. * ab/retire-pcre1: Remove support for v1 of the PCRE library config.mak.uname: remove redundant NO_LIBPCRE1_JIT flag
2021-02-10Merge branch 'jk/pretty-lazy-load-commit'Libravatar Junio C Hamano2-12/+13
Some pretty-format specifiers do not need the data in commit object (e.g. "%H"), but we were over-eager to load and parse it, which has been made even lazier. * jk/pretty-lazy-load-commit: pretty: lazy-load commit data when expanding user-format
2021-02-10Merge branch 'ds/more-index-cleanups'Libravatar Junio C Hamano15-53/+408
Cleaning various codepaths up. * ds/more-index-cleanups: t1092: test interesting sparse-checkout scenarios test-lib: test_region looks for trace2 regions sparse-checkout: load sparse-checkout patterns name-hash: use trace2 regions for init repository: add repo reference to index_state fsmonitor: de-duplicate BUG()s around dirty bits cache-tree: extract subtree_pos() cache-tree: simplify verify_cache() prototype cache-tree: clean up cache_tree_update()
2021-02-10Merge branch 'rs/worktree-list-verbose'Libravatar Junio C Hamano5-80/+314
`git worktree list` now annotates worktrees as prunable, shows locked and prunable attributes in --porcelain mode, and gained a --verbose option. * rs/worktree-list-verbose: worktree: teach `list` verbose mode worktree: teach `list` to annotate prunable worktree worktree: teach `list --porcelain` to annotate locked worktree t2402: ensure locked worktree is properly cleaned up worktree: teach worktree_lock_reason() to gently handle main worktree worktree: teach worktree to lazy-load "prunable" reason worktree: libify should_prune_worktree()
2021-02-10Merge branch 'js/rebase-i-commit-cleanup-fix'Libravatar Junio C Hamano2-1/+20
When "git rebase -i" processes "fixup" insn, there is no reason to clean up the commit log message, but we did the usual stripspace processing. This has been corrected. * js/rebase-i-commit-cleanup-fix: rebase -i: do leave commit message intact in fixup! chains
2021-02-10Merge branch 'jk/t0000-cleanups'Libravatar Junio C Hamano1-286/+284
Code clean-up. * jk/t0000-cleanups: t0000: consistently use single quotes for outer tests t0000: run cleaning test inside sub-test t0000: run prereq tests inside sub-test t0000: keep clean-up tests together
2021-02-10Merge branch 'sg/t7800-difftool-robustify'Libravatar Junio C Hamano1-19/+19
Test fix. * sg/t7800-difftool-robustify: t7800-difftool: don't accidentally match tmp dirs
2021-02-10Merge branch 'ab/lose-grep-debug'Libravatar Junio C Hamano4-107/+2
Lose the debugging aid that may have been useful in the past, but no longer is, in the "grep" codepaths. * ab/lose-grep-debug: grep/log: remove hidden --debug and --grep-debug options
2021-02-10Merge branch 'jk/use-oid-pos'Libravatar Junio C Hamano10-94/+87
Code clean-up to ensure our use of hashtables using object names as keys use the "struct object_id" objects, not the raw hash values. * jk/use-oid-pos: oid_pos(): access table through const pointers hash_pos(): convert to oid_pos() rerere: use strmap to store rerere directories rerere: tighten rr-cache dirname check rerere: check dirname format while iterating rr_cache directory commit_graft_pos(): take an oid instead of a bare hash
2021-02-08Sync with 2.30.1Libravatar Junio C Hamano1-0/+8
2021-02-08.github/workflows/main.yml: run static-analysis on bionicLibravatar Taylor Blau1-1/+1
GitHub Actions is transitioning workflow steps that run on 'ubuntu-latest' from 18.04 to 20.04 [1]. This works fine in all steps except the static-analysis one, since Coccinelle isn't available on Ubuntu focal (it is only available in the universe suite). Until Coccinelle can be installed from 20.04's main suite, pin the static-analysis build to run on 18.04, where it can be installed by default. [1]: https://github.com/actions/virtual-environments/issues/1816 Reported-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-02-08Git 2.30.1Libravatar Junio C Hamano2-1/+9
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-02-08Merge branch 'pb/ci-matrix-wo-shortcut' into maintLibravatar Junio C Hamano1-0/+4
Our setting of GitHub CI test jobs were a bit too eager to give up once there is even one failure found. Tweak the knob to allow other jobs keep running even when we see a failure, so that we can find more failures in a single run. * pb/ci-matrix-wo-shortcut: ci: do not cancel all jobs of a matrix if one fails
2021-02-08Merge branch 'pb/blame-funcname-range-userdiff' into maintLibravatar Junio C Hamano1-4/+4
Test fix. * pb/blame-funcname-range-userdiff: annotate-tests: quote variable expansions containing path names
2021-02-08Merge branch 'jk/p5303-sed-portability-fix' into maintLibravatar Junio C Hamano1-4/+8
A perf script was made more portable. * jk/p5303-sed-portability-fix: p5303: avoid sed GNU-ism
2021-02-08Merge branch 'ab/branch-sort' into maintLibravatar Junio C Hamano8-44/+111
The implementation of "git branch --sort" wrt the detached HEAD display has always been hacky, which has been cleaned up. * ab/branch-sort: branch: show "HEAD detached" first under reverse sort branch: sort detached HEAD based on a flag ref-filter: move ref_sorting flags to a bitfield ref-filter: move "cmp_fn" assignment into "else if" arm ref-filter: add braces to if/else if/else chain branch tests: add to --sort tests branch: change "--local" to "--list" in comment
2021-02-08Merge branch 'ma/more-opaque-lock-file' into maintLibravatar Junio C Hamano5-15/+15
Code clean-up. * ma/more-opaque-lock-file: read-cache: try not to peek into `struct {lock_,temp}file` refs/files-backend: don't peek into `struct lock_file` midx: don't peek into `struct lock_file` commit-graph: don't peek into `struct lock_file` builtin/gc: don't peek into `struct lock_file`
2021-02-08Merge branch 'dl/p4-encode-after-kw-expansion' into maintLibravatar Junio C Hamano1-1/+1
Text encoding fix for "git p4". * dl/p4-encode-after-kw-expansion: git-p4: fix syncing file types with pattern
2021-02-08Merge branch 'ar/t6016-modernise' into maintLibravatar Junio C Hamano1-187/+167
Test update. * ar/t6016-modernise: t6016: move to lib-log-graph.sh framework
2021-02-08Merge branch 'zh/arg-help-format' into maintLibravatar Junio C Hamano8-64/+64
Clean up option descriptions in "git cmd --help". * zh/arg-help-format: builtin/*: update usage format parse-options: format argh like error messages
2021-02-08Merge branch 'ma/doc-pack-format-varint-for-sizes' into maintLibravatar Junio C Hamano1-1/+16
Doc update. * ma/doc-pack-format-varint-for-sizes: pack-format.txt: document sizes at start of delta data
2021-02-08Merge branch 'ma/t1300-cleanup' into maintLibravatar Junio C Hamano1-40/+32
Code clean-up. * ma/t1300-cleanup: t1300: don't needlessly work with `core.foo` configs t1300: remove duplicate test for `--file no-such-file` t1300: remove duplicate test for `--file ../foo`
2021-02-08Merge branch 'fc/t6030-bisect-reset-removes-auxiliary-files' into maintLibravatar Junio C Hamano1-8/+8
A 3-year old test that was not testing anything useful has been corrected. * fc/t6030-bisect-reset-removes-auxiliary-files: test: bisect-porcelain: fix location of files
2021-02-05Sync with maintLibravatar Junio C Hamano1-0/+47
2021-02-05The sixth batchLibravatar Junio C Hamano1-0/+40
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-02-05Merge branch 'sg/test-stress-jobs'Libravatar Junio C Hamano1-4/+4
Test framework fix. * sg/test-stress-jobs: test-lib: prevent '--stress-jobs=X' from being ignored