summaryrefslogtreecommitdiff
AgeCommit message (Collapse)AuthorFilesLines
2022-03-30Merge branch 'vd/stash-silence-reset'Libravatar Junio C Hamano11-30/+87
"git stash" does not allow subcommands it internally runs as its implementation detail, except for "git reset", to emit messages; now "git reset" part has also been squelched. * vd/stash-silence-reset: reset: show --no-refresh in the short-help reset: remove 'reset.refresh' config option reset: remove 'reset.quiet' config option reset: do not make '--quiet' disable index refresh stash: make internal resets quiet and refresh index reset: suppress '--no-refresh' advice if logging is silenced reset: replace '--quiet' with '--no-refresh' in performance advice reset: introduce --[no-]refresh option to --mixed reset: revise index refresh advice
2022-03-30Merge branch 'ab/racy-hooks'Libravatar Junio C Hamano1-0/+1
Regression fix. * ab/racy-hooks: hooks: fix "invoked hook" regression in a8cc5943338
2022-03-29The 16th batchLibravatar Junio C Hamano1-0/+7
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-29Merge branch 'jc/rebase-detach-fix'Libravatar Junio C Hamano2-7/+13
"git rebase $base $non_branch_commit", when $base is an ancestor or the $non_branch_commit, modified the current branch, which has been corrected. * jc/rebase-detach-fix: rebase: set REF_HEAD_DETACH in checkout_up_to_date() rebase: use test_commit helper in setup
2022-03-29Merge branch 'jt/reset-grafts-when-resetting-shallow'Libravatar Junio C Hamano5-0/+22
When "shallow" information is updated, we forgot to update the in-core equivalent, which has been corrected. * jt/reset-grafts-when-resetting-shallow: shallow: reset commit grafts when shallow is reset
2022-03-29Merge branch 'vd/cache-bottom-fix'Libravatar Junio C Hamano2-26/+20
Correct a bug in unpack-trees introduced earlier. * vd/cache-bottom-fix: Revert "unpack-trees: improve performance of next_cache_entry" unpack-trees: increment cache_bottom for sparse directories t1092: add sparse directory before cone in test repo
2022-03-29Merge branch 'ab/refs-various-fixes'Libravatar Junio C Hamano9-188/+135
Code clean-up. * ab/refs-various-fixes: refs debug: add a wrapper for "read_symbolic_ref" packed-backend: remove stub BUG(...) functions misc *.c: use designated initializers for struct assignments refs: use designated initializers for "struct ref_iterator_vtable" refs: use designated initializers for "struct ref_storage_be"
2022-03-25The 15th batchLibravatar Junio C Hamano1-0/+12
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-25Merge branch 'gc/recursive-fetch-with-unused-submodules'Libravatar Junio C Hamano6-312/+761
When "git fetch --recurse-submodules" grabbed submodule commits that would be needed to recursively check out newly fetched commits in the superproject, it only paid attention to submodules that are in the current checkout of the superproject. We now do so for all submodules that have been run "git submodule init" on. * gc/recursive-fetch-with-unused-submodules: submodule: fix latent check_has_commit() bug fetch: fetch unpopulated, changed submodules submodule: move logic into fetch_task_create() submodule: extract get_fetch_task() submodule: store new submodule commits oid_array in a struct submodule: inline submodule_commits() into caller submodule: make static functions read submodules from commits t5526: create superproject commits with test helper t5526: stop asserting on stderr literally t5526: introduce test helper to assert on fetches
2022-03-25Merge branch 'ps/fsync-refs'Libravatar Junio C Hamano5-3/+10
Updates to refs traditionally weren't fsync'ed, but we can configure using core.fsync variable to do so. * ps/fsync-refs: core.fsync: new option to harden references
2022-03-25Merge branch 'ns/core-fsyncmethod'Libravatar Junio C Hamano26-60/+444
Replace core.fsyncObjectFiles with two new configuration variables, core.fsync and core.fsyncMethod. * ns/core-fsyncmethod: core.fsync: documentation and user-friendly aggregate options core.fsync: new option to harden the index core.fsync: add configuration parsing core.fsync: introduce granular fsync control infrastructure core.fsyncmethod: add writeout-only mode wrapper: make inclusion of Windows csprng header tightly scoped
2022-03-24reset: show --no-refresh in the short-helpLibravatar Junio C Hamano1-3/+3
In the short help output from "git reset -h", the recently added "--[no-]refresh" option is shown like so: --refresh skip refreshing the index after reset which explains what happens when the option is given in the negative form, i.e. "--no-refresh". We could rephrase the explanation to read "refresh the index after reset (default)" to hint that the user can say "--no-refresh" to override the default, but listing the "--no-refresh" form in the list of options would be more helpful. Helped-by: Phillip Wood <phillip.wood@dunelm.org.uk> Acked-by: Victoria Dye <vdye@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-23reset: remove 'reset.refresh' config optionLibravatar Victoria Dye3-18/+4
Remove the 'reset.refresh' option, requiring that users explicitly specify '--no-refresh' if they want to skip refreshing the index. The 'reset.refresh' option was introduced in 101cee42dd (reset: introduce --[no-]refresh option to --mixed, 2022-03-11) as a replacement for the refresh-skipping behavior originally controlled by 'reset.quiet'. Although 'reset.refresh=false' functionally served the same purpose as 'reset.quiet=true', it exposed [1] the fact that the existence of a global "skip refresh" option could potentially cause problems for users. Allowing a global config option to avoid refreshing the index forces scripts using 'git reset --mixed' to defensively use '--refresh' if index refresh is expected; if that option is missing, behavior of a script could vary from user-to-user without explanation. Furthermore, globally disabling index refresh in 'reset --mixed' was initially devised as a passive performance improvement; since the introduction of the option, other changes have been made to Git (e.g., the sparse index) with a greater potential performance impact without sacrificing index correctness. Therefore, we can more aggressively err on the side of correctness and limit the cases of skipping index refresh to only when a user specifies the '--no-refresh' option. [1] https://lore.kernel.org/git/xmqqy2179o3c.fsf@gitster.g/ Signed-off-by: Victoria Dye <vdye@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-23reset: remove 'reset.quiet' config optionLibravatar Victoria Dye6-12/+2
Remove the 'reset.quiet' config option, remove '--no-quiet' documentation in 'Documentation/git-reset.txt'. In 4c3abd0551 (reset: add new reset.quiet config setting, 2018-10-23), 'reset.quiet' was introduced as a way to globally change the default behavior of 'git reset --mixed' to skip index refresh. However, now that '--quiet' does not affect index refresh, 'reset.quiet' would only serve to globally silence logging. This was not the original intention of the config setting, and there's no precedent for such a setting in other commands with a '--quiet' option, so it appears to be obsolete. In addition to the options & its documentation, remove 'reset.quiet' from the recommended config for 'scalar'. Signed-off-by: Victoria Dye <vdye@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-23reset: do not make '--quiet' disable index refreshLibravatar Victoria Dye3-39/+7
Update '--quiet' to no longer implicitly skip refreshing the index in a mixed reset. Users now have the ability to explicitly disable refreshing the index with the '--no-refresh' option, so they no longer need to use '--quiet' to do so. Moreover, we explicitly remove the refresh-skipping behavior from '--quiet' because it is completely unrelated to the stated purpose of the option: "Be quiet, only report errors." Helped-by: Phillip Wood <phillip.wood@dunelm.org.uk> Signed-off-by: Victoria Dye <vdye@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-23The 14th batchLibravatar Junio C Hamano1-0/+13
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-23Merge branch 'ab/plug-random-leaks'Libravatar Junio C Hamano2-4/+19
Double-free fix for a recently merged topic. * ab/plug-random-leaks: diff.c: fix a double-free regression in a18d66cefb tests: demonstrate "show --word-diff --color-moved" regression
2022-03-23Merge branch 'dc/complete-restore'Libravatar Junio C Hamano1-0/+4
The command line completion support (in contrib/) learns to give modified paths to the "git restore" command. * dc/complete-restore: completion: tab completion of filenames for 'git restore'
2022-03-23Merge branch 'jc/cat-file-batch-default-format-optim'Libravatar Junio C Hamano2-6/+35
Optimize away strbuf_expand() call with a hardcoded formatting logic specific for the default format in the --batch and --batch-check options of "git cat-file". * jc/cat-file-batch-default-format-optim: cat-file: skip expanding default format
2022-03-23Merge branch 'js/in-place-reverse-in-sequencer'Libravatar Junio C Hamano1-6/+4
Code clean-up. * js/in-place-reverse-in-sequencer: sequencer: use reverse_commit_list() helper
2022-03-23Merge branch 'ac/test-lazy-fetch'Libravatar Junio C Hamano1-0/+19
A new test to ensure a lazy fetching is not triggered when it should not be. * ac/test-lazy-fetch: partial-clone: add a partial-clone test case
2022-03-23Merge branch 'ps/repack-with-server-info'Libravatar Junio C Hamano3-4/+63
"git repack" learned a new configuration to disable triggering of age-old "update-server-info" command, which is rarely useful these days. * ps/repack-with-server-info: repack: add config to skip updating server info repack: refactor to avoid double-negation of update-server-info
2022-03-23Merge branch 'ds/doc-maintenance-synopsis-fix'Libravatar Junio C Hamano1-18/+20
Doc update. * ds/doc-maintenance-synopsis-fix: maintenance: fix synopsis in documentation
2022-03-23Merge branch 'ab/reflog-prep-fix'Libravatar Junio C Hamano2-0/+11
Regression fix. * ab/reflog-prep-fix: reflog: don't be noisy on empty reflogs
2022-03-23Merge branch 'ep/remove-duplicated-includes'Libravatar Junio C Hamano6-6/+0
Code clean-up. * ep/remove-duplicated-includes: attr.h: remove duplicate struct definition t/helper/test-run-command.c: delete duplicate include builtin/stash.c: delete duplicate include builtin/sparse-checkout.c: delete duplicate include builtin/gc.c: delete duplicate include attr.c: delete duplicate include
2022-03-23Merge branch 'ep/t6423-modernize'Libravatar Junio C Hamano1-5/+5
Code clean-up. * ep/t6423-modernize: t6423-merge-rename-directories.sh: use the $(...) construct
2022-03-23Merge branch 'jk/name-rev-w-genno'Libravatar Junio C Hamano2-14/+175
"git name-rev" learned to use the generation numbers when setting the lower bound of searching commits used to explain the revision, when available, instead of committer time. * jk/name-rev-w-genno: name-rev: use generation numbers if available
2022-03-23Merge branch 'jd/userdiff-kotlin'Libravatar Junio C Hamano15-0/+167
A new built-in userdiff driver for kotlin. * jd/userdiff-kotlin: userdiff: add builtin diff driver for kotlin language.
2022-03-23Merge branch 'bc/block-sha1-without-gcc-asm-extension'Libravatar Junio C Hamano1-17/+0
Get rid of one use of __asm__() GCC extension that does not help us much these days, which has an added advantage of not having to worry about -pedantic complaining. * bc/block-sha1-without-gcc-asm-extension: block-sha1: remove use of obsolete x86 assembly
2022-03-23Merge branch 'gc/submodule-update-part1'Libravatar Junio C Hamano4-159/+183
Rewrite of "git submodule update" in C (early part). * gc/submodule-update-part1: submodule--helper update-clone: check for --filter and --init submodule update: add tests for --filter submodule--helper: remove ensure-core-worktree submodule--helper update-clone: learn --init submodule--helper: allow setting superprefix for init_submodule() submodule--helper: refactor get_submodule_displaypath() submodule--helper run-update-procedure: learn --remote submodule--helper: don't use bitfield indirection for parse_options() submodule--helper: get remote names from any repository submodule--helper run-update-procedure: remove --suboid submodule--helper: reorganize code for sh to C conversion submodule--helper: remove update-module-mode submodule tests: test for init and update failure output
2022-03-23hooks: fix "invoked hook" regression in a8cc5943338Libravatar Ævar Arnfjörð Bjarmason1-0/+1
Fix a regression in a8cc5943338 (hooks: fix an obscure TOCTOU "did we just run a hook?" race, 2022-03-07): The "invoked_hook" variable passed to run_commit_hook() wasn't passed forward to run_hooks_opt(), as push_to_checkout() in that commit correctly did. Whether we ran the code contingent on having run the hook or not was thus undefined, but in practice on most (all?) modern platforms we'd have run it (almost?) all the time, since stack variables will get initialized to some random value, which most of the time isn't "0". This bug was revealed by running e.g. "t5537-fetch-shallow.sh" with the --valgrind option. Unfortunately running the whole test suite with --valgrind is really slow, so we didn't have a CI job that spotted this. The --valgrind output was: ==31275== Conditional jump or move depends on uninitialised value(s) ==31275== at 0x43C63F: prepare_to_commit (commit.c:1058) ==31275== by 0x4396A5: cmd_commit (commit.c:1722) ==31275== by 0x407C8A: run_builtin (git.c:465) ==31275== by 0x406741: handle_builtin (git.c:718) ==31275== by 0x407665: run_argv (git.c:785) ==31275== by 0x406500: cmd_main (git.c:916) ==31275== by 0x510839: main (common-main.c:56) ==31275== Uninitialised value was created by a stack allocation ==31275== at 0x43B344: prepare_to_commit (commit.c:719) Reported-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-21The thirteenth batchLibravatar Junio C Hamano1-0/+15
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-21Merge branch 'jy/gitweb-no-need-for-meta'Libravatar Junio C Hamano2-4/+15
Remove unneeded <meta http-equiv=content-type...> from gitweb output. * jy/gitweb-no-need-for-meta: gitweb: remove invalid http-equiv="content-type" comment: fix typo
2022-03-21Merge branch 'pw/single-key-interactive'Libravatar Junio C Hamano3-6/+33
The single-key interactive operation used by "git add -p" has been made more robust. * pw/single-key-interactive: add -p: disable stdin buffering when interactive.singlekey is set terminal: set VMIN and VTIME in non-canonical mode terminal: pop signal handler when terminal is restored terminal: always reset terminal when reading without echo
2022-03-21Merge branch 'ds/partial-bundles'Libravatar Junio C Hamano20-133/+317
Bundle file format gets extended to allow a partial bundle, filtered by similar criteria you would give when making a partial/lazy clone. * ds/partial-bundles: clone: fail gracefully when cloning filtered bundle bundle: unbundle promisor packs bundle: create filtered bundles rev-list: move --filter parsing into revision.c bundle: parse filter capability list-objects: handle NULL function pointers MyFirstObjectWalk: update recommended usage list-objects: consolidate traverse_commit_list[_filtered] pack-bitmap: drop filter in prepare_bitmap_walk() pack-objects: use rev.filter when possible revision: put object filter into struct rev_info list-objects-filter-options: create copy helper index-pack: document and test the --promisor option
2022-03-21Merge branch 'ep/test-malloc-check-with-glibc-2.34'Libravatar Junio C Hamano1-0/+18
The method to trigger malloc check used in our tests no longer work with newer versions of glibc. * ep/test-malloc-check-with-glibc-2.34: test-lib: declare local variables as local test-lib.sh: Use GLIBC_TUNABLES instead of MALLOC_CHECK_ on glibc >= 2.34
2022-03-21Merge branch 'sm/no-git-in-upstream-of-pipe-in-tests'Libravatar Junio C Hamano7-45/+72
Test fixes. * sm/no-git-in-upstream-of-pipe-in-tests: t0030-t0050: avoid pipes with Git on LHS t0001-t0028: avoid pipes with Git on LHS t0003: avoid pipes with Git on LHS
2022-03-18rebase: set REF_HEAD_DETACH in checkout_up_to_date()Libravatar John Cai2-0/+11
"git rebase A B" where B is not a commit should behave as if the HEAD got detached at B and then the detached HEAD got rebased on top of A. A bug however overwrites the current branch to point at B, when B is a descendant of A (i.e. the rebase ends up being a fast-forward). See [1] for the original bug report. The callstack from checkout_up_to_date() is the following: cmd_rebase() -> checkout_up_to_date() -> reset_head() -> update_refs() -> update_ref() When B is not a valid branch but an oid, rebase sets the head_name of rebase_options to NULL. This value gets passed down this call chain through the branch member of reset_head_opts also getting set to NULL all the way to update_refs(). Then update_refs() checks ropts.branch to decide whether or not to switch branches. If ropts.branch is NULL, it calls update_ref() to update HEAD. At this point however, from rebase's point of view, we want a detached HEAD. But, since checkout_up_to_date() does not set the RESET_HEAD_DETACH flag, the update_ref() call will deference HEAD and update the branch its pointing to. We want the HEAD detached at B instead. Fix this bug by adding the RESET_HEAD_DETACH flag in checkout_up_to_date if B is not a valid branch, so that once reset_head() calls update_refs(), it calls update_ref() with REF_NO_DEREF which updates HEAD directly intead of deferencing it and updating the branch that HEAD points to. Also add a test to ensure the correct behavior. [1] https://lore.kernel.org/git/YiokTm3GxIZQQUow@newk/ Reported-by: Michael McClimon <michael@mcclimon.org> Signed-off-by: John Cai <johncai86@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-18rebase: use test_commit helper in setupLibravatar John Cai1-7/+2
To prepare for the next commit that will test rebase with oids instead of branch names, update the rebase setup test to add a couple of tags we can use. This uses the test_commit helper so we can replace some lines that add a commit manually. Setting logAllRefUpdates is not necessary because it's on by default for repositories with a working tree. Helped-by: Phillip Wood <phillip.wood123@gmail.com> Signed-off-by: John Cai <johncai86@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-17shallow: reset commit grafts when shallow is resetLibravatar Jonathan Tan5-0/+22
When reset_repository_shallow() is called, Git clears its cache of shallow information, so that if shallow information is re-requested, Git will read fresh data from disk instead of reusing its stale cached data. However, the cache of commit grafts is not likewise cleared, even though there are commit grafts created from shallow information. This means that if on-disk shallow information were to be updated and then a commit-graft-using codepath were run (for example, a revision walk), Git would be using stale commit graft information. This can be seen from the test in this patch, in which Git performs a revision walk (to check for changed submodules) after a fetch with --update-shallow. Therefore, clear the cache of commit grafts whenever reset_repository_shallow() is called. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-17refs debug: add a wrapper for "read_symbolic_ref"Libravatar Ævar Arnfjörð Bjarmason2-13/+27
In cd475b3b038 (refs: add ability for backends to special-case reading of symbolic refs, 2022-03-01) when the "read_symbolic_ref" callback was added we'd fall back on "refs_read_raw_ref" if there wasn't any backend implementation of "read_symbolic_ref". As discussed in the preceding commit this would only happen if we were running the "debug" backend, e.g. in the "setup for ref completion" test in t9902-completion.sh with: GIT_TRACE_REFS=1 git fetch --no-tags other Let's improve the trace output, but and also eliminate the now-redundant refs_read_raw_ref() fallback case. As noted in the preceding commit the "packed" backend will never call refs_read_symbolic_ref() (nor is it ever going to). For any future backend such as reftable it's OK to ask that they either implement this (or a wrapper) themselves. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-17packed-backend: remove stub BUG(...) functionsLibravatar Ævar Arnfjörð Bjarmason1-79/+9
Remove the stub BUG(...) functions previously used by the "struct ref_storage_be refs_be_packed" backend. We never call any functions in the packed backend by using it as a "normal" primary ref store, instead we'll always initialize a "files" backend ref-store. It will then via the "packed_ref_store" member of "struct files_ref_store" call selected functions in the "packed" backend, and we'll in addition call others via wrappers in refs.c. So while these would arguably give us *slightly* more meaningful error messages we'll NULL the missing members in the initializer anyway, so we'll reliably get a segfault if we're ever changing the backend and having it call something it doesn't have. So there's no need for this verbose boilerplate, and as shown in a subsequent commit it might even lead to some confusion about the packed backend being a "real" backend. Let's make it clear that it's not. As an aside, this also fixes a warning emitted by SunCC in at least versions 12.5 and 12.6 of Oracle Developer Studio: "refs/packed-backend.c", line 1599: warning: Function has no return statement : packed_create_symref "refs/packed-backend.c", line 1606: warning: Function has no return statement : packed_rename_ref) "refs/packed-backend.c", line 1613: warning: Function has no return statement : packed_copy_ref "refs/packed-backend.c", line 1648: warning: Function has no return statement : packed_create_reflog Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-17misc *.c: use designated initializers for struct assignmentsLibravatar Ævar Arnfjörð Bjarmason3-5/+7
Change a few miscellaneous non-designated initializer assignments to use designated initializers. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-17refs: use designated initializers for "struct ref_iterator_vtable"Libravatar Ævar Arnfjörð Bjarmason5-23/+24
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-17refs: use designated initializers for "struct ref_storage_be"Libravatar Ævar Arnfjörð Bjarmason3-78/+78
Change the definition of the three refs backends we currently carry to use designated initializers. The "= NULL" assignments being retained here are redundant, and could be removed, but let's keep them for clarity. All of these backends define almost all fields, so we're not saving much in terms of line count by omitting these, but e.g. for "refs_be_debug" it's immediately apparent that we're omitting "init" when comparing its assignment to the others. This is a follow-up to similar work merged in bd4232fac33 (Merge branch 'ab/struct-init', 2021-07-16), a4b9fb6a5cf (Merge branch 'ab/designated-initializers-more', 2021-10-18) and a30321b9eae (Merge branch 'ab/designated-initializers' into ab/designated-initializers-more, 2021-09-27). Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-17Revert "unpack-trees: improve performance of next_cache_entry"Libravatar Victoria Dye1-17/+6
This reverts commit f2a454e0a5 (unpack-trees: improve performance of next_cache_entry, 2021-11-29). The "hint" value was originally needed to improve performance in 'git reset -- <pathspec>' caused by 'cache_bottom' lagging behind its correct value when using a sparse index. The 'cache_bottom' tracking has since been corrected, removing the need for an additional "pseudo-cache_bottom" tracking variable. Signed-off-by: Victoria Dye <vdye@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-17unpack-trees: increment cache_bottom for sparse directoriesLibravatar Victoria Dye2-11/+11
Correct tracking of the 'cache_bottom' for cases where sparse directories are present in the index. BACKGROUND ---------- The 'unpack_trees_options.cache_bottom' is a variable that tracks the in-progress "bottom" of the cache as 'unpack_trees()' iterates through the contents of the index. Most importantly, this value informs the sequential return values of 'next_cache_entry()' which, in the "diff cache" usage of 'unpack_callback()', are either unpacked as-is or are passed into the diff machinery. The 'cache_bottom' is intended to track the position of the first entry in the index that has not yet been diffed or unpacked. It is advanced in two main ways: either it is incremented when an index entry is marked as "used" (in 'mark_ce_used()'), indicating that it was unpacked or diffed, or when a directory is unpacked, in which case it is increased by an amount equaling the number of index entries inside that tree. In 17a1bb570b (unpack-trees: preserve cache_bottom, 2021-07-14), it was identified that sparse directories posed a problem to the above 'cache_bottom' advancement logic - because a sparse directory was both an index entry that could be "used" and a directory that can be unpacked, the 'cache_bottom' would be incremented too many times. To solve this problem, the 'mark_ce_used()' advancement of 'cache_bottom' was skipped for sparse directories. INCORRECT CACHE_BOTTOM TRACKING ------------------------------- Skipping the 'cache_bottom' advancement for sparse directories in 'mark_ce_used()' breaks down in two cases: 1. When the 'unpack_trees()' operation is *not* a "cache diff" (because the directory contents-based incrementing of 'cache_bottom' does not happen). 2. When a cache diff is performed with a pathspec (because 'unpack_index_entry()' will unpack a sparse directory not matched by the pathspec without performing the directory contents-based increment). The former luckily does not appear to affect 'git' behavior, likely because 'cache_bottom' is largely unused (non-"cache diff" 'unpack_trees()' uses 'find_index_entry()' - rather than 'next_cache_entry()' - to find the index entries to unpack). The latter, however, causes 'cache_bottom' to "lag behind" its intended position by an amount equal to the number of sparse directories unpacked so far with 'unpack_index_entry()'. If a repository is structured such that any sparse directories are ordered lexicographically *after* any pathspec-matching directories, though, this issue won't present any adverse behavior. This was the case with the 't1092-sparse-checkout-compatibility.sh' tests before the addition of the 'before/' sparse directory (ordered *before* the in-cone 'deep/' directory), therefore sidestepping the issue. Once the 'before/' directory was added, though, 'cache_bottom' began to lag behind its intended position, causing 'next_cache_entry()' to return index entries it had already processed and, ultimately, an incorrect diff. CORRECTING CACHE_BOTTOM ----------------------- The problems observed in 't1092' come from 'cache_bottom' lagging behind in cases where the cache tree-based advancement doesn't occur. To solve this, then, the fix in 17a1bb570b is "reversed"; rather than skipping 'cache_bottom' advancement in 'mark_ce_used()', we skip the directory contents-based advancement for sparse directories. Now, every index entry can be accounted for in 'cache_bottom': * if you're working with a single index entry, 'cache_bottom' is incremented in 'mark_ce_used()' * if you're working with a directory that contains index entries (but is not one itself), 'cache_bottom' is incremented by the number of entries in that directory. Finally, change the 'test_expect_failure' tests in 't1092' failing due to this bug back to 'test_expect_success'. Signed-off-by: Victoria Dye <vdye@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-17t1092: add sparse directory before cone in test repoLibravatar Victoria Dye1-4/+9
Add a sparse directory 'before/' containing files 'a' and 'b' to the test repo used in 't/t1092-sparse-checkout-compatibility.sh'. This is meant to ensure that no sparse index integrations rely on the in-cone path(s) being lexicographically first in the repo. Unfortunately, some existing tests do not handle this repo architecture properly: * 'add outside sparse cone' * 'status/add: outside sparse cone' * 'reset with pathspecs inside sparse definition' All three of these are due to the incorrect handling of the 'unpack_trees_options.cache_bottom' when performing a cache diff via 'unpack_trees'. This will be corrected in a future patch; in the meantime, mark the tests with 'test_expect_failure'. Finally, update the 'ls-files' and 'root directory cannot be sparse' tests to include the 'before/' directory in their expected index contents. Co-authored-by: Derrick Stolee <derrickstolee@github.com> Signed-off-by: Victoria Dye <vdye@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-17diff.c: fix a double-free regression in a18d66cefbLibravatar Ævar Arnfjörð Bjarmason2-4/+13
My a18d66cefb9 (diff.c: free "buf" in diff_words_flush(), 2022-03-04) has what it retrospect is a rather obvious bug (I don't know what I was thinking, if it all): We use the "emitted_symbols" allocation in append_emitted_diff_symbol() N times, but starting with a18d66cefb9 we'd free it after its first use! The correct way to free this data would have been to add the free() to the existing free_diff_words_data() function, so let's do that. The "ecbdata->diff_words->opt->emitted_symbols" might be NULL, so let's add a trivial free_emitted_diff_symbols() helper next to the function that appends to it. This fixes the "no effect on show from" leak tested for in the preceding commit. Perhaps confusingly this change will skip that test under SANITIZE=leak, but otherwise opt-in the "t4015-diff-whitespace.sh" test. The reason is that a18d66cefb9 "fixed" the leak in the preceding "no effect on diff" test, but for the first call to diff_words_flush() the "wol->buf" would be NULL, so we wouldn't double-free (and SANITIZE=address would see nothing amiss). With this change we'll still pass that test, showing that we've also fixed leaks on this codepath. We then have to skip the new "no effect on show" test because it happens to trip over an unrelated memory leak (in revision.c). The same goes for "move detection with submodules". Both of them pass with SANITIZE=address though, which would error on the "no effect on show" test before this change. Reported-by: Michael J Gruber <git@grubix.eu> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-17tests: demonstrate "show --word-diff --color-moved" regressionLibravatar Michael J Gruber1-1/+7
Add a failing test which demonstrates a regression in a18d66cefb ("diff.c: free "buf" in diff_words_flush()", 2022-03-04), the regression is discussed in detail in the subsequent commit. With it running `git show --word-diff --color-moved` with SANITIZE=address would emit: ==31191==ERROR: AddressSanitizer: attempting double-free on 0x617000021100 in thread T0: #0 0x49f0a2 in free (git+0x49f0a2) #1 0x9b0e4d in diff_words_flush diff.c:2153:3 #2 0x9aed5d in fn_out_consume diff.c:2354:3 #3 0xe092ab in consume_one xdiff-interface.c:43:9 #4 0xe072eb in xdiff_outf xdiff-interface.c:76:10 #5 0xec7014 in xdl_emit_diffrec xdiff/xutils.c:53:6 [...] 0x617000021100 is located 0 bytes inside of 768-byte region [0x617000021100,0x617000021400) freed by thread T0 here: #0 0x49f0a2 in free (git+0x49f0a2) [...(same stacktrace)...] previously allocated by thread T0 here: #0 0x49f603 in __interceptor_realloc (git+0x49f603) #1 0xde4da4 in xrealloc wrapper.c:126:8 #2 0x995dc5 in append_emitted_diff_symbol diff.c:794:2 #3 0x96c44a in emit_diff_symbol diff.c:1527:3 [...] This was not caught by the test suite because we test `diff --word-diff --color-moved` only so far. Therefore, add a test for `show`, too. Signed-off-by: Michael J Gruber <git@grubix.eu> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>