summaryrefslogtreecommitdiff
path: root/t
AgeCommit message (Collapse)AuthorFilesLines
2021-05-04fsck: warn about symlinked dotfiles we'll open with O_NOFOLLOWLibravatar Jeff King1-2/+27
In the commits merged in via 204333b015 (Merge branch 'jk/open-dotgitx-with-nofollow', 2021-03-22), we stopped following symbolic links for .gitattributes, .gitignore, and .mailmap files. Let's teach fsck to warn that these symlinks are not going to do anything. Note that this is just a warning, and won't block the objects via transfer.fsckObjects, since there are reported to be cases of this in the wild (and even once fixed, they will continue to exist in the commit history of those projects, but are not particularly dangerous). Note that we won't add these to the existing gitmodules block in the fsck code. The logic for gitmodules is a bit more complicated, as we also check the content of non-symlink instances we find. But for these new files, there is no content check; we're just looking at the name and mode of the tree entry (and we can avoid even the complicated name checks in the common case that the mode doesn't indicate a symlink). We can reuse the test helper function we defined for .gitmodules, though (it needs some slight adjustments for the fsck error code, and because we don't block these symlinks via verify_path()). Note that I didn't explicitly test the transfer.fsckObjects case here (nor does the existing .gitmodules test that it blocks a push). The translation of fsck severities to outcomes is covered in general in t5504. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-04t0060: test ntfs/hfs-obscured dotfilesLibravatar Jeff King2-13/+63
We have tests that cover various filesystem-specific spellings of ".gitmodules", because we need to reliably identify that path for some security checks. These are from dc2d9ba318 (is_{hfs,ntfs}_dotgitmodules: add tests, 2018-05-12), with the actual code coming from e7cb0b4455 (is_ntfs_dotgit: match other .git files, 2018-05-11) and 0fc333ba20 (is_hfs_dotgit: match other .git files, 2018-05-02). Those latter two commits also added similar matching functions for .gitattributes and .gitignore. These ended up not being used in the final series, and are currently dead code. But in preparation for them being used in some fsck checks, let's make sure they actually work by throwing a few basic tests at them. Likewise, let's cover .mailmap (which does need matching code added). I didn't bother with the whole battery of tests that we cover for .gitmodules. These functions are all based on the same generic matcher, so it's sufficient to test most of the corner cases just once. Note that the ntfs magic prefix names in the tests come from the algorithm described in e7cb0b4455 (and are different for each file). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-04t7450: test .gitmodules symlink matching against obscured namesLibravatar Jeff King1-38/+53
In t7450 we check that both verify_path() and fsck catch malformed .gitmodules entries in trees. However, we don't check that we catch filesystem-equivalent forms of these (e.g., ".GITMOD~1" on Windows). Our name-matching functions are exercised well in t0060, but there's nothing to test that we correctly call the matching functions from the actual fsck and verify_path() code. So instead of testing just .gitmodules, let's repeat our tests for a few basic cases. We don't need to be exhaustive here (t0060 handles that), but just make sure we hit one name of each type. Besides pushing the tests into a function that takes the path as a parameter, we'll need to do a few things: - adjust the directory name to accommodate the tests running multiple times - set core.protecthfs for index checks. Fsck always protects all types by default, but we want to be able to exercise the HFS routines on every system. Note that core.protectntfs is already the default these days, but it doesn't hurt to explicitly label our need for it. - we'll also take the filename ("gitmodules") as a parameter. All calls use the same name for now, but a future patch will extend this to handle other .gitfoo files. Note that our fake-content symlink destination is somewhat .gitmodules specific. But it isn't necessary for other files (which don't do a content check). And it happens to be a valid attribute and ignore file anyway. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-04t7450: test verify_path() handling of gitmodulesLibravatar Jeff King1-3/+16
Commit 10ecfa7649 (verify_path: disallow symlinks in .gitmodules, 2018-05-04) made it impossible to load a symlink .gitmodules file into the index. However, there are no tests of this behavior. Let's make sure this case is covered. We can easily reuse the test setup created by the matching b7b1fca175 (fsck: complain when .gitmodules is a symlink, 2018-05-04). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-03t7415: rename to expand scopeLibravatar Jeff King1-3/+10
This script has already expanded beyond its original intent of ".. in submodule names" to include other malicious submodule bits. Let's update the name and description to reflect that, as well as the fact that we'll soon be adding similar tests for other dotfiles (.gitattributes, etc). We'll also renumber it to move it out of the group of submodule-specific tests. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-03fsck_tree(): fix shadowed variableLibravatar Jeff King1-2/+3
Commit b2f2039c2b (fsck: accept an oid instead of a "struct tree" for fsck_tree(), 2019-10-18) introduced a new "oid" parameter to fsck_tree(), and we pass it to the report() function when we find problems. However, that is shadowed within the tree-walking loop by the existing "oid" variable which we use to store the oid of each tree entry. As a result, we may report the wrong oid for some problems we detect within the loop (the entry oid, instead of the tree oid). Our tests didn't catch this because they checked only that we found the expected fsck problem, not that it was attached to the correct object. Let's rename both variables in the function to avoid confusion. This makes the diff a little noisy (e.g., all of the report() calls outside the loop were already correct but need to be touched), but makes sure we catch all cases and will avoid similar confusion in the future. And we can update the test to be a bit more specific and catch this problem. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-03t7415: remove out-dated comment about translationLibravatar Jeff King1-3/+2
Since GETTEXT_POISON does not exist anymore, there is no point warning people about whether we should use test_i18ngrep. This is doubly confusing because the comment was describing why it was OK to use grep, but it got caught up in the mass conversion of 674ba34038 (fsck: mark strings for translation, 2018-11-10). Note there are other uses of test_i18ngrep in this script which are now obsolete; I'll save those for a mass-cleanup. My goal here was just to fix the confusing comment in code I'm about to refactor. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-04-30Merge branch 'so/log-diff-merge'Libravatar Junio C Hamano2-0/+34
"git log" learned "--diff-merges=<style>" option, with an associated configuration variable log.diffMerges. * so/log-diff-merge: doc/diff-options: document new --diff-merges features diff-merges: introduce log.diffMerges config variable diff-merges: adapt -m to enable default diff format diff-merges: refactor set_diff_merges() diff-merges: introduce --diff-merges=on
2021-04-30Merge branch 'ds/sparse-index-protections'Libravatar Junio C Hamano5-24/+302
Builds on top of the sparse-index infrastructure to mark operations that are not ready to mark with the sparse index, causing them to fall back on fully-populated index that they always have worked with. * ds/sparse-index-protections: (47 commits) name-hash: use expand_to_path() sparse-index: expand_to_path() name-hash: don't add directories to name_hash revision: ensure full index resolve-undo: ensure full index read-cache: ensure full index pathspec: ensure full index merge-recursive: ensure full index entry: ensure full index dir: ensure full index update-index: ensure full index stash: ensure full index rm: ensure full index merge-index: ensure full index ls-files: ensure full index grep: ensure full index fsck: ensure full index difftool: ensure full index commit: ensure full index checkout: ensure full index ...
2021-04-30Merge branch 'ds/maintenance-prefetch-fix'Libravatar Junio C Hamano2-8/+57
The prefetch task in "git maintenance" assumed that "git fetch" from any remote would fetch all its local branches, which would fetch too much if the user is interested in only a subset of branches there. * ds/maintenance-prefetch-fix: maintenance: respect remote.*.skipFetchAll maintenance: use 'git fetch --prefetch' fetch: add --prefetch option maintenance: simplify prefetch logic
2021-04-30Merge branch 'ow/push-quiet-set-upstream'Libravatar Junio C Hamano1-0/+7
"git push --quiet --set-upstream" was not quiet when setting the upstream branch configuration, which has been corrected. * ow/push-quiet-set-upstream: transport: respect verbosity when setting upstream
2021-04-30Merge branch 'jk/promisor-optim'Libravatar Junio C Hamano2-3/+15
Handling of "promisor packs" that allows certain objects to be missing and lazily retrievable has been optimized (a bit). * jk/promisor-optim: revision: avoid parsing with --exclude-promisor-objects lookup_unknown_object(): take a repository argument is_promisor_object(): free tree buffer after parsing
2021-04-20Merge branch 'ab/detox-gettext-tests'Libravatar Junio C Hamano3-13/+6
Test clean-up. * ab/detox-gettext-tests: tests: remove all uses of test_i18cmp
2021-04-20Merge branch 'jk/pack-objects-bitmap-progress-fix'Libravatar Junio C Hamano1-0/+23
When "git pack-objects" makes a literal copy of a part of existing packfile using the reachability bitmaps, its update to the progress meter was broken. * jk/pack-objects-bitmap-progress-fix: pack-objects: update "nr_seen" progress based on pack-reused count
2021-04-20Merge branch 'ab/userdiff-tests'Libravatar Junio C Hamano6-52/+87
A bit of code clean-up and a lot of test clean-up around userdiff area. * ab/userdiff-tests: blame tests: simplify userdiff driver test blame tests: don't rely on t/t4018/ directory userdiff: remove support for "broken" tests userdiff tests: list builtin drivers via test-tool userdiff tests: explicitly test "default" pattern userdiff: add and use for_each_userdiff_driver() userdiff style: normalize pascal regex declaration userdiff style: declare patterns with consistent style userdiff style: re-order drivers in alphabetical order
2021-04-20Merge branch 'ar/userdiff-scheme'Libravatar Junio C Hamano16-0/+90
Userdiff patterns for "Scheme" has been added. * ar/userdiff-scheme: userdiff: add support for Scheme
2021-04-16diff-merges: introduce log.diffMerges config variableLibravatar Sergey Organov2-0/+26
New log.diffMerges configuration variable sets the format that --diff-merges=on will be using. The default is "separate". t4013: add the following tests for log.diffMerges config: * Test that wrong values are denied. * Test that the value of log.diffMerges properly affects both --diff-merges=on and -m. t9902: fix completion tests for log.d* to match log.diffMerges. Added documentation for log.diffMerges. Signed-off-by: Sergey Organov <sorganov@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-04-16diff-merges: introduce --diff-merges=onLibravatar Sergey Organov1-0/+8
Introduce the notion of default diff format for merges, and the option "on" to select it. The default format is "separate" and can't yet be changed, so effectively "on" is just a synonym for "separate" for now. Add corresponding test to t4013. This is in preparation for introducing log.diffMerges configuration option that will let --diff-merges=on to be configured to any supported format. Signed-off-by: Sergey Organov <sorganov@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-04-16Merge branch 'en/ort-readiness'Libravatar Junio C Hamano8-10/+183
Plug the ort merge backend throughout the rest of the system, and start testing it as a replacement for the recursive backend. * en/ort-readiness: Add testing with merge-ort merge strategy t6423: mark remaining expected failure under merge-ort as such Revert "merge-ort: ignore the directory rename split conflict for now" merge-recursive: add a bunch of FIXME comments documenting known bugs merge-ort: write $GIT_DIR/AUTO_MERGE whenever we hit a conflict t: mark several submodule merging tests as fixed under merge-ort merge-ort: implement CE_SKIP_WORKTREE handling with conflicted entries t6428: new test for SKIP_WORKTREE handling and conflicts merge-ort: support subtree shifting merge-ort: let renormalization change modify/delete into clean delete merge-ort: have ll_merge() use a special attr_index for renormalization merge-ort: add a special minimal index just for renormalization merge-ort: use STABLE_QSORT instead of QSORT where required
2021-04-16maintenance: respect remote.*.skipFetchAllLibravatar Derrick Stolee1-1/+7
If a remote has the skipFetchAll setting enabled, then that remote is not intended for frequent fetching. It makes sense to not fetch that data during the 'prefetch' maintenance task. Skip that remote in the iteration without error. The skip_default_update member is initialized in remote.c:handle_config() as part of initializing the 'struct remote'. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-04-16maintenance: use 'git fetch --prefetch'Libravatar Derrick Stolee1-7/+7
The 'prefetch' maintenance task previously forced the following refspec for each remote: +refs/heads/*:refs/prefetch/<remote>/* If a user has specified a more strict refspec for the remote, then this prefetch task downloads more objects than necessary. The previous change introduced the '--prefetch' option to 'git fetch' which manipulates the remote's refspec to place all resulting refs into refs/prefetch/, with further partitioning based on the destinations of those refspecs. Update the documentation to be more generic about the destination refs. Do not mention custom refspecs explicitly, as that does not need to be highlighted in this documentation. The important part of placing refs in refs/prefetch/ remains. Reported-by: Tom Saeger <tom.saeger@oracle.com> Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-04-16fetch: add --prefetch optionLibravatar Derrick Stolee1-0/+43
The --prefetch option will be used by the 'prefetch' maintenance task instead of sending refspecs explicitly across the command-line. The intention is to modify the refspec to place all results in refs/prefetch/ instead of anywhere else. Create helper method filter_prefetch_refspec() to modify a given refspec to fit the rules expected of the prefetch task: * Negative refspecs are preserved. * Refspecs without a destination are removed. * Refspecs whose source starts with "refs/tags/" are removed. * Other refspecs are placed within "refs/prefetch/". Finally, we add the 'force' option to ensure that prefetch refs are replaced as necessary. There are some interesting cases that are worth testing. An earlier version of this change dropped the "i--" from the loop that deletes a refspec item and shifts the remaining entries down. This allowed some refspecs to not be modified. The subtle part about the first --prefetch test is that the "refs/tags/*" refspec appears directly before the "refs/heads/bogus/*" refspec. Without that "i--", this ordering would remove the "refs/tags/*" refspec and leave the last one unmodified, placing the result in "refs/heads/*". It is possible to have an empty refspec. This is typically the case for remotes other than the origin, where users want to fetch a specific tag or branch. To correctly test this case, we need to further remove the upstream remote for the local branch. Thus, we are testing a refspec that will be deleted, leaving nothing to fetch. Helped-by: Tom Saeger <tom.saeger@oracle.com> Helped-by: Ramsay Jones <ramsay@ramsayjones.plus.com> Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-04-15Merge branch 'jz/apply-3way-cached'Libravatar Junio C Hamano1-0/+50
"git apply" now takes "--3way" and "--cached" at the same time, and work and record results only in the index. * jz/apply-3way-cached: git-apply: allow simultaneous --cached and --3way options
2021-04-15Merge branch 'jz/apply-run-3way-first'Libravatar Junio C Hamano1-0/+20
"git apply --3way" has always been "to fall back to 3-way merge only when straight application fails". Swap the order of falling back so that 3-way is always attempted first (only when the option is given, of course) and then straight patch application is used as a fallback when it fails. * jz/apply-run-3way-first: git-apply: try threeway first when "--3way" is used
2021-04-15transport: respect verbosity when setting upstreamLibravatar Øystein Walle1-0/+7
A command such as `git push -qu origin feature` will print "Branch 'feature' set up to track remote branch 'feature' from 'origin'." even when --quiet is passed. In this case it's because install_branch_config() is always called with BRANCH_CONFIG_VERBOSE. struct transport keeps track of the desired verbosity. Fix the above issue by passing BRANCH_CONFIG_VERBOSE conditionally based on that. Signed-off-by: Øystein Walle <oystwa@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-04-13Merge branch 'cc/test-helper-bloom-usage-fix'Libravatar Junio C Hamano1-1/+1
Usage message fix for a test helper. * cc/test-helper-bloom-usage-fix: test-bloom: fix missing 'bloom' from usage string
2021-04-13Merge branch 'ab/send-email-validate-errors'Libravatar Junio C Hamano1-8/+27
Clean-up codepaths that implements "git send-email --validate" option and improves the message from it. * ab/send-email-validate-errors: git-send-email: improve --validate error output git-send-email: refactor duplicate $? checks into a function git-send-email: test full --validate output
2021-04-13Merge branch 'tb/pack-preferred-tips-to-give-bitmap'Libravatar Junio C Hamano4-0/+64
A configuration variable has been added to force tips of certain refs to be given a reachability bitmap. * tb/pack-preferred-tips-to-give-bitmap: builtin/pack-objects.c: respect 'pack.preferBitmapTips' t/helper/test-bitmap.c: initial commit pack-bitmap: add 'test_bitmap_commits()' helper
2021-04-13Merge branch 'jk/ref-filter-segfault-fix'Libravatar Junio C Hamano1-0/+10
A NULL-dereference bug has been corrected in an error codepath in "git for-each-ref", "git branch --list" etc. * jk/ref-filter-segfault-fix: ref-filter: fix NULL check for parse object failure
2021-04-13tests: remove all uses of test_i18cmpLibravatar Ævar Arnfjörð Bjarmason3-13/+6
Finish the removal I started in 1108cea7f8e (tests: remove most uses of test_i18ncmp, 2021-02-11). At that time the function wasn't removed due to disruption with in-flight changes, remove the occurrences that have landed since then. As of writing this there are no test_i18ncmp uses between "master" and "seen", so let's also remove the function to finally put it to rest. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-04-13revision: avoid parsing with --exclude-promisor-objectsLibravatar Jeff King1-0/+8
When --exclude-promisor-objects is given, before traversing any objects we iterate over all of the objects in any promisor packs, marking them as UNINTERESTING and SEEN. We turn the oid we get from iterating the pack into an object with parse_object(), but this has two problems: - it's slow; we are zlib inflating (and reconstructing from deltas) every byte of every object in the packfile - it leaves the tree buffers attached to their structs, which means our heap usage will grow to store every uncompressed tree simultaneously. This can be gigabytes. We can obviously fix the second by freeing the tree buffers after we've parsed them. But we can observe that the function doesn't look at the object contents at all! The only reason we call parse_object() is that we need a "struct object" on which to set the flags. There are two options here: - we can look up just the object type via oid_object_info(), and then call the appropriate lookup_foo() function - we can call lookup_unknown_object(), which gives us an OBJ_NONE struct (which will get auto-converted later by object_as_type() via calls to lookup_commit(), etc). The first one is closer to the current code, but we do pay the price to look up the type for each object. The latter should be more efficient in CPU, though it wastes a little bit of memory (the "unknown" object structs are a union of all object types, so some of the structs are bigger than they need to be). It also runs the risk of triggering a latent bug in code that calls lookup_object() directly but isn't ready to handle OBJ_NONE (such code would already be buggy, but we use lookup_unknown_object() infrequently enough that it might be hiding). I went with the second option here. I don't think the risk is high (and we'd want to find and fix any such bugs anyway), and it should be more efficient overall. The new tests in p5600 show off the improvement (this is on git.git): Test HEAD^ HEAD ------------------------------------------------------------------------------- 5600.5: count commits 0.37(0.37+0.00) 0.38(0.38+0.00) +2.7% 5600.6: count non-promisor commits 11.74(11.37+0.37) 0.04(0.03+0.00) -99.7% The improvement is particularly big in this script because _every_ object in the newly-cloned partial repo is a promisor object. So after marking them all, there's nothing left to traverse. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-04-13lookup_unknown_object(): take a repository argumentLibravatar Jeff King1-3/+3
All of the other lookup_foo() functions take a repository argument, but lookup_unknown_object() was never converted, and it uses the_repository internally. Let's fix that. We could leave a wrapper that uses the_repository, but there aren't that many calls, so we'll just convert them all. I looked briefly at each site to see if we had a repository struct (besides the_repository) we could pass, but none of them do (so this conversion to pass the_repository is a pure noop in each case, though it does take us one step closer to eventually getting rid of the_repository). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-04-13is_promisor_object(): free tree buffer after parsingLibravatar Jeff King1-0/+4
To get the list of all promisor objects, we not only include all objects in promisor packs, but also parse each of those objects to see which objects they reference. After parsing a tree object, the tree->buffer field will remain populated until we explicitly free it. So in a partial clone of blob:none, for example, we are essentially reading every tree in the repository (since they're all in the initial promisor pack), and keeping all of their uncompressed contents in memory at once. This patch frees the tree buffers after we've finished marking all of their reachable objects. We shouldn't need to do this for any other object type. While we are using some extra memory to store the structs, no other object type stores the whole contents in its parsed form (we do sometimes hold on to commit buffers, but less so these days due to commit graphs, plus most commands which care about promisor objects turn off the save_commit_buffer global). Even for a moderate-sized repository like git.git, this patch drops the peak heap (as measured by massif) for git-fsck from ~1.7GB to ~138MB. Fsck is a good candidate for measuring here because it doesn't interact with the promisor code except to call is_promisor_object(), so we can isolate just this problem. The added perf test shows only a tiny improvement on my machine for git.git, since 1.7GB isn't enough to cause any real memory pressure: Test HEAD^ HEAD -------------------------------------------------------------------------------- 5600.4: fsck 21.26(20.90+0.35) 20.84(20.79+0.04) -2.0% With linux.git the absolute change is a bit bigger, though still a small percentage: Test HEAD^ HEAD ----------------------------------------------------------------------------- 5600.4: fsck 262.26(259.13+3.12) 254.92(254.62+0.29) -2.8% I didn't have the patience to run it under massif with linux.git, but it's probably on the order of about 14GB improvement, since that's the sum of the sizes of all of the uncompressed trees (but still isn't enough to create memory pressure on this particular machine, which has 64GB of RAM). Smaller machines would probably see a bigger effect on runtime (and sadly our perf suite does not measure peak heap). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-04-12pack-objects: update "nr_seen" progress based on pack-reused countLibravatar Jeff King1-0/+23
When serving a clone or fetch with bitmaps, after deciding which objects need to be sent our "pack reuse" mechanism kicks in: we try to send more-or-less verbatim a bunch of objects from the beginning of the bitmapped packfile without even adding them to the to_pack.objects array. After deciding which objects will be in the "reused" portion, we update nr_result to account for those, and then trigger display_progress() to show the user (who is undoubtedly dazzled that we managed to enumerate so many objects so quickly). But then something confusing happens: the "Enumerating objects" progress meter jumps _backwards_, counting up from zero the number of objects we actually add into to_pack.objects. This worked correctly once upon a time, but was broken in 5af050437a (pack-objects: show some progress when counting kept objects, 2018-04-15), when the latter half of that progress meter switched to using a separate nr_seen counter, rather than nr_result. Nobody noticed for two reasons: - prior to the pack-reuse fixes from a14aebeac3 (Merge branch 'jk/packfile-reuse-cleanup', 2020-02-14), the reuse code almost never kicked in anyway - the output looks _kind of_ correct. The "backwards" moment is hard to catch, because we overwrite the old progress number with the new one, and the larger number is displayed only for a second. So unless you look at that exact second, you just see the much smaller value, counting up to the number of non-reused objects (though of course if you catch it in stderr, or look at GIT_TRACE_PACKET from a server with bitmaps, you can see both values). This smaller output isn't wrong per se, but isn't counting what we ever intended to. We should give the user the whole number of objects we considered (which, as per 5af050437a's original purpose, is already _not_ a count of what goes into to_pack.objects). The follow-on "Counting objects" meter shows the actual number of objects we feed into that array. We can easily fix this by bumping (and showing) nr_seen for the pack-reused objects. When the included test is run without this patch, the second pack-objects invocation produces "Enumerating objects: 1" to show the one loose object, even though the resulting pack has hundreds of objects in it. With it, we jump to "Enumerating objects: 674" after deciding on reuse, and then "675" when we add in the loose object. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-04-08userdiff: add support for SchemeLibravatar Atharva Raykar16-0/+90
Add a diff driver for Scheme-like languages which recognizes top level and local `define` forms, whether it is a function definition, binding, syntax definition or a user-defined `define-xyzzy` form. Also supports R6RS `library` forms, `module` forms along with class and struct declarations used in Racket (PLT Scheme). Alternate "def" syntax such as those in Gerbil Scheme are also supported, like defstruct, defsyntax and so on. The rationale for picking `define` forms for the hunk headers is because it is usually the only significant form for defining the structure of the program, and it is a common pattern for schemers to have local function definitions to hide their visibility, so it is not only the top level `define`'s that are of interest. Schemers also extend the language with macros to provide their own define forms (for example, something like a `define-test-suite`) which is also captured in the hunk header. Since it is common practice to extend syntax with variants of a form like `module+`, `class*` etc, those have been supported as well. The word regex is a best-effort attempt to conform to R7RS[1] valid identifiers, symbols and numbers. [1] https://small.r7rs.org/attachment/r7rs.pdf (section 2.1) Signed-off-by: Atharva Raykar <raykar.ath@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-04-08Merge branch 'en/ort-perf-batch-9'Libravatar Junio C Hamano1-0/+71
The ort merge backend has been optimized by skipping irrelevant renames. * en/ort-perf-batch-9: diffcore-rename: avoid doing basename comparisons for irrelevant sources merge-ort: skip rename detection entirely if possible merge-ort: use relevant_sources to filter possible rename sources merge-ort: precompute whether directory rename detection is needed merge-ort: introduce wrappers for alternate tree traversal merge-ort: add data structures for an alternate tree traversal merge-ort: precompute subset of sources for which we need rename detection diffcore-rename: enable filtering possible rename sources
2021-04-08Merge branch 'en/sequencer-edit-upon-conflict-fix'Libravatar Junio C Hamano1-1/+31
"git cherry-pick/revert" with or without "--[no-]edit" did not spawn the editor as expected (e.g. "revert --no-edit" after a conflict still asked to edit the message), which has been corrected. * en/sequencer-edit-upon-conflict-fix: sequencer: fix edit handling for cherry-pick and revert messages
2021-04-08Merge branch 'll/clone-reject-shallow'Libravatar Junio C Hamano3-1/+60
"git clone --reject-shallow" option fails the clone as soon as we notice that we are cloning from a shallow repository. * ll/clone-reject-shallow: builtin/clone.c: add --reject-shallow option
2021-04-08Merge branch 'tb/reverse-midx'Libravatar Junio C Hamano2-4/+62
An on-disk reverse-index to map the in-pack location of an object back to its object name across multiple packfiles is introduced. * tb/reverse-midx: midx.c: improve cache locality in midx_pack_order_cmp() pack-revindex: write multi-pack reverse indexes pack-write.c: extract 'write_rev_file_order' pack-revindex: read multi-pack reverse indexes Documentation/technical: describe multi-pack reverse indexes midx: make some functions non-static midx: keep track of the checksum midx: don't free midx_name early midx: allow marking a pack as preferred t/helper/test-read-midx.c: add '--show-objects' builtin/multi-pack-index.c: display usage on unrecognized command builtin/multi-pack-index.c: don't enter bogus cmd_mode builtin/multi-pack-index.c: split sub-commands builtin/multi-pack-index.c: define common usage with a macro builtin/multi-pack-index.c: don't handle 'progress' separately builtin/multi-pack-index.c: inline 'flags' with options
2021-04-08blame tests: simplify userdiff driver testLibravatar Ævar Arnfjörð Bjarmason1-21/+15
Simplify the test added in 9466e3809d (blame: enable funcname blaming with userdiff driver, 2020-11-01) to use the --author support recently added in 999cfc4f45 (test-lib functions: add --author support to test_commit, 2021-01-12). We also did not need the full fortran-external-function content. Let's cut it down to just the important parts. I'm modifying it to demonstrate that the fortran-specific userdiff function is in effect by adding "DO NOT MATCH ..." and "AS THE ..." lines surrounding the "RIGHT" one. This is to check that we're using the userdiff "fortran" driver, as opposed to the default driver which would match on those lines as part of the general heuristic of matching a line that doesn't begin with whitespace. The test had also been leaving behind a .gitattributes file for later tests to possibly trip over, let's clean it up with "test_when_finished". Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-04-08blame tests: don't rely on t/t4018/ directoryLibravatar Ævar Arnfjörð Bjarmason1-3/+13
Refactor a test added in 9466e3809d (blame: enable funcname blaming with userdiff driver, 2020-11-01) so that the blame tests don't rely on stealing the contents of "t/t4018/fortran-external-function". I have another patch series that'll possibly (or not) refactor that file, but having this test inter-dependency makes things simple in any case by making this test more readable. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-04-08userdiff: remove support for "broken" testsLibravatar Ævar Arnfjörð Bjarmason2-10/+1
There have been no "broken" tests since 75c3b6b2e8 (userdiff: improve Fortran xfuncname regex, 2020-08-12). Let's remove the test support for them. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-04-08userdiff tests: list builtin drivers via test-toolLibravatar Ævar Arnfjörð Bjarmason4-27/+67
Change the userdiff test to list the builtin drivers via the test-tool, using the new for_each_userdiff_driver() API function. This gets rid of the need to modify this part of the test every time a new pattern is added, see 2ff6c34612 (userdiff: support Bash, 2020-10-22) and 09dad9256a (userdiff: support Markdown, 2020-05-02) for two recent examples. I only need the "list-builtin-drivers "argument here, but let's add "list-custom-drivers" and "list-drivers" too, just because it's easy. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-04-08userdiff tests: explicitly test "default" patternLibravatar Ævar Arnfjörð Bjarmason1-0/+1
Since 122aa6f9c0 (diff: introduce diff.<driver>.binary, 2008-10-05) the internals of the userdiff.c code have understood a "default" name, which is invoked as userdiff_find_by_name("default") and present in the "builtin_drivers" struct. Let's test for this special case. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-04-07git-apply: allow simultaneous --cached and --3way optionsLibravatar Jerry Zhang1-0/+50
"git apply" does not allow "--cached" and "--3way" to be used together, since "--3way" writes conflict markers into the working tree. Allow "git apply" to accept "--cached" and "--3way" at the same time. When a single file auto-resolves cleanly, the result is placed in the index at stage #0 and the command exits with 0 status. For a file that has a conflict which cannot be cleanly auto-resolved, the original contents from common ancestor (stage conflict at the content level, and the command exists with non-zero status, because there is no place (like the working tree) to leave a half-resolved merge for the user to resolve. The user can use `git diff` to view the contents of the conflict, or `git checkout -m -- .` to regenerate the conflict markers in the working directory. Don't attempt rerere in this case since it depends on conflict markers written to file for its database storage and lookup. There would be two main changes required to get rerere working: 1. Allow the rerere api to accept in memory object rather than files, which would allow us to pass in the conflict markers contained in the result from ll_merge(). 2. Rerere can't write to the working directory, so it would have to apply the result to cache stage #0 directly. A flag would be needed to control this. Signed-off-by: Jerry Zhang <jerry@skydio.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-04-07Merge branch 'ps/pack-bitmap-optim'Libravatar Junio C Hamano1-0/+14
Optimize "rev-list --use-bitmap-index --objects" corner case that uses negative tags as the stopping points. * ps/pack-bitmap-optim: pack-bitmap: avoid traversal of objects referenced by uninteresting tag
2021-04-07Merge branch 'zh/commit-trailer'Libravatar Junio C Hamano1-0/+312
"git commit" learned "--trailer <key>[=<value>]" option; together with the interpret-trailers command, this will make it easier to support custom trailers. * zh/commit-trailer: commit: add --trailer option
2021-04-06git-apply: try threeway first when "--3way" is usedLibravatar Jerry Zhang1-0/+20
The apply_fragments() method of "git apply" can silently apply patches incorrectly if a file has repeating contents. In these cases a three-way merge is capable of applying it correctly in more situations, and will show a conflict rather than applying it incorrectly. However, because the patches apply "successfully" using apply_fragments(), git will never fall back to the merge, even if the "--3way" flag is used, and the user has no way to ensure correctness by forcing the three-way merge method. Change the behavior so that when "--3way" is used, git will always try the three-way merge first and will only fall back to apply_fragments() in cases where blobs are not available or some other error (but not in the case of a merge conflict). Since user-facing results will be different, this has backwards compatibility implications for users depending on the old behavior. In addition, the three-way merge will be slower than direct patch application. Signed-off-by: Jerry Zhang <jerry@skydio.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-04-06git-send-email: improve --validate error outputLibravatar Ævar Arnfjörð Bjarmason1-5/+12
Improve the output we emit on --validate error to: * Say "FILE:LINE" instead of "FILE: LINE", to match "grep -n", compiler error messages etc. * Don't say "patch contains a" after just mentioning the filename, just leave it at "FILE:LINE: is longer than[...]. The "contains a" sounded like we were talking about the file in general, when we're actually checking it line-by-line. * Don't just say "rejected by sendemail-validate hook", but combine that with the system_or_msg() output to say what exit code the hook died with. I had an aborted attempt to make the line length checker note all lines that were longer than the limit. I didn't think that was worth the effort, but I've left in the testing change to check that we die as soon as we spot the first long line. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-04-06git-send-email: test full --validate outputLibravatar Ævar Arnfjörð Bjarmason1-6/+18
Change the tests that grep substrings out of the output to use a full test_cmp, in preparation for improving the output. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>