summaryrefslogtreecommitdiff
path: root/t
AgeCommit message (Collapse)AuthorFilesLines
2021-09-20Merge branch 'tb/multi-pack-bitmaps'Libravatar Junio C Hamano11-296/+753
The reachability bitmap file used to be generated only for a single pack, but now we've learned to generate bitmaps for history that span across multiple packfiles. * tb/multi-pack-bitmaps: (29 commits) pack-bitmap: drop bitmap_index argument from try_partial_reuse() pack-bitmap: drop repository argument from prepare_midx_bitmap_git() p5326: perf tests for MIDX bitmaps p5310: extract full and partial bitmap tests midx: respect 'GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP' t7700: update to work with MIDX bitmap test knob t5319: don't write MIDX bitmaps in t5319 t5310: disable GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP t0410: disable GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP t5326: test multi-pack bitmap behavior t/helper/test-read-midx.c: add --checksum mode t5310: move some tests to lib-bitmap.sh pack-bitmap: write multi-pack bitmaps pack-bitmap: read multi-pack bitmaps pack-bitmap.c: avoid redundant calls to try_partial_reuse pack-bitmap.c: introduce 'bitmap_is_preferred_refname()' pack-bitmap.c: introduce 'nth_bitmap_object_oid()' pack-bitmap.c: introduce 'bitmap_num_objects()' midx: avoid opening multiple MIDXs when writing midx: close linked MIDXs, avoid leaking memory ...
2021-09-15Merge branch 'pb/test-use-user-env'Libravatar Junio C Hamano3-21/+103
Teach "test_pause" and "debug" helpers to allow using the HOME and TERM environment variables the user usually uses. * pb/test-use-user-env: test-lib-functions: keep user's debugger config files and TERM in 'debug' test-lib-functions: optionally keep HOME, TERM and SHELL in 'test_pause' test-lib-functions: use 'TEST_SHELL_PATH' in 'test_pause'
2021-09-15Merge branch 'jc/trivial-threeway-binary-merge'Libravatar Junio C Hamano1-0/+45
The "git apply -3" code path learned not to bother the lower level merge machinery when the three-way merge can be trivially resolved without the content level merge. * jc/trivial-threeway-binary-merge: apply: resolve trivial merge without hitting ll-merge with "--3way"
2021-09-15Merge branch 'ab/send-email-config-fix'Libravatar Junio C Hamano1-0/+15
Regression fix. * ab/send-email-config-fix: send-email: fix a "first config key wins" regression in v2.33.0
2021-09-10Merge branch 'jk/log-warn-on-bogus-encoding'Libravatar Junio C Hamano1-0/+7
Doc update plus improved error reporting. * jk/log-warn-on-bogus-encoding: docs: use "character encoding" to refer to commit-object encoding logmsg_reencode(): warn when iconv() fails
2021-09-10Merge branch 'mk/clone-recurse-submodules'Libravatar Junio C Hamano1-0/+12
After "git clone --recurse-submodules", all submodules are cloned but they are not by default recursed into by other commands. With submodule.stickyRecursiveClone configuration set, submodule.recurse configuration is set to true in a repository created by "clone" with "--recurse-submodules" option. * mk/clone-recurse-submodules: clone: set submodule.recurse=true if submodule.stickyRecursiveClone enabled
2021-09-10Merge branch 'uk/userdiff-php-enum'Libravatar Junio C Hamano1-0/+4
Update the userdiff pattern for PHP. * uk/userdiff-php-enum: userdiff: support enum keyword in PHP hunk header
2021-09-10Merge branch 'tk/fast-export-anonymized-tag-fix'Libravatar Junio C Hamano1-4/+6
The output from "git fast-export", when its anonymization feature is in use, showed an annotated tag incorrectly. * tk/fast-export-anonymized-tag-fix: fast-export: fix anonymized tag using original length
2021-09-10Merge branch 'ab/commit-graph-usage'Libravatar Junio C Hamano1-0/+19
Fixes on usage message from "git commit-graph". * ab/commit-graph-usage: commit-graph: show "unexpected subcommand" error commit-graph: show usage on "commit-graph [write|verify] garbage" commit-graph: early exit to "usage" on !argc multi-pack-index: refactor "goto usage" pattern commit-graph: use parse_options_concat() commit-graph: remove redundant handling of -h commit-graph: define common usage with a macro
2021-09-10Merge branch 'mh/send-email-reset-in-reply-to'Libravatar Junio C Hamano1-0/+45
Even when running "git send-email" without its own threaded discussion support, a threading related header in one message is carried over to the subsequent message to result in an unwanted threading, which has been corrected. * mh/send-email-reset-in-reply-to: send-email: avoid incorrect header propagation
2021-09-10Merge branch 'sg/set-ceiling-during-tests'Libravatar Junio C Hamano1-1/+2
Buggy tests could damage repositories outside the throw-away test area we created. We now by default export GIT_CEILING_DIRECTORIES to limit the damage from such a stray test. * sg/set-ceiling-during-tests: test-lib: set GIT_CEILING_DIRECTORIES to protect the surrounding repository
2021-09-10Merge branch 'ka/want-ref-in-namespace'Libravatar Junio C Hamano1-36/+172
"git upload-pack" which runs on the other side of "git fetch" forgot to take the ref namespaces into account when handling want-ref requests. * ka/want-ref-in-namespace: docs: clarify the interaction of transfer.hideRefs and namespaces upload-pack.c: treat want-ref relative to namespace t5730: introduce fetch command helper
2021-09-10Merge branch 'zh/cherry-pick-advice'Libravatar Junio C Hamano2-7/+26
The advice message that "git cherry-pick" gives when it asks conflicted replay of a commit to be resolved by the end user has been updated. * zh/cherry-pick-advice: cherry-pick: use better advice message
2021-09-08Merge branch 'sg/column-nl'Libravatar Junio C Hamano1-0/+18
The parser for the "--nl" option of "git column" has been corrected. * sg/column-nl: column: fix parsing of the '--nl' option
2021-09-08Merge branch 'rs/branch-allow-deleting-dangling'Libravatar Junio C Hamano1-0/+13
"git branch -D <branch>" used to refuse to remove a broken branch ref that points at a missing commit, which has been corrected. * rs/branch-allow-deleting-dangling: branch: allow deleting dangling branches with --force
2021-09-08Merge branch 'mt/quiet-with-delayed-checkout'Libravatar Junio C Hamano1-0/+71
The delayed checkout code path in "git checkout" etc. were chatty even when --quiet and/or --no-progress options were given. * mt/quiet-with-delayed-checkout: checkout: make delayed checkout respect --quiet and --no-progress
2021-09-08Merge branch 'dd/diff-files-unmerged-fix'Libravatar Junio C Hamano1-0/+53
"git diff --relative" segfaulted and/or produced incorrect result when there are unmerged paths. * dd/diff-files-unmerged-fix: diff-lib: ignore paths that are outside $cwd if --relative asked
2021-09-08Merge branch 'dd/t6300-wo-gpg-fix'Libravatar Junio C Hamano1-11/+18
Test fix. * dd/t6300-wo-gpg-fix: t6300: check for cat-file exit status code t6300: don't run cat-file on non-existent object
2021-09-08Merge branch 'jk/t5323-no-pack-test-fix'Libravatar Junio C Hamano1-2/+2
Test fix. * jk/t5323-no-pack-test-fix: t5323: drop mentions of "master"
2021-09-08Merge branch 'js/maintenance-launchctl-fix'Libravatar Junio C Hamano1-0/+17
"git maintenance" scheduler fix for macOS. * js/maintenance-launchctl-fix: maintenance: skip bootout/bootstrap when plist is registered maintenance: create `launchctl` configuration using a lock file
2021-09-08Merge branch 'ab/ls-remote-packet-trace'Libravatar Junio C Hamano1-3/+3
Debugging aid fix. * ab/ls-remote-packet-trace: ls-remote: set packet_trace_identity(<name>)
2021-09-08Merge branch 'ga/send-email-sendmail-cmd'Libravatar Junio C Hamano1-1/+1
Test fix. * ga/send-email-sendmail-cmd: t9001: PATH must not use Windows-style paths
2021-09-08Merge branch 'me/t5582-cleanup'Libravatar Junio C Hamano1-1/+0
Test fix. * me/t5582-cleanup: t5582: remove spurious 'cd "$D"' line
2021-09-07send-email: fix a "first config key wins" regression in v2.33.0Libravatar Ævar Arnfjörð Bjarmason1-0/+15
Fix a regression in my c95e3a3f0b8 (send-email: move trivial config handling to Perl, 2021-05-28) where we'd pick the first config key out of multiple defined ones, instead of using the normal "last key wins" semantics of "git config --get". This broke e.g. cases where a .git/config would have a different sendemail.smtpServer than ~/.gitconfig. We'd pick the ~/.gitconfig over .git/config, instead of preferring the repository-local version. The same would go for /etc/gitconfig etc. The full list of impacted config keys (the %config_settings values which are references to scalars, not arrays) is: sendemail.smtpencryption sendemail.smtpserver sendemail.smtpserverport sendemail.smtpuser sendemail.smtppass sendemail.smtpdomain sendemail.smtpauth sendemail.smtpbatchsize sendemail.smtprelogindelay sendemail.tocmd sendemail.cccmd sendemail.aliasfiletype sendemail.envelopesender sendemail.confirm sendemail.from sendemail.assume8bitencoding sendemail.composeencoding sendemail.transferencoding sendemail.sendmailcmd I.e. having any of these set in say ~/.gitconfig and in-repo .git/config regressed in v2.33.0 to prefer the --global one over the --local. To test this add a test of config priority to one of these config variables, most don't have tests at all, but there was an existing one for sendemail.8bitEncoding. The "git config" (instead of "test_config") is somewhat of an anti-pattern, but follows established conventions in t9001-send-email.sh, likewise with any other pattern or idiom in this test. The populating of home/.gitconfig and setting of HOME= is copied from a test in t0017-env-helper.sh added in 1ff750b128e (tests: make GIT_TEST_GETTEXT_POISON a boolean, 2019-06-21). This test fails without this bugfix, but now it works. Reported-by: Eli Schwartz <eschwartz@archlinux.org> Tested-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-07test-lib-functions: keep user's debugger config files and TERM in 'debug'Libravatar Philippe Blain2-16/+50
The 'debug' function in test-lib-functions.sh is used to invoke a debugger at a specific line in a test. It inherits the value of HOME and TERM set by 'test-lib.sh': HOME="$TRASH_DIRECTORY" and TERM=dumb. Changing the value of HOME means that any customization configured in a developers' debugger configuration file (like $HOME/.gdbinit or $HOME/.lldbinit) are not available in the debugger invoked by 'test_pause'. Changing the value of TERM to 'dumb' means that colored output is disabled in the debugger. To make the debugging experience with 'debug' more pleasant, leverage the variable USER_HOME, added in the previous commit, to copy a developer's ~/.gdbinit and ~/.lldbinit to the test HOME. We do not set HOME to USER_HOME as in 'test_pause' to avoid user configuration in $USER_HOME/.gitconfig from interfering with the command being debugged. Also, add a flag to launch the debugger with the original value of TERM, and add the same warning as for 'test_pause'. Helped-by: Carlo Marcelo Arenas Belón <carenas@gmail.com> Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Acked-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-07test-lib-functions: optionally keep HOME, TERM and SHELL in 'test_pause'Libravatar Philippe Blain3-5/+53
The 'test_pause' function, which is designed to help interactive debugging and exploration of tests, currently inherits the value of HOME and TERM set by 'test-lib.sh': HOME="$TRASH_DIRECTORY" and TERM=dumb. It also invokes the shell defined by TEST_SHELL_PATH, which defaults to /bin/sh (through SHELL_PATH). Changing the value of HOME means that any customization configured in a developers' shell startup files and any Git aliases defined in their global Git configuration file are not available in the shell invoked by 'test_pause'. Changing the value of TERM to 'dumb' means that colored output is disabled for all commands in that shell. Using /bin/sh as the shell invoked by 'test_pause' is not ideal since some platforms (i.e. Debian and derivatives) use Dash as /bin/sh, and this shell is usually compiled without readline support, which makes for a poor interactive command line experience. To make the interactive command line experience in the shell invoked by 'test_pause' more pleasant, save the values of HOME and TERM in USER_HOME and USER_TERM before changing them in test-lib.sh, and add options to 'test_pause' to optionally use these variables to invoke the shell. Also add an option to invoke SHELL instead of TEST_SHELL_PATH, so that developer's interactive shell is used. We use options instead of changing the behaviour unconditionally since these three variables can slightly change command behaviour. Moreover, using the original HOME means commands could overwrite files in a user's home directory. Be explicit about these caveats in the new 'Usage' section in test-lib-functions.sh. Finally, add '[options]' to the test_pause synopsys in t/README, and mention that the full list of helper functions and their options can be found in test-lib-functions.sh. Helped-by: Elijah Newren <newren@gmail.com> Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Acked-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-07test-lib-functions: use 'TEST_SHELL_PATH' in 'test_pause'Libravatar Philippe Blain1-1/+1
3f824e91c8 (t/Makefile: introduce TEST_SHELL_PATH, 2017-12-08) made it easy to use a different shell for the tests than 'SHELL_PATH' used at compile time. But 'test_pause' still invokes 'SHELL_PATH'. If TEST_SHELL_PATH is set, invoke that shell in 'test_pause' for consistency. Suggested-by: Carlo Marcelo Arenas Belón <carenas@gmail.com> Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Acked-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-05apply: resolve trivial merge without hitting ll-merge with "--3way"Libravatar Junio C Hamano1-0/+45
The ll_binary_merge() function assumes that the ancestor blob is different from either side of the new versions, and always fails the merge in conflict, unless -Xours or -Xtheirs is in effect. The normal "merge" machineries all resolve the trivial cases (e.g. if our side changed while their side did not, the result is ours) without triggering the file-level merge drivers, so the assumption is warranted. The code path in "git apply --3way", however, does not check for the trivial three-way merge situation and always calls the file-level merge drivers. This used to be perfectly OK back when we always first attempted a straight patch application and used the three-way code path only as a fallback. Any binary patch that can be applied as a trivial three-way merge (e.g. the patch is based exactly on the version we happen to have) would always cleanly apply, so the ll_binary_merge() that is not prepared to see the trivial case would not have to handle such a case. This no longer is true after we made "--3way" to mean "first try three-way and then fall back to straight application", and made "git apply -3" on a binary patch that is based on the current version no longer apply. Teach "git apply -3" to first check for the trivial merge cases and resolve them without hitting the file-level merge drivers. Signed-off-by: Jerry Zhang <jerry@skydio.com> [jc: stolen tests from Jerry's patch] Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-03Merge branch 'fc/completion-updates'Libravatar Junio C Hamano1-0/+15
Command line completion updates. * fc/completion-updates: completion: bash: add correct suffix in variables completion: bash: fix for multiple dash commands completion: bash: fix for suboptions with value completion: bash: fix prefix detection in branch.*
2021-09-03Merge branch 'pw/rebase-r-fixes'Libravatar Junio C Hamano3-9/+98
Various bugs in "git rebase -r" have been fixed. * pw/rebase-r-fixes: rebase -r: fix merge -c with a merge strategy rebase -r: don't write .git/MERGE_MSG when fast-forwarding rebase -i: add another reword test rebase -r: make 'merge -c' behave like reword
2021-09-03Merge branch 'pw/rebase-skip-final-fix'Libravatar Junio C Hamano2-6/+25
Checking out all the paths from HEAD during the last conflicted step in "git rebase" and continuing would cause the step to be skipped (which is expected), but leaves MERGE_MSG file behind in $GIT_DIR and confuses the next "git commit", which has been corrected. * pw/rebase-skip-final-fix: rebase --continue: remove .git/MERGE_MSG rebase --apply: restore some tests t3403: fix commit authorship
2021-09-03Merge branch 'jk/commit-edit-fixup-fix'Libravatar Junio C Hamano1-1/+8
"git commit --fixup" now works with "--edit" again, after it was broken in v2.32. * jk/commit-edit-fixup-fix: commit: restore --edit when combined with --fixup
2021-09-03Merge branch 'ps/connectivity-optim'Libravatar Junio C Hamano1-0/+31
The revision traversal API has been optimized by taking advantage of the commit-graph, when available, to determine if a commit is reachable from any of the existing refs. * ps/connectivity-optim: revision: avoid hitting packfiles when commits are in commit-graph commit-graph: split out function to search commit position revision: stop retrieving reference twice connected: do not sort input revisions revision: separate walk and unsorted flags
2021-09-01p5326: perf tests for MIDX bitmapsLibravatar Taylor Blau1-0/+43
These new performance tests demonstrate effectively the same behavior as p5310, but use a multi-pack bitmap instead of a single-pack one. Notably, p5326 does not create a MIDX bitmap with multiple packs. This is so we can measure a direct comparison between it and p5310. Any difference between the two is measuring just the overhead of using MIDX bitmaps. Here are the results of p5310 and p5326 together, measured at the same time and on the same machine (using a Xenon W-2255 CPU): Test HEAD ------------------------------------------------------------------------ 5310.2: repack to disk 96.78(93.39+11.33) 5310.3: simulated clone 9.98(9.79+0.19) 5310.4: simulated fetch 1.75(4.26+0.19) 5310.5: pack to file (bitmap) 28.20(27.87+8.70) 5310.6: rev-list (commits) 0.41(0.36+0.05) 5310.7: rev-list (objects) 1.61(1.54+0.07) 5310.8: rev-list count with blob:none 0.25(0.21+0.04) 5310.9: rev-list count with blob:limit=1k 2.65(2.54+0.10) 5310.10: rev-list count with tree:0 0.23(0.19+0.04) 5310.11: simulated partial clone 4.34(4.21+0.12) 5310.13: clone (partial bitmap) 11.05(12.21+0.48) 5310.14: pack to file (partial bitmap) 31.25(34.22+3.70) 5310.15: rev-list with tree filter (partial bitmap) 0.26(0.22+0.04) versus the same tests (this time using a multi-pack index): Test HEAD ------------------------------------------------------------------------ 5326.2: setup multi-pack index 78.99(75.29+11.58) 5326.3: simulated clone 11.78(11.56+0.22) 5326.4: simulated fetch 1.70(4.49+0.13) 5326.5: pack to file (bitmap) 28.02(27.72+8.76) 5326.6: rev-list (commits) 0.42(0.36+0.06) 5326.7: rev-list (objects) 1.65(1.58+0.06) 5326.8: rev-list count with blob:none 0.26(0.21+0.05) 5326.9: rev-list count with blob:limit=1k 2.97(2.86+0.10) 5326.10: rev-list count with tree:0 0.25(0.20+0.04) 5326.11: simulated partial clone 5.65(5.49+0.16) 5326.13: clone (partial bitmap) 12.22(13.43+0.38) 5326.14: pack to file (partial bitmap) 30.05(31.57+7.25) 5326.15: rev-list with tree filter (partial bitmap) 0.24(0.20+0.04) There is slight overhead in "simulated clone", "simulated partial clone", and "clone (partial bitmap)". Unsurprisingly, that overhead is due to using the MIDX's reverse index to map between bit positions and MIDX positions. This can be reproduced by running "git repack -adb" along with "git multi-pack-index write --bitmap" in a large-ish repository. Then run: $ perf record -o pack.perf git -c core.multiPackIndex=false \ pack-objects --all --stdout >/dev/null </dev/null $ perf record -o midx.perf git -c core.multiPackIndex=true \ pack-objects --all --stdout >/dev/null </dev/null and compare the two with "perf diff -c delta -o 1 pack.perf midx.perf". The most notable results are below (the next largest positive delta is +0.14%): # Event 'cycles' # # Baseline Delta Shared Object Symbol # ........ ....... .................. .......................... # +5.86% git [.] nth_midxed_offset +5.24% git [.] nth_midxed_pack_int_id 3.45% +0.97% git [.] offset_to_pack_pos 3.30% +0.57% git [.] pack_pos_to_offset +0.30% git [.] pack_pos_to_midx Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-01p5310: extract full and partial bitmap testsLibravatar Taylor Blau2-62/+72
A new p5326 introduced by the next patch will want these same tests, interjecting its own setup in between. Move them out so that both perf tests can reuse them. Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-01midx: respect 'GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP'Libravatar Taylor Blau1-0/+4
Introduce a new 'GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP' environment variable to also write a multi-pack bitmap when 'GIT_TEST_MULTI_PACK_INDEX' is set. Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-01t7700: update to work with MIDX bitmap test knobLibravatar Taylor Blau1-6/+12
A number of these tests are focused only on pack-based bitmaps and need to be updated to disable 'GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP' where necessary. Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-01t5319: don't write MIDX bitmaps in t5319Libravatar Taylor Blau1-1/+2
This test is specifically about generating a midx still respecting a pack-based bitmap file. Generating a MIDX bitmap would confuse the test. Let's override the 'GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP' variable to make sure we don't do so. Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-01t5310: disable GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAPLibravatar Jeff King1-0/+4
Generating a MIDX bitmap confuses many of the tests in t5310, which expect to control whether and how bitmaps are written. Since the relevant MIDX-bitmap tests here are covered already in t5326, let's just disable the flag for the whole t5310 script. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-01t0410: disable GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAPLibravatar Jeff King1-0/+3
Generating a MIDX bitmap causes tests which repack in a partial clone to fail because they are missing objects. Missing objects is an expected component of tests in t0410, so disable this knob altogether. Graceful degradation when writing a bitmap with missing objects is tested in t5326. Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-01t5326: test multi-pack bitmap behaviorLibravatar Taylor Blau1-0/+286
This patch introduces a new test, t5326, which tests the basic functionality of multi-pack bitmaps. Some trivial behavior is tested, such as: - Whether bitmaps can be generated with more than one pack. - Whether clones can be served with all objects in the bitmap. - Whether follow-up fetches can be served with some objects outside of the server's bitmap These use lib-bitmap's tests (which in turn were pulled from t5310), and we cover cases where the MIDX represents both a single pack and multiple packs. In addition, some non-trivial and MIDX-specific behavior is tested, too, including: - Whether multi-pack bitmaps behave correctly with respect to the pack-reuse machinery when the base for some object is selected from a different pack than the delta. - Whether multi-pack bitmaps correctly respect the pack.preferBitmapTips configuration. Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-01t/helper/test-read-midx.c: add --checksum modeLibravatar Taylor Blau2-1/+19
Subsequent tests will want to check for the existence of a multi-pack bitmap which matches the multi-pack-index stored in the pack directory. The multi-pack bitmap includes the hex checksum of the MIDX it corresponds to in its filename (for example, '$packdir/multi-pack-index-<checksum>.bitmap'). As a result, some tests want a way to learn what '<checksum>' is. This helper addresses that need by printing the checksum of the repository's multi-pack-index. Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-01t5310: move some tests to lib-bitmap.shLibravatar Taylor Blau2-224/+241
We'll soon be adding a test script that will cover many of the same bitmap concepts as t5310, but for MIDX bitmaps. Let's pull out as many of the applicable tests as we can so we don't have to rewrite them. There should be no functional change to t5310; we still run the same operations in the same order. Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-01midx: avoid opening multiple MIDXs when writingLibravatar Taylor Blau1-1/+9
Opening multiple instance of the same MIDX can lead to problems like two separate packed_git structures which represent the same pack being added to the repository's object store. The above scenario can happen because prepare_midx_pack() checks if `m->packs[pack_int_id]` is NULL in order to determine if a pack has been opened and installed in the repository before. But a caller can construct two copies of the same MIDX by calling get_multi_pack_index() and load_multi_pack_index() since the former manipulates the object store directly but the latter is a lower-level routine which allocates a new MIDX for each call. So if prepare_midx_pack() is called on multiple MIDXs with the same pack_int_id, then that pack will be installed twice in the object store's packed_git pointer. This can lead to problems in, for e.g., the pack-bitmap code, which does something like the following (in pack-bitmap.c:open_pack_bitmap()): struct bitmap_index *bitmap_git = ...; for (p = get_all_packs(r); p; p = p->next) { if (open_pack_bitmap_1(bitmap_git, p) == 0) ret = 0; } which is a problem if two copies of the same pack exist in the packed_git list because pack-bitmap.c:open_pack_bitmap_1() contains a conditional like the following: if (bitmap_git->pack || bitmap_git->midx) { /* ignore extra bitmap file; we can only handle one */ warning("ignoring extra bitmap file: %s", packfile->pack_name); close(fd); return -1; } Avoid this scenario by not letting write_midx_internal() open a MIDX that isn't also pointed at by the object store. So long as this is the case, other routines should prefer to open MIDXs with get_multi_pack_index() or reprepare_packed_git() instead of creating instances on their own. Because get_multi_pack_index() returns `r->object_store->multi_pack_index` if it is non-NULL, we'll only have one instance of a MIDX open at one time, avoiding these problems. To encourage this, drop the `struct multi_pack_index *` parameter from `write_midx_internal()`, and rely instead on the `object_dir` to find (or initialize) the correct MIDX instance. Likewise, replace the call to `close_midx()` with `close_object_store()`, since we're about to replace the MIDX with a new one and should invalidate the object store's memory of any MIDX that might have existed beforehand. Note that this now forbids passing object directories that don't belong to alternate repositories over `--object-dir`, since before we would have happily opened a MIDX in any directory, but now restrict ourselves to only those reachable by `r->objects->multi_pack_index` (and alternate MIDXs that we can see by walking the `next` pointer). As far as I can tell, supporting arbitrary directories with `--object-dir` was a historical accident, since even the documentation says `<alt>` when referring to the value passed to this option. Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-01midx: reject empty `--preferred-pack`'sLibravatar Taylor Blau1-0/+17
The soon-to-be-implemented multi-pack bitmap treats object in the first bit position specially by assuming that all objects in the pack it was selected from are also represented from that pack in the MIDX. In other words, the pack from which the first object was selected must also have all of its other objects selected from that same pack in the MIDX in case of any duplicates. But this assumption relies on the fact that there is at least one object in that pack to begin with; otherwise the object in the first bit position isn't from a preferred pack, in which case we can no longer assume that all objects in that pack were also selected from the same pack. Guard this assumption by checking the number of objects in the given preferred pack, and failing if the given pack is empty. To make sure we can safely perform this check, open any packs which are contained in an existing MIDX via prepare_midx_pack(). The same is done for new packs via the add_pack_to_midx() callback, but packs picked up from a previous MIDX will not yet have these opened. Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-01midx: fix `*.rev` cleanups with `--object-dir`Libravatar Taylor Blau1-0/+28
If using --object-dir to point into an object directory which belongs to a different repository than the one in the current working directory, such as: git init repo git -C repo ... # add some objects cd alternate git multi-pack-index --object-dir ../repo/.git/objects write the binary will segfault trying to access the object-dir via the repo it found, but that's not fully initialized. Worse, if we later call clear_midx_files_ext(), we will use `the_repository` and remove files out of the wrong object directory. Fix this by using the given object_dir (or the object directory of `the_repository` if `--object-dir` wasn't given) to properly to clean up the *.rev files, avoiding the crash. Original-patch-by: Johannes Berg <johannes@sipsolutions.net> Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-01midx: disallow running outside of a repositoryLibravatar Taylor Blau1-0/+5
The multi-pack-index command supports working with arbitrary object directories via the `--object-dir` flag. Though this has historically worked in arbitrary repositories (including when the command itself was run outside of a Git repository), this has been somewhat of an accident. For example, running: git multi-pack-index write --object-dir=/path/to/repo/objects outside of a Git repository causes a BUG(). This is because the top-level `cmd_multi_pack_index()` function stops parsing when it sees "write", and then fills in the default object directory (the result of calling `get_object_directory()`) before handing off to `cmd_multi_pack_index_write()`. But there is no repository to initialize, and so calling `get_object_directory()` results in a BUG() (indicating that the current repository is not initialized). Another case where this doesn't quite work as expected is when operating in a SHA-256 repository. To see the failure, try this in your shell: git init --object-format=sha256 repo git -C repo commit --allow-empty base git -C repo repack -d git multi-pack-index --object-dir=$(pwd)/repo/.git/objects write and observe that we cannot open the `.idx` file in "repo", because the outermost process assumes that any repository that it works in also uses the default value of `the_hash_algo` (at the time of writing, SHA-1). There may be compelling reasons for trying to work around these bugs, but working in arbitrary `--object-dir`'s is non-standard enough (and likewise, these bugs prevalent enough) that I don't think any workflows would be broken by abandoning this behavior. Accordingly, restrict the `multi-pack-index` builtin to only work when inside of a Git repository (i.e., its main utility becomes selecting which alternate to operate in), which avoids both of the bugs above. (Note that you can still trigger a bug when writing a MIDX in an alternate which does not use the same object format as the repository which it is an alternate of, but that is an unrelated bug to this one). Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-01upload-pack.c: treat want-ref relative to namespaceLibravatar Kim Altintop1-0/+135
When 'upload-pack' runs within the context of a git namespace, treat any 'want-ref' lines the client sends as relative to that namespace. Also check if the wanted ref is hidden via 'hideRefs'. If it is hidden, respond with an error as if the ref didn't exist. Helped-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Kim Altintop <kim@eagain.st> Reviewed-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-01t5730: introduce fetch command helperLibravatar Kim Altintop1-36/+37
Assembling a "raw" fetch command to be fed directly to "test-tool serve-v2" is extracted into a test helper. Suggested-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Kim Altintop <kim@eagain.st> Reviewed-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-08-31userdiff: support enum keyword in PHP hunk headerLibravatar USAMI Kenta1-0/+4
"enum" keyword will be introduced in PHP 8.1. https://wiki.php.net/rfc/enumerations Signed-off-by: USAMI Kenta <tadsan@zonu.me> Signed-off-by: Junio C Hamano <gitster@pobox.com>