summaryrefslogtreecommitdiff
path: root/t/t5500-fetch-pack.sh
AgeCommit message (Collapse)AuthorFilesLines
2020-11-19t550*: adjust the references to the default branch name "main"Libravatar Johannes Schindelin1-15/+15
This trick was performed via $ (cd t && sed -i -e 's/master/main/g' -e 's/MASTER/MAIN/g' \ -e 's/Master/Main/g' -- t550*.sh) This allows us to define `GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main` for those tests. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-11-19tests: mark tests relying on the current default for `init.defaultBranch`Libravatar Johannes Schindelin1-0/+3
In addition to the manual adjustment to let the `linux-gcc` CI job run the test suite with `master` and then with `main`, this patch makes sure that GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME is set in all test scripts that currently rely on the initial branch name being `master by default. To determine which test scripts to mark up, the first step was to force-set the default branch name to `master` in - all test scripts that contain the keyword `master`, - t4211, which expects `t/t4211/history.export` with a hard-coded ref to initialize the default branch, - t5560 because it sources `t/t556x_common` which uses `master`, - t8002 and t8012 because both source `t/annotate-tests.sh` which also uses `master`) This trick was performed by this command: $ sed -i '/^ *\. \.\/\(test-lib\|lib-\(bash\|cvs\|git-svn\)\|gitweb-lib\)\.sh$/i\ GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=master\ export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME\ ' $(git grep -l master t/t[0-9]*.sh) \ t/t4211*.sh t/t5560*.sh t/t8002*.sh t/t8012*.sh After that, careful, manual inspection revealed that some of the test scripts containing the needle `master` do not actually rely on a specific default branch name: either they mention `master` only in a comment, or they initialize that branch specificially, or they do not actually refer to the current default branch. Therefore, the aforementioned modification was undone in those test scripts thusly: $ git checkout HEAD -- \ t/t0027-auto-crlf.sh t/t0060-path-utils.sh \ t/t1011-read-tree-sparse-checkout.sh \ t/t1305-config-include.sh t/t1309-early-config.sh \ t/t1402-check-ref-format.sh t/t1450-fsck.sh \ t/t2024-checkout-dwim.sh \ t/t2106-update-index-assume-unchanged.sh \ t/t3040-subprojects-basic.sh t/t3301-notes.sh \ t/t3308-notes-merge.sh t/t3423-rebase-reword.sh \ t/t3436-rebase-more-options.sh \ t/t4015-diff-whitespace.sh t/t4257-am-interactive.sh \ t/t5323-pack-redundant.sh t/t5401-update-hooks.sh \ t/t5511-refspec.sh t/t5526-fetch-submodules.sh \ t/t5529-push-errors.sh t/t5530-upload-pack-error.sh \ t/t5548-push-porcelain.sh \ t/t5552-skipping-fetch-negotiator.sh \ t/t5572-pull-submodule.sh t/t5608-clone-2gb.sh \ t/t5614-clone-submodules-shallow.sh \ t/t7508-status.sh t/t7606-merge-custom.sh \ t/t9302-fast-import-unpack-limit.sh We excluded one set of test scripts in these commands, though: the range of `git p4` tests. The reason? `git p4` stores the (foreign) remote branch in the branch called `p4/master`, which is obviously not the default branch. Manual analysis revealed that only five of these tests actually require a specific default branch name to pass; They were modified thusly: $ sed -i '/^ *\. \.\/lib-git-p4\.sh$/i\ GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=master\ export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME\ ' t/t980[0167]*.sh t/t9811*.sh Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-06Merge branch 'bc/sha-256-part-2'Libravatar Junio C Hamano1-2/+3
SHA-256 migration work continues. * bc/sha-256-part-2: (44 commits) remote-testgit: adapt for object-format bundle: detect hash algorithm when reading refs t5300: pass --object-format to git index-pack t5704: send object-format capability with SHA-256 t5703: use object-format serve option t5702: offer an object-format capability in the test t/helper: initialize the repository for test-sha1-array remote-curl: avoid truncating refs with ls-remote t1050: pass algorithm to index-pack when outside repo builtin/index-pack: add option to specify hash algorithm remote-curl: detect algorithm for dumb HTTP by size builtin/ls-remote: initialize repository based on fetch t5500: make hash independent serve: advertise object-format capability for protocol v2 connect: parse v2 refs with correct hash algorithm connect: pass full packet reader when parsing v2 refs Documentation/technical: document object-format for protocol v2 t1302: expect repo format version 1 for SHA-256 builtin/show-index: provide options to determine hash algo t5302: modernize test formatting ...
2020-06-05fetch: allow adding a filter after initial cloneLibravatar Xin Li1-1/+0
Retroactively adding a filter can be useful for existing shallow clones as they allow users to see earlier change histories without downloading all git objects in a regular --unshallow fetch. Without this patch, users can make a clone partial by editing the repository configuration to convert the remote into a promisor, like:   git config core.repositoryFormatVersion 1   git config extensions.partialClone origin   git fetch --unshallow --filter=blob:none origin Since the hard part of making this work is already in place and such edits can be error-prone, teach Git to perform the required configuration change automatically instead. Note that this change does not modify the existing git behavior which recognizes setting extensions.partialClone without changing repositoryFormatVersion. Signed-off-by: Xin Li <delphij@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-05-27t5500: make hash independentLibravatar brian m. carlson1-2/+3
This test has hard-coded pkt-lines with object IDs. The pkt-line lengths necessarily differ between hash algorithms, so generate these lines with the packetize helper so they're always the right size. In addition, we will require an object-format capability for SHA-256, so pass that capability on to the upload-pack process. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-05-14Merge branch 'jt/t5500-unflake'Libravatar Junio C Hamano1-6/+6
Test fix for a topic already in 'master' and meant for 'maint'. * jt/t5500-unflake: t5500: count objects through stderr, not trace
2020-05-06t5500: count objects through stderr, not traceLibravatar Jonathan Tan1-6/+6
In two tests introduced by 4fa3f00abb ("fetch-pack: in protocol v2, in_vain only after ACK", 2020-04-28) and 2f0a093dd6 ("fetch-pack: in protocol v2, reset in_vain upon ACK", 2020-04-28), the count of objects downloaded is checked by grepping for a specific message in the packet trace. However, this is flaky as that specific message may be delivered over 2 or more packet lines. Instead, grep over stderr, just like the "fetch creating new shallow root" test in the same file. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-05-01Merge branch 'jt/v2-fetch-nego-fix'Libravatar Junio C Hamano1-0/+48
The upload-pack protocol v2 gave up too early before finding a common ancestor, resulting in a wasteful fetch from a fork of a project. This has been corrected to match the behaviour of v0 protocol. * jt/v2-fetch-nego-fix: fetch-pack: in protocol v2, reset in_vain upon ACK fetch-pack: in protocol v2, in_vain only after ACK fetch-pack: return enum from process_acks()
2020-04-28fetch-pack: in protocol v2, reset in_vain upon ACKLibravatar Jonathan Tan1-0/+30
In the function process_acks() in fetch-pack.c, the variable received_ack is meant to track that an ACK was received, but it was never set. This results in negotiation terminating prematurely through the in_vain counter, when the counter should have been reset upon every ACK. Therefore, reset the in_vain counter upon every ACK. Helped-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Reviewed-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-28fetch-pack: in protocol v2, in_vain only after ACKLibravatar Jonathan Tan1-0/+18
When fetching, Git stops negotiation when it has sent at least MAX_IN_VAIN (which is 256) "have" lines without having any of them ACK-ed. But this is supposed to trigger only after the first ACK, as pack-protocol.txt says: However, the 256 limit *only* turns on in the canonical client implementation if we have received at least one "ACK %s continue" during a prior round. This helps to ensure that at least one common ancestor is found before we give up entirely. The code path for protocol v0 observes this, but not protocol v2, resulting in shorter negotiation rounds but significantly larger packfiles. Teach the code path for protocol v2 to check this criterion only after at least one ACK was received. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Reviewed-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-15test: request GIT_TEST_PROTOCOL_VERSION=0 when appropriateLibravatar Jonathan Nieder1-2/+3
Since 8cbeba0632 (tests: define GIT_TEST_PROTOCOL_VERSION, 2019-02-25), it has been possible to run tests with a newer protocol version by setting the GIT_TEST_PROTOCOL_VERSION envvar to a version number. Tests that assume protocol v0 handle this by explicitly setting GIT_TEST_PROTOCOL_VERSION= or similar constructs like 'test -z "$GIT_TEST_PROTOCOL_VERSION" || return 0' to declare that they only handle the default (v0) protocol. The emphasis there is a bit off: it would be clearer to specify GIT_TEST_PROTOCOL_VERSION=0 to inform the reader that these tests are specifically testing and relying on details of protocol v0. Do so. This way, a reader does not need to know what the default protocol version is, and the tests can continue to work when the default protocol version used by Git advances past v0. Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-15fetch test: use more robust test for filtered objectsLibravatar Jonathan Nieder1-5/+13
"git cat-file -e" uses has_object_file, which can fetch from promisor remotes when an object is missing. These tests end up checking that that fetch fails instead of for the object being missing. By luck, the tests pass anyway: - in one of these tests ("filtering by size"), the fetch fails because (in protocol v0) the server does not support fetches by SHA-1 - in the second, the object is present but the test could pass even if it weren't if the fetch succeeds - in the third, the test sets extensions.partialClone to "arbitrary string" so that when it tries to fetch, it looks up the "arbitrary string" remote which does not exist Use "git rev-list --objects --missing=allow-any", so that the tests pass for the right reason. Noticed while testing with protocol v2, which allows fetching by sha1 by default, causing the first fetch to succeed and the test to fail. Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-10-07Merge branch 'jk/disable-commit-graph-during-upload-pack'Libravatar Junio C Hamano1-0/+38
The "upload-pack" (the counterpart of "git fetch") needs to disable commit-graph when responding to a shallow clone/fetch request, but the way this was done made Git panic, which has been corrected. * jk/disable-commit-graph-during-upload-pack: upload-pack: disable commit graph more gently for shallow traversal commit-graph: bump DIE_ON_LOAD check to actual load-time
2019-09-12upload-pack: disable commit graph more gently for shallow traversalLibravatar Jeff King1-0/+38
When the client has asked for certain shallow options like "deepen-since", we do a custom rev-list walk that pretends to be shallow. Before doing so, we have to disable the commit-graph, since it is not compatible with the shallow view of the repository. That's handled by 829a321569 (commit-graph: close_commit_graph before shallow walk, 2018-08-20). That commit literally closes and frees our repo->objects->commit_graph struct. That creates an interesting problem for commits that have _already_ been parsed using the commit graph. Their commit->object.parsed flag is set, their commit->graph_pos is set, but their commit->maybe_tree may still be NULL. When somebody later calls repo_get_commit_tree(), we see that we haven't loaded the tree oid yet and try to get it from the commit graph. But since it has been freed, we segfault! So the root of the issue is a data dependency between the commit's lazy-load of the tree oid and the fact that the commit graph can go away mid-process. How can we resolve it? There are a couple of general approaches: 1. The obvious answer is to avoid loading the tree from the graph when we see that it's NULL. But then what do we return for the tree oid? If we return NULL, our caller in do_traverse() will rightly complain that we have no tree. We'd have to fallback to loading the actual commit object and re-parsing it. That requires teaching parse_commit_buffer() to understand re-parsing (i.e., not starting from a clean slate and not leaking any allocated bits like parent list pointers). 2. When we close the commit graph, walk through the set of in-memory objects and clear any graph_pos pointers. But this means we also have to "unparse" any such commits so that we know they still need to open the commit object to fill in their trees. So it's no less complicated than (1), and is more expensive (since we clear objects we might not later need). 3. Stop freeing the commit-graph struct. Continue to let it be used for lazy-loads of tree oids, but let upload-pack specify that it shouldn't be used for further commit parsing. 4. Push the whole shallow rev-list out to its own sub-process, with the commit-graph disabled from the start, giving it a clean memory space to work from. I've chosen (3) here. Options (1) and (2) would work, but are non-trivial to implement. Option (4) is more expensive, and I'm not sure how complicated it is (shelling out for the actual rev-list part is easy, but we do then parse the resulting commits internally, and I'm not clear which parts need to be handling shallow-ness). The new test in t5500 triggers this segfault, but see the comments there for how horribly intimate it has to be with how both upload-pack and commit graphs work. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-26mingw: support UNC in git clone file://server/share/repoLibravatar Torsten Bögershausen1-2/+11
Extend the parser to accept file://server/share/repo in the way that Windows users expect it to be parsed who are used to referring to file shares by UNC paths of the form \\server\share\folder. [jes: tightened check to avoid handling file://C:/some/path as a UNC path.] This closes https://github.com/git-for-windows/git/issues/1264. Signed-off-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-02t: warn against adding non-httpd-specific tests after sourcing 'lib-httpd'Libravatar SZEDER Gábor1-0/+3
We have a couple of test scripts that are not completely httpd-specific, but do run a few httpd-specific tests at the end. These test scripts source 'lib-httpd.sh' somewhere mid-script, which then skips all the rest of the test script if the dependencies for running httpd tests are not fulfilled. As the previous two patches in this series show, already on two occasions non-httpd-specific tests were appended at the end of such test scripts, and, consequently, they were skipped as well when httpd tests couldn't be run. Add a comment at the end of these test scripts to warn against adding non-httpd-specific tests at the end, in the hope that they will help prevent similar issues in the future. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-25Merge branch 'sg/test-atexit'Libravatar Junio C Hamano1-3/+0
Test framework update to more robustly clean up leftover files and processes after tests are done. * sg/test-atexit: t9811-git-p4-label-import: fix pipeline negation git p4 test: disable '-x' tracing in the p4d watchdog loop git p4 test: simplify timeout handling git p4 test: clean up the p4d cleanup functions git p4 test: use 'test_atexit' to kill p4d and the watchdog process t0301-credential-cache: use 'test_atexit' to stop the credentials helper tests: use 'test_atexit' to stop httpd git-daemon: use 'test_atexit` to stop 'git-daemon' test-lib: introduce 'test_atexit' t/lib-git-daemon: make sure to kill the 'git-daemon' process test-lib: fix interrupt handling with 'dash' and '--verbose-log -x'
2019-03-14tests: use 'test_atexit' to stop httpdLibravatar SZEDER Gábor1-3/+0
Use 'test_atexit' to run cleanup commands to stop httpd at the end of the test script or upon interrupt or failure, as it is shorter, simpler, and more robust than registering such cleanup commands in the trap on EXIT in the test scripts. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-07tests: always test fetch of unreachable with v0Libravatar Jonathan Tan1-1/+3
Some tests check that fetching an unreachable object fails, but protocol v2 allows such fetches. Unset GIT_TEST_PROTOCOL_VERSION so that these tests are always run using protocol v0. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-29Merge branch 'jt/fetch-pack-v2'Libravatar Junio C Hamano1-7/+15
"git fetch-pack" now can talk the version 2 protocol. * jt/fetch-pack-v2: fetch-pack: support protocol version 2
2019-01-10fetch-pack: support protocol version 2Libravatar Jonathan Tan1-7/+15
When the scaffolding for protocol version 2 was initially added in 8f6982b4e1 ("protocol: introduce enum protocol_version value protocol_v2", 2018-03-14). As seen in: git log -p -G'support for protocol v2 not implemented yet' --full-diff --reverse v2.17.0..v2.20.0 Many of those scaffolding "die" placeholders were removed, but we hadn't gotten around to fetch-pack yet. The test here for "fetch refs from cmdline" is very minimal. There's much better coverage when running the entire test suite under the WIP GIT_TEST_PROTOCOL_VERSION=2 mode[1], we should ideally have better coverage without needing to invoke a special test mode. 1. https://public-inbox.org/git/20181213155817.27666-1-avarab@gmail.com/ Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-07tests: standardize pipe placementLibravatar Matthew DeVore1-2/+5
Instead of using a line-continuation and pipe on the second line, take advantage of the shell's implicit line continuation after a pipe character. So for example, instead of some long line \ | next line use some long line | next line And add a blank line before and after the pipe where it aids readability (it usually does). This better matches the coding style documented in Documentation/CodingGuidelines and used in shell scripts elsewhere in the tree. Signed-off-by: Matthew DeVore <matvore@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-27Merge branch 'sg/test-must-be-empty'Libravatar Junio C Hamano1-1/+1
Test fixes. * sg/test-must-be-empty: tests: use 'test_must_be_empty' instead of 'test_cmp <empty> <out>' tests: use 'test_must_be_empty' instead of 'test_cmp /dev/null <out>' tests: use 'test_must_be_empty' instead of 'test ! -s' tests: use 'test_must_be_empty' instead of '! test -s'
2018-08-21tests: use 'test_must_be_empty' instead of '! test -s'Libravatar SZEDER Gábor1-1/+1
Using 'test_must_be_empty' is preferable to '! test -s', because it gives a helpful error message if the given file is unexpectedly not empty, while the latter remains completely silent. Furthermore, it also catches cases when the given file unexpectedly does not exist at all. This patch was basically created by: sed -i -e 's/! test -s/test_must_be_empty/' t[0-9]*.sh with the following notable exceptions: - The '! test -s' check in '.gitmodules ignore=dirty suppresses submodules with untracked content' in 't7508-status.sh' is left as-is, because it's bogus and, therefore, it's subject of a dedicated patch. - The '! test -s' checks in 't9131-git-svn-empty-symlink.sh' and 't9135-git-svn-moved-branch-empty-file.sh' are immediately preceeded by a 'test -f' to ensure that the files exist in the first place. 'test_must_be_empty' ensures that as well, so those 'test -f' commands are removed as well. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-15Merge branch 'nd/i18n'Libravatar Junio C Hamano1-1/+1
Many more strings are prepared for l10n. * nd/i18n: (23 commits) transport-helper.c: mark more strings for translation transport.c: mark more strings for translation sha1-file.c: mark more strings for translation sequencer.c: mark more strings for translation replace-object.c: mark more strings for translation refspec.c: mark more strings for translation refs.c: mark more strings for translation pkt-line.c: mark more strings for translation object.c: mark more strings for translation exec-cmd.c: mark more strings for translation environment.c: mark more strings for translation dir.c: mark more strings for translation convert.c: mark more strings for translation connect.c: mark more strings for translation config.c: mark more strings for translation commit-graph.c: mark more strings for translation builtin/replace.c: mark more strings for translation builtin/pack-objects.c: mark more strings for translation builtin/grep.c: mark strings for translation builtin/config.c: mark more strings for translation ...
2018-08-02Merge branch 'jt/fetch-pack-negotiator'Libravatar Junio C Hamano1-0/+33
Code restructuring and a small fix to transport protocol v2 during fetching. * jt/fetch-pack-negotiator: fetch-pack: introduce negotiator API fetch-pack: move common check and marking together fetch-pack: make negotiation-related vars local fetch-pack: use ref adv. to prune "have" sent fetch-pack: directly end negotiation if ACK ready fetch-pack: clear marks before re-marking fetch-pack: split up everything_local()
2018-08-02Merge branch 'es/test-fixes'Libravatar Junio C Hamano1-1/+1
Test clean-up and corrections. * es/test-fixes: (26 commits) t5608: fix broken &&-chain t9119: fix broken &&-chains t9000-t9999: fix broken &&-chains t7000-t7999: fix broken &&-chains t6000-t6999: fix broken &&-chains t5000-t5999: fix broken &&-chains t4000-t4999: fix broken &&-chains t3030: fix broken &&-chains t3000-t3999: fix broken &&-chains t2000-t2999: fix broken &&-chains t1000-t1999: fix broken &&-chains t0000-t0999: fix broken &&-chains t9814: simplify convoluted check that command correctly errors out t9001: fix broken "invoke hook" test t7810: use test_expect_code() instead of hand-rolled comparison t7400: fix broken "submodule add/reconfigure --force" test t7201: drop pointless "exit 0" at end of subshell t6036: fix broken "merge fails but has appropriate contents" tests t5505: modernize and simplify hard-to-digest test t5406: use write_script() instead of birthing shell script manually ...
2018-07-24Merge branch 'jk/fetch-all-peeled-fix'Libravatar Junio C Hamano1-6/+13
Test modernization. * jk/fetch-all-peeled-fix: t5500: prettify non-commit tag tests
2018-07-23builtin/pack-objects.c: mark more strings for translationLibravatar Nguyễn Thái Ngọc Duy1-1/+1
Most of these are straight forward. GETTEXT_POISON does catch the last string in cmd_pack_objects(), but since this is --progress output, it's not supposed to be machine-readable. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-16t5000-t5999: fix broken &&-chainsLibravatar Eric Sunshine1-1/+1
Signed-off-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-06t5500: prettify non-commit tag testsLibravatar Jeff King1-6/+13
We don't need to use backslash continuation, as the "&&" already provides continuation (and happily soaks up empty lines between commands). We can also expand the multi-line printf into a here-document, which lets us use line breaks more naturally (and avoids another continuation that required us to break the natural indentation). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-06-28Merge branch 'jk/fetch-all-peeled-fix'Libravatar Junio C Hamano1-0/+41
"git fetch-pack --all" used to unnecessarily fail upon seeing an annotated tag that points at an object other than a commit. * jk/fetch-all-peeled-fix: fetch-pack: test explicitly that --all can fetch tag references pointing to non-commits fetch-pack: don't try to fetch peel values with --all
2018-06-25Merge branch 'nd/reject-empty-shallow-request'Libravatar Junio C Hamano1-0/+11
"git fetch --shallow-since=<cutoff>" that specifies the cut-off point that is newer than the existing history used to end up grabbing the entire history. Such a request now errors out. * nd/reject-empty-shallow-request: upload-pack: reject shallow requests that would return nothing
2018-06-15fetch-pack: use ref adv. to prune "have" sentLibravatar Jonathan Tan1-0/+33
In negotiation using protocol v2, fetch-pack sometimes does not make full use of the information obtained in the ref advertisement: specifically, that if the server advertises a commit that the client also has, the client never needs to inform the server that it has the commit's parents, since it can just tell the server that it has the advertised commit and it knows that the server can and will infer the rest. This is because, in do_fetch_pack_v2(), rev_list_insert_ref_oid() is invoked before mark_complete_and_common_ref(). This means that if we have a commit that is both our ref and their ref, it would be enqueued by rev_list_insert_ref_oid() as SEEN, and since it is thus already SEEN, mark_complete_and_common_ref() would not enqueue it. If mark_complete_and_common_ref() were invoked first, as it is in do_fetch_pack() for protocol v0, then mark_complete_and_common_ref() would enqueue it with COMMON_REF | SEEN. The addition of COMMON_REF ensures that its parents are not sent as "have" lines. Change the order in do_fetch_pack_v2() to be consistent with do_fetch_pack(), and to avoid sending unnecessary "have" lines. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-06-13fetch-pack: test explicitly that --all can fetch tag references pointing to ↵Libravatar Kirill Smelkov1-0/+31
non-commits Fetch-pack --all became broken with respect to unusual tags in 5f0fc64513 (fetch-pack: eliminate spurious error messages, 2012-09-09), and was fixed only recently in e9502c0a7f (fetch-pack: don't try to fetch peel values with --all, 2018-06-11). However the test added in e9502c0a7f does not explicitly cover all funky cases. In order to be sure fetching funky tags will never break, let's explicitly test all relevant cases with 4 tag objects pointing to 1) a blob, 2) a tree, 3) a commit, and 4) another tag objects. The referenced tag objects themselves are referenced from under regular refs/tags/* namespace. Before e9502c0a7f `fetch-pack --all` was failing e.g. this way: .../git/t/trash directory.t5500-fetch-pack/fetchall$ git ls-remote .. 44085874... HEAD ... bc4e9e1f... refs/tags/tag-to-blob 038f48ad... refs/tags/tag-to-blob^{} # peeled 520db1f5... refs/tags/tag-to-tree 7395c100... refs/tags/tag-to-tree^{} # peeled .../git/t/trash directory.t5500-fetch-pack/fetchall$ git fetch-pack --all .. fatal: A git upload-pack: not our ref 038f48ad... fatal: The remote end hung up unexpectedly Helped-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Kirill Smelkov <kirr@nexedi.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-06-11fetch-pack: don't try to fetch peel values with --allLibravatar Jeff King1-0/+10
When "fetch-pack --all" sees a tag-to-blob on the remote, it tries to fetch both the tag itself ("refs/tags/foo") and the peeled value that the remote advertises ("refs/tags/foo^{}"). Asking for the object pointed to by the latter can cause upload-pack to complain with "not our ref", since it does not mark the peeled objects with the OUR_REF (unless they were at the tip of some other ref). Arguably upload-pack _should_ be marking those peeled objects. But it never has in the past, since clients would generally just ask for the tag and expect to get the peeled value along with it. And that's how "git fetch" works, as well as older versions of "fetch-pack --all". The problem was introduced by 5f0fc64513 (fetch-pack: eliminate spurious error messages, 2012-09-09). Before then, the matching logic was something like: if (refname is ill-formed) do nothing else if (doing --all) always consider it matched else look through list of sought refs for a match That commit wanted to flip the order of the second two arms of that conditional. But we ended up with: if (refname is ill-formed) do nothing else look through list of sought refs for a match if (--all and no match so far) always consider it matched That means tha an ill-formed ref will trigger the --all conditional block, even though we should just be ignoring it. We can fix that by having a single "else" with all of the well-formed logic, that checks the sought refs and "--all" in the correct order. Reported-by: Kirill Smelkov <kirr@nexedi.com> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-06-04upload-pack: reject shallow requests that would return nothingLibravatar Nguyễn Thái Ngọc Duy1-0/+11
Shallow clones with --shallow-since or --shalow-exclude work by running rev-list to get all reachable commits, then draw a boundary between reachable and unreachable and send "shallow" requests based on that. The code does miss one corner case: if rev-list returns nothing, we'll have no border and we'll send no shallow requests back to the client (i.e. no history cuts). This essentially means a full clone (or a full branch if the client requests just one branch). One example is the oldest commit is older than what is specified by --shallow-since. To avoid this, if rev-list returns nothing, we abort the clone/fetch. The user could adjust their request (e.g. --shallow-since further back in the past) and retry. Another possible option for this case is to fall back to a default depth (like depth 1). But I don't like too much magic that way because we may return something unexpected to the user. If they request "history since 2008" and we return a single depth at 2000, that might break stuff for them. It is better to tell them that something is wrong and let them take the best course of action. Note that we need to die() in get_shallow_commits_by_rev_list() instead of just checking for empty result from its caller deepen_by_rev_list() and handling the error there. The reason is, empty result could be a valid case: if you have commits in year 2013 and you request --shallow-since=year.2000 then you should get a full clone (i.e. empty result). Reported-by: Andreas Krey <a.krey@gmx.de> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-05-23t: make many tests depend less on the refs being filesLibravatar David Turner1-5/+5
Many tests are very focused on the file system representation of the loose and packed refs code. As there are plans to implement other ref storage systems, let's migrate these tests to a form that test the intent of the refs storage system instead of it internals. This will make clear to readers that these tests do not depend on which ref backend is used. The internals of the loose refs backend are still tested in t1400-update-ref.sh, whereas the tests changed in this patch focus on testing other aspects. This patch just takes care of many low hanging fruits. It does not try to completely solves the issue. Helped-by: Stefan Beller <sbeller@google.com> Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de> Signed-off-by: David Turner <dturner@twopensource.com> Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-02-28t5500-fetch-pack: don't check the stderr of a subshellLibravatar SZEDER Gábor1-6/+6
Three "missing reference" tests in 't5500-fetch-pack.sh' fail when the test script is run with '-x' tracing (and using a shell other than a Bash version supporting BASH_XTRACEFD). The reason for those failures is that the tests check a subshell's stderr, which includes the trace of executing commands in that subshell as well, throwing off the comparison with the expected output. Save the stderr of 'git fetch-pack' only instead of the whole subshell, so it remains free from tracing output. After this change t5500 passes with '-x', even when running with /bin/sh. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-12-08fetch: support filtersLibravatar Jeff Hostetler1-0/+36
Teach fetch to support filters. This is only allowed for the remote configured in extensions.partialcloneremote. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-12-08fetch-pack: test support excluding large blobsLibravatar Jonathan Tan1-0/+27
Created tests to verify fetch-pack and upload-pack support for excluding large blobs using --filter=blobs:limit=<n> parameter. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-16fetch-pack: always allow fetching of literal SHA1sLibravatar Jonathan Tan1-0/+35
fetch-pack, when fetching a literal SHA-1 from a server that is not configured with uploadpack.allowtipsha1inwant (or similar), always returns an error message of the form "Server does not allow request for unadvertised object %s". However, it is sometimes the case that such object is advertised. This situation would occur, for example, if a user or a script was provided a SHA-1 instead of a branch or tag name for fetching, and wanted to invoke "git fetch" or "git fetch-pack" using that SHA-1. Teach fetch-pack to also check the SHA-1s of the refs in the received ref advertisement if a literal SHA-1 was given by the user. Helped-by: Jeff King <peff@peff.net> Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Reviewed-by: Jonathan Nieder <jrnieder@gmail.com> Reviewed-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-02fetch-pack: move code to report unmatched refs to a functionLibravatar Matt McCutchen1-3/+3
Prepare to reuse this code in transport.c for "git fetch". While we're here, internationalize the existing error message. Signed-off-by: Matt McCutchen <matt@mattmccutchen.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-10-10Merge branch 'nd/shallow-deepen'Libravatar Junio C Hamano1-0/+68
The existing "git fetch --depth=<n>" option was hard to use correctly when making the history of an existing shallow clone deeper. A new option, "--deepen=<n>", has been added to make this easier to use. "git clone" also learned "--shallow-since=<date>" and "--shallow-exclude=<tag>" options to make it easier to specify "I am interested only in the recent N months worth of history" and "Give me only the history since that version". * nd/shallow-deepen: (27 commits) fetch, upload-pack: --deepen=N extends shallow boundary by N commits upload-pack: add get_reachable_list() upload-pack: split check_unreachable() in two, prep for get_reachable_list() t5500, t5539: tests for shallow depth excluding a ref clone: define shallow clone boundary with --shallow-exclude fetch: define shallow boundary with --shallow-exclude upload-pack: support define shallow boundary by excluding revisions refs: add expand_ref() t5500, t5539: tests for shallow depth since a specific date clone: define shallow clone boundary based on time with --shallow-since fetch: define shallow boundary with --shallow-since upload-pack: add deepen-since to cut shallow repos based on time shallow.c: implement a generic shallow boundary finder based on rev-list fetch-pack: use a separate flag for fetch in deepening mode fetch-pack.c: mark strings for translating fetch-pack: use a common function for verbose printing fetch-pack: use skip_prefix() instead of starts_with() upload-pack: move rev-list code out of check_non_tip() upload-pack: make check_non_tip() clean things up on error upload-pack: tighten number parsing at "deepen" lines ...
2016-06-13fetch, upload-pack: --deepen=N extends shallow boundary by N commitsLibravatar Nguyễn Thái Ngọc Duy1-0/+23
In git-fetch, --depth argument is always relative with the latest remote refs. This makes it a bit difficult to cover this use case, where the user wants to make the shallow history, say 3 levels deeper. It would work if remote refs have not moved yet, but nobody can guarantee that, especially when that use case is performed a couple months after the last clone or "git fetch --depth". Also, modifying shallow boundary using --depth does not work well with clones created by --since or --not. This patch fixes that. A new argument --deepen=<N> will add <N> more (*) parent commits to the current history regardless of where remote refs are. Have/Want negotiation is still respected. So if remote refs move, the server will send two chunks: one between "have" and "want" and another to extend shallow history. In theory, the client could send no "want"s in order to get the second chunk only. But the protocol does not allow that. Either you send no want lines, which means ls-remote; or you have to send at least one want line that carries deep-relative to the server.. The main work was done by Dongcan Jiang. I fixed it up here and there. And of course all the bugs belong to me. (*) We could even support --deepen=<N> where <N> is negative. In that case we can cut some history from the shallow clone. This operation (and --depth=<shorter depth>) does not require interaction with remote side (and more complicated to implement as a result). Helped-by: Duy Nguyen <pclouds@gmail.com> Helped-by: Eric Sunshine <sunshine@sunshineco.com> Helped-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Dongcan Jiang <dongcan.jiang@gmail.com> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-13t5500, t5539: tests for shallow depth excluding a refLibravatar Nguyễn Thái Ngọc Duy1-0/+21
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-13t5500, t5539: tests for shallow depth since a specific dateLibravatar Nguyễn Thái Ngọc Duy1-0/+24
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-10Merge branch 'jk/shell-portability'Libravatar Junio C Hamano1-1/+0
test fixes. * jk/shell-portability: t5500 & t7403: lose bash-ism "local" test-lib: add in-shell "env" replacement
2016-06-01t5500 & t7403: lose bash-ism "local"Libravatar Junio C Hamano1-1/+0
In t5500::check_prot_host_port_path(), diagport is not a variable used elsewhere and the function is not recursively called so this can simply lose the "local", which may not be supported by shell (besides, the function liberally clobbers other variables without making them "local"). t7403::reset_submodule_urls() overrides the "root" variable used in the test framework for no good reason; its use is not about temporarily relocating where the test repositories are created. This assignment can be made not to clobber the variable by moving them into the subshells it already uses. Its value is always $TRASH_DIRECTORY, so we could use it instead there, and this function that is called only once and its two subshells may not be necessary (instead, the caller can use "git -C $there config" and set a value that is derived from $TRASH_DIRECTORY), but this is a minimum fix that is needed to lose "local". Helped-by: John Keeping <john@keeping.me.uk> Helped-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-04-08Merge branch 'jc/merge-refuse-new-root'Libravatar Junio C Hamano1-2/+4
"git merge" used to allow merging two branches that have no common base by default, which led to a brand new history of an existing project created and then get pulled by an unsuspecting maintainer, which allowed an unnecessary parallel history merged into the existing project. The command has been taught not to allow this by default, with an escape hatch "--allow-unrelated-histories" option to be used in a rare event that merges histories of two projects that started their lives independently. * jc/merge-refuse-new-root: merge: refuse to create too cool a merge by default