summaryrefslogtreecommitdiff
path: root/t/t5601-clone.sh
AgeCommit message (Collapse)AuthorFilesLines
2021-02-10tests: remove most uses of C_LOCALE_OUTPUTLibravatar Ævar Arnfjörð Bjarmason1-1/+1
As a follow-up to d162b25f956 (tests: remove support for GIT_TEST_GETTEXT_POISON, 2021-01-20) remove those uses of the now always true C_LOCALE_OUTPUT prerequisite from those tests which declare it as an argument to test_expect_{success,failure}. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-11-19t5[6-9]*: adjust the references to the default branch name "main"Libravatar Johannes Schindelin1-3/+3
This trick was performed via $ (cd t && sed -i -e 's/master/main/g' -e 's/MASTER/MAIN/g' \ -e 's/Master/Main/g' -- t5[6-9]*.sh) This allows us to define `GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main` for those tests. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-11-19tests: mark tests relying on the current default for `init.defaultBranch`Libravatar Johannes Schindelin1-0/+3
In addition to the manual adjustment to let the `linux-gcc` CI job run the test suite with `master` and then with `main`, this patch makes sure that GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME is set in all test scripts that currently rely on the initial branch name being `master by default. To determine which test scripts to mark up, the first step was to force-set the default branch name to `master` in - all test scripts that contain the keyword `master`, - t4211, which expects `t/t4211/history.export` with a hard-coded ref to initialize the default branch, - t5560 because it sources `t/t556x_common` which uses `master`, - t8002 and t8012 because both source `t/annotate-tests.sh` which also uses `master`) This trick was performed by this command: $ sed -i '/^ *\. \.\/\(test-lib\|lib-\(bash\|cvs\|git-svn\)\|gitweb-lib\)\.sh$/i\ GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=master\ export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME\ ' $(git grep -l master t/t[0-9]*.sh) \ t/t4211*.sh t/t5560*.sh t/t8002*.sh t/t8012*.sh After that, careful, manual inspection revealed that some of the test scripts containing the needle `master` do not actually rely on a specific default branch name: either they mention `master` only in a comment, or they initialize that branch specificially, or they do not actually refer to the current default branch. Therefore, the aforementioned modification was undone in those test scripts thusly: $ git checkout HEAD -- \ t/t0027-auto-crlf.sh t/t0060-path-utils.sh \ t/t1011-read-tree-sparse-checkout.sh \ t/t1305-config-include.sh t/t1309-early-config.sh \ t/t1402-check-ref-format.sh t/t1450-fsck.sh \ t/t2024-checkout-dwim.sh \ t/t2106-update-index-assume-unchanged.sh \ t/t3040-subprojects-basic.sh t/t3301-notes.sh \ t/t3308-notes-merge.sh t/t3423-rebase-reword.sh \ t/t3436-rebase-more-options.sh \ t/t4015-diff-whitespace.sh t/t4257-am-interactive.sh \ t/t5323-pack-redundant.sh t/t5401-update-hooks.sh \ t/t5511-refspec.sh t/t5526-fetch-submodules.sh \ t/t5529-push-errors.sh t/t5530-upload-pack-error.sh \ t/t5548-push-porcelain.sh \ t/t5552-skipping-fetch-negotiator.sh \ t/t5572-pull-submodule.sh t/t5608-clone-2gb.sh \ t/t5614-clone-submodules-shallow.sh \ t/t7508-status.sh t/t7606-merge-custom.sh \ t/t9302-fast-import-unpack-limit.sh We excluded one set of test scripts in these commands, though: the range of `git p4` tests. The reason? `git p4` stores the (foreign) remote branch in the branch called `p4/master`, which is obviously not the default branch. Manual analysis revealed that only five of these tests actually require a specific default branch name to pass; They were modified thusly: $ sed -i '/^ *\. \.\/lib-git-p4\.sh$/i\ GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=master\ export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME\ ' t/t980[0167]*.sh t/t9811*.sh Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-10-05Merge branch 'jt/keep-partial-clone-filter-upon-lazy-fetch'Libravatar Junio C Hamano1-1/+2
The lazy fetching done internally to make missing objects available in a partial clone incorrectly made permanent damage to the partial clone filter in the repository, which has been corrected. * jt/keep-partial-clone-filter-upon-lazy-fetch: fetch: do not override partial clone filter promisor-remote: remove unused variable
2020-09-28fetch: do not override partial clone filterLibravatar Jonathan Tan1-1/+2
When a fetch with the --filter argument is made, the configured default filter is set even if one already exists. This change was made in 5e46139376 ("builtin/fetch: remove unique promisor remote limitation", 2019-06-25) - in particular, changing from: * If this is the FIRST partial-fetch request, we enable partial * on this repo and remember the given filter-spec as the default * for subsequent fetches to this remote. to: * If this is a partial-fetch request, we enable partial on * this repo if not already enabled and remember the given * filter-spec as the default for subsequent fetches to this * remote. (The given filter-spec is "remembered" even if there is already an existing one.) This is problematic whenever a lazy fetch is made, because lazy fetches are made using "git fetch --filter=blob:none", but this will also happen if the user invokes "git fetch --filter=<filter>" manually. Therefore, restore the behavior prior to 5e46139376, which writes a filter-spec only if the current fetch request is the first partial-fetch one (for that remote). Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-09-22builtin/clone: avoid failure with GIT_DEFAULT_HASHLibravatar brian m. carlson1-0/+14
If a user is cloning a SHA-1 repository with GIT_DEFAULT_HASH set to "sha256", then we can end up with a repository where the repository format version is 0 but the extensions.objectformat key is set to "sha256". This is both wrong (the user has a SHA-1 repository) and nonfunctional (because the extension cannot be used in a v0 repository). This happens because in a clone, we initially set up the repository, and then change its algorithm based on what the remote side tells us it's using. We've initially set up the repository as SHA-256 in this case, and then later on reset the repository version without clearing the extension. We could just always set the extension in this case, but that would mean that our SHA-1 repositories weren't compatible with older Git versions, even though there's no reason why they shouldn't be. And we also don't want to initialize the repository as SHA-1 initially, since that means if we're cloning an empty repository, we'll have failed to honor the GIT_DEFAULT_HASH variable and will end up with a SHA-1 repository, not a SHA-256 repository. Neither of those are appealing, so let's tell the repository initialization code if we're doing a reinit like this, and if so, to clear the extension if we're using SHA-1. This makes sure we produce a valid and functional repository and doesn't break any of our other use cases. Reported-by: Matheus Tavares <matheus.bernardino@usp.br> Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-08-18promisor-remote: lazy-fetch objects in subprocessLibravatar Jonathan Tan1-1/+1
Teach Git to lazy-fetch missing objects in a subprocess instead of doing it in-process. This allows any fatal errors that occur during the fetch to be isolated and converted into an error return value, instead of causing the current command being run to terminate. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-10git clone: don't clone into non-empty directoryLibravatar Ben Wijen1-1/+3
When using git clone with --separate-git-dir realgitdir and realgitdir already exists, it's content is destroyed. So, make sure we don't clone into an existing non-empty directory. When d45420c1 (clone: do not clean up directories we didn't create, 2018-01-02) tightened the clean-up procedure after a failed cloning into an empty directory, it assumed that the existing directory given is an empty one so it is OK to keep that directory, while running the clean-up procedure that is designed to remove everything in it (since there won't be any, anyway). Check and make sure that the $GIT_DIR is empty even cloning into an existing repository. Signed-off-by: Ben Wijen <ben@wijen.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-15t5601: switch into repository to hash objectLibravatar brian m. carlson1-2/+2
This test performs a clone from outside any repository. Consequently, the hash algorithm used defaults to SHA-1. When the test is running with SHA-256, this results in an object ID that is not usable by the rest of the test. In order to ensure that we provide a usable value, switch into the source repository before hashing. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-09-30Merge branch 'dl/use-sq-from-test-lib'Libravatar Junio C Hamano1-1/+0
Code cleanup. * dl/use-sq-from-test-lib: t: use common $SQ variable
2019-09-18Merge branch 'cc/multi-promisor'Libravatar Junio C Hamano1-1/+2
Teach the lazy clone machinery that there can be more than one promisor remote and consult them in order when downloading missing objects on demand. * cc/multi-promisor: Move core_partial_clone_filter_default to promisor-remote.c Move repository_format_partial_clone to promisor-remote.c Remove fetch-object.{c,h} in favor of promisor-remote.{c,h} remote: add promisor and partial clone config to the doc partial-clone: add multiple remotes in the doc t0410: test fetching from many promisor remotes builtin/fetch: remove unique promisor remote limitation promisor-remote: parse remote.*.partialclonefilter Use promisor_remote_get_direct() and has_promisor_remote() promisor-remote: use repository_format_partial_clone promisor-remote: add promisor_remote_reinit() promisor-remote: implement promisor_remote_get_direct() Add initial support for many promisor remotes fetch-object: make functions return an error code t0410: remove pipes after git commands
2019-09-06t: use common $SQ variableLibravatar Denton Liu1-1/+0
In many test scripts, there are bespoke definitions of the single quote that are some variation of this: SQ="'" Define a common $SQ variable in test-lib.sh and replace all usages of these bespoke variables with the common one. This change was done by running `git grep =\"\'\" t/` and `git grep =\\\\\'` and manually changing the resulting definitions and corresponding usages. Signed-off-by: Denton Liu <liu.denton@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-02t: warn against adding non-httpd-specific tests after sourcing 'lib-httpd'Libravatar SZEDER Gábor1-0/+3
We have a couple of test scripts that are not completely httpd-specific, but do run a few httpd-specific tests at the end. These test scripts source 'lib-httpd.sh' somewhere mid-script, which then skips all the rest of the test script if the dependencies for running httpd tests are not fulfilled. As the previous two patches in this series show, already on two occasions non-httpd-specific tests were appended at the end of such test scripts, and, consequently, they were skipped as well when httpd tests couldn't be run. Add a comment at the end of these test scripts to warn against adding non-httpd-specific tests at the end, in the hope that they will help prevent similar issues in the future. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-06-25promisor-remote: parse remote.*.partialclonefilterLibravatar Christian Couder1-0/+1
This makes it possible to specify a different partial clone filter for each promisor remote. Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-06-25Use promisor_remote_get_direct() and has_promisor_remote()Libravatar Christian Couder1-1/+1
Instead of using the repository_format_partial_clone global and fetch_objects() directly, let's use has_promisor_remote() and promisor_remote_get_direct(). This way all the configured promisor remotes will be taken into account, not only the one specified by extensions.partialClone. Also when cloning or fetching using a partial clone filter, remote.origin.promisor will be set to "true" instead of setting extensions.partialClone to "origin". This makes it possible to use many promisor remote just by fetching from them. Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-05-29clone: respect user supplied origin name when setting up partial cloneLibravatar Xin Li1-3/+13
Signed-off-by: Xin Li <delphij@google.com> Reviewed-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-05-09Merge branch 'tb/unexpected'Libravatar Junio C Hamano1-4/+0
Code tightening against a "wrong" object appearing where an object of a different type is expected, instead of blindly assuming that the connection between objects are correctly made. * tb/unexpected: rev-list: detect broken root trees rev-list: let traversal die when --missing is not in use get_commit_tree(): return NULL for broken tree list-objects.c: handle unexpected non-tree entries list-objects.c: handle unexpected non-blob entries t: introduce tests for unexpected object types t: move 'hex2oct' into test-lib-functions.sh
2019-04-25Merge branch 'sg/test-atexit'Libravatar Junio C Hamano1-2/+0
Test framework update to more robustly clean up leftover files and processes after tests are done. * sg/test-atexit: t9811-git-p4-label-import: fix pipeline negation git p4 test: disable '-x' tracing in the p4d watchdog loop git p4 test: simplify timeout handling git p4 test: clean up the p4d cleanup functions git p4 test: use 'test_atexit' to kill p4d and the watchdog process t0301-credential-cache: use 'test_atexit' to stop the credentials helper tests: use 'test_atexit' to stop httpd git-daemon: use 'test_atexit` to stop 'git-daemon' test-lib: introduce 'test_atexit' t/lib-git-daemon: make sure to kill the 'git-daemon' process test-lib: fix interrupt handling with 'dash' and '--verbose-log -x'
2019-04-05t: move 'hex2oct' into test-lib-functions.shLibravatar Taylor Blau1-4/+0
The helper 'hex2oct' is used to convert base-16 encoded data into a base-8 binary form, and is useful for preparing data for commands that accept input in a binary format, such as 'git hash-object', via 'printf'. This helper is defined identically in three separate places throughout 't'. Move the definition to test-lib-function.sh, so that it can be used in new test suites, and its definition is not redundant. This will likewise make our job easier in the subsequent commit, which also uses 'hex2oct'. Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-14tests: use 'test_atexit' to stop httpdLibravatar SZEDER Gábor1-2/+0
Use 'test_atexit' to run cleanup commands to stop httpd at the end of the test script or upon interrupt or failure, as it is shorter, simpler, and more robust than registering such cleanup commands in the trap on EXIT in the test scripts. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-07t5601: check ssh command only with protocol v0Libravatar Jonathan Tan1-8/+8
When running the SSH command as part of a fetch, Git will write "SendEnv GIT_PROTOCOL" as an option if protocol v1 or v2 is used, but not v0. Update all tests that check this to run Git with GIT_TEST_PROTOCOL_VERSION=0. I chose not to do a more thorough fix (for example, checking the value of GIT_TEST_PROTOCOL_VERSION to see if the SendEnv check needs to be done) because a set of patches [1] that unifies the handling of SSH options, including writing "SendEnv GIT_PROTOCOL" regardless of protocol version, is in progress. When that is done, this patch should be reverted, since the functionality in here is no longer needed. As of this patch, all tests pass if GIT_TEST_PROTOCOL_VERSION is set to 1. [1] https://public-inbox.org/git/cover.1545342797.git.steadmon@google.com/ Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-14Merge branch 'tb/use-common-win32-pathfuncs-on-cygwin'Libravatar Junio C Hamano1-1/+1
Cygwin update. * tb/use-common-win32-pathfuncs-on-cygwin: git clone <url> C:\cygwin\home\USER\repo' is working (again)
2018-12-26git clone <url> C:\cygwin\home\USER\repo' is working (again)Libravatar Torsten Bögershausen1-1/+1
A regression for cygwin users was introduced with commit 05b458c, "real_path: resolve symlinks by hand". In the the commit message we read: The current implementation of real_path uses chdir() in order to resolve symlinks. Unfortunately this isn't thread-safe as chdir() affects a process as a whole... The old (and non-thread-save) OS calls chdir()/pwd() had been replaced by a string operation. The cygwin layer "knows" that "C:\cygwin" is an absolute path, but the new string operation does not. "git clone <url> C:\cygwin\home\USER\repo" fails like this: fatal: Invalid path '/home/USER/repo/C:\cygwin\home\USER\repo' The solution is to implement has_dos_drive_prefix(), skip_dos_drive_prefix() is_dir_sep(), offset_1st_component() and convert_slashes() for cygwin in the same way as it is done in 'Git for Windows' in compat/mingw.[ch] Extract the needed code into compat/win32/path-utils.[ch] and use it for cygwin as well. Reported-by: Steven Penny <svnpenn@gmail.com> Helped-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-11-26Merge branch 'tb/clone-case-smashing-warning-test'Libravatar Junio C Hamano1-1/+1
The code recently added to "git clone" to see if the platform's filesystem is adequate to check out and use the project code correctly (e.g. a case smashing filesystem cannot be used for a project with two files whose paths are different only in case) was meant to help Windows users, but the test for it was not enabled for that platform, which has been corrected. * tb/clone-case-smashing-warning-test: t5601-99: Enable colliding file detection for MINGW
2018-11-24t5601-99: Enable colliding file detection for MINGWLibravatar Torsten Bögershausen1-1/+1
Commit b878579ae7 (clone: report duplicate entries on case-insensitive filesystems - 2018-08-17) adds a warning to user when cloning a repo with case-sensitive file names on a case-insensitive file system. This test has never been enabled for MINGW. It had been working since day 1, but I forget to report that to the author. Enable it after a re-test. Signed-off-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-11-21Merge branch 'nd/clone-case-smashing-warning'Libravatar Junio C Hamano1-1/+1
Recently added check for case smashing filesystems did not correctly utilize the cached stat information, leading to false breakage detected by our test suite, which has been corrected. * nd/clone-case-smashing-warning: clone: fix colliding file detection on APFS
2018-11-21clone: fix colliding file detection on APFSLibravatar Nguyễn Thái Ngọc Duy1-1/+1
Commit b878579ae7 (clone: report duplicate entries on case-insensitive filesystems - 2018-08-17) adds a warning to user when cloning a repo with case-sensitive file names on a case-insensitive file system. The "find duplicate file" check was doing by comparing inode number (and only fall back to fspathcmp() when inode is known to be unreliable because fspathcmp() can't cover all case folding cases). The inode check is very simple, and wrong. It compares between a 32-bit number (sd_ino) and potentially a 64-bit number (st_ino). When an inode is larger than 2^32 (which seems to be the case for APFS), it will be truncated and stored in sd_ino, but comparing with itself will fail. As a result, instead of showing a pair of files that have the same name, we show just one file (marked before the beginning of the loop). We fail to find the original one. The fix could be just a simple type cast (*) dup->ce_stat_data.sd_ino == (unsigned int)st->st_ino but this is no longer a reliable test, there are 4G possible inodes that can match sd_ino because we only match the lower 32 bits instead of full 64 bits. There are two options to go. Either we ignore inode and go with fspathcmp() on Apple platform. This means we can't do accurate inode check on HFS anymore, or even on APFS when inode numbers are still below 2^32. Or we just to to reduce the odds of matching a wrong file by checking more attributes, counting mostly on st_size because st_xtime is likely the same. This patch goes with this direction, hoping that false positive chances are too small to be seen in practice. While at there, enable the test on Cygwin (verified working by Ramsay Jones) (*) this is also already done inside match_stat_data() Reported-by: Carlo Arenas <carenas@gmail.com> Helped-by: Ramsay Jones <ramsay@ramsayjones.plus.com> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-17Merge branch 'nd/clone-case-smashing-warning'Libravatar Junio C Hamano1-1/+7
Running "git clone" against a project that contain two files with pathnames that differ only in cases on a case insensitive filesystem would result in one of the files lost because the underlying filesystem is incapable of holding both at the same time. An attempt is made to detect such a case and warn. * nd/clone-case-smashing-warning: clone: report duplicate entries on case-insensitive filesystems
2018-08-17clone: report duplicate entries on case-insensitive filesystemsLibravatar Duy Nguyen1-1/+7
Paths that only differ in case work fine in a case-sensitive filesystems, but if those repos are cloned in a case-insensitive one, you'll get problems. The first thing to notice is "git status" will never be clean with no indication what exactly is "dirty". This patch helps the situation a bit by pointing out the problem at clone time. Even though this patch talks about case sensitivity, the patch makes no assumption about folding rules by the filesystem. It simply observes that if an entry has been already checked out at clone time when we're about to write a new path, some folding rules are behind this. In the case that we can't rely on filesystem (via inode number) to do this check, fall back to fspathcmp() which is not perfect but should not give false positives. This patch is tested with vim-colorschemes and Sublime-Gitignore repositories on a JFS partition with case insensitive support on Linux. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-16t5000-t5999: fix broken &&-chainsLibravatar Eric Sunshine1-1/+1
Signed-off-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-02-13Merge branch 'jh/partial-clone'Libravatar Junio C Hamano1-0/+101
The machinery to clone & fetch, which in turn involves packing and unpacking objects, have been told how to omit certain objects using the filtering mechanism introduced by the jh/object-filtering topic, and also mark the resulting pack as a promisor pack to tolerate missing objects, taking advantage of the mechanism introduced by the jh/fsck-promisors topic. * jh/partial-clone: t5616: test bulk prefetch after partial fetch fetch: inherit filter-spec from partial clone t5616: end-to-end tests for partial clone fetch-pack: restore save_commit_buffer after use unpack-trees: batch fetching of missing blobs clone: partial clone partial-clone: define partial clone settings in config fetch: support filters fetch: refactor calculation of remote list fetch-pack: test support excluding large blobs fetch-pack: add --no-filter fetch-pack, index-pack, transport: partial clone upload-pack: add object filtering for partial clone
2018-01-21Merge branch 'bc/hash-algo' into maintLibravatar Junio C Hamano1-0/+17
* bc/hash-algo: t5601-clone: test case-conflicting files on case-insensitive filesystem repository: pre-initialize hash algo pointer
2018-01-21t5601-clone: test case-conflicting files on case-insensitive filesystemLibravatar Eric Sunshine1-0/+17
A recently introduced regression caused a segfault at clone time on case-insensitive filesystems when filenames differing only in case are present. This bug has already been fixed (repository: pre-initialize hash algo pointer, 2018-01-18), but it's not the first time similar problems have arisen. Therefore, introduce a test to catch this case and protect against future regressions. Signed-off-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-12-08unpack-trees: batch fetching of missing blobsLibravatar Jonathan Tan1-0/+52
When running checkout, first prefetch all blobs that are to be updated but are missing. This means that only one pack is downloaded during such operations, instead of one per missing blob. This operates only on the blob level - if a repository has a missing tree, they are still fetched one at a time. This does not use the delayed checkout mechanism introduced in commit 2841e8f ("convert: add "status=delayed" to filter process protocol", 2017-06-30) due to significant conceptual differences - in particular, for partial clones, we already know what needs to be fetched based on the contents of the local repo alone, whereas for status=delayed, it is the filter process that tells us what needs to be checked in the end. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-12-08clone: partial cloneLibravatar Jonathan Tan1-0/+49
Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-12-06Merge branch 'jn/ssh-wrappers'Libravatar Junio C Hamano1-24/+46
The ssh-variant 'simple' introduced earlier broke existing installations by not passing --port/-4/-6 and not diagnosing an attempt to pass these as an error. Instead, default to automatically detect how compatible the GIT_SSH/GIT_SSH_COMMAND is to OpenSSH convention and then error out an invocation to make it easier to diagnose connection errors. * jn/ssh-wrappers: connect: correct style of C-style comment ssh: 'simple' variant does not support --port ssh: 'simple' variant does not support -4/-6 ssh: 'auto' variant to select between 'ssh' and 'simple' connect: split ssh option computation to its own function connect: split ssh command line options into separate function connect: split git:// setup into a separate function connect: move no_fork fallback to git_tcp_connect ssh test: make copy_ssh_wrapper_as clean up after itself
2017-12-06Merge branch 'bw/protocol-v1'Libravatar Junio C Hamano1-6/+22
A new mechanism to upgrade the wire protocol in place is proposed and demonstrated that it works with the older versions of Git without harming them. * bw/protocol-v1: Documentation: document Extra Parameters ssh: introduce a 'simple' ssh variant i5700: add interop test for protocol transition http: tell server that the client understands v1 connect: tell server that the client understands v1 connect: teach client to recognize v1 server response upload-pack, receive-pack: introduce protocol version 1 daemon: recognize hidden request arguments protocol: introduce protocol extension mechanisms pkt-line: add packet_write function connect: in ref advertisement, shallows are last
2017-11-21ssh: 'simple' variant does not support --portLibravatar Jonathan Nieder1-2/+8
When trying to connect to an ssh:// URL with port explicitly specified and the ssh command configured with GIT_SSH does not support such a setting, it is less confusing to error out than to silently suppress the port setting and continue. This requires updating the GIT_SSH setting in t5603-clone-dirname.sh. That test is about the directory name produced when cloning various URLs. It uses an ssh wrapper that ignores all its arguments but does not declare that it supports a port argument; update it to set GIT_SSH_VARIANT=ssh to do so. (Real-life ssh wrappers that pass a port argument to OpenSSH would also support -G and would not require such an update.) Reported-by: William Yan <wyan@google.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Acked-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-11-21ssh: 'simple' variant does not support -4/-6Libravatar Jonathan Nieder1-6/+6
If the user passes -4/--ipv4 or -6/--ipv6 to "git fetch" or "git push" and the ssh command configured with GIT_SSH does not support such a setting, error out instead of ignoring the option and continuing. Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Acked-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-11-21ssh: 'auto' variant to select between 'ssh' and 'simple'Libravatar Jonathan Nieder1-0/+21
Android's "repo" tool is a tool for managing a large codebase consisting of multiple smaller repositories, similar to Git's submodule feature. Starting with Git 94b8ae5a (ssh: introduce a 'simple' ssh variant, 2017-10-16), users noticed that it stopped handling the port in ssh:// URLs. The cause: when it encounters ssh:// URLs, repo pre-connects to the server and sets GIT_SSH to a helper ".repo/repo/git_ssh" that reuses that connection. Before 94b8ae5a, the helper was assumed to support OpenSSH options for lack of a better guess and got passed a -p option to set the port. After that patch, it uses the new default of a simple helper that does not accept an option to set the port. The next release of "repo" will set GIT_SSH_VARIANT to "ssh" to avoid that. But users of old versions and of other similar GIT_SSH implementations would not get the benefit of that fix. So update the default to use OpenSSH options again, with a twist. As observed in 94b8ae5a, we cannot assume that $GIT_SSH always handles OpenSSH options: common helpers such as travis-ci's dpl[*] are configured using GIT_SSH and do not accept OpenSSH options. So make the default a new variant "auto", with the following behavior: 1. First, check for a recognized basename, like today. 2. If the basename is not recognized, check whether $GIT_SSH supports OpenSSH options by running $GIT_SSH -G <options> <host> This returns status 0 and prints configuration in OpenSSH if it recognizes all <options> and returns status 255 if it encounters an unrecognized option. A wrapper script like exec ssh -- "$@" would fail with ssh: Could not resolve hostname -g: Name or service not known , correctly reflecting that it does not support OpenSSH options. The command is run with stdin, stdout, and stderr redirected to /dev/null so even a command that expects a terminal would exit immediately. 3. Based on the result from step (2), behave like "ssh" (if it succeeded) or "simple" (if it failed). This way, the default ssh variant for unrecognized commands can handle both the repo and dpl cases as intended. This autodetection has been running on Google workstations since 2017-10-23 with no reported negative effects. [*] https://github.com/travis-ci/dpl/blob/6c3fddfda1f2a85944c544446b068bac0a77c049/lib/dpl/provider.rb#L215 Reported-by: William Yan <wyan@google.com> Improved-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-11-21ssh test: make copy_ssh_wrapper_as clean up after itselfLibravatar Jonathan Nieder1-16/+12
Simplify by not allowing the copied ssh wrapper to persist between tests. This way, tests can be safely reordered, added, and removed with less fear of hidden side effects. This also avoids having to call setup_ssh_wrapper to restore the value of GIT_SSH after this battery of tests, since it means each test will restore it individually. Noticed because on Windows, if `uplink.exe` exists, the MSYS2 Bash will overwrite that when redirecting via `>uplink`. A proposed test wrote a script to 'uplink' after a previous test created uplink.exe using copy_ssh_wrapper_as, so the script written with '>uplink' had the wrong filename and failed. Reported-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Acked-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-10-17t5601: rm the target file of cp that could still be executingLibravatar Junio C Hamano1-0/+2
"while sh t5601-clone.sh; do :; done" seems to fail sporadically at around test #45 where fake-ssh wrapper is copied create plink.exe, with an error message that says the "text is busy". I have a mild suspicion that the root cause of the bug is that the fake SSH process from the previous test is still running by the time the next test wants to replace it with a new binary, but in the meantime, removing the target that could still be executing before copying something else over seems to work it around. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-10-17ssh: introduce a 'simple' ssh variantLibravatar Brandon Williams1-5/+21
When using the 'ssh' transport, the '-o' option is used to specify an environment variable which should be set on the remote end. This allows git to send additional information when contacting the server, requesting the use of a different protocol version via the 'GIT_PROTOCOL' environment variable like so: "-o SendEnv=GIT_PROTOCOL". Unfortunately not all ssh variants support the sending of environment variables to the remote end. To account for this, only use the '-o' option for ssh variants which are OpenSSH compliant. This is done by checking that the basename of the ssh command is 'ssh' or the ssh variant is overridden to be 'ssh' (via the ssh.variant config). Other options like '-p' and '-P', which are used to specify a specific port to use, or '-4' and '-6', which are used to indicate that IPV4 or IPV6 addresses should be used, may also not be supported by all ssh variants. Currently if an ssh command's basename wasn't 'plink' or 'tortoiseplink' git assumes that the command is an OpenSSH variant. Since user configured ssh commands may not be OpenSSH compliant, tighten this constraint and assume a variant of 'simple' if the basename of the command doesn't match the variants known to git. The new ssh variant 'simple' will only have the host and command to execute ([username@]host command) passed as parameters to the ssh command. Update the Documentation to better reflect the command-line options sent to ssh commands based on their variant. Reported-by: Jeffrey Yasskin <jyasskin@google.com> Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-04-16connect.c: handle errors from split_cmdlineLibravatar Jeff King1-0/+6
Commit e9d9a8a4d (connect: handle putty/plink also in GIT_SSH_COMMAND, 2017-01-02) added a call to split_cmdline(), but checks only for a non-zero return to see if we got any output. Since the function returns negative values (and a NULL argv) on error, we end up dereferencing NULL and segfaulting. Arguably we could report on the parsing error here, but it's probably not worth it. This is a best-effort attempt to see if we are using plink. So we can simply return here with "no, it wasn't plink" and let the shell actually complain about the bogus quoting. Reported-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-02-01connect: Add the envvar GIT_SSH_VARIANT and ssh.variant configLibravatar Segev Finer1-0/+26
This environment variable and configuration value allow to override the autodetection of plink/tortoiseplink in case that Git gets it wrong. [jes: wrapped overly-long lines, factored out and changed get_ssh_variant() to handle_ssh_variant() to accomodate the change from the putty/tortoiseplink variables to port_option/needs_batch, adjusted the documentation, free()d value obtained from the config.] Signed-off-by: Segev Finer <segev208@gmail.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-01-25connect: handle putty/plink also in GIT_SSH_COMMANDLibravatar Segev Finer1-0/+15
Git for Windows has special support for the popular SSH client PuTTY: when using PuTTY's non-interactive version ("plink.exe"), we use the -P option to specify the port rather than OpenSSH's -p option. TortoiseGit ships with its own, forked version of plink.exe, that adds support for the -batch option, and for good measure we special-case that, too. However, this special-casing of PuTTY only covers the case where the user overrides the SSH command via the environment variable GIT_SSH (which allows specifying the name of the executable), not GIT_SSH_COMMAND (which allows specifying a full command, including additional command-line options). When users want to pass any additional arguments to (Tortoise-)Plink, such as setting a private key, they are required to either use a shell script named plink or tortoiseplink or duplicate the logic that is already in Git for passing the correct style of command line arguments, which can be difficult, error prone and annoying to get right. This patch simply reuses the existing logic and expands it to cover GIT_SSH_COMMAND, too. Note: it may look a little heavy-handed to duplicate the entire command-line and then split it, only to extract the name of the executable. However, this is not a performance-critical code path, and the code is much more readable this way. Signed-off-by: Segev Finer <segev208@gmail.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-01-07don't use test_must_fail with grepLibravatar Pranit Bauva1-1/+1
test_must_fail should only be used for testing git commands. To test the failure of other commands use `!`. Reported-by: Stefan Beller <sbeller@google.com> Signed-off-by: Pranit Bauva <pranit.bauva@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-17Merge branch 'tb/t5601-sed-fix'Libravatar Junio C Hamano1-1/+1
Test fix. * tb/t5601-sed-fix: t5601: Remove trailing space in sed expression
2016-05-09t5601: Remove trailing space in sed expressionLibravatar Torsten Bögershausen1-1/+1
The sed expression for IPv6, "Tested User And Host" or "tuah" used a wrong sed expression, which doesn't work under all versions of sed. Reported-By: Armin Kunaschik <megabreit@googlemail.com> Signed-off-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-04-15test helpers: move test-* to t/helper/ subdirectoryLibravatar Nguyễn Thái Ngọc Duy1-1/+1
This keeps top dir a bit less crowded. And because these programs are for testing purposes, it makes sense that they stay somewhere in t/ Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>