summaryrefslogtreecommitdiff
path: root/t/t5526-fetch-submodules.sh
AgeCommit message (Collapse)AuthorFilesLines
2019-01-29Merge branch 'sb/submodule-recursive-fetch-gets-the-tip'Libravatar Junio C Hamano1-0/+117
"git fetch --recurse-submodules" may not fetch the necessary commit that is bound to the superproject, which is getting corrected. * sb/submodule-recursive-fetch-gets-the-tip: fetch: ensure submodule objects fetched submodule.c: fetch in submodules git directory instead of in worktree submodule: migrate get_next_submodule to use repository structs repository: repo_submodule_init to take a submodule struct submodule: store OIDs in changed_submodule_names submodule.c: tighten scope of changed_submodule_names struct submodule.c: sort changed_submodule_names before searching it submodule.c: fix indentation sha1-array: provide oid_array_filter
2018-12-14submodule update: run at most one fetch job unless otherwise setLibravatar Junio C Hamano1-0/+2
In a028a1930c (fetching submodules: respect `submodule.fetchJobs` config option, 2016-02-29), we made sure to keep the default behavior of fetching at most one submodule at once when not setting the newly introduced `submodule.fetchJobs` config. This regressed in 90efe595c5 (builtin/submodule--helper: factor out submodule updating, 2018-08-03). Fix it. Reported-by: Sjon Hortensius <sjon@parse.nl> Signed-off-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-12-09fetch: ensure submodule objects fetchedLibravatar Stefan Beller1-0/+117
Currently when git-fetch is asked to recurse into submodules, it dispatches a plain "git-fetch -C <submodule-dir>" (with some submodule related options such as prefix and recusing strategy, but) without any information of the remote or the tip that should be fetched. But this default fetch is not sufficient, as a newly fetched commit in the superproject could point to a commit in the submodule that is not in the default refspec. This is common in workflows like Gerrit's. When fetching a Gerrit change under review (from refs/changes/??), the commits in that change likely point to submodule commits that have not been merged to a branch yet. Fetch a submodule object by id if the object that the superproject points to, cannot be found. For now this object is fetched from the 'origin' remote as we defer getting the default remote to a later patch. A list of new submodule commits are already generated in certain conditions (by check_for_new_submodule_commits()); this new feature invokes that function in more situations. The submodule checks were done only when a ref in the superproject changed, these checks were extended to also be performed when fetching into FETCH_HEAD for completeness, and add a test for that too. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-27Merge branch 'sg/test-must-be-empty'Libravatar Junio C Hamano1-23/+22
Test fixes. * sg/test-must-be-empty: tests: use 'test_must_be_empty' instead of 'test_cmp <empty> <out>' tests: use 'test_must_be_empty' instead of 'test_cmp /dev/null <out>' tests: use 'test_must_be_empty' instead of 'test ! -s' tests: use 'test_must_be_empty' instead of '! test -s'
2018-08-21tests: use 'test_must_be_empty' instead of 'test_cmp <empty> <out>'Libravatar SZEDER Gábor1-2/+1
Using 'test_must_be_empty' is shorter and more idiomatic than >empty && test_cmp empty out as it saves the creation of an empty file. Furthermore, sometimes the expected empty file doesn't have such a descriptive name like 'empty', and its creation is far away from the place where it's finally used for comparison (e.g. in 't7600-merge.sh', where two expected empty files are created in the 'setup' test, but are used only about 500 lines later). These cases were found by instrumenting 'test_cmp' to error out the test script when it's used to compare empty files, and then converted manually. Note that even after this patch there still remain a lot of cases where we use 'test_cmp' to check empty files: - Sometimes the expected output is not hard-coded in the test, but 'test_cmp' is used to ensure that two similar git commands produce the same output, and that output happens to be empty, e.g. the test 'submodule update --merge - ignores --merge for new submodules' in 't7406-submodule-update.sh'. - Repetitive common tasks, including preparing the expected results and running 'test_cmp', are often extracted into a helper function, and some of this helper's callsites expect no output. - For the same reason as above, the whole 'test_expect_success' block is within a helper function, e.g. in 't3070-wildmatch.sh'. - Or 'test_cmp' is invoked in a loop, e.g. the test 'cvs update (-p)' in 't9400-git-cvsserver-server.sh'. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-21tests: use 'test_must_be_empty' instead of '! test -s'Libravatar SZEDER Gábor1-21/+21
Using 'test_must_be_empty' is preferable to '! test -s', because it gives a helpful error message if the given file is unexpectedly not empty, while the latter remains completely silent. Furthermore, it also catches cases when the given file unexpectedly does not exist at all. This patch was basically created by: sed -i -e 's/! test -s/test_must_be_empty/' t[0-9]*.sh with the following notable exceptions: - The '! test -s' check in '.gitmodules ignore=dirty suppresses submodules with untracked content' in 't7508-status.sh' is left as-is, because it's bogus and, therefore, it's subject of a dedicated patch. - The '! test -s' checks in 't9131-git-svn-empty-symlink.sh' and 't9135-git-svn-moved-branch-empty-file.sh' are immediately preceeded by a 'test -f' to ensure that the files exist in the first place. 'test_must_be_empty' ensures that as well, so those 'test -f' commands are removed as well. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-16t5000-t5999: fix broken &&-chainsLibravatar Eric Sunshine1-1/+1
Signed-off-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-06-14t5526: test recursive submodules when fetching moved submodulesLibravatar Stefan Beller1-5/+1
The topic merged in 0c7ecb7c311 (Merge branch 'sb/submodule-move-nested', 2018-05-08) provided support for moving nested submodules. Remove the NEEDSWORK comment and implement the nested submodules test as the comment hinted at. Signed-off-by: Stefan Beller <sbeller@google.com> Acked-by: Heiko Voigt <hvoigt@hvoigt.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-03-14Merge branch 'sg/test-x'Libravatar Junio C Hamano1-1/+1
Running test scripts under -x option of the shell is often not a useful way to debug them, because the error messages from the commands tests try to capture and inspect are contaminated by the tracing output by the shell. An earlier work done to make it more pleasant to run tests under -x with recent versions of bash is extended to cover posix shells that do not support BASH_XTRACEFD. * sg/test-x: travis-ci: run tests with '-x' tracing t/README: add a note about don't saving stderr of compound commands t1510-repo-setup: mark as untraceable with '-x' t9903-bash-prompt: don't check the stderr of __git_ps1() t5570-git-daemon: don't check the stderr of a subshell t5526: use $TRASH_DIRECTORY to specify the path of GIT_TRACE log file t5500-fetch-pack: don't check the stderr of a subshell t3030-merge-recursive: don't check the stderr of a subshell t1507-rev-parse-upstream: don't check the stderr of a shell function t: add means to disable '-x' tracing for individual test scripts t: prevent '-x' tracing from interfering with test helpers' stderr
2018-02-28t5526: use $TRASH_DIRECTORY to specify the path of GIT_TRACE log fileLibravatar SZEDER Gábor1-1/+1
The test 'fetch --recurse-submodules -j2 has the same output behaviour' in 't5526-fetch-submodules.sh' fails when the test script is run with '-x' tracing (and using a shell other than a Bash version supporting BASH_XTRACEFD). The reason of that failure is the following command: GIT_TRACE=$(pwd)/../trace.out git fetch <...> 2>../actual.err because the trace of executing 'pwd' in the command substitution ends up in 'actual.err' as well, throwing off the subsequent 'test_i18ncmp'. Use $TRASH_DIRECTORY to specify the path of the GIT_TRACE log file instead of $(pwd), so the command's stderr remains free from tracing output. After this change t5526 passes with '-x', even when running with /bin/sh. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-02-14Correct mispellings of ".gitmodule" to ".gitmodules"Libravatar Robert P. J. Day1-1/+1
There are a small number of misspellings, ".gitmodule", scattered throughout the code base, correct them ... no apparent functional changes. Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-10-17implement fetching of moved submodulesLibravatar Heiko Voigt1-0/+35
We store the changed submodules paths to calculate which submodule needs fetching. This does not work for moved submodules since their paths do not stay the same in case of a moved submodules. In case of new submodules we do not have a path in the current checkout, since they just appeared in this fetch. It is more general to collect the submodule names for changes instead of their paths to include the above cases. If we do not have a configuration for a gitlink we rely on constructing a default name from the path if a git repository can be found at its path. We skip non-configured gitlinks whose default name collides with a configured one. With the change described above we implement 'on-demand' fetching of changes in moved submodules. Signed-off-by: Heiko Voigt <hvoigt@hvoigt.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-10-08fetch: add test to make sure we stay backwards compatibleLibravatar Heiko Voigt1-1/+41
The current implementation of submodules supports on-demand fetch if there is no .gitmodules entry for a submodule. Let's add a test to document this behavior. Signed-off-by: Heiko Voigt <hvoigt@hvoigt.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-08-23Merge branch 'hv/t5526-andand-chain-fix'Libravatar Junio C Hamano1-4/+4
Test fix. * hv/t5526-andand-chain-fix: t5526: fix some broken && chains
2017-08-17t5526: fix some broken && chainsLibravatar Heiko Voigt1-4/+4
Signed-off-by: Heiko Voigt <hvoigt@hvoigt.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-01builtin/fetch.c: respect 'submodule.recurse' optionLibravatar Stefan Beller1-0/+10
Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-09-01submodule: avoid auto-discovery in prepare_submodule_repo_env()Libravatar Junio C Hamano1-0/+35
The function is used to set up the environment variable used in a subprocess we spawn in a submodule directory. The callers set up a child_process structure, find the working tree path of one submodule and set .dir field to it, and then use start_command() API to spawn the subprocess like "status", "fetch", etc. When this happens, we expect that the ".git" (either a directory or a gitfile that points at the real location) in the current working directory of the subprocess MUST be the repository for the submodule. If this ".git" thing is a corrupt repository, however, because prepare_submodule_repo_env() unsets GIT_DIR and GIT_WORK_TREE, the subprocess will see ".git", thinks it is not a repository, and attempt to find one by going up, likely to end up in finding the repository of the superproject. In some codepaths, this will cause a command run with the "--recurse-submodules" option to recurse forever. By exporting GIT_DIR=.git, disable the auto-discovery logic in the subprocess, which would instead stop it and report an error. The test illustrates existing problems in a few callsites of this function. Without this fix, "git fetch --recurse-submodules", "git status" and "git diff" keep recursing forever. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-01fetching submodules: respect `submodule.fetchJobs` config optionLibravatar Stefan Beller1-0/+14
This allows to configure fetching and updating in parallel without having the command line option. This moved the responsibility to determine how many parallel processes to start from builtin/fetch to submodule.c as we need a way to communicate "The user did not specify the number of parallel processes in the command line options" in the builtin fetch. The submodule code takes care of the precedence (CLI > config > default). Reviewed-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-12-16submodules: allow parallel fetching, add tests and documentationLibravatar Stefan Beller1-0/+20
This enables the work of the previous patches. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-12-16submodule.c: write "Fetching submodule <foo>" to stderrLibravatar Jonathan Nieder1-25/+26
The "Pushing submodule <foo>" progress output correctly goes to stderr, but "Fetching submodule <foo>" is going to stdout by mistake. Fix it to write to stderr. Noticed while trying to implement a parallel submodule fetch. When this particular output line went to a different file descriptor, it was buffered separately, resulting in wrongly interleaved output if we copied it to the terminal naively. Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-03-20t: fix trivial &&-chain breakageLibravatar Jeff King1-5/+5
These are tests which are missing a link in their &&-chain, but during a setup phase. We may fail to notice failure in commands that build the test environment, but these are typically not expected to fail at all (but it's still good to double-check that our test environment is what we expect). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-06-20fetch: Also fetch submodules in subdirectories in on-demand modeLibravatar Jens Lehmann1-8/+8
When on-demand mode was active examining the new commits just fetched in the superproject (to check if they record commits for submodules which are not downloaded yet) wasn't done recursively. Because of that fetch did not recursively fetch submodules living in subdirectories even when it should have. Fix that by adding the RECURSIVE flag to the diff_options used to check the new commits and avoid future regressions in this area by moving a submodule in t5526 into a subdirectory. Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-04-12i18n: do not overuse C_LOCALE_OUTPUTLibravatar Junio C Hamano1-57/+36
It is too coarse-grained way that led to artificial splitting of a logically single test case into "do" and "check only without poison". As the majority of check is done by comparing expected and actual output stored in a file with test_cmp anyway, just introduce test_i18ncmp that pretends the actual output matched the expected one when gettext-poison is in effect. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-04-04Merge branch 'jl/submodule-fetch-on-demand'Libravatar Junio C Hamano1-0/+255
* jl/submodule-fetch-on-demand: fetch/pull: Describe --recurse-submodule restrictions in the BUGS section submodule update: Don't fetch when the submodule commit is already present fetch/pull: Don't recurse into a submodule when commits are already present Submodules: Add 'on-demand' value for the 'fetchRecurseSubmodule' option config: teach the fetch.recurseSubmodules option the 'on-demand' value fetch/pull: Add the 'on-demand' value to the --recurse-submodules option fetch/pull: recurse into submodules when necessary Conflicts: builtin/fetch.c submodule.c
2011-03-09i18n: git-fetch formatting messagesLibravatar Ævar Arnfjörð Bjarmason1-8/+32
Translate some of the formatting messages that appear on git-fetch showing how branches/tags etc. were updated. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-03-09fetch/pull: Don't recurse into a submodule when commits are already presentLibravatar Jens Lehmann1-0/+19
When looking for submodules where new commits have been recorded in the superproject ignore those cases where the submodules commits are already present locally. This can happen e.g. when the submodule has been rewound to an earlier state. Then there is no need to fetch the submodule again as the commit recorded in the newly fetched superproject commit has already been fetched earlier into the submodule. Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-03-09Submodules: Add 'on-demand' value for the 'fetchRecurseSubmodule' optionLibravatar Jens Lehmann1-0/+28
Now the behavior of fetch and pull can be configured to the recently added 'on-demand' mode separately for each submodule too. Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-03-09config: teach the fetch.recurseSubmodules option the 'on-demand' valueLibravatar Jens Lehmann1-0/+28
To enable the user to change the default behavior of "git fetch" and "git pull" regarding submodule recursion add the new "on-demand" value which has just been added to the "--recurse-submodules" command line option. Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-03-09fetch/pull: Add the 'on-demand' value to the --recurse-submodules optionLibravatar Jens Lehmann1-0/+71
Until now the --recurse-submodules option could only be used to either fetch all populated submodules recursively or to disable recursion completely. As fetch and pull now by default just fetch those submodules for which new commits have been fetched in the superproject, a command line option to enforce that behavior is needed to be able to override configuration settings. Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-03-09fetch/pull: recurse into submodules when necessaryLibravatar Jens Lehmann1-0/+109
To be able to access all commits of populated submodules referenced by the superproject it is sufficient to only then let "git fetch" recurse into a submodule when the new commits fetched in the superproject record new commits for it. Having these commits present is extremely useful when using the "--submodule" option to "git diff" (which is what "git gui" and "gitk" do since 1.6.6), as all submodule commits needed for creating a descriptive output can be accessed. Also merging submodule commits (added in 1.7.3) depends on the submodule commits in question being present to work. Last but not least this enables disconnected operation when using submodules, as all commits necessary for a successful "git submodule update -N" will have been fetched automatically. So we choose this mode as the default for fetch and pull. Before a new or changed ref from upstream is updated in update_local_ref() "git rev-list <new-sha1> --not --branches --remotes" is used to determine all newly fetched commits. These are then walked and diffed against their parent(s) to see if a submodule has been changed. If that is the case, its path is stored to be fetched after the superproject fetch is completed. Using the "--recurse-submodules" or the "--no-recurse-submodules" option disables the examination of the fetched refs because the result will be ignored anyway. There is currently no infrastructure for storing deleted and new submodules in the .git directory of the superproject. That's why fetch and pull for now only fetch submodules that are already checked out and are not renamed. In t7403 the "--no-recurse-submodules" argument had to be added to "git pull" to avoid failure because of the moved upstream submodule repo. Thanks-to: Jonathan Nieder <jrnieder@gmail.com> Thanks-to: Heiko Voigt <hvoigt@hvoigt.net> Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-01-31t5526: Fix wrong argument order in "git config"Libravatar Jens Lehmann1-1/+1
This fixes a typo where the "git config" arguments "-f" and "--unset" were swapped leading to the creation of a "--unset" file. Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-11-12Submodules: Add the "fetchRecurseSubmodules" config optionLibravatar Jens Lehmann1-0/+50
The new boolean "fetchRecurseSubmodules" config option controls the behavior for "git fetch" and "git pull". It specifies if these commands should recurse into submodules and fetch new commits there too and can be set separately for each submodule. In the .gitmodules file "submodule.<name>.fetchRecurseSubmodules" entries are read before looking for them in .git/config. Thus settings found in .git/config will override those from .gitmodules, thereby allowing the user to ignore settings given by the remote side while also letting upstream set reasonable defaults for those users who don't have special needs. This configuration can be overridden by the command line option "--[no-]recurse-submodules" of "git fetch" and "git pull". Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-11-12Add the 'fetch.recurseSubmodules' config settingLibravatar Jens Lehmann1-0/+36
This new boolean option can be used to override the default for "git fetch" and "git pull", which is to not recurse into populated submodules and fetch all new commits there too. Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-11-12fetch/pull: Add the --recurse-submodules optionLibravatar Jens Lehmann1-0/+109
Until now you had to call "git submodule update" (without -N|--no-fetch option) or something like "git submodule foreach git fetch" to fetch new commits in populated submodules from their remote. This could lead to "(commits not present)" messages in the output of "git diff --submodule" (which is used by "git gui" and "gitk") after fetching or pulling new commits in the superproject and is an obstacle for implementing recursive checkout of submodules. Also "git submodule update" cannot fetch changes when disconnected, so it was very easy to forget to fetch the submodule changes before disconnecting only to discover later that they are needed. This patch adds the "--recurse-submodules" option to recursively fetch each populated submodule from the url configured in the .git/config of the submodule at the end of each "git fetch" or during "git pull" in the superproject. The submodule paths are taken from the index. The hidden option "--submodule-prefix" is added to "git fetch" to be able to print out the full paths of nested submodules. Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>