summaryrefslogtreecommitdiff
path: root/git-submodule.sh
AgeCommit message (Collapse)AuthorFilesLines
2020-06-02submodule: port subcommand 'set-branch' from shell to CLibravatar Shourya Shukla1-29/+3
Convert submodule subcommand 'set-branch' to a builtin and call it via 'git-submodule.sh'. Mentored-by: Christian Couder <chriscool@tuxfamily.org> Mentored-by: Kaartic Sivaraam <kaartic.sivaraam@gmail.com> Helped-by: Denton Liu <liu.denton@gmail.com> Helped-by: Eric Sunshine <sunshine@sunshineco.com> Helped-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Shourya Shukla <shouryashukla.oo@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-05-08submodule: port subcommand 'set-url' from shell to CLibravatar Shourya Shukla1-21/+1
Convert submodule subcommand 'set-url' to a builtin. Port 'set-url' to 'submodule--helper.c' and call the latter via 'git-submodule.sh'. Signed-off-by: Shourya Shukla <shouryashukla.oo@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-28Merge branch 'lx/submodule-clear-variables'Libravatar Junio C Hamano1-0/+2
The "git submodule" command did not initialize a few variables it internally uses and was affected by variable settings leaked from the environment. * lx/submodule-clear-variables: git-submodule.sh: setup uninitialized variables
2020-04-02git-submodule.sh: setup uninitialized variablesLibravatar Li Xuejiang1-0/+2
We have an environment variable `jobs=16` defined in our CI system, and this environment makes our build job failed with the following message: error: pathspec '16' did not match any file(s) known to git The pathspec '16' for Git command is from the environment variable "jobs". This is because "git-submodule" command is implemented in shell script, and environment variables may change its behavior. Set values for uninitialized variables, such as "jobs" and "recommend_shallow" will fix this issue. Helped-by: Jiang Xin <worldhello.net@gmail.com> Signed-off-by: Li Xuejiang <xuejiang@alibaba-inc.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-03-05Merge branch 'es/recursive-single-branch-clone'Libravatar Junio C Hamano1-1/+9
"git clone --recurse-submodules --single-branch" now uses the same single-branch option when cloning the submodules. * es/recursive-single-branch-clone: clone: pass --single-branch during --recurse-submodules submodule--helper: use C99 named initializer
2020-02-25clone: pass --single-branch during --recurse-submodulesLibravatar Emily Shaffer1-1/+9
Previously, performing "git clone --recurse-submodules --single-branch" resulted in submodules cloning all branches even though the superproject cloned only one branch. Pipe --single-branch through the submodule helper framework to make it to 'clone' later on. Signed-off-by: Emily Shaffer <emilyshaffer@google.com> Acked-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-15submodule add: show 'add --dry-run' stderr when abortingLibravatar Kyle Meyer1-6/+8
Unless --force is specified, 'submodule add' checks if the destination path is ignored by calling 'git add --dry-run --ignore-missing', and, if that call fails, aborts with a custom "path is ignored" message (a slight variant of what 'git add' shows). Aborting early rather than letting the downstream 'git add' call fail is done so that the command exits before cloning into the destination path. However, in rare cases where the dry-run call fails for a reason other than the path being ignored---for example, due to a preexisting index.lock file---displaying the "ignored path" error message hides the real source of the failure. Instead of displaying the tailored "ignored path" message, let's report the standard error from the dry run to give the caller more accurate information about failures that are not due to an ignored path. For the ignored path case, this leads to the following change in the error message: The following [-path is-]{+paths are+} ignored by one of your .gitignore files: <destination path> Use -f if you really want to add [-it.-]{+them.+} The new phrasing is a bit awkward, because 'submodule add' is only dealing with one destination path. Alternatively, we could continue to use the tailored message when the exit code is 1 (the expected status for a failure due to an ignored path) and relay the standard error for all other non-zero exits. That, however, risks hiding the message of unrelated failures that share an exit code of 1, so it doesn't seem worth doing just to avoid a clunkier, but still clear, error message. Signed-off-by: Kyle Meyer <kyle@kyleam.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-10Merge branch 'dl/submodule-set-url'Libravatar Junio C Hamano1-1/+51
"git submodule" learned a subcommand "set-url". * dl/submodule-set-url: submodule: teach set-url subcommand
2019-12-06Sync with 2.21.1Libravatar Johannes Schindelin1-0/+6
* maint-2.21: (42 commits) Git 2.21.1 mingw: sh arguments need quoting in more circumstances mingw: fix quoting of empty arguments for `sh` mingw: use MSYS2 quoting even when spawning shell scripts mingw: detect when MSYS2's sh is to be spawned more robustly t7415: drop v2.20.x-specific work-around Git 2.20.2 t7415: adjust test for dubiously-nested submodule gitdirs for v2.20.x Git 2.19.3 Git 2.18.2 Git 2.17.3 Git 2.16.6 test-drop-caches: use `has_dos_drive_prefix()` Git 2.15.4 Git 2.14.6 mingw: handle `subst`-ed "DOS drives" mingw: refuse to access paths with trailing spaces or periods mingw: refuse to access paths with illegal characters unpack-trees: let merged_entry() pass through do_add_entry()'s errors quote-stress-test: offer to test quoting arguments for MSYS2 sh ...
2019-12-06Sync with 2.20.2Libravatar Johannes Schindelin1-0/+6
* maint-2.20: (36 commits) Git 2.20.2 t7415: adjust test for dubiously-nested submodule gitdirs for v2.20.x Git 2.19.3 Git 2.18.2 Git 2.17.3 Git 2.16.6 test-drop-caches: use `has_dos_drive_prefix()` Git 2.15.4 Git 2.14.6 mingw: handle `subst`-ed "DOS drives" mingw: refuse to access paths with trailing spaces or periods mingw: refuse to access paths with illegal characters unpack-trees: let merged_entry() pass through do_add_entry()'s errors quote-stress-test: offer to test quoting arguments for MSYS2 sh t6130/t9350: prepare for stringent Win32 path validation quote-stress-test: allow skipping some trials quote-stress-test: accept arguments to test via the command-line tests: add a helper to stress test argument quoting mingw: fix quoting of arguments Disallow dubiously-nested submodule git directories ...
2019-12-06Sync with 2.19.3Libravatar Johannes Schindelin1-0/+6
* maint-2.19: (34 commits) Git 2.19.3 Git 2.18.2 Git 2.17.3 Git 2.16.6 test-drop-caches: use `has_dos_drive_prefix()` Git 2.15.4 Git 2.14.6 mingw: handle `subst`-ed "DOS drives" mingw: refuse to access paths with trailing spaces or periods mingw: refuse to access paths with illegal characters unpack-trees: let merged_entry() pass through do_add_entry()'s errors quote-stress-test: offer to test quoting arguments for MSYS2 sh t6130/t9350: prepare for stringent Win32 path validation quote-stress-test: allow skipping some trials quote-stress-test: accept arguments to test via the command-line tests: add a helper to stress test argument quoting mingw: fix quoting of arguments Disallow dubiously-nested submodule git directories protect_ntfs: turn on NTFS protection by default path: also guard `.gitmodules` against NTFS Alternate Data Streams ...
2019-12-06Sync with 2.18.2Libravatar Johannes Schindelin1-0/+6
* maint-2.18: (33 commits) Git 2.18.2 Git 2.17.3 Git 2.16.6 test-drop-caches: use `has_dos_drive_prefix()` Git 2.15.4 Git 2.14.6 mingw: handle `subst`-ed "DOS drives" mingw: refuse to access paths with trailing spaces or periods mingw: refuse to access paths with illegal characters unpack-trees: let merged_entry() pass through do_add_entry()'s errors quote-stress-test: offer to test quoting arguments for MSYS2 sh t6130/t9350: prepare for stringent Win32 path validation quote-stress-test: allow skipping some trials quote-stress-test: accept arguments to test via the command-line tests: add a helper to stress test argument quoting mingw: fix quoting of arguments Disallow dubiously-nested submodule git directories protect_ntfs: turn on NTFS protection by default path: also guard `.gitmodules` against NTFS Alternate Data Streams is_ntfs_dotgit(): speed it up ...
2019-12-06Sync with 2.17.3Libravatar Johannes Schindelin1-0/+6
* maint-2.17: (32 commits) Git 2.17.3 Git 2.16.6 test-drop-caches: use `has_dos_drive_prefix()` Git 2.15.4 Git 2.14.6 mingw: handle `subst`-ed "DOS drives" mingw: refuse to access paths with trailing spaces or periods mingw: refuse to access paths with illegal characters unpack-trees: let merged_entry() pass through do_add_entry()'s errors quote-stress-test: offer to test quoting arguments for MSYS2 sh t6130/t9350: prepare for stringent Win32 path validation quote-stress-test: allow skipping some trials quote-stress-test: accept arguments to test via the command-line tests: add a helper to stress test argument quoting mingw: fix quoting of arguments Disallow dubiously-nested submodule git directories protect_ntfs: turn on NTFS protection by default path: also guard `.gitmodules` against NTFS Alternate Data Streams is_ntfs_dotgit(): speed it up mingw: disallow backslash characters in tree objects' file names ...
2019-12-06Sync with 2.16.6Libravatar Johannes Schindelin1-0/+6
* maint-2.16: (31 commits) Git 2.16.6 test-drop-caches: use `has_dos_drive_prefix()` Git 2.15.4 Git 2.14.6 mingw: handle `subst`-ed "DOS drives" mingw: refuse to access paths with trailing spaces or periods mingw: refuse to access paths with illegal characters unpack-trees: let merged_entry() pass through do_add_entry()'s errors quote-stress-test: offer to test quoting arguments for MSYS2 sh t6130/t9350: prepare for stringent Win32 path validation quote-stress-test: allow skipping some trials quote-stress-test: accept arguments to test via the command-line tests: add a helper to stress test argument quoting mingw: fix quoting of arguments Disallow dubiously-nested submodule git directories protect_ntfs: turn on NTFS protection by default path: also guard `.gitmodules` against NTFS Alternate Data Streams is_ntfs_dotgit(): speed it up mingw: disallow backslash characters in tree objects' file names path: safeguard `.git` against NTFS Alternate Streams Accesses ...
2019-12-06Sync with 2.15.4Libravatar Johannes Schindelin1-0/+6
* maint-2.15: (29 commits) Git 2.15.4 Git 2.14.6 mingw: handle `subst`-ed "DOS drives" mingw: refuse to access paths with trailing spaces or periods mingw: refuse to access paths with illegal characters unpack-trees: let merged_entry() pass through do_add_entry()'s errors quote-stress-test: offer to test quoting arguments for MSYS2 sh t6130/t9350: prepare for stringent Win32 path validation quote-stress-test: allow skipping some trials quote-stress-test: accept arguments to test via the command-line tests: add a helper to stress test argument quoting mingw: fix quoting of arguments Disallow dubiously-nested submodule git directories protect_ntfs: turn on NTFS protection by default path: also guard `.gitmodules` against NTFS Alternate Data Streams is_ntfs_dotgit(): speed it up mingw: disallow backslash characters in tree objects' file names path: safeguard `.git` against NTFS Alternate Streams Accesses clone --recurse-submodules: prevent name squatting on Windows is_ntfs_dotgit(): only verify the leading segment ...
2019-12-04clone --recurse-submodules: prevent name squatting on WindowsLibravatar Johannes Schindelin1-0/+6
In addition to preventing `.git` from being tracked by Git, on Windows we also have to prevent `git~1` from being tracked, as the default NTFS short name (also known as the "8.3 filename") for the file name `.git` is `git~1`, otherwise it would be possible for malicious repositories to write directly into the `.git/` directory, e.g. a `post-checkout` hook that would then be executed _during_ a recursive clone. When we implemented appropriate protections in 2b4c6efc821 (read-cache: optionally disallow NTFS .git variants, 2014-12-16), we had analyzed carefully that the `.git` directory or file would be guaranteed to be the first directory entry to be written. Otherwise it would be possible e.g. for a file named `..git` to be assigned the short name `git~1` and subsequently, the short name generated for `.git` would be `git~2`. Or `git~3`. Or even `~9999999` (for a detailed explanation of the lengths we have to go to protect `.gitmodules`, see the commit message of e7cb0b4455c (is_ntfs_dotgit: match other .git files, 2018-05-11)). However, by exploiting two issues (that will be addressed in a related patch series close by), it is currently possible to clone a submodule into a non-empty directory: - On Windows, file names cannot end in a space or a period (for historical reasons: the period separating the base name from the file extension was not actually written to disk, and the base name/file extension was space-padded to the full 8/3 characters, respectively). Helpfully, when creating a directory under the name, say, `sub.`, that trailing period is trimmed automatically and the actual name on disk is `sub`. This means that while Git thinks that the submodule names `sub` and `sub.` are different, they both access `.git/modules/sub/`. - While the backslash character is a valid file name character on Linux, it is not so on Windows. As Git tries to be cross-platform, it therefore allows backslash characters in the file names stored in tree objects. Which means that it is totally possible that a submodule `c` sits next to a file `c\..git`, and on Windows, during recursive clone a file called `..git` will be written into `c/`, of course _before_ the submodule is cloned. Note that the actual exploit is not quite as simple as having a submodule `c` next to a file `c\..git`, as we have to make sure that the directory `.git/modules/b` already exists when the submodule is checked out, otherwise a different code path is taken in `module_clone()` that does _not_ allow a non-empty submodule directory to exist already. Even if we will address both issues nearby (the next commit will disallow backslash characters in tree entries' file names on Windows, and another patch will disallow creating directories/files with trailing spaces or periods), it is a wise idea to defend in depth against this sort of attack vector: when submodules are cloned recursively, we now _require_ the directory to be empty, addressing CVE-2019-1349. Note: the code path we patch is shared with the code path of `git submodule update --init`, which must not expect, in general, that the directory is empty. Hence we have to introduce the new option `--force-init` and hand it all the way down from `git submodule` to the actual `git submodule--helper` process that performs the initial clone. Reported-by: Nicolas Joly <Nicolas.Joly@microsoft.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2019-10-30submodule: teach set-url subcommandLibravatar Denton Liu1-1/+51
Currently, in the event that a submodule's upstream URL changes, users have to manually alter the URL in the .gitmodules file then run `git submodule sync`. Let's make that process easier. Teach submodule the set-url subcommand which will automatically change the `submodule.$name.url` property in the .gitmodules file and then run `git submodule sync` to complete the process. Signed-off-by: Denton Liu <liu.denton@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-05-09Merge branch 'km/empty-repo-is-still-a-repo'Libravatar Junio C Hamano1-0/+7
Running "git add" on a repository created inside the current repository is an explicit indication that the user wants to add it as a submodule, but when the HEAD of the inner repository is on an unborn branch, it cannot be added as a submodule. Worse, the files in its working tree can be added as if they are a part of the outer repository, which is not what the user wants. These problems are being addressed. * km/empty-repo-is-still-a-repo: add: error appropriately on repository with no commits dir: do not traverse repositories with no commits submodule: refuse to add repository with no commits
2019-04-25Merge branch 'nd/submodule-foreach-quiet'Libravatar Junio C Hamano1-5/+6
"git submodule foreach <command> --quiet" did not pass the option down correctly, which has been corrected. * nd/submodule-foreach-quiet: submodule foreach: fix "<command> --quiet" not being respected
2019-04-25Merge branch 'dl/submodule-set-branch'Libravatar Junio C Hamano1-4/+71
"git submodule" learns "set-branch" subcommand that allows the submodule.*.branch settings to be modified. * dl/submodule-set-branch: submodule: teach set-branch subcommand submodule--helper: teach config subcommand --unset git-submodule.txt: "--branch <branch>" option defaults to 'master'
2019-04-15submodule foreach: fix "<command> --quiet" not being respectedLibravatar Nguyễn Thái Ngọc Duy1-5/+6
Robin reported that git submodule foreach --quiet git pull --quiet origin is not really quiet anymore [1]. "git pull" behaves as if --quiet is not given. This happens because parseopt in submodule--helper will try to parse both --quiet options as if they are foreach's options, not git-pull's. The parsed options are removed from the command line. So when we do pull later, we execute just this git pull origin When calling submodule helper, adding "--" in front of "git pull" will stop parseopt for parsing options that do not really belong to submodule--helper foreach. PARSE_OPT_KEEP_UNKNOWN is removed as a safety measure. parseopt should never see unknown options or something has gone wrong. There are also a couple usage string update while I'm looking at them. While at it, I also add "--" to other subcommands that pass "$@" to submodule--helper. "$@" in these cases are paths and less likely to be --something-like-this. But the point still stands, git-submodule has parsed and classified what are options, what are paths. submodule--helper should never consider paths passed by git-submodule to be options even if they look like one. The test case is also contributed by Robin. [1] it should be quiet before fc1b9243cd (submodule: port submodule subcommand 'foreach' from shell to C, 2018-05-10) because parseopt can't accidentally eat options then. Reported-by: Robin H. Johnson <robbat2@gentoo.org> Tested-by: Robin H. Johnson <robbat2@gentoo.org> Signed-off-by: Robin H. Johnson <robbat2@gentoo.org> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-10submodule: refuse to add repository with no commitsLibravatar Kyle Meyer1-0/+7
When the path given to 'git submodule add' is an existing repository that is not in the index, the repository is passed to 'git add'. If this repository doesn't have a commit checked out, we don't get a useful result: there is no subproject OID to track, and any untracked files in the sub-repository are added as blobs in the top-level repository. To avoid getting into this state, abort if the path is a repository that doesn't have a commit checked out. Note that this check must come before the 'git add --dry-run' check because the next commit will make 'git add' fail when given a repository that doesn't have a commit checked out. Signed-off-by: Kyle Meyer <kyle@kyleam.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-10submodule: teach set-branch subcommandLibravatar Denton Liu1-4/+71
This teaches git-submodule the set-branch subcommand which allows the branch of a submodule to be set through a porcelain command without having to manually manipulate the .gitmodules file. Signed-off-by: Denton Liu <liu.denton@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-04-10Merge branch 'jt/submodule-fetch-errmsg'Libravatar Junio C Hamano1-1/+1
Error message update. * jt/submodule-fetch-errmsg: submodule: explain first attempt failure clearly
2019-03-14submodule: explain first attempt failure clearlyLibravatar Jonathan Tan1-1/+1
When cloning with --recurse-submodules a superproject with at least one submodule with HEAD pointing to an unborn branch, the clone goes something like this: Cloning into 'test'... <messages about cloning of superproject> Submodule '<name>' (<uri>) registered for path '<submodule path>' Cloning into '<submodule path>'... fatal: Couldn't find remote ref HEAD Unable to fetch in submodule path '<submodule path>' <messages about fetching with SHA-1> From <uri> * branch <hash> -> FETCH_HEAD Submodule path '<submodule path>': checked out '<hash>' In other words, first, a fetch is done with no hash arguments (that is, a fetch of HEAD) resulting in a "Couldn't find remote ref HEAD" error; then, a fetch is done given a hash, which succeeds. The fetch given a hash was added in fb43e31f2b ("submodule: try harder to fetch needed sha1 by direct fetching sha1", 2016-02-24), and the "Unable to fetch..." message was downgraded from a fatal error to a notice in e30d833671 ("git-submodule.sh: try harder to fetch a submodule", 2018-05-16). This commit improves the notice to be clearer that we are retrying the fetch, and that the previous messages (in particular, the fatal errors from fetch) do not necessarily indicate that the whole command fails. In other words: - If the HEAD-fetch succeeds and we then have the commit we want, git-submodule prints no explanation. - If the HEAD-fetch succeeds and we do not have the commit we want, but the hash-fetch succeeds, git-submodule prints no explanation. - If the HEAD-fetch succeeds and we do not have the commit we want, but the hash-fetch fails, git-submodule prints a fatal error. - If the HEAD-fetch fails, fetch prints a fatal error, and git-submodule informs the user that it will retry by fetching specific commits by hash. - If the hash-fetch then succeeds, git-submodule prints no explanation (besides the ones already printed). - If the HEAD-fetch then fails, git-submodule prints a fatal error. It could be said that we should just eliminate the HEAD-fetch altogether, but that changes some behavior (in particular, some refs that were opportunistically updated would no longer be), so I have left that alone for now. There is an analogous situation with the fetching code in fetch_finish() and surrounding functions. For now, I have added a NEEDSWORK. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-15submodule: document default behaviorLibravatar Denton Liu1-1/+2
submodule's default behavior wasn't documented in both git-submodule.txt and in the usage text of git-submodule. Document the default behavior similar to how git-remote does it. Signed-off-by: Denton Liu <liu.denton@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-06Merge branch 'sh/submodule-summary-abbrev-fix'Libravatar Junio C Hamano1-2/+5
The "git submodule summary" subcommand showed shortened commit object names by mechanically truncating them at 7-hexdigit, which has been improved to let "rev-parse --short" scale the length of the abbreviation with the size of the repository. * sh/submodule-summary-abbrev-fix: git-submodule.sh: shorten submodule SHA-1s using rev-parse
2019-02-04git-submodule.sh: shorten submodule SHA-1s using rev-parseLibravatar Sven van Haastregt1-2/+5
Until now, `git submodule summary` was always emitting 7-character SHA-1s that have a higher chance of being ambiguous for larger repositories. Use `git rev-parse --short` instead, which will determine suitable short SHA-1 lengths. When a submodule hasn't been initialized with "submodule init" or not cloned, `git rev-parse` would not work in it yet; as a fallback, use the original method of cutting at 7 hexdigits. Signed-off-by: Sven van Haastregt <svenvh@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-01-18git-submodule: abort if core.worktree could not be set correctlyLibravatar Stefan Beller1-1/+1
74d4731da1f (submodule--helper: replace connect-gitdir-workingtree by ensure-core-worktree, 2018-08-13) forgot to exit the submodule operation if the helper could not ensure that core.worktree is set correctly. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-11-13Merge branch 'ao/submodule-wo-gitmodules-checked-out'Libravatar Junio C Hamano1-4/+9
The submodule support has been updated to read from the blob at HEAD:.gitmodules when the .gitmodules file is missing from the working tree. * ao/submodule-wo-gitmodules-checked-out: t/helper: add test-submodule-nested-repo-config submodule: support reading .gitmodules when it's not in the working tree submodule: add a helper to check if it is safe to write to .gitmodules t7506: clean up .gitmodules properly before setting up new scenario submodule: use the 'submodule--helper config' command submodule--helper: add a new 'config' subcommand t7411: be nicer to future tests and really clean things up t7411: merge tests 5 and 6 submodule: factor out a config_set_in_gitmodules_file_gently function submodule: add a print_config_from_gitmodules() helper
2018-10-31submodule: support reading .gitmodules when it's not in the working treeLibravatar Antonio Ospite1-0/+5
When the .gitmodules file is not available in the working tree, try using the content from the index and from the current branch. This covers the case when the file is part of the repository but for some reason it is not checked out, for example because of a sparse checkout. This makes it possible to use at least the 'git submodule' commands which *read* the gitmodules configuration file without fully populating the working tree. Writing to .gitmodules will still require that the file is checked out, so check for that before calling config_set_in_gitmodules_file_gently. Add a similar check also in git-submodule.sh::cmd_add() to anticipate the eventual failure of the "git submodule add" command when .gitmodules is not safely writeable; this prevents the command from leaving the repository in a spurious state (e.g. the submodule repository was cloned but .gitmodules was not updated because config_set_in_gitmodules_file_gently failed). Moreover, since config_from_gitmodules() now accesses the global object store, it is necessary to protect all code paths which call the function against concurrent access to the global object store. Currently this only happens in builtin/grep.c::grep_submodules(), so call grep_read_lock() before invoking code involving config_from_gitmodules(). Finally, add t7418-submodule-sparse-gitmodules.sh to verify that reading from .gitmodules succeeds and that writing to it fails when the file is not checked out. NOTE: there is one rare case where this new feature does not work properly yet: nested submodules without .gitmodules in their working tree. This has been documented with a warning and a test_expect_failure item in t7814, and in this case the current behavior is not altered: no config is read. Signed-off-by: Antonio Ospite <ao2@ao2.it> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-15submodule: make zero-oid comparison hash function agnosticLibravatar brian m. carlson1-1/+6
With SHA-256, the length of the all-zeros object ID is longer. Add a function to git-submodule.sh to check if a full hex object ID is the all-zeros value, and use it to check the output we're parsing from git diff-files or diff-index. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-09submodule: use the 'submodule--helper config' commandLibravatar Antonio Ospite1-4/+4
Use the 'submodule--helper config' command in git-submodules.sh to avoid referring explicitly to .gitmodules by the hardcoded file path. This makes it possible to access the submodules configuration in a more controlled way. Signed-off-by: Antonio Ospite <ao2@ao2.it> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-17Merge branch 'sb/submodule-update-in-c'Libravatar Junio C Hamano1-18/+6
"git submodule update" is getting rewritten piece-by-piece into C. * sb/submodule-update-in-c: submodule--helper: introduce new update-module-mode helper submodule--helper: replace connect-gitdir-workingtree by ensure-core-worktree builtin/submodule--helper: factor out method to update a single submodule builtin/submodule--helper: store update_clone information in a struct builtin/submodule--helper: factor out submodule updating git-submodule.sh: rename unused variables git-submodule.sh: align error reporting for update mode to use path
2018-09-07Revert "Merge branch 'sb/submodule-core-worktree'"Libravatar Jonathan Nieder1-5/+0
This reverts commit 7e25437d35a70791b345872af202eabfb3e1a8bc, reversing changes made to 00624d608cc69bd62801c93e74d1ea7a7ddd6598. v2.19.0-rc0~165^2~1 (submodule: ensure core.worktree is set after update, 2018-06-18) assumes an "absorbed" submodule layout, where the submodule's Git directory is in the superproject's .git/modules/ directory and .git in the submodule worktree is a .git file pointing there. In particular, it uses $GIT_DIR/modules/$name to find the submodule to find out whether it already has core.worktree set, and it uses connect_work_tree_and_git_dir if not, resulting in fatal: could not open sub/.git for writing The context behind that patch: v2.19.0-rc0~165^2~2 (submodule: unset core.worktree if no working tree is present, 2018-06-12) unsets core.worktree when running commands like "git checkout --recurse-submodules" to switch to a branch without the submodule. If a user then uses "git checkout --no-recurse-submodules" to switch back to a branch with the submodule and runs "git submodule update", this patch is needed to ensure that commands using the submodule directly are aware of the path to the worktree. It is late in the release cycle, so revert the whole 3-patch series. We can try again later for 2.20. Reported-by: Allan Sandfeld Jensen <allan.jensen@qt.io> Helped-by: Stefan Beller <sbeller@google.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-20Merge branch 'sb/pull-rebase-submodule'Libravatar Junio C Hamano1-0/+3
"git pull --rebase -v" in a repository with a submodule barfed as an intermediate process did not understand what "-v(erbose)" flag meant, which has been fixed. * sb/pull-rebase-submodule: git-submodule.sh: accept verbose flag in cmd_update to be non-quiet
2018-08-14submodule--helper: introduce new update-module-mode helperLibravatar Stefan Beller1-15/+1
This chews off a bit of the shell part of the update command in git-submodule.sh. When writing the C code, keep in mind that the submodule--helper part will go away eventually and we want to have a C function that is able to determine the submodule update strategy, it as a nicety, make determine_submodule_update_strategy accessible for arbitrary repositories. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-14submodule--helper: replace connect-gitdir-workingtree by ensure-core-worktreeLibravatar Stefan Beller1-5/+2
e98317508c0 (submodule: ensure core.worktree is set after update, 2018-06-18) was overly aggressive in calling connect_work_tree_and_git_dir as that ensures both the 'core.worktree' configuration is set as well as setting up correct gitlink file pointing at the git directory. We do not need to check for the gitlink in this part of the cmd_update in git-submodule.sh, as the initial call to update-clone will have ensured that. So we can reduce the work to only (check and potentially) set the 'core.worktree' setting. While at it move the check from shell to C as that proves to be useful in a follow up patch, as we do not need the 'name' in shell now. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-14git-submodule.sh: accept verbose flag in cmd_update to be non-quietLibravatar Stefan Beller1-0/+3
In a56771a668d (builtin/pull: respect verbosity settings in submodules, 2018-01-25), we made sure to pass on both quiet and verbose flag from builtin/pull.c to the submodule shell script. However git-submodule doesn't understand a verbose flag, which results in a bug when invoking git pull --recurse-submodules -v [...] There are a few different approaches to fix this bug: 1) rewrite 'argv_push_verbosity' or its caller in builtin/pull.c to cap opt_verbosity at 0. Then 'argv_push_verbosity' would only add '-q' if any. 2) Have a flag in 'argv_push_verbosity' that specifies if we allow adding -q or -v (or both). 3) Add -v to git-submodule.sh and make it a no-op (1) seems like a maintenance burden: What if we add code after the submodule operations or move submodule operations higher up, then we have altered the opt_verbosity setting further down the line in builtin/pull.c. (2) seems like it could work reasonably well without more regressions (3) seems easiest to implement as well as actually is a feature with the last-one-wins rule of passing flags to Git commands. Reported-by: Jochen Kühner Signed-off-by: Stefan Beller <sbeller@google.com> Reviewed-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-03git-submodule.sh: rename unused variablesLibravatar Stefan Beller1-2/+2
The 'mode' variable is not used in cmd_update for its original purpose, rename it to 'dummy' as it only serves the purpose to abort quickly documenting this knowledge. The variable 'stage' is also not used any more in cmd_update, so remove it. This went unnoticed as first each function used the commonly used submodule listing, which was converted in 74703a1e4df (submodule: rewrite `module_list` shell function in C, 2015-09-02). When cmd_update was using its own function starting in 48308681b07 (git submodule update: have a dedicated helper for cloning, 2016-02-29), its removal was missed. A later patch in this series also touches the communication between the submodule helper and git-submodule.sh, but let's have this as a preparatory patch, as it eases the next patch, which stores the raw data instead of the line printed for this communication. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-03git-submodule.sh: align error reporting for update mode to use pathLibravatar Stefan Beller1-1/+1
All other error messages in cmd_update are reporting the submodule based on its path, so let's do that for invalid update modes, too. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-18Merge branch 'sb/submodule-core-worktree'Libravatar Junio C Hamano1-0/+5
"git submodule" did not correctly adjust core.worktree setting that indicates whether/where a submodule repository has its associated working tree across various state transitions, which has been corrected. * sb/submodule-core-worktree: submodule deinit: unset core.worktree submodule: ensure core.worktree is set after update submodule: unset core.worktree if no working tree is present
2018-06-25Merge branch 'pc/submodule-helper-foreach'Libravatar Junio C Hamano1-39/+1
The bulk of "git submodule foreach" has been rewritten in C. * pc/submodule-helper-foreach: submodule: port submodule subcommand 'foreach' from shell to C submodule foreach: document variable '$displaypath' submodule foreach: document '$sm_path' instead of '$path' submodule foreach: correct '$path' in nested submodules from a subdirectory
2018-06-19submodule: ensure core.worktree is set after updateLibravatar Stefan Beller1-0/+5
Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-05-30Merge branch 'sb/submodule-update-try-harder'Libravatar Junio C Hamano1-1/+1
"git submodule update" attempts two different kinds of "git fetch" against the upstream repository to grab a commit bound at the submodule's path, but it incorrectly gave up if the first kind (i.e. a normal fetch) failed, making the second "last resort" one (i.e. fetching an exact commit object by object name) ineffective. This has been corrected. * sb/submodule-update-try-harder: git-submodule.sh: try harder to fetch a submodule
2018-05-30Merge branch 'cf/submodule-progress-dissociate'Libravatar Junio C Hamano1-5/+16
"git submodule update" and "git submodule add" supported the "--reference" option to borrow objects from a neighbouring local repository like "git clone" does, but lacked the more recent invention "--dissociate". Also "git submodule add" has been taught to take the "--progress" option. * cf/submodule-progress-dissociate: submodule: add --dissociate option to add/update commands submodule: add --progress option to add command submodule: clean up substitutions in script
2018-05-22Sync with Git 2.16.4Libravatar Junio C Hamano1-0/+5
* maint-2.16: Git 2.16.4 Git 2.15.2 Git 2.14.4 Git 2.13.7 verify_path: disallow symlinks in .gitmodules update-index: stat updated files earlier verify_dotfile: mention case-insensitivity in comment verify_path: drop clever fallthrough skip_prefix: add case-insensitive variant is_{hfs,ntfs}_dotgitmodules: add tests is_ntfs_dotgit: match other .git files is_hfs_dotgit: match other .git files is_ntfs_dotgit: use a size_t for traversing string submodule-config: verify submodule names as paths
2018-05-22Sync with Git 2.15.2Libravatar Junio C Hamano1-0/+5
* maint-2.15: Git 2.15.2 Git 2.14.4 Git 2.13.7 verify_path: disallow symlinks in .gitmodules update-index: stat updated files earlier verify_dotfile: mention case-insensitivity in comment verify_path: drop clever fallthrough skip_prefix: add case-insensitive variant is_{hfs,ntfs}_dotgitmodules: add tests is_ntfs_dotgit: match other .git files is_hfs_dotgit: match other .git files is_ntfs_dotgit: use a size_t for traversing string submodule-config: verify submodule names as paths
2018-05-22Sync with Git 2.13.7Libravatar Junio C Hamano1-0/+5
* maint-2.13: Git 2.13.7 verify_path: disallow symlinks in .gitmodules update-index: stat updated files earlier verify_dotfile: mention case-insensitivity in comment verify_path: drop clever fallthrough skip_prefix: add case-insensitive variant is_{hfs,ntfs}_dotgitmodules: add tests is_ntfs_dotgit: match other .git files is_hfs_dotgit: match other .git files is_ntfs_dotgit: use a size_t for traversing string submodule-config: verify submodule names as paths
2018-05-21submodule-config: verify submodule names as pathsLibravatar Jeff King1-0/+5
Submodule "names" come from the untrusted .gitmodules file, but we blindly append them to $GIT_DIR/modules to create our on-disk repo paths. This means you can do bad things by putting "../" into the name (among other things). Let's sanity-check these names to avoid building a path that can be exploited. There are two main decisions: 1. What should the allowed syntax be? It's tempting to reuse verify_path(), since submodule names typically come from in-repo paths. But there are two reasons not to: a. It's technically more strict than what we need, as we really care only about breaking out of the $GIT_DIR/modules/ hierarchy. E.g., having a submodule named "foo/.git" isn't actually dangerous, and it's possible that somebody has manually given such a funny name. b. Since we'll eventually use this checking logic in fsck to prevent downstream repositories, it should be consistent across platforms. Because verify_path() relies on is_dir_sep(), it wouldn't block "foo\..\bar" on a non-Windows machine. 2. Where should we enforce it? These days most of the .gitmodules reads go through submodule-config.c, so I've put it there in the reading step. That should cover all of the C code. We also construct the name for "git submodule add" inside the git-submodule.sh script. This is probably not a big deal for security since the name is coming from the user anyway, but it would be polite to remind them if the name they pick is invalid (and we need to expose the name-checker to the shell anyway for our test scripts). This patch issues a warning when reading .gitmodules and just ignores the related config entry completely. This will generally end up producing a sensible error, as it works the same as a .gitmodules file which is missing a submodule entry (so "submodule update" will barf, but "git clone --recurse-submodules" will print an error but not abort the clone. There is one minor oddity, which is that we print the warning once per malformed config key (since that's how the config subsystem gives us the entries). So in the new test, for example, the user would see three warnings. That's OK, since the intent is that this case should never come up outside of malicious repositories (and then it might even benefit the user to see the message multiple times). Credit for finding this vulnerability and the proof of concept from which the test script was adapted goes to Etienne Stalmans. Signed-off-by: Jeff King <peff@peff.net>