summaryrefslogtreecommitdiff
path: root/git.c
AgeCommit message (Collapse)AuthorFilesLines
2017-11-28Merge branch 'ma/branch-list-paginate'Libravatar Junio C Hamano1-1/+1
"git branch --list" learned to show its output through the pager by default when the output is going to a terminal, which is controlled by the pager.branch configuration variable. This is similar to a recent change to "git tag --list". * ma/branch-list-paginate: branch: change default of `pager.branch` to "on" branch: respect `pager.branch` in list-mode only t7006: add tests for how git branch paginates
2017-11-20branch: respect `pager.branch` in list-mode onlyLibravatar Martin Ågren1-1/+1
Similar to de121ffe5 (tag: respect `pager.tag` in list-mode only, 2017-08-02), use the DELAY_PAGER_CONFIG-mechanism to only respect `pager.branch` when we are listing branches. We have two possibilities of generalizing what that earlier commit made to `git tag`. One is to interpret, e.g., --set-upstream-to as "it does not use an editor, so we should page". Another, the one taken by this commit, is to say "it does not list, so let's not page". That is in line with the approach of the series on `pager.tag` and in particular the wording in Documentation/git-tag.txt, which this commit reuses for git-branch.txt. This fixes the failing test added in the previous commit. Also adapt the test for whether `git branch --set-upstream-to` respects `pager.branch`. Signed-off-by: Martin Ågren <martin.agren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-10-03Merge branch 'jk/no-optional-locks'Libravatar Junio C Hamano1-0/+4
Some commands (most notably "git status") makes an opportunistic update when performing a read-only operation to help optimize later operations in the same repository. The new "--no-optional-locks" option can be passed to Git to disable them. * jk/no-optional-locks: git: add --no-optional-locks option
2017-09-27git: add --no-optional-locks optionLibravatar Jeff King1-0/+4
Some tools like IDEs or fancy editors may periodically run commands like "git status" in the background to keep track of the state of the repository. Some of these commands may refresh the index and write out the result in an opportunistic way: if they can get the index lock, then they update the on-disk index with any updates they find. And if not, then their in-core refresh is lost and just has to be recomputed by the next caller. But taking the index lock may conflict with other operations in the repository. Especially ones that the user is doing themselves, which _aren't_ opportunistic. In other words, "git status" knows how to back off when somebody else is holding the lock, but other commands don't know that status would be happy to drop the lock if somebody else wanted it. There are a couple possible solutions: 1. Have some kind of "pseudo-lock" that allows other commands to tell status that they want the lock. This is likely to be complicated and error-prone to implement (and maybe even impossible with just dotlocks to work from, as it requires some inter-process communication). 2. Avoid background runs of commands like "git status" that want to do opportunistic updates, preferring instead plumbing like diff-files, etc. This is awkward for a couple of reasons. One is that "status --porcelain" reports a lot more about the repository state than is available from individual plumbing commands. And two is that we actually _do_ want to see the refreshed index. We just don't want to take a lock or write out the result. Whereas commands like diff-files expect us to refresh the index separately and write it to disk so that they can depend on the result. But that write is exactly what we're trying to avoid. 3. Ask "status" not to lock or write the index. This is easy to implement. The big downside is that any work done in refreshing the index for such a call is lost when the process exits. So a background process may end up re-hashing a changed file multiple times until the user runs a command that does an index refresh themselves. This patch implements the option 3. The idea (and the test) is largely stolen from a Git for Windows patch by Johannes Schindelin, 67e5ce7f63 (status: offer *not* to lock the index and update it, 2016-08-12). The twist here is that instead of making this an option to "git status", it becomes a "git" option and matching environment variable. The reason there is two-fold: 1. An environment variable is carried through to sub-processes. And whether an invocation is a background process or not should apply to the whole process tree. So you could do "git --no-optional-locks foo", and if "foo" is a script or alias that calls "status", you'll still get the effect. 2. There may be other programs that want the same treatment. I've punted here on finding more callers to convert, since "status" is the obvious one to call as a repeated background job. But "git diff"'s opportunistic refresh of the index may be a good candidate. The test is taken from 67e5ce7f63, and it's worth repeating Johannes's explanation: Note that the regression test added in this commit does not *really* verify that no index.lock file was written; that test is not possible in a portable way. Instead, we verify that .git/index is rewritten *only* when `git status` is run without `--no-optional-locks`. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-08-22Merge branch 'bw/grep-recurse-submodules'Libravatar Junio C Hamano1-1/+1
"git grep --recurse-submodules" has been reworked to give a more consistent output across submodule boundary (and do its thing without having to fork a separate process). * bw/grep-recurse-submodules: grep: recurse in-process using 'struct repository' submodule: merge repo_read_gitmodules and gitmodules_config submodule: check for unmerged .gitmodules outside of config parsing submodule: check for unstaged .gitmodules outside of config parsing submodule: remove fetch.recursesubmodules from submodule-config parsing submodule: remove submodule.fetchjobs from submodule-config parsing config: add config_from_gitmodules cache.h: add GITMODULES_FILE macro repository: have the_repository use the_index repo_read_index: don't discard the index
2017-08-03git.c: ignore pager.* when launching builtin as dashed externalLibravatar Martin Ågren1-1/+1
When running, e.g., `git -c alias.bar=foo bar`, we expand the alias and execute `git-foo` as a dashed external. This is true even if git foo is a builtin. That is on purpose, and is motivated in a comment which was added in commit 441981bc ("git: simplify environment save/restore logic", 2016-01-26). Shortly before we launch a dashed external, and unless we have already found out whether we should use a pager, we check `pager.foo`. This was added in commit 92058e4d ("support pager.* for external commands", 2011-08-18). If the dashed external is a builtin, this does not match that commit's intention and is arguably wrong, since it would be cleaner if we let the "dashed external builtin" handle `pager.foo`. This has not mattered in practice, but a recent patch taught `git-tag` to ignore `pager.tag` under certain circumstances. But, when started using an alias, it doesn't get the chance to do so, as outlined above. That recent patch added a test to document this breakage. Do not check `pager.foo` before launching a builtin as a dashed external, i.e., if we recognize the name of the external as a builtin. Change the test to use `test_expect_success`. Signed-off-by: Martin Ågren <martin.agren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-08-03tag: respect `pager.tag` in list-mode onlyLibravatar Martin Ågren1-1/+1
Using, e.g., `git -c pager.tag tag -a new-tag` results in errors such as "Vim: Warning: Output is not to a terminal" and a garbled terminal. Someone who makes use of both `git tag -a` and `git tag -l` will probably not set `pager.tag`, so that `git tag -a` will actually work, at the cost of not paging output of `git tag -l`. Use the mechanisms introduced in two earlier patches to ignore `pager.tag` in git.c and let the `git tag` builtin handle it on its own. Only respect `pager.tag` when running in list-mode. There is a window between where the pager is started before and after this patch. This means that early errors can behave slightly different before and after this patch. Since operation-parsing has to happen inside this window, this can be seen with `git -c pager.tag="echo pager is used" tag -l --unknown-option`. This change in paging-behavior should be acceptable since it only affects erroneous usages. Update the documentation and update tests. If an alias is used to run `git tag -a`, then `pager.tag` will still be respected. Document this known breakage. It will be fixed in a later commit. Add a similar test for `-l`, which works. Noticed-by: Anatoly Borodin <anatoly.borodin@gmail.com> Suggested-by: Jeff King <peff@peff.net> Signed-off-by: Martin Ågren <martin.agren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-08-03git.c: provide setup_auto_pager()Libravatar Martin Ågren1-0/+10
The previous patch introduced a way for builtins to declare that they will take responsibility for handling the `pager.foo`-config item. (See the commit message of that patch for why that could be useful.) Provide setup_auto_pager(), which builtins can call in order to handle `pager.<cmd>`, including possibly starting the pager. Make this function don't do anything if a pager has already been started, as indicated by use_pager or pager_in_use(). Whenever this function is called from a builtin, git.c will already have called commit_pager_choice(). Since commit_pager_choice() treats the special value -1 as "punt" or "not yet decided", it is not a problem that we might end up calling commit_pager_choice() once in git.c and once (or more) in the builtin. Make the new function use -1 in the same way and document it as "punt". Don't add any users of setup_auto_pager just yet, one will follow in a later patch. Suggested-by: Jeff King <peff@peff.net> Signed-off-by: Martin Ågren <martin.agren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-08-03git.c: let builtins opt for handling `pager.foo` themselvesLibravatar Martin Ågren1-1/+3
Before launching a builtin git foo and unless mechanisms with precedence are in use, we check for and handle the `pager.foo` config. This is done without considering exactly how git foo is being used, and indeed, git.c cannot (and should not) know what the arguments to git foo are supposed to achieve. In practice this means that, e.g., `git -c pager.tag tag -a new-tag` results in errors such as "Vim: Warning: Output is not to a terminal" and a garbled terminal. Someone who makes use of both `git tag -a` and `git tag -l` will probably not set `pager.tag`, so that `git tag -a` will actually work, at the cost of not paging output of `git tag -l`. To allow individual builtins to make more informed decisions about when to respect `pager.foo`, introduce a flag DELAY_PAGER_CONFIG. If the flag is set, do not check `pager.foo`. Do not check for DELAY_PAGER_CONFIG in `execv_dashed_external()`. That call site is arguably wrong, although in a way that is not yet visible, and will be changed in a slightly different direction in a later patch. Don't add any users of DELAY_PAGER_CONFIG just yet, one will follow in a later patch. Suggested-by: Jeff King <peff@peff.net> Signed-off-by: Martin Ågren <martin.agren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-08-02grep: recurse in-process using 'struct repository'Libravatar Brandon Williams1-1/+1
Convert grep to use 'struct repository' which enables recursing into submodules to be handled in-process. Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-23ls-files: use repository objectLibravatar Brandon Williams1-1/+1
Convert ls-files to use a repository struct and recurse submodules inprocess. Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-23Merge branches 'bw/ls-files-sans-the-index' and 'bw/config-h' into ↵Libravatar Junio C Hamano1-51/+5
bw/repo-object * bw/ls-files-sans-the-index: ls-files: factor out tag calculation ls-files: factor out debug info into a function ls-files: convert show_files to take an index ls-files: convert show_ce_entry to take an index ls-files: convert prune_cache to take an index ls-files: convert ce_excluded to take an index ls-files: convert show_ru_info to take an index ls-files: convert show_other_files to take an index ls-files: convert show_killed_files to take an index ls-files: convert write_eolinfo to take an index ls-files: convert overlay_tree_on_cache to take an index tree: convert read_tree to take an index parameter convert: convert renormalize_buffer to take an index convert: convert convert_to_git to take an index convert: convert convert_to_git_filter_fd to take an index convert: convert crlf_to_git to take an index convert: convert get_cached_convert_stats_ascii to take an index * bw/config-h: config: don't implicitly use gitdir or commondir config: respect commondir setup: teach discover_git_directory to respect the commondir config: don't include config.h by default config: remove git_config_iter config: create config.h alias: use the early config machinery to expand aliases t7006: demonstrate a problem with aliases in subdirectories t1308: relax the test verifying that empty alias values are disallowed help: use early config when autocorrecting aliases config: report correct line number upon error discover_git_directory(): avoid setting invalid git_dir
2017-06-15config: don't include config.h by defaultLibravatar Brandon Williams1-0/+1
Stop including config.h by default in cache.h. Instead only include config.h in those files which require use of the config system. Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-15alias: use the early config machinery to expand aliasesLibravatar Johannes Schindelin1-51/+4
Instead of discovering the .git/ directory, reading the config and then trying to painstakingly reset all the global state if we did not find a matching alias, let's use the early config machinery instead. It may look like unnecessary work to discover the .git/ directory in the early config machinery and then call setup_git_directory_gently() in the case of a shell alias, repeating the very same discovery *again*. However, we have to do this as the early config machinery takes pains *not* to touch any global state, while shell aliases expect a possibly changed working directory and at least the GIT_PREFIX and GIT_DIR variables to be set. This change also fixes a known issue where Git tried to read the pager config from an incorrect path in a subdirectory of a Git worktree if an alias expanded to a shell command. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Reviewed-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-05git: add hidden --list-builtins optionLibravatar Jeff King1-0/+12
It can be useful in the test suite to be able to iterate over the list of builtins. We could do this with some Makefile magic. But since the authoritative list is in the commands array inside git.c, and since this could also be handy for debugging, let's add a hidden command-line option to dump that list. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-30Merge branch 'bw/recurse-submodules-relative-fix'Libravatar Junio C Hamano1-2/+0
A few commands that recently learned the "--recurse-submodule" option misbehaved when started from a subdirectory of the superproject. * bw/recurse-submodules-relative-fix: ls-files: fix bug when recursing with relative pathspec ls-files: fix typo in variable name grep: fix bug when recursing with relative pathspec setup: allow for prefix to be passed to git commands grep: fix help text typo
2017-03-17setup: allow for prefix to be passed to git commandsLibravatar Brandon Williams1-2/+0
In a future patch child processes which act on submodules need a little more context about the original command that was invoked. This patch teaches git to use the prefix stored in `GIT_INTERNAL_TOPLEVEL_PREFIX` instead of the prefix that was potentally found during the git directory setup process. Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-02-09rebase--helper: add a builtin helper for interactive rebasesLibravatar Johannes Schindelin1-0/+1
Git's interactive rebase is still implemented as a shell script, despite its complexity. This implies that it suffers from the portability point of view, from lack of expressibility, and of course also from performance. The latter issue is particularly serious on Windows, where we pay a hefty price for relying so much on POSIX. Unfortunately, being such a huge shell script also means that we missed the train when it would have been relatively easy to port it to C, and instead piled feature upon feature onto that poor script that originally never intended to be more than a slightly pimped cherry-pick in a loop. To open the road toward better performance (in addition to all the other benefits of C over shell scripts), let's just start *somewhere*. The approach taken here is to add a builtin helper that at first intends to take care of the parts of the interactive rebase that are most affected by the performance penalties mentioned above. In particular, after we spent all those efforts on preparing the sequencer to process rebase -i's git-rebase-todo scripts, we implement the `git rebase -i --continue` functionality as a new builtin, git-rebase--helper. Once that is in place, we can work gradually on tackling the rest of the technical debt. Note that the rebase--helper needs to learn about the transient --ff/--no-ff options of git-rebase, as the corresponding flag is not persisted to, and re-read from, the state directory. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-02-03Merge branch 'sb/unpack-trees-super-prefix'Libravatar Junio C Hamano1-1/+1
"git read-tree" and its underlying unpack_trees() machinery learned to report problematic paths prefixed with the --super-prefix option. * sb/unpack-trees-super-prefix: unpack-trees: support super-prefix option t1001: modernize style t1000: modernize style read-tree: use OPT_BOOL instead of OPT_SET_INT
2017-01-31Merge branch 'js/difftool-builtin'Libravatar Junio C Hamano1-0/+1
Rewrite a scripted porcelain "git difftool" in C. * js/difftool-builtin: difftool: hack around -Wzero-length-format warning difftool: retire the scripted version difftool: implement the functionality in the builtin difftool: add a skeleton for the upcoming builtin
2017-01-25unpack-trees: support super-prefix optionLibravatar Stefan Beller1-1/+1
In the future we want to support working tree operations within submodules, e.g. "git checkout --recurse-submodules", which will update the submodule to the commit as recorded in its superproject. In the submodule the unpack-tree operation is carried out as usual, but the reporting to the user needs to prefix any path with the superproject. The mechanism for this is the super-prefix. (see 74866d757, git: make super-prefix option) Add support for the super-prefix option for commands that unpack trees by wrapping any path output in unpacking trees in the newly introduced super_prefixed function. This new function prefixes any path with the super-prefix if there is one. Assuming the submodule case doesn't happen in the majority of the cases, we'd want to have a fast behavior for no super prefix, i.e. no reallocation/copying, but just returning path. Another aspect of introducing the `super_prefixed` function is to consider who owns the memory and if this is the right place where the path gets modified. As the super prefix ought to change the output behavior only and not the actual unpack tree part, it is fine to be that late in the line. As we get passed in 'const char *path', we cannot change the path itself, which means in case of a super prefix we have to copy over the path. We need two static buffers in that function as the error messages contain at most two paths. For testing purposes enable it in read-tree, which has no output of paths other than an unpack-trees.c. These are all converted in this patch. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-01-19difftool: retire the scripted versionLibravatar Johannes Schindelin1-6/+1
It served its purpose, but now we have a builtin difftool. Time for the Perl script to enjoy Florida. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-01-18Merge branch 'jk/execv-dashed-external'Libravatar Junio C Hamano1-21/+15
Typing ^C to pager, which usually does not kill it, killed Git and took the pager down as a collateral damage in certain process-tree structure. This has been fixed. * jk/execv-dashed-external: execv_dashed_external: wait for child on signal death execv_dashed_external: stop exiting with negative code execv_dashed_external: use child_process struct
2017-01-18Merge branch 'bw/grep-recurse-submodules'Libravatar Junio C Hamano1-1/+1
"git grep" has been taught to optionally recurse into submodules. * bw/grep-recurse-submodules: grep: search history of moved submodules grep: enable recurse-submodules to work on <tree> objects grep: optionally recurse into submodules grep: add submodules as a grep source type submodules: load gitmodules file from commit sha1 submodules: add helper to determine if a submodule is initialized submodules: add helper to determine if a submodule is populated real_path: canonicalize directory separators in root parts real_path: have callers use real_pathdup and strbuf_realpath real_path: create real_pathdup real_path: convert real_path_internal to strbuf_realpath real_path: resolve symlinks by hand
2017-01-17difftool: add a skeleton for the upcoming builtinLibravatar Johannes Schindelin1-0/+6
This adds a builtin difftool that still falls back to the legacy Perl version, which has been renamed to `legacy-difftool`. The idea is that the new, experimental, builtin difftool immediately hands off to the legacy difftool for now, unless the config variable difftool.useBuiltin is set to true. This feature flag will be used in the upcoming Git for Windows v2.11.0 release, to allow early testers to opt-in to use the builtin difftool and flesh out any bugs. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-01-10Merge branch 'sb/submodule-embed-gitdir'Libravatar Junio C Hamano1-1/+1
A new submodule helper "git submodule embedgitdirs" to make it easier to move embedded .git/ directory for submodules in a superproject to .git/modules/ (and point the latter with the former that is turned into a "gitdir:" file) has been added. * sb/submodule-embed-gitdir: worktree: initialize return value for submodule_uses_worktrees submodule: add absorb-git-dir function move connect_work_tree_and_git_dir to dir.h worktree: check if a submodule uses worktrees test-lib-functions.sh: teach test_commit -C <dir> submodule helper: support super prefix submodule: use absolute path for computing relative path connecting
2017-01-09execv_dashed_external: wait for child on signal deathLibravatar Jeff King1-0/+1
When you hit ^C to interrupt a git command going to a pager, this usually leaves the pager running. But when a dashed external is in use, the pager ends up in a funny state and quits (but only after eating one more character from the terminal!). This fixes it. Explaining the reason will require a little background. When git runs a pager, it's important for the git process to hang around and wait for the pager to finish, even though it has no more data to feed it. This is because git spawns the pager as a child, and thus the git process is the session leader on the terminal. After it dies, the pager will finish its current read from the terminal (eating the one character), and then get EIO trying to read again. When you hit ^C, that sends SIGINT to git and to the pager, and it's a similar situation. The pager ignores it, but the git process needs to hang around until the pager is done. We addressed that long ago in a3da882120 (pager: do wait_for_pager on signal death, 2009-01-22). But when you have a dashed external (or an alias pointing to a builtin, which will re-exec git for the builtin), there's an extra process in the mix. For instance, running: $ git -c alias.l=log l will end up with a process tree like: git (parent) \ git-log (child) \ less (pager) If you hit ^C, SIGINT goes to all of them. The pager ignores it, and the child git process will end up in wait_for_pager(). But the parent git process will die, and the usual EIO trouble happens. So we really want the parent git process to wait_for_pager(), but of course it doesn't know anything about the pager at all, since it was started by the child. However, we can have it wait on the git-log child, which in turn is waiting on the pager. And that's what this patch does. There are a few design decisions here worth explaining: 1. The new feature is attached to run-command's clean_on_exit feature. Partly this is convenience, since that feature already has a signal handler that deals with child cleanup. But it's also a meaningful connection. The main reason that dashed externals use clean_on_exit is to bind the two processes together. If somebody kills the parent with a signal, we propagate that to the child (in this instance with SIGINT, we do propagate but it doesn't matter because the original signal went to the whole process group). Likewise, we do not want the parent to go away until the child has done so. In a traditional Unix world, we'd probably accomplish this binding by just having the parent execve() the child directly. But since that doesn't work on Windows, everything goes through run_command's more spawn-like interface. 2. We do _not_ automatically waitpid() on any clean_on_exit children. For dashed externals this makes sense; we know that the parent is doing nothing but waiting for the child to exit anyway. But with other children, it's possible that the child, after getting the signal, could be waiting on the parent to do something (like closing a descriptor). If we were to wait on such a child, we'd end up in a deadlock. So this errs on the side of caution, and lets callers enable the feature explicitly. 3. When we send children the cleanup signal, we send all the signals first, before waiting on any children. This is to avoid the case where one child might be waiting on another one to exit, causing a deadlock. We inform all of them that it's time to die before reaping any. In practice, there is only ever one dashed external run from a given process, so this doesn't matter much now. But it future-proofs us if other callers start using the wait_after_clean mechanism. There's no automated test here, because it would end up racy and unportable. But it's easy to reproduce the situation by running the log command given above and hitting ^C. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-01-09execv_dashed_external: stop exiting with negative codeLibravatar Jeff King1-3/+7
When we try to exec a git sub-command, we pass along the status code from run_command(). But that may return -1 if we ran into an error with pipe() or execve(). This tends to work (and end up as 255 due to twos-complement wraparound and truncation), but in general it's probably a good idea to avoid negative exit codes for portability. We can easily translate to the normal generic "128" code we get when syscalls cause us to die. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-01-09execv_dashed_external: use child_process structLibravatar Jeff King1-18/+7
When we run a dashed external, we use the one-liner run_command_v_opt() to do so. Let's switch to using a child_process struct, which has two advantages: 1. We can drop all of the allocation and cleanup code for building our custom argv array, and just rely on the builtin argv_array (at the minor cost of doing a few extra mallocs). 2. We have access to the complete range of child_process options, not just the ones that the "_opt()" form can forward. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-22grep: optionally recurse into submodulesLibravatar Brandon Williams1-1/+1
Allow grep to recognize submodules and recursively search for patterns in each submodule. This is done by forking off a process to recursively call grep on each submodule. The top level --super-prefix option is used to pass a path to the submodule which can in turn be used to prepend to output or in pathspec matching logic. Recursion only occurs for submodules which have been initialized and checked out by the parent project. If a submodule hasn't been initialized and checked out it is simply skipped. In order to support the existing multi-threading infrastructure in grep, output from each child process is captured in a strbuf so that it can be later printed to the console in an ordered fashion. To limit the number of theads that are created, each child process has half the number of threads as its parents (minimum of 1), otherwise we potentailly have a fork-bomb. Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-09submodule helper: support super prefixLibravatar Stefan Beller1-1/+1
Just like main commands in Git, the submodule helper needs access to the superproject prefix. Enable this in the git.c but have its own fuse in the helper code by having a flag to turn on the super prefix. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-11-29Merge branch 'jk/common-main'Libravatar Junio C Hamano1-0/+5
Fix for a small regression in a topic already in 'master'. * jk/common-main: common-main: stop munging argv[0] path
2016-11-29common-main: stop munging argv[0] pathLibravatar Jeff King1-0/+5
Since 650c44925 (common-main: call git_extract_argv0_path(), 2016-07-01), the argv[0] that is seen in cmd_main() of individual programs is always the basename of the executable, as common-main strips off the full path. This can produce confusing results for git-daemon, which wants to re-exec itself. For instance, if the program was originally run as "/usr/lib/git/git-daemon", it will try just re-execing "git-daemon", which will find the first instance in $PATH. If git's exec-path has not been prepended to $PATH, we may find the git-daemon from a different version (or no git-daemon at all). Normally this isn't a problem. Git commands are run as "git daemon", the git wrapper puts the exec-path at the front of $PATH, and argv[0] is already "daemon" anyway. But running git-daemon via its full exec-path, while not really a recommended method, did work prior to 650c44925. Let's make it work again. The real goal of 650c44925 was not to munge argv[0], but to reliably set the argv0_path global. The only reason it munges at all is that one caller, the git.c wrapper, piggy-backed on that computation to find the command basename. Instead, let's leave argv[0] untouched in common-main, and have git.c do its own basename computation. While we're at it, let's drop the return value from git_extract_argv0_path(). It was only ever used in this one callsite, and its dual purposes is what led to this confusion in the first place. Note that by changing the interface, the compiler can confirm for us that there are no other callers storing the return value. But the compiler can't tell us whether any of the cmd_main() functions (besides git.c) were relying on the basename munging. However, we can observe that prior to 650c44925, no other cmd_main() functions did that munging, and no new cmd_main() functions have been introduced since then. So we can't be regressing any of those cases. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-11-22archive: read local configurationLibravatar Junio C Hamano1-1/+1
Since b9605bc4f2 ("config: only read .git/config from configured repos", 2016-09-12), we do not read from ".git/config" unless we know we are in a repository. "git archive" however didn't do the repository discovery and instead relied on the old behaviour. Teach the command to run a "gentle" version of repository discovery so that local configuration variables are honoured. [jc: stole tests from peff] Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-11-22mailinfo: read local configurationLibravatar Junio C Hamano1-1/+1
Since b9605bc4f2 ("config: only read .git/config from configured repos", 2016-09-12), we do not read from ".git/config" unless we know we are in a repository. "git mailinfo" however didn't do the repository discovery and instead relied on the old behaviour. This was mostly OK because it was merely run as a helper program by other porcelain scripts that first chdir's up to the root of the working tree. Teach the command to run a "gentle" version of repository discovery so that local configuration variables like mailinfo.scissors are honoured. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-10-26Merge branch 'jc/cocci-xstrdup-or-null'Libravatar Junio C Hamano1-2/+1
Code cleanup. * jc/cocci-xstrdup-or-null: cocci: refactor common patterns to use xstrdup_or_null()
2016-10-26Merge branch 'bw/ls-files-recurse-submodules'Libravatar Junio C Hamano1-1/+26
"git ls-files" learned "--recurse-submodules" option that can be used to get a listing of tracked files across submodules (i.e. this only works with "--cached" option, not for listing untracked or ignored files). This would be a useful tool to sit on the upstream side of a pipe that is read with xargs to work on all working tree files from the top-level superproject. * bw/ls-files-recurse-submodules: ls-files: add pathspec matching for submodules ls-files: pass through safe options for --recurse-submodules ls-files: optionally recurse into submodules git: make super-prefix option
2016-10-12cocci: refactor common patterns to use xstrdup_or_null()Libravatar Junio C Hamano1-2/+1
d64ea0f83b ("git-compat-util: add xstrdup_or_null helper", 2015-01-12) added a handy wrapper that allows us to get a duplicate of a string or NULL if the original is NULL, but a handful of codepath predate its introduction or just weren't aware of it. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-10-10ls-files: optionally recurse into submodulesLibravatar Brandon Williams1-1/+1
Allow ls-files to recognize submodules in order to retrieve a list of files from a repository's submodules. This is done by forking off a process to recursively call ls-files on all submodules. Use top-level --super-prefix option to pass a path to the submodule which it can use to prepend to output or pathspec matching logic. Signed-off-by: Brandon Williams <bmwill@google.com> Reviewed-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-10-10git: make super-prefix optionLibravatar Brandon Williams1-0/+25
Add a super-prefix environment variable 'GIT_INTERNAL_SUPER_PREFIX' which can be used to specify a path from above a repository down to its root. When such a super-prefix is specified, the paths reported by Git are prefixed with it to make them relative to that directory "above". The paths given by the user on the command line (e.g. "git subcmd --output-file=path/to/a/file" and pathspecs) are taken relative to the directory "above" to match. The immediate use of this option is by commands which have a --recurse-submodule option in order to give context to submodules about how they were invoked. This option is currently only allowed for builtins which support a super-prefix. Signed-off-by: Brandon Williams <bmwill@google.com> Reviewed-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-09-21Merge branch 'jk/setup-sequence-update'Libravatar Junio C Hamano1-1/+1
There were numerous corner cases in which the configuration files are read and used or not read at all depending on the directory a Git command was run, leading to inconsistent behaviour. The code to set-up repository access at the beginning of a Git process has been updated to fix them. * jk/setup-sequence-update: t1007: factor out repeated setup init: reset cached config when entering new repo init: expand comments explaining config trickery config: only read .git/config from configured repos test-config: setup git directory t1302: use "git -C" pager: handle early config pager: use callbacks instead of configset pager: make pager_program a file-local static pager: stop loading git_default_config() pager: remove obsolete comment diff: always try to set up the repository diff: handle --no-index prefixes consistently diff: skip implicit no-index check when given --no-index patch-id: use RUN_SETUP_GENTLY hash-object: always try to set up the git repository
2016-09-13patch-id: use RUN_SETUP_GENTLYLibravatar Jeff King1-1/+1
Patch-id does not require a repository because it is just processing the incoming diff on stdin, but it may look at git config for keys like patchid.stable. Even though we do not setup_git_directory(), this works from the top-level of a repository because we blindly look at ".git/config" in this case. But as the included test demonstrates, it does not work from a subdirectory. We can fix it by using RUN_SETUP_GENTLY. We do not take any filenames from the user on the command line, so there's no need to adjust them via prefix_filename(). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-09-08Merge branch 'rt/help-unknown'Libravatar Junio C Hamano1-1/+14
"git nosuchcommand --help" said "No manual entry for gitnosuchcommand", which was not intuitive, given that "git nosuchcommand" said "git: 'nosuchcommand' is not a git command". * rt/help-unknown: help: make option --help open man pages only for Git commands help: introduce option --exclude-guides
2016-08-30help: make option --help open man pages only for Git commandsLibravatar Ralf Thielow1-1/+14
If option --help is passed to a Git command, we try to open the man page of that command. However, we do it for both commands and concepts. Make sure it is an actual command. This makes "git <concept> --help" not working anymore, while "git help <concept>" still works. Signed-off-by: Ralf Thielow <ralf.thielow@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-01common-main: call git_setup_gettext()Libravatar Jeff King1-2/+0
This should be part of every program, as otherwise users do not get translated error messages. However, some external commands forgot to do so (e.g., git-credential-store). This fixes them, and eliminates the repeated code in programs that did remember to use it. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-01common-main: call restore_sigpipe_to_default()Libravatar Jeff King1-23/+0
This is another safety/sanity setup that should be in force everywhere, but which we only applied in git.c. This did catch most cases, since even external commands are typically run via "git ..." (and the restoration applies to sub-processes, too). But there were cases we missed, such as somebody calling git-upload-pack directly via ssh, or scripts which use dashed external commands directly. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-01common-main: call sanitize_stdfds()Libravatar Jeff King1-7/+0
This is setup that should be done in every program for safety, but we never got around to adding it everywhere (so builtins benefited from the call in git.c, but any external commands did not). Putting it in the common main() gives us this safety everywhere. Note that the case in daemon.c is a little funny. We wait until we know whether we want to daemonize, and then either: - call daemonize(), which will close stdio and reopen it to /dev/null under the hood - sanitize_stdfds(), to fix up any odd cases But that is way too late; the point of sanitizing is to give us reliable descriptors on 0/1/2, and we will already have executed code, possibly called die(), etc. The sanitizing should be the very first thing that happens. With this patch, git-daemon will sanitize first, and can remove the call in the non-daemonize case. It does mean that daemonize() may just end up closing the descriptors we opened, but that's not a big deal (it's not wrong to do so, nor is it really less optimal than the case where our parent process redirected us from /dev/null ahead of time). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-01common-main: call git_extract_argv0_path()Libravatar Jeff King1-1/+1
Every program which links against libgit.a must call this function, or risk hitting an assert() in system_path() that checks whether we have configured argv0_path (though only when RUNTIME_PREFIX is defined, so essentially only on Windows). Looking at the diff, you can see that putting it into the common main() saves us having to do it individually in each of the external commands. But what you can't see are the cases where we _should_ have been doing so, but weren't (e.g., git-credential-store, and all of the t/helper test programs). This has been an accident-waiting-to-happen for a long time, but wasn't triggered until recently because it involves one of those programs actually calling system_path(). That happened with git-credential-store in v2.8.0 with ae5f677 (lazily load core.sharedrepository, 2016-03-11). The program: - takes a lock file, which... - opens a tempfile, which... - calls adjust_shared_perm to fix permissions, which... - lazy-loads the config (as of ae5f677), which... - calls system_path() to find the location of /etc/gitconfig On systems with RUNTIME_PREFIX, this means credential-store reliably hits that assert() and cannot be used. We never noticed in the test suite, because we set GIT_CONFIG_NOSYSTEM there, which skips the system_path() lookup entirely. But if we were to tweak git_config() to find /etc/gitconfig even when we aren't going to open it, then the test suite shows multiple failures (for credential-store, and for some other test helpers). I didn't include that tweak here because it's way too specific to this particular call to be worth carrying around what is essentially dead code. The implementation is fairly straightforward, with one exception: there is exactly one caller (git.c) that actually cares about the result of the function, and not the side-effect of setting up argv0_path. We can accommodate that by simply replacing the value of argv[0] in the array we hand down to cmd_main(). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-01add an extra level of indirection to main()Libravatar Jeff King1-2/+1
There are certain startup tasks that we expect every git process to do. In some cases this is just to improve the quality of the program (e.g., setting up gettext()). In others it is a requirement for using certain functions in libgit.a (e.g., system_path() expects that you have called git_extract_argv0_path()). Most commands are builtins and are covered by the git.c version of main(). However, there are still a few external commands that use their own main(). Each of these has to remember to include the correct startup sequence, and we are not always consistent. Rather than just fix the inconsistencies, let's make this harder to get wrong by providing a common main() that can run this standard startup. We basically have two options to do this: - the compat/mingw.h file already does something like this by adding a #define that replaces the definition of main with a wrapper that calls mingw_startup(). The upside is that the code in each program doesn't need to be changed at all; it's rewritten on the fly by the preprocessor. The downside is that it may make debugging of the startup sequence a bit more confusing, as the preprocessor is quietly inserting new code. - the builtin functions are all of the form cmd_foo(), and git.c's main() calls them. This is much more explicit, which may make things more obvious to somebody reading the code. It's also more flexible (because of course we have to figure out _which_ cmd_foo() to call). The downside is that each of the builtins must define cmd_foo(), instead of just main(). This patch chooses the latter option, preferring the more explicit approach, even though it is more invasive. We introduce a new file common-main.c, with the "real" main. It expects to call cmd_main() from whatever other objects it is linked against. We link common-main.o against anything that links against libgit.a, since we know that such programs will need to do this setup. Note that common-main.o can't actually go inside libgit.a, as the linker would not pick up its main() function automatically (it has no callers). The rest of the patch is just adjusting all of the various external programs (mostly in t/helper) to use cmd_main(). I've provided a global declaration for cmd_main(), which means that all of the programs also need to match its signature. In particular, many functions need to switch to "const char **" instead of "char **" for argv. This effect ripples out to a few other variables and functions, as well. This makes the patch even more invasive, but the end result is much better. We should be treating argv strings as const anyway, and now all programs conform to the same signature (which also matches the way builtins are defined). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-06setup: make startup_info available everywhereLibravatar Jeff King1-3/+0
Commit a60645f (setup: remember whether repository was found, 2010-08-05) introduced the startup_info structure, which records some parts of the setup_git_directory() process (notably, whether we actually found a repository or not). One of the uses of this data is for functions to behave appropriately based on whether we are in a repo. But the startup_info struct is just a pointer to storage provided by the main program, and the only program that sets it up is the git.c wrapper. Thus builtins have access to startup_info, but externally linked programs do not. Worse, library code which is accessible from both has to be careful about accessing startup_info. This can be used to trigger a die("BUG") via get_sha1(): $ git fast-import <<-\EOF tag foo from HEAD:./whatever EOF fatal: BUG: startup_info struct is not initialized. Obviously that's fairly nonsensical input to feed to fast-import, but we should never hit a die("BUG"). And there may be other ways to trigger it if other non-builtins resolve sha1s. So let's point the storage for startup_info to a static variable in setup.c, making it available to all users of the library code. We _could_ turn startup_info into a regular extern struct, but doing so would mean tweaking all of the existing use sites. So let's leave the pointer indirection in place. We can, however, drop any checks for NULL, as they will always be false (and likewise, we can drop the test covering this case, which was a rather artificial situation using one of the test-* programs). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>