summaryrefslogtreecommitdiff
path: root/builtin/grep.c
AgeCommit message (Collapse)AuthorFilesLines
2018-04-11Merge branch 'sb/object-store'Libravatar Junio C Hamano1-1/+2
Refactoring the internal global data structure to make it possible to open multiple repositories, work with and then close them. Rerolled by Duy on top of a separate preliminary clean-up topic. The resulting structure of the topics looked very sensible. * sb/object-store: (27 commits) sha1_file: allow sha1_loose_object_info to handle arbitrary repositories sha1_file: allow map_sha1_file to handle arbitrary repositories sha1_file: allow map_sha1_file_1 to handle arbitrary repositories sha1_file: allow open_sha1_file to handle arbitrary repositories sha1_file: allow stat_sha1_file to handle arbitrary repositories sha1_file: allow sha1_file_name to handle arbitrary repositories sha1_file: add repository argument to sha1_loose_object_info sha1_file: add repository argument to map_sha1_file sha1_file: add repository argument to map_sha1_file_1 sha1_file: add repository argument to open_sha1_file sha1_file: add repository argument to stat_sha1_file sha1_file: add repository argument to sha1_file_name sha1_file: allow prepare_alt_odb to handle arbitrary repositories sha1_file: allow link_alt_odb_entries to handle arbitrary repositories sha1_file: add repository argument to prepare_alt_odb sha1_file: add repository argument to link_alt_odb_entries sha1_file: add repository argument to read_info_alternates sha1_file: add repository argument to link_alt_odb_entry sha1_file: add raw_object_store argument to alt_odb_usable pack: move approximate object count to object store ...
2018-04-10Merge branch 'bc/object-id'Libravatar Junio C Hamano1-3/+3
Conversion from uchar[20] to struct object_id continues. * bc/object-id: (36 commits) convert: convert to struct object_id sha1_file: introduce a constant for max header length Convert lookup_replace_object to struct object_id sha1_file: convert read_sha1_file to struct object_id sha1_file: convert read_object_with_reference to object_id tree-walk: convert tree entry functions to object_id streaming: convert istream internals to struct object_id tree-walk: convert get_tree_entry_follow_symlinks internals to object_id builtin/notes: convert static functions to object_id builtin/fmt-merge-msg: convert remaining code to object_id sha1_file: convert sha1_object_info* to object_id Convert remaining callers of sha1_object_info_extended to object_id packfile: convert unpack_entry to struct object_id sha1_file: convert retry_bad_packed_offset to struct object_id sha1_file: convert assert_sha1_type to object_id builtin/mktree: convert to struct object_id streaming: convert open_istream to use struct object_id sha1_file: convert check_sha1_signature to struct object_id sha1_file: convert read_loose_object to use struct object_id builtin/index-pack: convert struct ref_delta_entry to object_id ...
2018-03-23repository: introduce raw object store fieldLibravatar Stefan Beller1-1/+2
The raw object store field will contain any objects needed for access to objects in a given repository. This patch introduces the raw object store and populates it with the `objectdir`, which used to be part of the repository struct. As the struct gains members, we'll also populate the function to clear the memory for these members. In a later step, we'll introduce a struct object_parser, that will complement the object parsing in a repository struct: The raw object parser is the layer that will provide access to raw object content, while the higher level object parser code will parse raw objects and keeps track of parenthood and other object relationships using 'struct object'. For now only add the lower level to the repository struct. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-03-14Merge branch 'nd/parseopt-completion'Libravatar Junio C Hamano1-5/+8
Teach parse-options API an option to help the completion script, and make use of the mechanism in command line completion. * nd/parseopt-completion: (45 commits) completion: more subcommands in _git_notes() completion: complete --{reuse,reedit}-message= for all notes subcmds completion: simplify _git_notes completion: don't set PARSE_OPT_NOCOMPLETE on --rerere-autoupdate completion: use __gitcomp_builtin in _git_worktree completion: use __gitcomp_builtin in _git_tag completion: use __gitcomp_builtin in _git_status completion: use __gitcomp_builtin in _git_show_branch completion: use __gitcomp_builtin in _git_rm completion: use __gitcomp_builtin in _git_revert completion: use __gitcomp_builtin in _git_reset completion: use __gitcomp_builtin in _git_replace remote: force completing --mirror= instead of --mirror completion: use __gitcomp_builtin in _git_remote completion: use __gitcomp_builtin in _git_push completion: use __gitcomp_builtin in _git_pull completion: use __gitcomp_builtin in _git_notes completion: use __gitcomp_builtin in _git_name_rev completion: use __gitcomp_builtin in _git_mv completion: use __gitcomp_builtin in _git_merge_base ...
2018-03-14sha1_file: convert read_sha1_file to struct object_idLibravatar brian m. carlson1-1/+1
Convert read_sha1_file to take a pointer to struct object_id and rename it read_object_file. Do the same for read_sha1_file_extended. Convert one use in grep.c to use the new function without any other code change, since the pointer being passed is a void pointer that is already initialized with a pointer to struct object_id. Update the declaration and definitions of the modified functions, and apply the following semantic patch to convert the remaining callers: @@ expression E1, E2, E3; @@ - read_sha1_file(E1.hash, E2, E3) + read_object_file(&E1, E2, E3) @@ expression E1, E2, E3; @@ - read_sha1_file(E1->hash, E2, E3) + read_object_file(E1, E2, E3) @@ expression E1, E2, E3, E4; @@ - read_sha1_file_extended(E1.hash, E2, E3, E4) + read_object_file_extended(&E1, E2, E3, E4) @@ expression E1, E2, E3, E4; @@ - read_sha1_file_extended(E1->hash, E2, E3, E4) + read_object_file_extended(E1, E2, E3, E4) Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-03-14sha1_file: convert read_object_with_reference to object_idLibravatar brian m. carlson1-2/+2
Convert read_object_with_reference to take pointers to struct object_id. Update the internals of the function accordingly. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-03-08Merge branch 'rv/grep-cleanup'Libravatar Junio C Hamano1-13/+20
Threaded "git grep" has been optimized to avoid allocation in code section that is covered under a mutex. * rv/grep-cleanup: grep: simplify grep_oid and grep_file grep: move grep_source_init outside critical section
2018-02-23grep: simplify grep_oid and grep_fileLibravatar Rasmus Villemoes1-4/+2
In the NO_PTHREADS or !num_threads case, this doesn't change anything. In the threaded case, note that grep_source_init duplicates its third argument, so there is no need to keep [path]buf.buf alive across the call of add_work(). Signed-off-by: Rasmus Villemoes <rv@rasmusvillemoes.dk> Reviewed-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-02-23grep: move grep_source_init outside critical sectionLibravatar Rasmus Villemoes1-9/+18
grep_source_init typically does three strdup()s, and in the threaded case, the call from add_work() happens while holding grep_mutex. We can thus reduce the time we hold grep_mutex by moving the grep_source_init() call out of add_work(), and simply have add_work() copy the initialized structure to the available slot in the todo array. This also simplifies the prototype of add_work(), since it no longer needs to duplicate all the parameters of grep_source_init(). In the callers of add_work(), we get to reduce the amount of code duplicated in the threaded and non-threaded cases slightly (avoiding repeating the long "GREP_SOURCE_OID, pathbuf.buf, path, oid" argument list); a subsequent cleanup patch will make that even more so. Signed-off-by: Rasmus Villemoes <rv@rasmusvillemoes.dk> Reviewed-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-02-14object: rename function 'typename' to 'type_name'Libravatar Brandon Williams1-1/+1
Rename C++ keyword in order to bring the codebase closer to being able to be compiled with a C++ compiler. Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-02-09completion: use __gitcomp_builtin in _git_grepLibravatar Nguyễn Thái Ngọc Duy1-5/+8
The new completable options are: --after-context= --before-context= --color --context --exclude-standard --quiet --recurse-submodules --textconv Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-12-19Merge branch 'bw/pathspec-match-submodule-boundary'Libravatar Junio C Hamano1-0/+1
An v2.12-era regression in pathspec match logic, which made it look into submodule tree even when it is not desired, has been fixed. * bw/pathspec-match-submodule-boundary: pathspec: only match across submodule boundaries when requested
2017-12-05pathspec: only match across submodule boundaries when requestedLibravatar Brandon Williams1-0/+1
Commit 74ed43711fd (grep: enable recurse-submodules to work on <tree> objects, 2016-12-16) taught 'tree_entry_interesting()' to be able to match across submodule boundaries in the presence of wildcards. This is done by performing literal matching up to the first wildcard and then punting to the submodule itself to perform more accurate pattern matching. Instead of introducing a new flag to request this behavior, commit 74ed43711fd overloaded the already existing 'recursive' flag in 'struct pathspec' to request this behavior. This leads to a bug where whenever any other caller has the 'recursive' flag set as well as a pathspec with wildcards that all submodules will be indicated as matches. One simple example of this is: git init repo cd repo git init submodule git -C submodule commit -m initial --allow-empty touch "[bracket]" git add "[bracket]" git commit -m bracket git add submodule git commit -m submodule git rev-list HEAD -- "[bracket]" Fix this by introducing the new flag 'recurse_submodules' in 'struct pathspec' and using this flag to determine if matches should be allowed to cross submodule boundaries. This fixes https://github.com/git-for-windows/git/issues/1371. Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-11-06Merge branch 'bw/grep-recurse-submodules'Libravatar Junio C Hamano1-0/+2
A broken access to object databases in recent update to "git grep --recurse-submodules" has been fixed. * bw/grep-recurse-submodules: grep: take the read-lock when adding a submodule
2017-11-02grep: take the read-lock when adding a submoduleLibravatar Martin Ågren1-0/+2
With --recurse-submodules, we add each submodule that we encounter to the list of alternate object databases. With threading, our changes to the list are not protected against races. Indeed, ThreadSanitizer reports a race when we call `add_to_alternates_memory()` around the same time that another thread is reading in the list through `read_sha1_file()`. Take the grep read-lock while adding the submodule. The lock is used to serialize uses of non-thread-safe parts of Git's API, including `read_sha1_file()`. Helped-by: Brandon Williams <bmwill@google.com> Signed-off-by: Martin Ågren <martin.agren@gmail.com> Acked-by: Brandon Williams <bmwill@google.com> Reviewed-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-10-18Merge branch 'jk/ref-filter-colors-fix'Libravatar Junio C Hamano1-1/+1
This is the "theoretically more correct" approach of simply stepping back to the state before plumbing commands started paying attention to "color.ui" configuration variable. Let's run with this one. * jk/ref-filter-colors-fix: tag: respect color.ui config Revert "color: check color.ui in git_default_config()" Revert "t6006: drop "always" color config tests" Revert "color: make "always" the same as "auto" in config"
2017-10-17Revert "color: check color.ui in git_default_config()"Libravatar Jeff King1-1/+1
This reverts commit 136c8c8b8fa39f1315713248473dececf20f8fe7. That commit was trying to address a bug caused by 4c7f1819b3 (make color.ui default to 'auto', 2013-06-10), in which plumbing like diff-tree defaulted to "auto" color, but did not respect a "color.ui" directive to disable it. But it also meant that we started respecting "color.ui" set to "always". This was a known problem, but 4c7f1819b3 argued that nobody ought to be doing that. However, that turned out to be wrong, and we got a number of bug reports related to "add -p" regressing in v2.14.2. Let's revert 136c8c8b8, fixing the regression to "add -p". This leaves the problem from 4c7f1819b3 unfixed, but: 1. It's a pretty obscure problem in the first place. I only noticed it while working on the color code, and we haven't got a single bug report or complaint about it. 2. We can make a more moderate fix on top by respecting "never" but not "always" for plumbing commands. This is just the minimal fix to go back to the working state we had before v2.14.2. Note that this isn't a pure revert. We now have a test in t3701 which shows off the "add -p" regression. This can be flipped to success. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-08-26Merge branch 'bw/submodule-config-cleanup'Libravatar Junio C Hamano1-4/+0
Code clean-up to avoid mixing values read from the .gitmodules file and values read from the .git/config file. * bw/submodule-config-cleanup: submodule: remove gitmodules_config unpack-trees: improve loading of .gitmodules submodule-config: lazy-load a repository's .gitmodules file submodule-config: move submodule-config functions to submodule-config.c submodule-config: remove support for overlaying repository config diff: stop allowing diff to have submodules configured in .git/config submodule: remove submodule_config callback routine unpack-trees: don't respect submodule.update submodule: don't rely on overlayed config when setting diffopts fetch: don't overlay config with submodule-config submodule--helper: don't overlay config in update-clone submodule--helper: don't overlay config in remote_submodule_branch add, reset: ensure submodules can be added or reset submodule: don't use submodule_from_name t7411: check configuration parsing errors
2017-08-23Merge branch 'jk/ref-filter-colors' into maintLibravatar Junio C Hamano1-1/+1
"%C(color name)" in the pretty print format always produced ANSI color escape codes, which was an early design mistake. They now honor the configuration (e.g. "color.ui = never") and also tty-ness of the output medium. * jk/ref-filter-colors: ref-filter: consult want_color() before emitting colors pretty: respect color settings for %C placeholders rev-list: pass diffopt->use_colors through to pretty-print for-each-ref: load config earlier color: check color.ui in git_default_config() ref-filter: pass ref_format struct to atom parsers ref-filter: factor out the parsing of sorting atoms ref-filter: make parse_ref_filter_atom a private function ref-filter: provide a function for parsing sort options ref-filter: move need_color_reset_at_eol into ref_format ref-filter: abstract ref format into its own struct ref-filter: simplify automatic color reset t: use test_decode_color rather than literal ANSI codes docs/for-each-ref: update pointer to color syntax check return value of verify_ref_format()
2017-08-22Merge branch 'bw/grep-recurse-submodules'Libravatar Junio C Hamano1-310/+86
"git grep --recurse-submodules" has been reworked to give a more consistent output across submodule boundary (and do its thing without having to fork a separate process). * bw/grep-recurse-submodules: grep: recurse in-process using 'struct repository' submodule: merge repo_read_gitmodules and gitmodules_config submodule: check for unmerged .gitmodules outside of config parsing submodule: check for unstaged .gitmodules outside of config parsing submodule: remove fetch.recursesubmodules from submodule-config parsing submodule: remove submodule.fetchjobs from submodule-config parsing config: add config_from_gitmodules cache.h: add GITMODULES_FILE macro repository: have the_repository use the_index repo_read_index: don't discard the index
2017-08-11Merge branch 'jk/ref-filter-colors'Libravatar Junio C Hamano1-1/+1
"%C(color name)" in the pretty print format always produced ANSI color escape codes, which was an early design mistake. They now honor the configuration (e.g. "color.ui = never") and also tty-ness of the output medium. * jk/ref-filter-colors: ref-filter: consult want_color() before emitting colors pretty: respect color settings for %C placeholders rev-list: pass diffopt->use_colors through to pretty-print for-each-ref: load config earlier color: check color.ui in git_default_config() ref-filter: pass ref_format struct to atom parsers ref-filter: factor out the parsing of sorting atoms ref-filter: make parse_ref_filter_atom a private function ref-filter: provide a function for parsing sort options ref-filter: move need_color_reset_at_eol into ref_format ref-filter: abstract ref format into its own struct ref-filter: simplify automatic color reset t: use test_decode_color rather than literal ANSI codes docs/for-each-ref: update pointer to color syntax check return value of verify_ref_format()
2017-08-11Merge branch 'bc/object-id'Libravatar Junio C Hamano1-4/+4
Conversion from uchar[20] to struct object_id continues. * bc/object-id: sha1_name: convert uses of 40 to GIT_SHA1_HEXSZ sha1_name: convert GET_SHA1* flags to GET_OID* sha1_name: convert get_sha1* to get_oid* Convert remaining callers of get_sha1 to get_oid. builtin/unpack-file: convert to struct object_id bisect: convert bisect_checkout to struct object_id builtin/update_ref: convert to struct object_id sequencer: convert to struct object_id remote: convert struct push_cas to struct object_id submodule: convert submodule config lookup to use object_id builtin/merge-tree: convert remaining caller of get_sha1 to object_id builtin/fsck: convert remaining caller of get_sha1 to object_id
2017-08-03submodule: remove gitmodules_configLibravatar Brandon Williams1-4/+0
Now that the submodule-config subsystem can lazily read the gitmodules file we no longer need to explicitly pre-read the gitmodules by calling 'gitmodules_config()' so let's remove it. Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-08-02Merge branch 'bc/object-id' into bw/submodule-config-cleanupLibravatar Junio C Hamano1-3/+3
* bc/object-id: sha1_name: convert uses of 40 to GIT_SHA1_HEXSZ sha1_name: convert GET_SHA1* flags to GET_OID* sha1_name: convert get_sha1* to get_oid* Convert remaining callers of get_sha1 to get_oid. builtin/unpack-file: convert to struct object_id bisect: convert bisect_checkout to struct object_id builtin/update_ref: convert to struct object_id sequencer: convert to struct object_id remote: convert struct push_cas to struct object_id submodule: convert submodule config lookup to use object_id builtin/merge-tree: convert remaining caller of get_sha1 to object_id builtin/fsck: convert remaining caller of get_sha1 to object_id tag: convert gpg_verify_tag to use struct object_id commit: convert lookup_commit_graft to struct object_id
2017-08-02grep: recurse in-process using 'struct repository'Libravatar Brandon Williams1-310/+86
Convert grep to use 'struct repository' which enables recursing into submodules to be handled in-process. Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-07-17sha1_name: convert GET_SHA1* flags to GET_OID*Libravatar brian m. carlson1-1/+1
Convert the flags for get_oid_with_context and friends to use "OID" instead of "SHA1" in their names. This transform was made by running the following one-liner on the affected files: perl -pi -e 's/GET_SHA1/GET_OID/g' Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-07-17sha1_name: convert get_sha1* to get_oid*Libravatar brian m. carlson1-2/+2
Now that all the callers of get_sha1 directly or indirectly use struct object_id, rename the functions starting with get_sha1 to start with get_oid. Convert the internals in sha1_name.c to use struct object_id as well, and eliminate explicit length checks where possible. Convert a use of 40 in get_oid_basic to GIT_SHA1_HEXSZ. Outside of sha1_name.c and cache.h, this transition was made with the following semantic patch: @@ expression E1, E2; @@ - get_sha1(E1, E2.hash) + get_oid(E1, &E2) @@ expression E1, E2; @@ - get_sha1(E1, E2->hash) + get_oid(E1, E2) @@ expression E1, E2; @@ - get_sha1_committish(E1, E2.hash) + get_oid_committish(E1, &E2) @@ expression E1, E2; @@ - get_sha1_committish(E1, E2->hash) + get_oid_committish(E1, E2) @@ expression E1, E2; @@ - get_sha1_treeish(E1, E2.hash) + get_oid_treeish(E1, &E2) @@ expression E1, E2; @@ - get_sha1_treeish(E1, E2->hash) + get_oid_treeish(E1, E2) @@ expression E1, E2; @@ - get_sha1_commit(E1, E2.hash) + get_oid_commit(E1, &E2) @@ expression E1, E2; @@ - get_sha1_commit(E1, E2->hash) + get_oid_commit(E1, E2) @@ expression E1, E2; @@ - get_sha1_tree(E1, E2.hash) + get_oid_tree(E1, &E2) @@ expression E1, E2; @@ - get_sha1_tree(E1, E2->hash) + get_oid_tree(E1, E2) @@ expression E1, E2; @@ - get_sha1_blob(E1, E2.hash) + get_oid_blob(E1, &E2) @@ expression E1, E2; @@ - get_sha1_blob(E1, E2->hash) + get_oid_blob(E1, E2) @@ expression E1, E2, E3, E4; @@ - get_sha1_with_context(E1, E2, E3.hash, E4) + get_oid_with_context(E1, E2, &E3, E4) @@ expression E1, E2, E3, E4; @@ - get_sha1_with_context(E1, E2, E3->hash, E4) + get_oid_with_context(E1, E2, E3, E4) Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-07-17submodule: convert submodule config lookup to use object_idLibravatar brian m. carlson1-2/+2
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-07-13Merge branch 'ab/grep-lose-opt-regflags'Libravatar Junio C Hamano1-2/+0
Code cleanup. * ab/grep-lose-opt-regflags: grep: remove redundant REG_NEWLINE when compiling fixed regex grep: remove regflags from the public grep_opt API grep: remove redundant and verbose re-assignments to 0 grep: remove redundant "fixed" field re-assignment to 0 grep: adjust a redundant grep pattern type assignment grep: remove redundant double assignment to 0
2017-07-13color: check color.ui in git_default_config()Libravatar Jeff King1-1/+1
Back in prehistoric times, our decision on whether or not to show color by default relied on using a config callback that either did or didn't load color config like color.diff. When we introduced color.ui, we put it in the same boat: commands had to manually respect it by using git_color_config() or its git_color_default_config() convenience wrapper. But in 4c7f1819b (make color.ui default to 'auto', 2013-06-10), that changed. Since then, we default color.ui to auto in all programs, meaning that even plumbing commands like "git diff-tree --pretty" might colorize the output. Nobody seems to have complained in the intervening years, presumably because the "is stdout a tty" check does a good job of catching the right cases. But that leaves an interesting curiosity: color.ui defaults to auto even in plumbing, but you can't actually _disable_ the color via config. So if you really hate color and set "color.ui" to false, diff-tree will still show color (but porcelain like git-diff won't). Nobody noticed that either, probably because very few people disable color. One could argue that the plumbing should _always_ disable color unless an explicit --color option is given on the command line. But in practice, this creates a lot of complications for scripts which do want plumbing to show user-visible output. They can't just pass "--color" blindly; they need to check the user's config and decide what to send. Given that nobody has complained about the current behavior, let's assume it's a good path, and follow it to its conclusion: supporting color.ui everywhere. Note that you can create havoc by setting color.ui=always in your config, but that's more or less already the case. We could disallow it entirely, but it is handy for one-offs like: git -c color.ui=always foo >not-a-tty when "foo" does not take a --color option itself. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-07-12Merge branch 'rs/use-div-round-up'Libravatar Junio C Hamano1-1/+1
Code cleanup. * rs/use-div-round-up: use DIV_ROUND_UP
2017-07-10use DIV_ROUND_UPLibravatar René Scharfe1-1/+1
Convert code that divides and rounds up to use DIV_ROUND_UP to make the intent clearer and reduce the number of magic constants. Signed-off-by: Rene Scharfe <l.s.r@web.de> Reviewed-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-07-05Merge branch 'bw/repo-object'Libravatar Junio C Hamano1-1/+2
Introduce a "repository" object to eventually make it easier to work in multiple repositories (the primary focus is to work with the superproject and its submodules) in a single process. * bw/repo-object: ls-files: use repository object repository: enable initialization of submodules submodule: convert is_submodule_initialized to work on a repository submodule: add repo_read_gitmodules submodule-config: store the_submodule_cache in the_repository repository: add index_state to struct repo config: read config from a repository object path: add repo_worktree_path and strbuf_repo_worktree_path path: add repo_git_path and strbuf_repo_git_path path: worktree_git_path() should not use file relocation path: convert do_git_path to take a 'struct repository' path: convert strbuf_git_common_path to take a 'struct repository' path: always pass in commondir to update_common_dir path: create path.h environment: store worktree in the_repository environment: place key repository state in the_repository repository: introduce the repository object environment: remove namespace_len variable setup: add comment indicating a hack setup: don't perform lazy initialization of repository state
2017-06-30grep: remove regflags from the public grep_opt APILibravatar Ævar Arnfjörð Bjarmason1-2/+0
Refactor calls to the grep machinery to always pass opt.ignore_case & opt.extended_regexp_option instead of setting the equivalent regflags bits. The bug fixed when making -i work with -P in commit 9e3cbc59d5 ("log: make --regexp-ignore-case work with --perl-regexp", 2017-05-20) was really just plastering over the code smell which this change fixes. The reason for adding the extensive commentary here is that I discovered some subtle complexity in implementing this that really should be called out explicitly to future readers. Before this change we'd rely on the difference between `extended_regexp_option` and `regflags` to serve as a membrane between our preliminary parsing of grep.extendedRegexp and grep.patternType, and what we decided to do internally. Now that those two are the same thing, it's necessary to unset `extended_regexp_option` just before we commit in cases where both of those config variables are set. See 84befcd0a4 ("grep: add a grep.patternType configuration setting", 2012-08-03) for the code and documentation related to that. The explanation of why the if/else branches in grep_commit_pattern_type() are ordered the way they are exists in that commit message, but I think it's worth calling this subtlety out explicitly with a comment for future readers. Even though grep_commit_pattern_type() is the only caller of grep_set_pattern_type_option() it's simpler to reset the extended_regexp_option flag in the latter, since 2/3 branches in the former would otherwise need to reset it, this way we can do it in one place. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-27Spelling fixesLibravatar Ville Skyttä1-1/+1
Signed-off-by: Ville Skyttä <ville.skytta@iki.fi> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-24Merge branch 'bw/config-h'Libravatar Junio C Hamano1-0/+1
Fix configuration codepath to pay proper attention to commondir that is used in multi-worktree situation, and isolate config API into its own header file. * bw/config-h: config: don't implicitly use gitdir or commondir config: respect commondir setup: teach discover_git_directory to respect the commondir config: don't include config.h by default config: remove git_config_iter config: create config.h
2017-06-23submodule: convert is_submodule_initialized to work on a repositoryLibravatar Brandon Williams1-1/+2
Convert 'is_submodule_initialized()' to take a repository object and while we're at it, lets rename the function to 'is_submodule_active()' and remove the NEEDSWORK comment. Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-23Merge branches 'bw/ls-files-sans-the-index' and 'bw/config-h' into ↵Libravatar Junio C Hamano1-0/+1
bw/repo-object * bw/ls-files-sans-the-index: ls-files: factor out tag calculation ls-files: factor out debug info into a function ls-files: convert show_files to take an index ls-files: convert show_ce_entry to take an index ls-files: convert prune_cache to take an index ls-files: convert ce_excluded to take an index ls-files: convert show_ru_info to take an index ls-files: convert show_other_files to take an index ls-files: convert show_killed_files to take an index ls-files: convert write_eolinfo to take an index ls-files: convert overlay_tree_on_cache to take an index tree: convert read_tree to take an index parameter convert: convert renormalize_buffer to take an index convert: convert convert_to_git to take an index convert: convert convert_to_git_filter_fd to take an index convert: convert crlf_to_git to take an index convert: convert get_cached_convert_stats_ascii to take an index * bw/config-h: config: don't implicitly use gitdir or commondir config: respect commondir setup: teach discover_git_directory to respect the commondir config: don't include config.h by default config: remove git_config_iter config: create config.h alias: use the early config machinery to expand aliases t7006: demonstrate a problem with aliases in subdirectories t1308: relax the test verifying that empty alias values are disallowed help: use early config when autocorrecting aliases config: report correct line number upon error discover_git_directory(): avoid setting invalid git_dir
2017-06-19Merge branch 'bw/object-id'Libravatar Junio C Hamano1-11/+11
Conversion from uchar[20] to struct object_id continues. * bw/object-id: (33 commits) diff: rename diff_fill_sha1_info to diff_fill_oid_info diffcore-rename: use is_empty_blob_oid tree-diff: convert path_appendnew to object_id tree-diff: convert diff_tree_paths to struct object_id tree-diff: convert try_to_follow_renames to struct object_id builtin/diff-tree: cleanup references to sha1 diff-tree: convert diff_tree_sha1 to struct object_id notes-merge: convert write_note_to_worktree to struct object_id notes-merge: convert verify_notes_filepair to struct object_id notes-merge: convert find_notes_merge_pair_ps to struct object_id notes-merge: convert merge_from_diffs to struct object_id notes-merge: convert notes_merge* to struct object_id tree-diff: convert diff_root_tree_sha1 to struct object_id combine-diff: convert find_paths_* to struct object_id combine-diff: convert diff_tree_combined to struct object_id diff: convert diff_flush_patch_id to struct object_id patch-ids: convert to struct object_id diff: finish conversion for prepare_temp_file to struct object_id diff: convert reuse_worktree_file to struct object_id diff: convert fill_filespec to struct object_id ...
2017-06-19Merge branch 'ab/pcre-v2'Libravatar Junio C Hamano1-3/+13
Update "perl-compatible regular expression" support to enable JIT and also allow linking with the newer PCRE v2 library. * ab/pcre-v2: grep: add support for PCRE v2 grep: un-break building with PCRE >= 8.32 without --enable-jit grep: un-break building with PCRE < 8.20 grep: un-break building with PCRE < 8.32 grep: add support for the PCRE v1 JIT API log: add -P as a synonym for --perl-regexp grep: skip pthreads overhead when using one thread grep: don't redundantly compile throwaway patterns under threading
2017-06-15config: don't include config.h by defaultLibravatar Brandon Williams1-0/+1
Stop including config.h by default in cache.h. Instead only include config.h in those files which require use of the config system. Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-13Merge branch 'sb/submodule-blanket-recursive'Libravatar Junio C Hamano1-0/+3
Many commands learned to pay attention to submodule.recurse configuration. * sb/submodule-blanket-recursive: builtin/fetch.c: respect 'submodule.recurse' option builtin/push.c: respect 'submodule.recurse' option builtin/grep.c: respect 'submodule.recurse' option Introduce 'submodule.recurse' option for worktree manipulators submodule loading: separate code path for .gitmodules and config overlay reset/checkout/read-tree: unify config callback for submodule recursion submodule test invocation: only pass additional arguments submodule recursing: do not write a config variable twice
2017-06-02Merge branch 'ab/grep-preparatory-cleanup'Libravatar Junio C Hamano1-4/+19
The internal implementation of "git grep" has seen some clean-up. * ab/grep-preparatory-cleanup: (31 commits) grep: assert that threading is enabled when calling grep_{lock,unlock} grep: given --threads with NO_PTHREADS=YesPlease, warn pack-objects: fix buggy warning about threads pack-objects & index-pack: add test for --threads warning test-lib: add a PTHREADS prerequisite grep: move is_fixed() earlier to avoid forward declaration grep: change internal *pcre* variable & function names to be *pcre1* grep: change the internal PCRE macro names to be PCRE1 grep: factor test for \0 in grep patterns into a function grep: remove redundant regflags assignments grep: catch a missing enum in switch statement perf: add a comparison test of log --grep regex engines with -F perf: add a comparison test of log --grep regex engines perf: add a comparison test of grep regex engines with -F perf: add a comparison test of grep regex engines perf: emit progress output when unpacking & building perf: add a GIT_PERF_MAKE_COMMAND for when *_MAKE_OPTS won't do grep: add tests to fix blind spots with \0 patterns grep: prepare for testing binary regexes containing rx metacharacters grep: add a test helper function for less verbose -f \0 tests ...
2017-06-02Merge branch 'jk/diff-blob'Libravatar Junio C Hamano1-1/+3
The result from "git diff" that compares two blobs, e.g. "git diff $commit1:$path $commit2:$path", used to be shown with the full object name as given on the command line, but it is more natural to use the $path in the output and use it to look up .gitattributes. * jk/diff-blob: diff: use blob path for blob/file diffs diff: use pending "path" if it is available diff: use the word "path" instead of "name" for blobs diff: pass whole pending entry in blobinfo handle_revision_arg: record paths for pending objects handle_revision_arg: record modes for "a..b" endpoints t4063: add tests of direct blob diffs get_sha1_with_context: dynamically allocate oc->path get_sha1_with_context: always initialize oc->symlink_path sha1_name: consistently refer to object_context as "oc" handle_revision_arg: add handle_dotdot() helper handle_revision_arg: hoist ".." check out of range parsing handle_revision_arg: stop using "dotdot" as a generic pointer handle_revision_arg: simplify commit reference lookups handle_revision_arg: reset "dotdot" consistently
2017-06-02grep: convert to struct object_idLibravatar Brandon Williams1-11/+11
Convert the remaining parts of grep to use struct object_id. Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-01builtin/grep.c: respect 'submodule.recurse' optionLibravatar Stefan Beller1-0/+3
In builtin/grep.c we parse the config before evaluating the command line options. This makes the task of teaching grep to respect the new config option 'submodule.recurse' very easy by just parsing that option. As an alternative I had implemented a similar structure to treat submodules as the fetch/push command have, including * aligning the meaning of the 'recurse_submodules' to possible submodule values RECURSE_SUBMODULES_* as defined in submodule.h. * having a callback to parse the value and * reacting to the RECURSE_SUBMODULES_DEFAULT state that was the initial state. However all this is not needed for a true boolean value, so let's keep it simple. However this adds another place where "submodule.recurse" is parsed. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-29Merge branch 'bc/object-id'Libravatar Junio C Hamano1-1/+1
Conversion from uchar[20] to struct object_id continues. * bc/object-id: (53 commits) object: convert parse_object* to take struct object_id tree: convert parse_tree_indirect to struct object_id sequencer: convert do_recursive_merge to struct object_id diff-lib: convert do_diff_cache to struct object_id builtin/ls-tree: convert to struct object_id merge: convert checkout_fast_forward to struct object_id sequencer: convert fast_forward_to to struct object_id builtin/ls-files: convert overlay_tree_on_cache to object_id builtin/read-tree: convert to struct object_id sha1_name: convert internals of peel_onion to object_id upload-pack: convert remaining parse_object callers to object_id revision: convert remaining parse_object callers to object_id revision: rename add_pending_sha1 to add_pending_oid http-push: convert process_ls_object and descendants to object_id refs/files-backend: convert many internals to struct object_id refs: convert struct ref_update to use struct object_id ref-filter: convert some static functions to struct object_id Convert struct ref_array_item to struct object_id Convert the verify_pack callback to struct object_id Convert lookup_tag to struct object_id ...
2017-05-26grep: skip pthreads overhead when using one threadLibravatar Ævar Arnfjörð Bjarmason1-0/+2
Skip the administrative overhead of using pthreads when only using one thread. Instead take the non-threaded path which would be taken under NO_PTHREADS. The threading support was initially added in commit 5b594f457a ("Threaded grep", 2010-01-25) with a hardcoded compile-time number of 8 threads. Later the number of threads was made configurable in commit 89f09dd34e ("grep: add --threads=<num> option and grep.threads configuration", 2015-12-15). That change did not add any special handling for --threads=1. Now we take a slightly faster path by skipping thread handling entirely when 1 thread is requested. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-26grep: don't redundantly compile throwaway patterns under threadingLibravatar Ævar Arnfjörð Bjarmason1-3/+11
Change the pattern compilation logic under threading so that grep doesn't compile a pattern it never ends up using on the non-threaded code path, only to compile it again N times for N threads which will each use their own copy, ignoring the initially compiled pattern. This redundant compilation dates back to the initial introduction of the threaded grep in commit 5b594f457a ("Threaded grep", 2010-01-25). There was never any reason for doing this redundant work other than an oversight in the initial commit. Jeff King suggested on-list in <20170414212325.fefrl3qdjigwyitd@sigill.intra.peff.net> that this might be needed to check the pattern for sanity before threaded execution commences. That's not the case. The pattern is compiled under threading in start_threads() before any concurrent execution has started by calling pthread_create(), so if the pattern contains an error we still do the right thing. I.e. die with one error before any threaded execution has commenced, instead of e.g. spewing out an error for each N threads, which could be a regression a change like this might inadvertently introduce. This change is not meant as an optimization, any performance gains from this are in the hundreds to thousands of nanoseconds at most. If we wanted more performance here we could just re-use the compiled patterns in multiple threads (regcomp(3) is thread-safe), or partially re-use them and the associated structures in the case of later PCRE JIT changes. Rather, it's just to make the code easier to reason about. It's confusing to debug this under threading & non-threading when the threading codepaths redundantly compile a pattern which is never used. The reason the patterns are recompiled is as a side-effect of duplicating the whole grep_opt structure, which is not thread safe, writable, and munged during execution. The grep_opt structure then points to the grep_pat structure where pattern or patterns are stored. I looked into e.g. splitting the API into some "do & alloc threadsafe stuff", "spawn thread", "do and alloc non-threadsafe stuff", but the execution time of grep_opt_dup() & pattern compilation is trivial compared to actually executing the grep, so there was no point. Even with the more expensive JIT changes to follow the most expensive PCRE patterns take something like 0.0X milliseconds to compile at most[1]. The undocumented --debug mode added in commit 17bf35a3c7 ("grep: teach --debug option to dump the parse tree", 2012-09-13) still works properly with this change. It only emits debugging info during pattern compilation, which is now dumped by the pattern compiled just before the first thread is started. 1. http://sljit.sourceforge.net/pcre.html Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-26grep: assert that threading is enabled when calling grep_{lock,unlock}Libravatar Ævar Arnfjörð Bjarmason1-4/+4
Change the grep_{lock,unlock} functions to assert that num_threads is true, instead of only locking & unlocking the pthread mutex lock when it is. These functions are never called when num_threads isn't true, this logic has gone through multiple iterations since the initial introduction of grep threading in commit 5b594f457a ("Threaded grep", 2010-01-25), but ever since then they'd only be called if num_threads was true, so this check made the code confusing to read. Replace the check with an assertion, so that it's clear to the reader that this code path is never taken unless we're spawning threads. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>