summaryrefslogtreecommitdiff
path: root/builtin
AgeCommit message (Collapse)AuthorFilesLines
2017-06-19Merge branch 'ab/pcre-v2'Libravatar Junio C Hamano1-3/+13
Update "perl-compatible regular expression" support to enable JIT and also allow linking with the newer PCRE v2 library. * ab/pcre-v2: grep: add support for PCRE v2 grep: un-break building with PCRE >= 8.32 without --enable-jit grep: un-break building with PCRE < 8.20 grep: un-break building with PCRE < 8.32 grep: add support for the PCRE v1 JIT API log: add -P as a synonym for --perl-regexp grep: skip pthreads overhead when using one thread grep: don't redundantly compile throwaway patterns under threading
2017-06-13Merge branch 'nd/fopen-errors'Libravatar Junio C Hamano9-25/+17
We often try to open a file for reading whose existence is optional, and silently ignore errors from open/fopen; report such errors if they are not due to missing files. * nd/fopen-errors: mingw_fopen: report ENOENT for invalid file names mingw: verify that paths are not mistaken for remote nicknames log: fix memory leak in open_next_file() rerere.c: move error_errno() closer to the source system call print errno when reporting a system call error wrapper.c: make warn_on_inaccessible() static wrapper.c: add and use fopen_or_warn() wrapper.c: add and use warn_on_fopen_errors() config.mak.uname: set FREAD_READS_DIRECTORIES for Darwin, too config.mak.uname: set FREAD_READS_DIRECTORIES for Linux and FreeBSD clone: use xfopen() instead of fopen() use xfopen() in more places git_fopen: fix a sparse 'not declared' warning
2017-06-13Merge branch 'jc/diff-tree-stale-comment'Libravatar Junio C Hamano1-3/+5
Comment fix. * jc/diff-tree-stale-comment: diff-tree: update stale in-code comments
2017-06-13Merge branch 'sb/submodule-blanket-recursive'Libravatar Junio C Hamano6-78/+38
Many commands learned to pay attention to submodule.recurse configuration. * sb/submodule-blanket-recursive: builtin/fetch.c: respect 'submodule.recurse' option builtin/push.c: respect 'submodule.recurse' option builtin/grep.c: respect 'submodule.recurse' option Introduce 'submodule.recurse' option for worktree manipulators submodule loading: separate code path for .gitmodules and config overlay reset/checkout/read-tree: unify config callback for submodule recursion submodule test invocation: only pass additional arguments submodule recursing: do not write a config variable twice
2017-06-13Merge branch 'jc/noent-notdir'Libravatar Junio C Hamano2-2/+2
Our code often opens a path to an optional file, to work on its contents when we can successfully open it. We can ignore a failure to open if such an optional file does not exist, but we do want to report a failure in opening for other reasons (e.g. we got an I/O error, or the file is there, but we lack the permission to open). The exact errors we need to ignore are ENOENT (obviously) and ENOTDIR (less obvious). Instead of repeating comparison of errno with these two constants, introduce a helper function to do so. * jc/noent-notdir: treewide: use is_missing_file_error() where ENOENT and ENOTDIR are checked compat-util: is_missing_file_error()
2017-06-05Merge branch 'tb/pull-ff-rebase-autostash'Libravatar Junio C Hamano1-11/+14
"git pull --rebase --autostash" didn't auto-stash when the local history fast-forwards to the upstream. * tb/pull-ff-rebase-autostash: pull: ff --rebase --autostash works in dirty repo
2017-06-05Merge branch 'js/blame-lib'Libravatar Junio C Hamano2-2048/+65
The internal logic used in "git blame" has been libified to make it easier to use by cgit. * js/blame-lib: (29 commits) blame: move entry prepend to libgit blame: move scoreboard setup to libgit blame: move scoreboard-related methods to libgit blame: move fake-commit-related methods to libgit blame: move origin-related methods to libgit blame: move core structures to header blame: create entry prepend function blame: create scoreboard setup function blame: create scoreboard init function blame: rework methods that determine 'final' commit blame: wrap blame_sort and compare_blame_final blame: move progress updates to a scoreboard callback blame: make sanity_check use a callback in scoreboard blame: move no_whole_file_rename flag to scoreboard blame: move xdl_opts flags to scoreboard blame: move show_root flag to scoreboard blame: move reverse flag to scoreboard blame: move contents_from to scoreboard blame: move copy/move thresholds to scoreboard blame: move stat counters to scoreboard ...
2017-06-05Merge branch 'mh/packed-ref-store-prep'Libravatar Junio C Hamano2-3/+3
The implementation of "ref" API around the "packed refs" have been cleaned up, in preparation for further changes. * mh/packed-ref-store-prep: (25 commits) cache_ref_iterator_begin(): avoid priming unneeded directories ref-filter: limit traversal to prefix create_ref_entry(): remove `check_name` option refs_ref_iterator_begin(): handle `GIT_REF_PARANOIA` read_packed_refs(): report unexpected fopen() failures read_packed_refs(): do more of the work of reading packed refs get_packed_ref_cache(): assume "packed-refs" won't change while locked should_pack_ref(): new function, extracted from `files_pack_refs()` ref_update_reject_duplicates(): add a sanity check ref_update_reject_duplicates(): use `size_t` rather than `int` ref_update_reject_duplicates(): expose function to whole refs module ref_transaction_prepare(): new optional step for reference updates ref_transaction_commit(): check for valid `transaction->state` files_transaction_cleanup(): new helper function files_ref_store: put the packed files lock directly in this struct files-backend: move `lock` member to `files_ref_store` lockfile: add a new method, is_lock_file_locked() ref_store: take a `msg` parameter when deleting references refs: use `size_t` indexes when iterating over ref transaction updates refs_ref_iterator_begin(): don't check prefixes redundantly ...
2017-06-05Merge branch 'mb/diff-default-to-indent-heuristics'Libravatar Junio C Hamano3-3/+3
Make the "indent" heuristics the default in "diff" and diff.indentHeuristics configuration variable an escape hatch for those who do no want it. * mb/diff-default-to-indent-heuristics: add--interactive: drop diff.indentHeuristic handling diff: enable indent heuristic by default diff: have the diff-* builtins configure diff before initializing revisions diff: make the indent heuristic part of diff's basic configuration
2017-06-04Merge branch 'ab/c-translators-comment-style'Libravatar Junio C Hamano3-10/+18
Update the C style recommendation for notes for translators, as recent versions of gettext tools can work with our style of multi-line comments. * ab/c-translators-comment-style: C style: use standard style for "TRANSLATORS" comments
2017-06-04Merge branch 'jk/unbreak-am-h'Libravatar Junio C Hamano1-0/+3
"git am -h" triggered a BUG(). * jk/unbreak-am-h: am: handle "-h" argument earlier
2017-06-02Merge branch 'ab/grep-preparatory-cleanup'Libravatar Junio C Hamano2-5/+22
The internal implementation of "git grep" has seen some clean-up. * ab/grep-preparatory-cleanup: (31 commits) grep: assert that threading is enabled when calling grep_{lock,unlock} grep: given --threads with NO_PTHREADS=YesPlease, warn pack-objects: fix buggy warning about threads pack-objects & index-pack: add test for --threads warning test-lib: add a PTHREADS prerequisite grep: move is_fixed() earlier to avoid forward declaration grep: change internal *pcre* variable & function names to be *pcre1* grep: change the internal PCRE macro names to be PCRE1 grep: factor test for \0 in grep patterns into a function grep: remove redundant regflags assignments grep: catch a missing enum in switch statement perf: add a comparison test of log --grep regex engines with -F perf: add a comparison test of log --grep regex engines perf: add a comparison test of grep regex engines with -F perf: add a comparison test of grep regex engines perf: emit progress output when unpacking & building perf: add a GIT_PERF_MAKE_COMMAND for when *_MAKE_OPTS won't do grep: add tests to fix blind spots with \0 patterns grep: prepare for testing binary regexes containing rx metacharacters grep: add a test helper function for less verbose -f \0 tests ...
2017-06-02Merge branch 'jk/diff-blob'Libravatar Junio C Hamano4-36/+42
The result from "git diff" that compares two blobs, e.g. "git diff $commit1:$path $commit2:$path", used to be shown with the full object name as given on the command line, but it is more natural to use the $path in the output and use it to look up .gitattributes. * jk/diff-blob: diff: use blob path for blob/file diffs diff: use pending "path" if it is available diff: use the word "path" instead of "name" for blobs diff: pass whole pending entry in blobinfo handle_revision_arg: record paths for pending objects handle_revision_arg: record modes for "a..b" endpoints t4063: add tests of direct blob diffs get_sha1_with_context: dynamically allocate oc->path get_sha1_with_context: always initialize oc->symlink_path sha1_name: consistently refer to object_context as "oc" handle_revision_arg: add handle_dotdot() helper handle_revision_arg: hoist ".." check out of range parsing handle_revision_arg: stop using "dotdot" as a generic pointer handle_revision_arg: simplify commit reference lookups handle_revision_arg: reset "dotdot" consistently
2017-06-02Merge branch 'sl/clean-d-ignored-fix'Libravatar Junio C Hamano1-0/+42
"git clean -d" used to clean directories that has ignored files, even though the command should not lose ignored ones without "-x". "git status --ignored" did not list ignored and untracked files without "-uall". These have been corrected. * sl/clean-d-ignored-fix: clean: teach clean -d to preserve ignored paths dir: expose cmp_name() and check_contains() dir: hide untracked contents of untracked dirs dir: recurse into untracked dirs for ignored files t7061: status --ignored should search untracked dirs t7300: clean -d should skip dirs with ignored files
2017-06-02diff-tree: update stale in-code commentsLibravatar Junio C Hamano1-3/+5
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-01pull: ff --rebase --autostash works in dirty repoLibravatar Tyler Brazier1-11/+14
When `git pull --rebase --autostash` in a dirty repository resulted in a fast-forward, nothing was being autostashed and the pull failed. This was due to a shortcut to avoid running rebase when we can fast-forward, but autostash is ignored on that codepath. Now we will only take the shortcut if autostash is not in effect. Based on a few tests against the git.git repo, the shortcut does not seem to give us significant performance benefits, on Linux at least. Regardless, it is more important to be correct than to be fast. Signed-off-by: Tyler Brazier <tyler@tylerbrazier.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-01builtin/fetch.c: respect 'submodule.recurse' optionLibravatar Stefan Beller1-0/+7
Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-01builtin/push.c: respect 'submodule.recurse' optionLibravatar Stefan Beller1-0/+4
The closest mapping from the boolean 'submodule.recurse' set to "yes" to the variety of submodule push modes is "on-demand", so implement that. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-01builtin/grep.c: respect 'submodule.recurse' optionLibravatar Stefan Beller1-0/+3
In builtin/grep.c we parse the config before evaluating the command line options. This makes the task of teaching grep to respect the new config option 'submodule.recurse' very easy by just parsing that option. As an alternative I had implemented a similar structure to treat submodules as the fetch/push command have, including * aligning the meaning of the 'recurse_submodules' to possible submodule values RECURSE_SUBMODULES_* as defined in submodule.h. * having a callback to parse the value and * reacting to the RECURSE_SUBMODULES_DEFAULT state that was the initial state. However all this is not needed for a true boolean value, so let's keep it simple. However this adds another place where "submodule.recurse" is parsed. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-01Introduce 'submodule.recurse' option for worktree manipulatorsLibravatar Stefan Beller3-3/+19
Any command that understands '--recurse-submodules' can have its default changed to true, by setting the new 'submodule.recurse' option. This patch includes read-tree/checkout/reset for working tree manipulating commands. Later patches will cover other commands. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-31C style: use standard style for "TRANSLATORS" commentsLibravatar Ævar Arnfjörð Bjarmason3-10/+18
Change all the "TRANSLATORS: [...]" comments in the C code to use the regular Git coding style, and amend the style guide so that the example there uses that style. This custom style was necessary back in 2010 when the gettext support was initially added, and was subsequently documented in commit cbcfd4e3ea ("i18n: mention "TRANSLATORS:" marker in Documentation/CodingGuidelines", 2014-04-18). GNU xgettext hasn't had the parsing limitation that necessitated this exception for almost 3 years. Since its 0.19 release on 2014-06-02 it's been able to recognize TRANSLATOR comments in the standard Git comment syntax[1]. Usually we'd like to keep compatibility with software that's that young, but in this case literally the only person who needs to be using a gettext newer than 3 years old is Jiang Xin (the only person who runs & commits "make pot" results), so I think in this case we can make an exception. This xgettext parsing feature was added after a thread on the Git mailing list[2] which continued on the bug-gettext[3] list, but we never subsequently changed our style & styleguide, do so. There are already longstanding changes in git that use the standard comment style & have their TRANSLATORS comments extracted properly without getting the literal "*"'s mixed up in the text, as would happen before xgettext 0.19. Commit 7ff2683253 ("builtin-am: implement -i/--interactive", 2015-08-04) added one such comment, which in commit df0617bfa7 ("l10n: git.pot: v2.6.0 round 1 (123 new, 41 removed)", 2015-09-05) got picked up in the po/git.pot file with the right format, showing that Jiang already runs a modern xgettext. The xgettext parser does not handle the sort of non-standard comment style that I'm amending here in sequencer.c, but that isn't standard Git comment syntax anyway. With this change to sequencer.c & "make pot" the comment in the pot file is now correct: #. TRANSLATORS: %s will be "revert", "cherry-pick" or -#. * "rebase -i". +#. "rebase -i". 1. http://git.savannah.gnu.org/cgit/gettext.git/commit/?id=10af7fe6bd 2. <2ce9ec406501d112e032c8208417f8100bed04c6.1397712142.git.worldhello.net@gmail.com> (https://public-inbox.org/git/2ce9ec406501d112e032c8208417f8100bed04c6.1397712142.git.worldhello.net@gmail.com/) 3. https://lists.gnu.org/archive/html/bug-gettext/2014-04/msg00016.html Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Acked-by: Jiang Xin <worldhello.net@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-30am: handle "-h" argument earlierLibravatar Jeff King1-0/+3
If the user provides "-h" on the command line, then our parse_options() invocation will show a usage message and quit. But if "-h" is the only argument, the git wrapper behaves specially: it ignores our RUN_SETUP flag and calls cmd_am() without having done repository setup at all. This is due to 99caeed05 (Let 'git <command> -h' show usage without a git dir, 2009-11-09). Before cmd_am() calls parse_options(), though, it runs a few other setup functions. One of these is am_state_init(), which uses git_pathdup() to set up the default rebase-apply path. But calling git_pathdup() when we haven't done repository setup will fall back to using ".git". That's mostly harmless (since we won't use the value anyway), but is forbidden since b1ef400eec ("setup_git_env: avoid blind fall-back to ".git"", 2016-10-20), and we now BUG(). We can't easily move that setup to after the parse_options() call; the point is to set up defaults that are overwritten by the option parsing. Instead, we'll detect the "-h" case early and show the usage then. This matches the behavior of other builtins which have a similar setup-ordering issue (e.g., git-branch). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-30reset/checkout/read-tree: unify config callback for submodule recursionLibravatar Stefan Beller3-76/+6
The callback function is essentially duplicated 3 times. Remove all of them and offer a new callback function, that lives in submodule.c By putting the callback function there, we no longer need the function 'set_config_update_recurse_submodules', nor duplicate the global variable in each builtin as well as submodule.c In the three builtins we have different 2 ways how to load the .gitmodules and config file, which are slightly different. git-checkout has to load the submodule config all the time due to 23b4c7bcc5 (checkout: Use submodule.*.ignore settings from .git/config and .gitmodules, 2010-08-28) git-reset and git-read-tree do not respect these diff settings, so loading the submodule configuration is optional. Also put that into submodule.c for code deduplication. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-30submodule recursing: do not write a config variable twiceLibravatar Stefan Beller3-3/+3
The command line option for '--recurse-submodules' is implemented using an OPTION_CALLBACK, which takes both the callback (that sets the file static global variable) as well as passes the same file static global variable to the option parsing machinery to assign it. This is fixed in this commit by passing NULL as the variable. The callback sets it instead Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-30Merge branch 'ab/grep-preparatory-cleanup' into sb/submodule-blanket-recursiveLibravatar Junio C Hamano2-5/+22
* ab/grep-preparatory-cleanup: (31 commits) grep: assert that threading is enabled when calling grep_{lock,unlock} grep: given --threads with NO_PTHREADS=YesPlease, warn pack-objects: fix buggy warning about threads pack-objects & index-pack: add test for --threads warning test-lib: add a PTHREADS prerequisite grep: move is_fixed() earlier to avoid forward declaration grep: change internal *pcre* variable & function names to be *pcre1* grep: change the internal PCRE macro names to be PCRE1 grep: factor test for \0 in grep patterns into a function grep: remove redundant regflags assignments grep: catch a missing enum in switch statement perf: add a comparison test of log --grep regex engines with -F perf: add a comparison test of log --grep regex engines perf: add a comparison test of grep regex engines with -F perf: add a comparison test of grep regex engines perf: emit progress output when unpacking & building perf: add a GIT_PERF_MAKE_COMMAND for when *_MAKE_OPTS won't do grep: add tests to fix blind spots with \0 patterns grep: prepare for testing binary regexes containing rx metacharacters grep: add a test helper function for less verbose -f \0 tests ...
2017-05-30Merge branch 'js/larger-timestamps'Libravatar Junio C Hamano1-1/+1
A follow-up hotfix for a topic already in 'master'. * js/larger-timestamps: name-rev: change a "long" variable to timestamp_t
2017-05-30Merge branch 'bw/pathspec-sans-the-index'Libravatar Junio C Hamano6-17/+39
Simplify parse_pathspec() codepath and stop it from looking at the default in-core index. * bw/pathspec-sans-the-index: pathspec: convert find_pathspecs_matching_against_index to take an index pathspec: remove PATHSPEC_STRIP_SUBMODULE_SLASH_CHEAP ls-files: prevent prune_cache from overeagerly pruning submodules pathspec: remove PATHSPEC_STRIP_SUBMODULE_SLASH_EXPENSIVE flag submodule: add die_in_unpopulated_submodule function pathspec: provide a more descriptive die message
2017-05-30Merge branch 'jc/name-rev-lw-tag'Libravatar Junio C Hamano1-7/+52
"git describe --contains" penalized light-weight tags so much that they were almost never considered. Instead, give them about the same chance to be considered as an annotated tag that is the same age as the underlying commit would. * jc/name-rev-lw-tag: name-rev: favor describing with tags and use committer date to tiebreak name-rev: refactor logic to see if a new candidate is a better name
2017-05-30treewide: use is_missing_file_error() where ENOENT and ENOTDIR are checkedLibravatar Junio C Hamano2-2/+2
Using the is_missing_file_error() helper introduced in the previous step, update all hits from $ git grep -e ENOENT --and -e ENOTDIR There are codepaths that only check ENOENT, and it is possible that some of them should be checking both. Updating them is kept out of this step deliberately, as we do not want to change behaviour in this step. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-29Merge branch 'bm/interpret-trailers-cut-line-is-eom'Libravatar Junio C Hamano1-1/+1
"git interpret-trailers", when used as GIT_EDITOR for "git commit -v", looked for and appended to a trailer block at the very end, i.e. at the end of the "diff" output. The command has been corrected to pay attention to the cut-mark line "commit -v" adds to the buffer---the real trailer block should appear just before it. * bm/interpret-trailers-cut-line-is-eom: interpret-trailers: honor the cut line
2017-05-29Merge branch 'ja/do-not-ask-needless-questions'Libravatar Junio C Hamano3-7/+8
Git sometimes gives an advice in a rhetorical question that does not require an answer, which can confuse new users and non native speakers. Attempt to rephrase them. * ja/do-not-ask-needless-questions: git-filter-branch: be more direct in an error message read-tree -m: make error message for merging 0 trees less smart aleck usability: don't ask questions if no reply is required
2017-05-29Merge branch 'jk/bug-to-abort'Libravatar Junio C Hamano1-0/+3
Introduce the BUG() macro to improve die("BUG: ..."). * jk/bug-to-abort: usage: add NORETURN to BUG() function definitions config: complain about --local outside of a git repo setup_git_env: convert die("BUG") to BUG() usage.c: add BUG() function
2017-05-29Merge branch 'jc/read-tree-empty-with-m'Libravatar Junio C Hamano1-1/+1
"git read-tree -m" (no tree-ish) gave a nonsense suggestion "use --empty if you want to clear the index". With "-m", such a request will still fail anyway, as you'd need to name at least one tree-ish to be merged. * jc/read-tree-empty-with-m: read-tree: "read-tree -m --empty" does not make sense
2017-05-29Merge branch 'js/plug-leaks'Libravatar Junio C Hamano11-35/+69
Fix memory leaks pointed out by Coverity (and people). * js/plug-leaks: (26 commits) checkout: fix memory leak submodule_uses_worktrees(): plug memory leak show_worktree(): plug memory leak name-rev: avoid leaking memory in the `deref` case remote: plug memory leak in match_explicit() add_reflog_for_walk: avoid memory leak shallow: avoid memory leak line-log: avoid memory leak receive-pack: plug memory leak in update() fast-export: avoid leaking memory in handle_tag() mktree: plug memory leaks reported by Coverity pack-redundant: plug memory leak setup_discovered_git_dir(): plug memory leak setup_bare_git_dir(): help static analysis split_commit_in_progress(): simplify & fix memory leak checkout: fix memory leak cat-file: fix memory leak mailinfo & mailsplit: check for EOF while parsing status: close file descriptor after reading git-rebase-todo difftool: address a couple of resource/memory leaks ...
2017-05-29Merge branch 'jk/disable-pack-reuse-when-broken'Libravatar Junio C Hamano1-1/+5
"pack-objects" can stream a slice of an existing packfile out when the pack bitmap can tell that the reachable objects are all needed in the output, without inspecting individual objects. This strategy however would not work well when "--local" and other options are in use, and need to be disabled. * jk/disable-pack-reuse-when-broken: t5310: fix "; do" style pack-objects: disable pack reuse for object-selection options
2017-05-29Merge branch 'bc/object-id'Libravatar Junio C Hamano38-311/+331
Conversion from uchar[20] to struct object_id continues. * bc/object-id: (53 commits) object: convert parse_object* to take struct object_id tree: convert parse_tree_indirect to struct object_id sequencer: convert do_recursive_merge to struct object_id diff-lib: convert do_diff_cache to struct object_id builtin/ls-tree: convert to struct object_id merge: convert checkout_fast_forward to struct object_id sequencer: convert fast_forward_to to struct object_id builtin/ls-files: convert overlay_tree_on_cache to object_id builtin/read-tree: convert to struct object_id sha1_name: convert internals of peel_onion to object_id upload-pack: convert remaining parse_object callers to object_id revision: convert remaining parse_object callers to object_id revision: rename add_pending_sha1 to add_pending_oid http-push: convert process_ls_object and descendants to object_id refs/files-backend: convert many internals to struct object_id refs: convert struct ref_update to use struct object_id ref-filter: convert some static functions to struct object_id Convert struct ref_array_item to struct object_id Convert the verify_pack callback to struct object_id Convert lookup_tag to struct object_id ...
2017-05-29Merge branch 'jc/repack-threads'Libravatar Junio C Hamano1-0/+5
"git repack" learned to accept the --threads=<n> option and pass it to pack-objects. * jc/repack-threads: repack: accept --threads=<n> and pass it down to pack-objects
2017-05-29Merge branch 'bw/dir-c-stops-relying-on-the-index'Libravatar Junio C Hamano5-9/+11
API update. * bw/dir-c-stops-relying-on-the-index: dir: convert fill_directory to take an index dir: convert read_directory to take an index dir: convert read_directory_recursive to take an index dir: convert open_cached_dir to take an index dir: convert is_excluded to take an index dir: convert prep_exclude to take an index dir: convert add_excludes to take an index dir: convert is_excluded_from_list to take an index dir: convert last_exclude_matching_from_list to take an index dir: convert dir_add* to take an index dir: convert get_dtype to take index dir: convert directory_exists_in_index to take index dir: convert read_skip_worktree_file_from_index to take an index dir: stop using the index compatibility macros
2017-05-29Merge branch 'sb/reset-recurse-submodules'Libravatar Junio C Hamano1-0/+30
"git reset" learned "--recurse-submodules" option. * sb/reset-recurse-submodules: builtin/reset: add --recurse-submodules switch submodule.c: submodule_move_head works with broken submodules submodule.c: uninitialized submodules are ignored in recursive commands entry.c: submodule recursing: respect force flag correctly
2017-05-26grep: skip pthreads overhead when using one threadLibravatar Ævar Arnfjörð Bjarmason1-0/+2
Skip the administrative overhead of using pthreads when only using one thread. Instead take the non-threaded path which would be taken under NO_PTHREADS. The threading support was initially added in commit 5b594f457a ("Threaded grep", 2010-01-25) with a hardcoded compile-time number of 8 threads. Later the number of threads was made configurable in commit 89f09dd34e ("grep: add --threads=<num> option and grep.threads configuration", 2015-12-15). That change did not add any special handling for --threads=1. Now we take a slightly faster path by skipping thread handling entirely when 1 thread is requested. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-26grep: don't redundantly compile throwaway patterns under threadingLibravatar Ævar Arnfjörð Bjarmason1-3/+11
Change the pattern compilation logic under threading so that grep doesn't compile a pattern it never ends up using on the non-threaded code path, only to compile it again N times for N threads which will each use their own copy, ignoring the initially compiled pattern. This redundant compilation dates back to the initial introduction of the threaded grep in commit 5b594f457a ("Threaded grep", 2010-01-25). There was never any reason for doing this redundant work other than an oversight in the initial commit. Jeff King suggested on-list in <20170414212325.fefrl3qdjigwyitd@sigill.intra.peff.net> that this might be needed to check the pattern for sanity before threaded execution commences. That's not the case. The pattern is compiled under threading in start_threads() before any concurrent execution has started by calling pthread_create(), so if the pattern contains an error we still do the right thing. I.e. die with one error before any threaded execution has commenced, instead of e.g. spewing out an error for each N threads, which could be a regression a change like this might inadvertently introduce. This change is not meant as an optimization, any performance gains from this are in the hundreds to thousands of nanoseconds at most. If we wanted more performance here we could just re-use the compiled patterns in multiple threads (regcomp(3) is thread-safe), or partially re-use them and the associated structures in the case of later PCRE JIT changes. Rather, it's just to make the code easier to reason about. It's confusing to debug this under threading & non-threading when the threading codepaths redundantly compile a pattern which is never used. The reason the patterns are recompiled is as a side-effect of duplicating the whole grep_opt structure, which is not thread safe, writable, and munged during execution. The grep_opt structure then points to the grep_pat structure where pattern or patterns are stored. I looked into e.g. splitting the API into some "do & alloc threadsafe stuff", "spawn thread", "do and alloc non-threadsafe stuff", but the execution time of grep_opt_dup() & pattern compilation is trivial compared to actually executing the grep, so there was no point. Even with the more expensive JIT changes to follow the most expensive PCRE patterns take something like 0.0X milliseconds to compile at most[1]. The undocumented --debug mode added in commit 17bf35a3c7 ("grep: teach --debug option to dump the parse tree", 2012-09-13) still works properly with this change. It only emits debugging info during pattern compilation, which is now dumped by the pattern compiled just before the first thread is started. 1. http://sljit.sourceforge.net/pcre.html Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-26grep: assert that threading is enabled when calling grep_{lock,unlock}Libravatar Ævar Arnfjörð Bjarmason1-4/+4
Change the grep_{lock,unlock} functions to assert that num_threads is true, instead of only locking & unlocking the pthread mutex lock when it is. These functions are never called when num_threads isn't true, this logic has gone through multiple iterations since the initial introduction of grep threading in commit 5b594f457a ("Threaded grep", 2010-01-25), but ever since then they'd only be called if num_threads was true, so this check made the code confusing to read. Replace the check with an assertion, so that it's clear to the reader that this code path is never taken unless we're spawning threads. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-26grep: given --threads with NO_PTHREADS=YesPlease, warnLibravatar Ævar Arnfjörð Bjarmason1-0/+13
Add a warning about missing thread support when grep.threads or --threads is set to a non 0 (default) or 1 (no parallelism) value under NO_PTHREADS=YesPlease. This is for consistency with the index-pack & pack-objects commands, which also take a --threads option & are configurable via pack.threads, and have long warned about the same under NO_PTHREADS=YesPlease. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-26pack-objects: fix buggy warning about threadsLibravatar Ævar Arnfjörð Bjarmason1-1/+3
Fix a buggy warning about threads under NO_PTHREADS=YesPlease. Due to re-using the delta_search_threads variable for both the state of the "pack.threads" config & the --threads option, setting "pack.threads" but not supplying --threads would trigger the warning for both "pack.threads" & --threads. Solve this bug by resetting the delta_search_threads variable in git_pack_config(), it might then be set by --threads again and be subsequently warned about, as the test I'm changing here asserts. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-26grep: catch a missing enum in switch statementLibravatar Ævar Arnfjörð Bjarmason1-0/+2
Add a die(...) to a default case for the switch statement selecting between grep pattern types under --recurse-submodules. Normally this would be caught by -Wswitch, but the grep_pattern_type type is converted to int by going through parse_options(). Changing the argument type passed to compile_submodule_options() won't work, the value will just get coerced. The -Wswitch-default warning will warn about it, but that produces a lot of noise across the codebase, this potential issue would be drowned in that noise. Thus catching this at runtime is the least bad option. This won't ever trigger in practice, but if a new pattern type were to be added this catches an otherwise silent bug during development. See commit 0281e487fd ("grep: optionally recurse into submodules", 2016-12-16) for the initial addition of this code. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-26log: fix memory leak in open_next_file()Libravatar Nguyễn Thái Ngọc Duy1-4/+8
Noticed-by: Jeff King <peff@peff.net> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-26print errno when reporting a system call errorLibravatar Nguyễn Thái Ngọc Duy1-1/+2
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-26wrapper.c: add and use fopen_or_warn()Libravatar Nguyễn Thái Ngọc Duy1-1/+1
When fopen() returns NULL, it could be because the given path does not exist, but it could also be some other errors and the caller has to check. Add a wrapper so we don't have to repeat the same error check everywhere. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-26clone: use xfopen() instead of fopen()Libravatar Nguyễn Thái Ngọc Duy1-1/+1
copy_alternates() called fopen() without handling errors. By switching to xfopen(), this bug is fixed. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-26use xfopen() in more placesLibravatar Nguyễn Thái Ngọc Duy6-20/+7
xfopen() - provides error details - explains error on reading, or writing, or whatever operation - has l10n support - prints file name in the error Some of these are missing in the places that are replaced with xfopen(), which is a clear win. In some other places, it's just less code (not as clearly a win as the previous case but still is). The only slight regresssion is in remote-testsvn, where we don't report the file class (marks files) in the error messages anymore. But since this is a _test_ svn remote transport, I'm not too concerned. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>