summaryrefslogtreecommitdiff
path: root/builtin
AgeCommit message (Collapse)AuthorFilesLines
2020-05-01Merge branch 'js/anonymise-push-url-in-errors'Libravatar Junio C Hamano1-2/+4
Error and verbose trace messages from "git push" did not redact credential material embedded in URLs. * js/anonymise-push-url-in-errors: push: anonymize URLs in error messages and warnings
2020-05-01Merge branch 'es/bugreport'Libravatar Junio C Hamano1-0/+86
The "bugreport" tool. * es/bugreport: bugreport: drop extraneous includes bugreport: add compiler info bugreport: add uname info bugreport: gather git version and build info bugreport: add tool to generate debugging info help: move list_config_help to builtin/help
2020-05-01Merge branch 'en/rebase-root-and-fork-point-are-incompatible'Libravatar Junio C Hamano1-0/+3
Incompatible options "--root" and "--fork-point" of "git rebase" have been marked and documented as being incompatible. * en/rebase-root-and-fork-point-are-incompatible: rebase: display an error if --root and --fork-point are both provided
2020-05-01Merge branch 'ds/blame-on-bloom'Libravatar Junio C Hamano3-5/+16
"git blame" learns to take advantage of the "changed-paths" Bloom filter stored in the commit-graph file. * ds/blame-on-bloom: test-bloom: check that we have expected arguments test-bloom: fix some whitespace issues blame: drop unused parameter from maybe_changed_path blame: use changed-path Bloom filters tests: write commit-graph with Bloom filters revision: complicated pathspecs disable filters
2020-05-01Merge branch 'gs/commit-graph-path-filter'Libravatar Junio C Hamano1-2/+8
Introduce an extension to the commit-graph to make it efficient to check for the paths that were modified at each commit using Bloom filters. * gs/commit-graph-path-filter: bloom: ignore renames when computing changed paths commit-graph: add GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS test flag t4216: add end to end tests for git log with Bloom filters revision.c: add trace2 stats around Bloom filter usage revision.c: use Bloom filters to speed up path based revision walks commit-graph: add --changed-paths option to write subcommand commit-graph: reuse existing Bloom filters during write commit-graph: write Bloom filters to commit graph file commit-graph: examine commits by generation number commit-graph: examine changed-path objects in pack order commit-graph: compute Bloom filters for changed paths diff: halt tree-diff early after max_changes bloom.c: core Bloom filter implementation for changed paths. bloom.c: introduce core Bloom filter constructs bloom.c: add the murmur3 hash implementation commit-graph: define and use MAX_NUM_CHUNKS
2020-05-01Merge branch 'tb/commit-graph-split-strategy'Libravatar Junio C Hamano1-7/+45
"git commit-graph write" learned different ways to write out split files. * tb/commit-graph-split-strategy: Revert "commit-graph.c: introduce '--[no-]check-oids'" commit-graph.c: introduce '--[no-]check-oids' commit-graph.h: replace 'commit_hex' with 'commits' oidset: introduce 'oidset_size' builtin/commit-graph.c: introduce split strategy 'replace' builtin/commit-graph.c: introduce split strategy 'no-merge' builtin/commit-graph.c: support for '--split[=<strategy>]' t/helper/test-read-graph.c: support commit-graph chains
2020-05-01Merge branch 'tb/reset-shallow'Libravatar Junio C Hamano1-2/+2
Fix in-core inconsistency after fetching into a shallow repository that broke the code to write out commit-graph. * tb/reset-shallow: shallow.c: use '{commit,rollback}_shallow_file' t5537: use test_write_lines and indented heredocs for readability
2020-04-29Merge branch 'ps/transactional-update-ref-stdin'Libravatar Junio C Hamano1-66/+179
"git update-ref --stdin" learned a handful of new verbs to let the user control ref update transactions more explicitly, which helps as an ingredient to implement two-phase commit-style atomic ref-updates across multiple repositories. * ps/transactional-update-ref-stdin: update-ref: implement interactive transaction handling update-ref: read commands in a line-wise fashion update-ref: move transaction handling into `update_refs_stdin()` update-ref: pass end pointer instead of strbuf update-ref: drop unused argument for `parse_refname` update-ref: organize commands in an array strbuf: provide function to append whole lines git-update-ref.txt: add missing word refs: fix segfault when aborting empty transaction
2020-04-29Merge branch 'en/fill-directory-exponential'Libravatar Junio C Hamano4-22/+8
The directory traversal code had redundant recursive calls which made its performance characteristics exponential with respect to the depth of the tree, which was corrected. * en/fill-directory-exponential: completion: fix 'git add' on paths under an untracked directory Fix error-prone fill_directory() API; make it only return matches dir: replace double pathspec matching with single in treat_directory() dir: include DIR_KEEP_UNTRACKED_CONTENTS handling in treat_directory() dir: replace exponential algorithm with a linear one dir: refactor treat_directory to clarify control flow dir: fix confusion based on variable tense dir: fix broken comment dir: consolidate treat_path() and treat_one_path() dir: fix simple typo in comment t3000: add more testcases testing a variety of ls-files issues t7063: more thorough status checking
2020-04-29Merge branch 'en/sparse-checkout'Libravatar Junio C Hamano1-33/+22
"sparse-checkout" UI improvements. * en/sparse-checkout: sparse-checkout: provide a new reapply subcommand unpack-trees: failure to set SKIP_WORKTREE bits always just a warning unpack-trees: provide warnings on sparse updates for unmerged paths too unpack-trees: make sparse path messages sound like warnings unpack-trees: split display_error_msgs() into two unpack-trees: rename ERROR_* fields meant for warnings to WARNING_* unpack-trees: move ERROR_WOULD_LOSE_SUBMODULE earlier sparse-checkout: use improved unpack_trees porcelain messages sparse-checkout: use new update_sparsity() function unpack-trees: add a new update_sparsity() function unpack-trees: pull sparse-checkout pattern reading into a new function unpack-trees: do not mark a dirty path with SKIP_WORKTREE unpack-trees: allow check_updates() to work on a different index t1091: make some tests a little more defensive against failures unpack-trees: simplify pattern_list freeing unpack-trees: simplify verify_absent_sparse() unpack-trees: remove unused error type unpack-trees: fix minor typo in comment
2020-04-29Merge branch 'dl/merge-autostash-rebase-quit-fix'Libravatar Junio C Hamano1-0/+1
The stash entry created by "git rebase --autosquash" to keep the initial dirty state were discarded by mistake upon "git rebase --quit", which has been corrected. * dl/merge-autostash-rebase-quit-fix: rebase: save autostash entry into stash reflog on --quit
2020-04-29Merge branch 'dl/merge-autostash'Libravatar Junio C Hamano4-268/+76
"git merge" learns the "--autostash" option. * dl/merge-autostash: (22 commits) pull: pass --autostash to merge t5520: make test_pull_autostash() accept expect_parent_num merge: teach --autostash option sequencer: implement apply_autostash_oid() sequencer: implement save_autostash() sequencer: unlink autostash in apply_autostash() sequencer: extract perform_autostash() from rebase rebase: generify create_autostash() rebase: extract create_autostash() reset: extract reset_head() from rebase rebase: generify reset_head() rebase: use apply_autostash() from sequencer.c sequencer: rename stash_sha1 to stash_oid sequencer: make apply_autostash() accept a path rebase: use read_oneliner() sequencer: make read_oneliner() extern sequencer: configurably warn on non-existent files sequencer: make read_oneliner() accept flags sequencer: make file exists check more efficient sequencer: stop leaking buf ...
2020-04-29Revert "commit-graph.c: introduce '--[no-]check-oids'"Libravatar Junio C Hamano1-8/+3
This reverts commit 7a9ce0269bc0f4ef230f930b3910b70ac3142552, which has not yet gained consensus.
2020-04-28Merge branch 'mt/grep-cquote-path'Libravatar Junio C Hamano1-12/+34
"git grep" did not quote a path with unusual character like other commands (like "git diff", "git status") do, but did quote when run from a subdirectory, both of which has been corrected. * mt/grep-cquote-path: grep: follow conventions for printing paths w/ unusual chars
2020-04-28Merge branch 'ds/log-exclude-decoration-config'Libravatar Junio C Hamano1-1/+15
The "--decorate-refs" and "--decorate-refs-exclude" options "git log" takes have learned a companion configuration variable log.excludeDecoration that sits at the lowest priority in the family. * ds/log-exclude-decoration-config: log: add log.excludeDecoration config option log-tree: make ref_filter_match() a helper method
2020-04-28Merge branch 'tb/diff-tree-with-notes'Libravatar Junio C Hamano1-0/+9
"git diff-tree --pretty --notes" used to hit an assertion failure, as it forgot to initialize the notes subsystem. * tb/diff-tree-with-notes: diff-tree.c: load notes machinery when required
2020-04-28Merge branch 'js/stash-p-fix'Libravatar Junio C Hamano1-1/+1
Allowing the user to split a patch hunk while "git stash -p" does not work well; a band-aid has been added to make this (partially) work better. * js/stash-p-fix: stash -p: (partially) fix bug concerning split hunks t3904: fix incorrect demonstration of a bug
2020-04-28Merge branch 'dl/libify-a-few'Libravatar Junio C Hamano4-700/+4
Code in builtin/*, i.e. those can only be called from within built-in subcommands, that implements bulk of a couple of subcommands have been moved to libgit.a so that they could be used by others. * dl/libify-a-few: Lib-ify prune-packed Lib-ify fmt-merge-msg
2020-04-28Merge branch 'jt/avoid-prefetch-when-able-in-diff'Libravatar Junio C Hamano1-3/+2
"git diff" in a partial clone learned to avoid lazy loading blob objects in more casese when they are not needed. * jt/avoid-prefetch-when-able-in-diff: diff: restrict when prefetching occurs diff: refactor object read diff: make diff_populate_filespec_options struct promisor-remote: accept 0 as oid_nr in function
2020-04-28Merge branch 'ds/commit-graph-expiry-fix'Libravatar Junio C Hamano1-1/+1
"git commit-graph write --expire-time=<timestamp>" did not use the given timestamp correctly, which has been corrected. * ds/commit-graph-expiry-fix: commit-graph: fix buggy --expire-time option
2020-04-28Merge branch 'jc/log-no-mailmap'Libravatar Junio C Hamano2-1/+2
"git log" learns "--[no-]mailmap" as a synonym to "--[no-]use-mailmap" * jc/log-no-mailmap: log: give --[no-]use-mailmap a more sensible synonym --[no-]mailmap clone: reorder --recursive/--recurse-submodules parse-options: teach "git cmd -h" to show alias as alias
2020-04-28Merge branch 'jk/config-use-size-t'Libravatar Junio C Hamano2-2/+2
The config API made mixed uses of int and size_t types to represent length of various pieces of text it parsed, which has been updated to use the correct type (i.e. size_t) throughout. * jk/config-use-size-t: config: reject parsing of files over INT_MAX config: use size_t to store parsed variable baselen git_config_parse_key(): return baselen as size_t config: drop useless length variable in write_pair() parse_config_key(): return subsection len as size_t remote: drop auto-strlen behavior of make_branch() and make_rewrite()
2020-04-28Merge branch 'bc/constant-memequal'Libravatar Junio C Hamano1-1/+22
Validation of push certificate has been made more robust against timing attacks. * bc/constant-memequal: receive-pack: compilation fix builtin/receive-pack: use constant-time comparison for HMAC value
2020-04-28push: anonymize URLs in error messages and warningsLibravatar Johannes Schindelin1-2/+4
Just like 47abd85ba0 (fetch: Strip usernames from url's before storing them, 2009-04-17) and later 882d49ca5c (push: anonymize URL in status output, 2016-07-13), and even later c1284b21f243 (curl: anonymize URLs in error messages and warnings, 2019-03-04) this change anonymizes URLs (read: strips them of user names and especially passwords) in user-facing error messages and warnings. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Reviewed-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-28rebase: save autostash entry into stash reflog on --quitLibravatar Denton Liu1-0/+1
In a03b55530a (merge: teach --autostash option, 2020-04-07), the --autostash option was introduced for `git merge`. Notably, when `git merge --quit` is run with an autostash entry present, it is saved into the stash reflog. This is contrasted with the current behaviour of `git rebase --quit` where the autostash entry is simply just dropped out of existence. Adopt the behaviour of `git merge --quit` in `git rebase --quit` and save the autostash entry into the stash reflog instead of just deleting it. Signed-off-by: Denton Liu <liu.denton@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-27rebase: display an error if --root and --fork-point are both providedLibravatar Elijah Newren1-0/+3
--root implies we want to rebase all commits since the beginning of history. --fork-point means we want to use the reflog of the specified upstream to find the best common ancestor between <upstream> and <branch> and only rebase commits since that common ancestor. These options are clearly contradictory, so throw an error (instead of segfaulting on a NULL pointer) if both are specified. Reported-by: Alexander Berg <alexander.berg@atos.net> Documentation-by: Alban Gruin <alban.gruin@gmail.com> Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-24shallow.c: use '{commit,rollback}_shallow_file'Libravatar Taylor Blau1-2/+2
In bd0b42aed3 (fetch-pack: do not take shallow lock unnecessarily, 2019-01-10), the author noted that 'is_repository_shallow' produces visible side-effect(s) by setting 'is_shallow' and 'shallow_stat'. This is a problem for e.g., fetching with '--update-shallow' in a shallow repository with 'fetch.writeCommitGraph' enabled, since the update to '.git/shallow' will cause Git to think that the repository isn't shallow when it is, thereby circumventing the commit-graph compatibility check. This causes problems in shallow repositories with at least shallow refs that have at least one ancestor (since the client won't have those objects, and therefore can't take the reachability closure over commits when writing a commit-graph). Address this by introducing thin wrappers over 'commit_lock_file' and 'rollback_lock_file' for use specifically when the lock is held over '.git/shallow'. These wrappers (appropriately called 'commit_shallow_file' and 'rollback_shallow_file') call into their respective functions in 'lockfile.h', but additionally reset validity checks used by the shallow machinery. Replace each instance of 'commit_lock_file' and 'rollback_lock_file' with 'commit_shallow_file' and 'rollback_shallow_file' when the lock being held is over the '.git/shallow' file. As a result, 'prune_shallow' can now only be called once (since 'check_shallow_file_for_update' will die after calling 'reset_repository_shallow'). But, this is OK since we only call 'prune_shallow' at most once per process. Helped-by: Jonathan Tan <jonathantanmy@google.com> Helped-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Taylor Blau <me@ttaylorr.com> Reviewed-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-22Merge branch 'jt/rebase-allow-duplicate'Libravatar Junio C Hamano1-0/+7
Allow "git rebase" to reapply all local commits, even if the may be already in the upstream, without checking first. * jt/rebase-allow-duplicate: rebase --merge: optionally skip upstreamed commits
2020-04-22Merge branch 'en/rebase-no-keep-empty'Libravatar Junio C Hamano1-7/+10
"git rebase" (again) learns to honor "--no-keep-empty", which lets the user to discard commits that are empty from the beginning (as opposed to the ones that become empty because of rebasing). The interactive rebase also marks commits that are empty in the todo. * en/rebase-no-keep-empty: rebase: fix an incompatible-options error message rebase: reinstate --no-keep-empty rebase -i: mark commits that begin empty in todo editor
2020-04-22Merge branch 'js/flush-prompt-before-interative-input'Libravatar Junio C Hamano1-10/+4
The interactive input from various codepaths are consolidated and any prompt possibly issued earlier are fflush()ed before we read. * js/flush-prompt-before-interative-input: interactive: explicitly `fflush` stdout before expecting input interactive: refactor code asking the user for interactive input
2020-04-22Merge branch 'ma/simplify-merge-config-parsing'Libravatar Junio C Hamano1-3/+5
Code simplification. * ma/simplify-merge-config-parsing: merge: use skip_prefix to parse config key
2020-04-22Merge branch 'eb/format-patch-no-encode-headers'Libravatar Junio C Hamano1-0/+7
The output from "git format-patch" uses RFC 2047 encoding for non-ASCII letters on From: and Subject: headers, so that it can directly be fed to e-mail programs. A new option has been added to produce these headers in raw. * eb/format-patch-no-encode-headers: format-patch: teach --no-encode-email-headers
2020-04-22Merge branch 'dd/no-gpg-sign'Libravatar Junio C Hamano1-3/+4
"git rebase" learned the "--no-gpg-sign" option to countermand commit.gpgSign the user may have. * dd/no-gpg-sign: Documentation: document merge option --no-gpg-sign Documentation: merge commit-tree --[no-]gpg-sign Documentation: reword commit --no-gpg-sign Documentation: document am --no-gpg-sign cherry-pick/revert: honour --no-gpg-sign in all case rebase.c: honour --no-gpg-sign
2020-04-22Merge branch 'jk/use-quick-lookup-in-clone-for-tag-following'Libravatar Junio C Hamano1-1/+3
The logic to auto-follow tags by "git clone --single-branch" was not careful to avoid lazy-fetching unnecessary tags, which has been corrected. * jk/use-quick-lookup-in-clone-for-tag-following: clone: use "quick" lookup while following tags
2020-04-22Merge branch 'jk/oid-array-cleanups'Libravatar Junio C Hamano8-8/+8
Code cleanup. * jk/oid-array-cleanups: oidset: stop referring to sha1-array ref-filter: stop referring to "sha1 array" bisect: stop referring to sha1_array test-tool: rename sha1-array to oid-array oid_array: rename source file from sha1-array oid_array: use size_t for iteration oid_array: use size_t for count and allocation
2020-04-22Merge branch 'en/pull-do-not-rebase-after-fast-forwarding'Libravatar Junio C Hamano1-1/+4
"git pull --rebase" tried to run a rebase even after noticing that the pull results in a fast-forward and no rebase is needed nor sensible, for the past few years due to a mistake nobody noticed. * en/pull-do-not-rebase-after-fast-forwarding: pull: avoid running both merge and rebase
2020-04-22Merge branch 'rs/pull-options-sync-code-and-doc'Libravatar Junio C Hamano1-0/+18
"git pull" shares many options with underlying "git fetch", but some of them were not documented and some of those that would make sense to pass down were not passed down. * rs/pull-options-sync-code-and-doc: pull: pass documented fetch options on pull: remove --update-head-ok from documentation
2020-04-22Merge branch 'jt/connectivity-check-optim-in-partial-clone'Libravatar Junio C Hamano2-12/+2
Simplify the commit ancestry connectedness check in a partial clone repository in which "promised" objects are assumed to be obtainable lazily on-demand from promisor remote repositories. * jt/connectivity-check-optim-in-partial-clone: connected: always use partial clone optimization
2020-04-22receive-pack: compilation fixLibravatar Junio C Hamano1-1/+3
We do not use C99 "for loop initial declaration" in our codebase (yet), but one snuck in. Reported-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-20diff-tree.c: load notes machinery when requiredLibravatar Taylor Blau1-0/+9
Since its introduction in 7249e91 (revision.c: support --notes command-line option, 2011-03-29), combining '--notes' with any option that causes us to format notes (e.g., '--pretty', '--format="%N"', etc) results in a failed assertion at runtime. $ git rev-list HEAD | git diff-tree --stdin --pretty=medium --notes commit 8f3d9f354286745c751374f5f1fcafee6b3f3136 git: notes.c:1308: format_display_notes: Assertion `display_notes_trees' failed. Aborted This failure is due to diff-tree not calling 'load_display_notes' to initialize the notes machinery. Ordinarily, this failure isn't triggered, because it requires passing both '--notes' and another of the above mentioned options. In the case of '--pretty', for example, we set 'opt->verbose_header', causing 'show_log()' to eventually call 'format_display_notes()', which expects a non-NULL 'display_note_trees'. Without initializing the notes machinery, 'display_note_trees' remains NULL, and thus triggers an assertion failure. Fix this by initializing the notes machinery after parsing our options, and harden this behavior against regression with a test in t4013. (Note that the added ref in this test requires updating two unrelated tests which use 'log --all', and thus need to learn about the new refs). Reported-by: Jeff King <peff@peff.net> Signed-off-by: Taylor Blau <me@ttaylorr.com> Acked-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-20grep: follow conventions for printing paths w/ unusual charsLibravatar Matheus Tavares1-12/+34
grep does not follow the conventions used by other Git commands when printing paths that contain unusual characters (as double-quotes or newlines). Commands such as ls-files, commit, status and diff will: - Quote and escape unusual pathnames, by default. - Print names verbatim and unquoted when "-z" is used. But grep *never* quotes/escapes absolute paths with unusual chars and *always* quotes/escapes relative ones, even with "-z". Besides being inconsistent in its own output, the deviation from other Git commands can be confusing. So let's make it follow the two rules above and add some tests for this new behavior. Note that, making grep quote/escape all unusual paths by default, also make it fully compliant with the core.quotePath configuration, which is currently ignored for absolute paths. Reported-by: Greg Hurrell <greg@hurrell.net> Helped-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-16blame: use changed-path Bloom filtersLibravatar Derrick Stolee1-0/+10
The changed-path Bloom filters help reduce the amount of tree parsing required during history queries. Before calculating a diff, we can ask the filter if a path changed between a commit and its first parent. If the filter says "no" then we can move on without parsing trees. If the filter says "maybe" then we parse trees to discover if the answer is actually "yes" or "no". When computing a blame, there is a section in find_origin() that computes a diff between a commit and one of its parents. When this is the first parent, we can check the Bloom filters before calling diff_tree_oid(). In order to make this work with the blame machinery, we need to initialize a struct bloom_key with the initial path. But also, we need to add more keys to a list if a rename is detected. We then check to see if _any_ of these keys answer "maybe" in the diff. During development, I purposefully left out this "add a new key when a rename is detected" to see if the test suite would catch my error. That is how I discovered the issues with GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS from the previous change. With that change, we can feel some confidence in the coverage of this change. If a user requests copy detection using "git blame -C", then there are more places where the set of "important" files can expand. I do not know enough about how this happens in the blame machinery. Thus, the Bloom filter integration is explicitly disabled in this mode. A later change could expand the bloom_key data with an appropriate call (or calls) to add_bloom_key(). If we did not disable this mode, then the following tests would fail: t8003-blame-corner-cases.sh t8011-blame-split-file.sh Generally, this is a performance enhancement and should not change the behavior of 'git blame' in any way. If a repo has a commit-graph file with computed changed-path Bloom filters, then they should notice improved performance for their 'git blame' commands. Here are some example timings that I found by blaming some paths in the Linux kernel repository: git blame arch/x86/kernel/topology.c >/dev/null Before: 0.83s After: 0.24s git blame kernel/time/time.c >/dev/null Before: 0.72s After: 0.24s git blame tools/perf/ui/stdio/hist.c >/dev/null Before: 0.27s After: 0.11s I specifically looked for "deep" paths that were also edited many times. As a counterpoint, the MAINTAINERS file was edited many times but is located in the root tree. This means that the cost of computing a diff relative to the pathspec is very small. Here are the timings for that command: git blame MAINTAINERS >/dev/null Before: 20.1s After: 18.0s These timings are the best of five. The worst-case runs were on the order of 2.5 minutes for both cases. Note that the MAINTAINERS file has 18,740 lines across 17,000+ commits. This happens to be one of the cases where this change provides the least improvement. The lack of improvement for the MAINTAINERS file and the relatively modest improvement for the other examples can be easily explained. The blame machinery needs to compute line-level diffs to determine which lines were changed by each commit. That makes up a large proportion of the computation time, and this change does not attempt to improve on that section of the algorithm. The MAINTAINERS file is large and changed often, so it takes time to determine which lines were updated by which commit. In contrast, the code files are much smaller, and it takes longer to comute the line-by-line diff for a single patch on the Linux mailing lists. Outside of the "-C" integration, I believe there is little more to gain from the changed-path Bloom filters for 'git blame' after this patch. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-16tests: write commit-graph with Bloom filtersLibravatar Derrick Stolee2-5/+6
The GIT_TEST_COMMIT_GRAPH environment variable updates the commit- graph file whenever "git commit" is run, ensuring that we always have an updated commit-graph throughout the test suite. The GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS environment variable was introduced to write the changed-path Bloom filters whenever "git commit-graph write" is run. However, the GIT_TEST_COMMIT_GRAPH trick doesn't launch a separate process and instead writes it directly. To expand the number of tests that have commits in the commit-graph file, add a helper method that computes the commit-graph and place that helper inside "git commit" and "git merge". In the helper method, check GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS to ensure we are writing changed-path Bloom filters whenever possible. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-16help: move list_config_help to builtin/helpLibravatar Emily Shaffer1-0/+86
Starting in 3ac68a93fd2, help.o began to depend on builtin/branch.o, builtin/clean.o, and builtin/config.o. This meant that help.o was unusable outside of the context of the main Git executable. To make help.o usable by other commands again, move list_config_help() into builtin/help.c (where it makes sense to assume other builtin libraries are present). When command-list.h is included but a member is not used, we start to hear a compiler warning. Since the config list is generated in a fairly different way than the command list, and since commands and config options are semantically different, move the config list into its own header and move the generator into its own script and build rule. For reasons explained in 976aaedc (msvc: add a Makefile target to pre-generate the Visual Studio solution, 2019-07-29), some build artifacts we consider non-source files cannot be generated in the Visual Studio environment, and we already have some Makefile tweaks to help Visual Studio to use generated command-list.h header file. Do the same to a new generated file, config-list.h, introduced by this change. Helped-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
2020-04-16log: add log.excludeDecoration config optionLibravatar Derrick Stolee1-1/+15
In 'git log', the --decorate-refs-exclude option appends a pattern to a string_list. This list is used to prevent showing some refs in the decoration output, or even by --simplify-by-decoration. Users may want to use their refs space to store utility refs that should not appear in the decoration output. For example, Scalar [1] runs a background fetch but places the "new" refs inside the refs/scalar/hidden/<remote>/* refspace instead of refs/<remote>/* to avoid updating remote refs when the user is not looking. However, these "hidden" refs appear during regular 'git log' queries. A similar idea to use "hidden" refs is under consideration for core Git [2]. Add the 'log.excludeDecoration' config option so users can exclude some refs from decorations by default instead of needing to use --decorate-refs-exclude manually. The config value is multi-valued much like the command-line option. The documentation is careful to point out that the config value can be overridden by the --decorate-refs option, even though --decorate-refs-exclude would always "win" over --decorate-refs. Since the 'log.excludeDecoration' takes lower precedence to --decorate-refs, and --decorate-refs-exclude takes higher precedence, the struct decoration_filter needed another field. This led also to new logic in load_ref_decorations() and ref_filter_match(). There are several tests in t4202-log.sh that test the --decorate-refs-(include|exclude) options, so these are extended. Since the expected output is already stored as a file, most tests could simply replace a "--decorate-refs-exclude" option with an in-line config setting. Other tests involve the precedence of the config option compared to command-line options and needed more modification. [1] https://github.com/microsoft/scalar [2] https://lore.kernel.org/git/77b1da5d3063a2404cd750adfe3bb8be9b6c497d.1585946894.git.gitgitgadget@gmail.com/ Helped-by: Junio C Hamano <gister@pobox.com> Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-15commit-graph.c: introduce '--[no-]check-oids'Libravatar Taylor Blau1-3/+8
When operating on a stream of commit OIDs on stdin, 'git commit-graph write' checks that each OID refers to an object that is indeed a commit. This is convenient to make sure that the given input is well-formed, but can sometimes be undesirable. For example, server operators may wish to feed the refnames that were updated during a push to 'git commit-graph write --input=stdin-commits', and silently discard refs that don't point at commits. This can be done by combing the output of 'git for-each-ref' with '--format %(*objecttype)', but this requires opening up a potentially large number of objects. Instead, it is more convenient to feed the updated refs to the commit-graph machinery, and let it throw out refs that don't point to commits. Introduce '--[no-]check-oids' to make such a behavior possible. With '--check-oids' (the default behavior to retain backwards compatibility), 'git commit-graph write' will barf on a non-commit line in its input. With 'no-check-oids', such lines will be silently ignored, making the above possible by specifying this option. No matter which is supplied, 'git commit-graph write' retains the behavior from the previous commit of rejecting non-OID inputs like "HEAD" and "refs/heads/foo" as before. Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-15commit-graph.h: replace 'commit_hex' with 'commits'Libravatar Taylor Blau1-3/+16
The 'write_commit_graph()' function takes in either a string list of pack indices, or a string list of hexadecimal commit OIDs. These correspond to the '--stdin-packs' and '--stdin-commits' mode(s) from 'git commit-graph write'. Using a string_list of hexadecimal commit IDs is not the most efficient use of memory, since we can instead use the 'struct oidset', which is more well-suited for this case. This has another benefit which will become apparent in the following commit. This is that we are about to disambiguate the kinds of errors we produce with '--stdin-commits' into "non-hex input" and "hex-input, but referring to a non-commit object". By having 'write_commit_graph' take in a 'struct oidset *' of commits, we place the burden on the caller (in this case, the builtin) to handle the first case, and the commit-graph machinery can handle the second case. Suggested-by: Jeff King <peff@peff.net> Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-15builtin/commit-graph.c: introduce split strategy 'replace'Libravatar Taylor Blau1-0/+2
When using split commit-graphs, it is sometimes useful to completely replace the commit-graph chain with a new base. For example, consider a scenario in which a repository builds a new commit-graph incremental for each push. Occasionally (say, after some fixed number of pushes), they may wish to rebuild the commit-graph chain with all reachable commits. They can do so with $ git commit-graph write --reachable but this removes the chain entirely and replaces it with a single commit-graph in 'objects/info/commit-graph'. Unfortunately, this means that the next push will have to move this commit-graph into the first layer of a new chain, and then write its new commits on top. Avoid such copying entirely by allowing the caller to specify that they wish to replace the entirety of their commit-graph chain, while also specifying that the new commit-graph should become the basis of a fresh, length-one chain. This addresses the above situation by making it possible for the caller to instead write: $ git commit-graph write --reachable --split=replace which writes a new length-one chain to 'objects/info/commit-graphs', making the commit-graph incremental generated by the subsequent push relatively cheap by avoiding the aforementioned copy. In order to do this, remove an assumption in 'write_commit_graph_file' that chains are always at least two incrementals long. Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-15builtin/commit-graph.c: introduce split strategy 'no-merge'Libravatar Taylor Blau1-1/+6
In the previous commit, we laid the groundwork for supporting different splitting strategies. In this commit, we introduce the first splitting strategy: 'no-merge'. Passing '--split=no-merge' is useful for callers which wish to write a new incremental commit-graph, but do not want to spend effort condensing the incremental chain [1]. Previously, this was possible by passing '--size-multiple=0', but this no longer the case following 63020f175f (commit-graph: prefer default size_mult when given zero, 2020-01-02). When '--split=no-merge' is given, the commit-graph machinery will never condense an existing chain, and it will always write a new incremental. [1]: This might occur when, for example, a server administrator running some program after each push may want to ensure that each job runs proportional in time to the size of the push, and does not "jump" when the commit-graph machinery decides to trigger a merge. Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-15builtin/commit-graph.c: support for '--split[=<strategy>]'Libravatar Taylor Blau1-4/+22
With '--split', the commit-graph machinery writes new commits in another incremental commit-graph which is part of the existing chain, and optionally decides to condense the chain into a single commit-graph. This is done to ensure that the asymptotic behavior of looking up a commit in an incremental chain is not dominated by the number of incrementals in that chain. It can be controlled by the '--max-commits' and '--size-multiple' options. In the next two commits, we will introduce additional splitting strategies that can exert additional control over: - when a split commit-graph is and isn't written, and - when the existing commit-graph chain is discarded completely and replaced with another graph To prepare for this, make '--split' take an optional strategy (as in '--split[=<strategy>]'), and add a new enum to describe which strategy is being used. For now, no strategies are given, and the only enumerated value is 'COMMIT_GRAPH_SPLIT_UNSPECIFIED', indicating the absence of a strategy. Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>