summaryrefslogtreecommitdiff
path: root/t
AgeCommit message (Collapse)AuthorFilesLines
2018-11-02prio-queue: add 'peek' operationLibravatar Derrick Stolee2-8/+32
When consuming a priority queue, it can be convenient to inspect the next object that will be dequeued without actually dequeueing it. Our existing library did not have such a 'peek' operation, so add it as prio_queue_peek(). Add a reference-level comparison in t/helper/test-prio-queue.c so this method is exercised by t0009-prio-queue.sh. Further, add a test that checks the behavior when the compare function is NULL (i.e. the queue becomes a stack). Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-17Merge branch 'ab/fetch-tags-noclobber'Libravatar Junio C Hamano2-4/+29
The rules used by "git push" and "git fetch" to determine if a ref can or cannot be updated were inconsistent; specifically, fetching to update existing tags were allowed even though tags are supposed to be unmoving anchoring points. "git fetch" was taught to forbid updates to existing tags without the "--force" option. * ab/fetch-tags-noclobber: fetch: stop clobbering existing tags without --force fetch: document local ref updates with/without --force push doc: correct lies about how push refspecs work push doc: move mention of "tag <tag>" later in the prose push doc: remove confusing mention of remote merger fetch tests: add a test for clobbering tag behavior push tests: use spaces in interpolated string push tests: make use of unused $1 in test description fetch: change "branch" to "reference" in --force -h output
2018-09-17Merge branch 'es/worktree-forced-ops-fix'Libravatar Junio C Hamano2-0/+62
Fix a bug in which the same path could be registered under multiple worktree entries if the path was missing (for instance, was removed manually). Also, as a convenience, expand the number of cases in which --force is applicable. * es/worktree-forced-ops-fix: doc-diff: force worktree add worktree: delete .git/worktrees if empty after 'remove' worktree: teach 'remove' to override lock when --force given twice worktree: teach 'move' to override lock when --force given twice worktree: teach 'add' to respect --force for registered but missing path worktree: disallow adding same path multiple times worktree: prepare for more checks of whether path can become worktree worktree: generalize delete_git_dir() to reduce code duplication worktree: move delete_git_dir() earlier in file for upcoming new callers worktree: don't die() in library function find_worktree()
2018-09-17Merge branch 'jk/patch-corrupted-delta-fix'Libravatar Junio C Hamano2-4/+93
Malformed or crafted data in packstream can make our code attempt to read or write past the allocated buffer and abort, instead of reporting an error, which has been fixed. * jk/patch-corrupted-delta-fix: t5303: use printf to generate delta bases patch-delta: handle truncated copy parameters patch-delta: consistently report corruption patch-delta: fix oob read t5303: test some corrupt deltas test-delta: read input into a heap buffer
2018-09-17Merge branch 'ds/commit-graph-tests'Libravatar Junio C Hamano5-10/+13
We can now optionally run tests with commit-graph enabled. * ds/commit-graph-tests: commit-graph: define GIT_TEST_COMMIT_GRAPH
2018-09-17Merge branch 'jk/pack-objects-with-bitmap-fix'Libravatar Junio C Hamano1-0/+93
Hotfix of the base topic. * jk/pack-objects-with-bitmap-fix: pack-bitmap: drop "loaded" flag traverse_bitmap_commit_list(): don't free result t5310: test delta reuse with bitmaps bitmap_has_sha1_in_uninteresting(): drop BUG check
2018-09-17Merge branch 'rs/mailinfo-format-flowed'Libravatar Junio C Hamano4-0/+2578
"git mailinfo" used in "git am" learned to make a best-effort recovery of a patch corrupted by MUA that sends text/plain with format=flawed option. * rs/mailinfo-format-flowed: mailinfo: support format=flowed
2018-09-17Merge branch 'jk/cocci'Libravatar Junio C Hamano1-1/+1
spatch transformation to replace boolean uses of !hashcmp() to newly introduced oideq() is added, and applied, to regain performance lost due to support of multiple hash algorithms. * jk/cocci: show_dirstat: simplify same-content check read-cache: use oideq() in ce_compare functions convert hashmap comparison functions to oideq() convert "hashcmp() != 0" to "!hasheq()" convert "oidcmp() != 0" to "!oideq()" convert "hashcmp() == 0" to hasheq() convert "oidcmp() == 0" to oideq() introduce hasheq() and oideq() coccinelle: use <...> for function exclusion
2018-09-17Merge branch 'es/format-patch-rangediff'Libravatar Junio C Hamano1-0/+12
"git format-patch" learned a new "--range-diff" option to explain the difference between this version and the previous attempt in the cover letter (or after the tree-dashes as a comment). * es/format-patch-rangediff: format-patch: allow --range-diff to apply to a lone-patch format-patch: add --creation-factor tweak for --range-diff format-patch: teach --range-diff to respect -v/--reroll-count format-patch: extend --range-diff to accept revision range format-patch: add --range-diff option to embed diff in cover letter range-diff: relieve callers of low-level configuration burden range-diff: publish default creation factor range-diff: respect diff_option.file rather than assuming 'stdout'
2018-09-17Merge branch 'es/format-patch-interdiff'Libravatar Junio C Hamano1-0/+34
"git format-patch" learned a new "--interdiff" option to explain the difference between this version and the previous atttempt in the cover letter (or after the tree-dashes as a comment). * es/format-patch-interdiff: format-patch: allow --interdiff to apply to a lone-patch log-tree: show_log: make commentary block delimiting reusable interdiff: teach show_interdiff() to indent interdiff format-patch: teach --interdiff to respect -v/--reroll-count format-patch: add --interdiff option to embed diff in cover letter format-patch: allow additional generated content in make_cover_letter()
2018-09-17Merge branch 'cc/delta-islands'Libravatar Junio C Hamano1-0/+143
Lift code from GitHub to restrict delta computation so that an object that exists in one fork is not made into a delta against another object that does not appear in the same forked repository. * cc/delta-islands: pack-objects: move 'layer' into 'struct packing_data' pack-objects: move tree_depth into 'struct packing_data' t5320: tests for delta islands repack: add delta-islands support pack-objects: add delta-islands support pack-objects: refactor code into compute_layer_order() Add delta-islands.{c,h}
2018-09-17Merge branch 'jk/trailer-fixes'Libravatar Junio C Hamano4-0/+104
"git interpret-trailers" and its underlying machinery had a buggy code that attempted to ignore patch text after commit log message, which triggered in various codepaths that will always get the log message alone and never get such an input. * jk/trailer-fixes: append_signoff: use size_t for string offsets sequencer: ignore "---" divider when parsing trailers pretty, ref-filter: format %(trailers) with no_divider option interpret-trailers: allow suppressing "---" divider interpret-trailers: tighten check for "---" patch boundary trailer: pass process_trailer_opts to trailer_info_get() trailer: use size_t for iterating trailer list trailer: use size_t for string offsets
2018-09-17Merge branch 'sb/range-diff-colors'Libravatar Junio C Hamano2-3/+44
The color output support for recently introduced "range-diff" command got tweaked a bit. * sb/range-diff-colors: range-diff: indent special lines as context range-diff: make use of different output indicators diff.c: add --output-indicator-{new, old, context} diff.c: rewrite emit_line_0 more understandably diff.c: omit check for line prefix in emit_line_0 diff: use emit_line_0 once per line diff.c: add set_sign to emit_line_0 diff.c: reorder arguments for emit_line_ws_markup diff.c: simplify caller of emit_line_0 t3206: add color test for range-diff --dual-color test_decode_color: understand FAINT and ITALIC
2018-09-17Merge branch 'jk/pack-delta-reuse-with-bitmap'Libravatar Junio C Hamano4-40/+173
When creating a thin pack, which allows objects to be made into a delta against another object that is not in the resulting pack but is known to be present on the receiving end, the code learned to take advantage of the reachability bitmap; this allows the server to send a delta against a base beyond the "boundary" commit. * jk/pack-delta-reuse-with-bitmap: pack-objects: reuse on-disk deltas for thin "have" objects pack-bitmap: save "have" bitmap from walk t/perf: add perf tests for fetches from a bitmapped server t/perf: add infrastructure for measuring sizes t/perf: factor out percent calculations t/perf: factor boilerplate out of test_perf
2018-09-17Merge branch 'nd/unpack-trees-with-cache-tree'Libravatar Junio C Hamano2-0/+10
The unpack_trees() API used in checking out a branch and merging walks one or more trees along with the index. When the cache-tree in the index tells us that we are walking a tree whose flattened contents is known (i.e. matches a span in the index), as linearly scanning a span in the index is much more efficient than having to open tree objects recursively and listing their entries, the walk can be optimized, which is done in this topic. * nd/unpack-trees-with-cache-tree: Document update for nd/unpack-trees-with-cache-tree cache-tree: verify valid cache-tree in the test suite unpack-trees: add missing cache invalidation unpack-trees: reuse (still valid) cache-tree from src_index unpack-trees: reduce malloc in cache-tree walk unpack-trees: optimize walking same trees with cache-tree unpack-trees: add performance tracing trace.h: support nested performance tracing
2018-09-17Merge branch 'ds/reachable'Libravatar Junio C Hamano4-0/+374
The code for computing history reachability has been shuffled, obtained a bunch of new tests to cover them, and then being improved. * ds/reachable: commit-reach: correct accidental #include of C file commit-reach: use can_all_from_reach commit-reach: make can_all_from_reach... linear commit-reach: replace ref_newer logic test-reach: test commit_contains test-reach: test can_all_from_reach_with_flags test-reach: test reduce_heads test-reach: test get_merge_bases_many test-reach: test is_descendant_of test-reach: test in_merge_bases test-reach: create new test tool for ref_newer commit-reach: move can_all_from_reach_with_flags upload-pack: generalize commit date cutoff upload-pack: refactor ok_to_give_up() upload-pack: make reachable() more generic commit-reach: move commit_contains from ref-filter commit-reach: move ref_newer from remote.c commit.h: remove method declarations commit-reach: move walk methods from commit.c
2018-09-17Merge branch 'tg/rerere'Libravatar Junio C Hamano1-0/+65
Fixes to "git rerere" corner cases, especially when conflict markers cannot be parsed in the file. * tg/rerere: rerere: recalculate conflict ID when unresolved conflict is committed rerere: teach rerere to handle nested conflicts rerere: return strbuf from handle path rerere: factor out handle_conflict function rerere: only return whether a path has conflicts or not rerere: fix crash with files rerere can't handle rerere: add documentation for conflict normalization rerere: mark strings for translation rerere: wrap paths in output in sq rerere: lowercase error messages rerere: unify error messages when read_cache fails
2018-09-17Merge branch 'ds/multi-pack-index'Libravatar Junio C Hamano4-0/+270
When there are too many packfiles in a repository (which is not recommended), looking up an object in these would require consulting many pack .idx files; a new mechanism to have a single file that consolidates all of these .idx files is introduced. * ds/multi-pack-index: (32 commits) pack-objects: consider packs in multi-pack-index midx: test a few commands that use get_all_packs treewide: use get_all_packs packfile: add all_packs list midx: fix bug that skips midx with alternates midx: stop reporting garbage midx: mark bad packed objects multi-pack-index: store local property multi-pack-index: provide more helpful usage info midx: clear midx on repack packfile: skip loading index if in multi-pack-index midx: prevent duplicate packfile loads midx: use midx in approximate_object_count midx: use existing midx when writing new one midx: use midx in abbreviation calculations midx: read objects from multi-pack-index config: create core.multiPackIndex setting midx: write object offsets midx: write object id fanout chunk midx: write object ids in a chunk ...
2018-09-17Merge branch 'jk/rev-list-stdin-noop-is-ok'Libravatar Junio C Hamano1-1/+1
"git rev-list --stdin </dev/null" used to be an error; it now shows no output without an error. "git rev-list --stdin --default HEAD" still falls back to the given default when nothing is given on the standard input. * jk/rev-list-stdin-noop-is-ok: rev-list: make empty --stdin not an error
2018-09-17Merge branch 'bp/checkout-new-branch-optim'Libravatar Junio C Hamano1-0/+14
"git checkout -b newbranch [HEAD]" should not have to do as much as checking out a commit different from HEAD. An attempt is made to optimize this special case. * bp/checkout-new-branch-optim: checkout: optimize "git checkout -b <new_branch>"
2018-09-17Merge branch 'sg/t1404-update-ref-test-timeout'Libravatar Junio C Hamano1-3/+3
An attempt to unflake a test a bit. * sg/t1404-update-ref-test-timeout: t1404: increase core.packedRefsTimeout to avoid occasional test failure
2018-09-17Merge branch 'nd/clone-case-smashing-warning'Libravatar Junio C Hamano1-1/+7
Running "git clone" against a project that contain two files with pathnames that differ only in cases on a case insensitive filesystem would result in one of the files lost because the underlying filesystem is incapable of holding both at the same time. An attempt is made to detect such a case and warn. * nd/clone-case-smashing-warning: clone: report duplicate entries on case-insensitive filesystems
2018-09-17Merge branch 'mk/http-backend-content-length'Libravatar Junio C Hamano1-2/+2
Test update. * mk/http-backend-content-length: http-backend test: make empty CONTENT_LENGTH test more realistic
2018-09-11http-backend test: make empty CONTENT_LENGTH test more realisticLibravatar Max Kirillov1-2/+2
This is a test of smart HTTP, so it should use the smart HTTP endpoints (e.g. /info/refs?service=git-receive-pack), not dumb HTTP (HEAD). Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Max Kirillov <max@max630.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-10Merge branch 'jn/submodule-core-worktree-revert'Libravatar Junio C Hamano2-8/+2
* jn/submodule-core-worktree-revert: Revert "Merge branch 'sb/submodule-core-worktree'"
2018-09-10Merge branch 'mk/http-backend-content-length'Libravatar Junio C Hamano1-0/+11
The earlier attempt barfed when given a CONTENT_LENGTH that is set to an empty string. RFC 3875 is fairly clear that in this case we should not read any message body, but we've been reading through to the EOF in previous versions (which did not even pay attention to the environment variable), so keep that behaviour for now in this late update. * mk/http-backend-content-length: http-backend: allow empty CONTENT_LENGTH
2018-09-07Revert "Merge branch 'sb/submodule-core-worktree'"Libravatar Jonathan Nieder2-8/+2
This reverts commit 7e25437d35a70791b345872af202eabfb3e1a8bc, reversing changes made to 00624d608cc69bd62801c93e74d1ea7a7ddd6598. v2.19.0-rc0~165^2~1 (submodule: ensure core.worktree is set after update, 2018-06-18) assumes an "absorbed" submodule layout, where the submodule's Git directory is in the superproject's .git/modules/ directory and .git in the submodule worktree is a .git file pointing there. In particular, it uses $GIT_DIR/modules/$name to find the submodule to find out whether it already has core.worktree set, and it uses connect_work_tree_and_git_dir if not, resulting in fatal: could not open sub/.git for writing The context behind that patch: v2.19.0-rc0~165^2~2 (submodule: unset core.worktree if no working tree is present, 2018-06-12) unsets core.worktree when running commands like "git checkout --recurse-submodules" to switch to a branch without the submodule. If a user then uses "git checkout --no-recurse-submodules" to switch back to a branch with the submodule and runs "git submodule update", this patch is needed to ensure that commands using the submodule directly are aware of the path to the worktree. It is late in the release cycle, so revert the whole 3-patch series. We can try again later for 2.20. Reported-by: Allan Sandfeld Jensen <allan.jensen@qt.io> Helped-by: Stefan Beller <sbeller@google.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-07http-backend: allow empty CONTENT_LENGTHLibravatar Max Kirillov1-0/+11
According to RFC3875, empty environment variable is equivalent to unset, and for CONTENT_LENGTH it should mean zero body to read. However, unset CONTENT_LENGTH is also used for chunked encoding to indicate reading until EOF. At least, the test "large fetch-pack requests can be split across POSTs" from t5551 starts faliing, if unset or empty CONTENT_LENGTH is treated as zero length body. So keep the existing behavior as much as possible. Add a test for the case. Reported-By: Jelmer Vernooij <jelmer@jelmer.uk> Signed-off-by: Max Kirillov <max@max630.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-04Merge branch 'es/chain-lint-more'Libravatar Junio C Hamano5-4/+18
The test linter code has learned that the end of here-doc mark "EOF" can be quoted in a double-quote pair, not just in a single-quote pair. * es/chain-lint-more: chainlint: match "quoted" here-doc tags
2018-09-04Merge branch 'ab/portable-more'Libravatar Junio C Hamano10-44/+53
Portability fix. * ab/portable-more: tests: fix non-portable iconv invocation tests: fix non-portable "${var:-"str"}" construct tests: fix and add lint for non-portable grep --file tests: fix version-specific portability issue in Perl JSON tests: use shorter labels in chainlint.sed for AIX sed tests: fix comment syntax in chainlint.sed for AIX sed tests: fix and add lint for non-portable seq tests: fix and add lint for non-portable head -c N
2018-09-04Merge branch 'en/directory-renames-nothanks'Libravatar Junio C Hamano1-1/+109
Recent addition of "directory rename" heuristics to the merge-recursive backend makes the command susceptible to false positives and false negatives. In the context of "git am -3", which does not know about surrounding unmodified paths and thus cannot inform the merge machinery about the full trees involved, this risk is particularly severe. As such, the heuristic is disabled for "git am -3" to keep the machinery "more stupid but predictable". * en/directory-renames-nothanks: am: avoid directory rename detection when calling recursive merge machinery merge-recursive: add ability to turn off directory rename detection t3401: add another directory rename testcase for rebase and am
2018-09-04Merge branch 'pw/rebase-i-author-script-fix'Libravatar Junio C Hamano1-3/+15
Recent "git rebase -i" update started to write bogusly formatted author-script, with a matching broken reading code. These are fixed. * pw/rebase-i-author-script-fix: sequencer: fix quoting in write_author_script sequencer: handle errors from read_author_ident()
2018-09-04t5310: test delta reuse with bitmapsLibravatar Jeff King1-0/+93
Commit 6a1e32d532 (pack-objects: reuse on-disk deltas for thin "have" objects, 2018-08-21) taught pack-objects a new optimization trick. Since this wasn't meant to change user-visible behavior, but only produce smaller packs more quickly, testing focused on t/perf/p5311. However, since people don't run perf tests very often, we should make sure that the feature is exercised in the regular test suite. This patch does so. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-31fetch: stop clobbering existing tags without --forceLibravatar Ævar Arnfjörð Bjarmason2-4/+5
Change "fetch" to treat "+" in refspecs (aka --force) to mean we should clobber a local tag of the same name. This changes the long-standing behavior of "fetch" added in 853a3697dc ("[PATCH] Multi-head fetch.", 2005-08-20). Before this change, all tag fetches effectively had --force enabled. See the git-fetch-script code in fast_forward_local() with the comment: > Tags need not be pointing at commits so there is no way to > guarantee "fast-forward" anyway. That commit and the rest of the history of "fetch" shows that the "+" (--force) part of refpecs was only conceived for branch updates, while tags have accepted any changes from upstream unconditionally and clobbered the local tag object. Changing this behavior has been discussed as early as 2011[1]. The current behavior doesn't make sense to me, it easily results in local tags accidentally being clobbered. We could namespace our tags per-remote and not locally populate refs/tags/*, but as with my 97716d217c ("fetch: add a --prune-tags option and fetch.pruneTags config", 2018-02-09) it's easier to work around the current implementation than to fix the root cause. So this change implements suggestion #1 from Jeff's 2011 E-Mail[1], "fetch" now only clobbers the tag if either "+" is provided as part of the refspec, or if "--force" is provided on the command-line. This also makes it nicely symmetrical with how "tag" itself works when creating tags. I.e. we refuse to clobber any existing tags unless "--force" is supplied. Now we can refuse all such clobbering, whether it would happen by clobbering a local tag with "tag", or by fetching it from the remote with "fetch". Ref updates outside refs/{tags,heads/* are still still not symmetrical with how "git push" works, as discussed in the recently changed pull-fetch-param.txt documentation. This change brings the two divergent behaviors more into line with one another. I don't think there's any reason "fetch" couldn't fully converge with the behavior used by "push", but that's a topic for another change. One of the tests added in 31b808a032 ("clone --single: limit the fetch refspec to fetched branch", 2012-09-20) is being changed to use --force where a clone would clobber a tag. This changes nothing about the existing behavior of the test. 1. https://public-inbox.org/git/20111123221658.GA22313@sigill.intra.peff.net/ Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-31fetch tests: add a test for clobbering tag behaviorLibravatar Ævar Arnfjörð Bjarmason1-0/+24
The test suite only incidentally (and unintentionally) tested for the current behavior of eager tag clobbering on "fetch". This is a followup to 380efb65df ("push tests: assert re-pushing annotated tags", 2018-07-31) which tests for it explicitly. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-31push tests: use spaces in interpolated stringLibravatar Ævar Arnfjörð Bjarmason1-1/+1
The quoted -m'msg' option would mean the same as -mmsg when passed through the test_force_push_tag helper. Let's instead use a string with spaces in it, to have a working example in case we need to pass other whitespace-delimited arguments to git-tag. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-31push tests: make use of unused $1 in test descriptionLibravatar Ævar Arnfjörð Bjarmason1-1/+1
Fix up a logic error in 380efb65df ("push tests: assert re-pushing annotated tags", 2018-07-31), where the $tag_type_description variable was assigned to but never used, unlike in the subsequently added companion test for fetches in 2d216a7ef6 ("fetch tests: add a test for clobbering tag behavior", 2018-04-29). Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-30t5303: use printf to generate delta basesLibravatar Jeff King1-10/+10
The exact byte count of the delta base file is important. The test-delta helper will feed it to patch_delta(), which will barf if it doesn't match the size byte given in the delta. Using "echo" may end up with unexpected line endings on some platforms (e.g,. "\r\n" instead of just "\n"). This actually wouldn't cause the test to fail (since we already expect test-delta to complain about these bogus deltas), but would mean that we're not exercising the code we think we are. Let's use printf instead (which we already trust to give us byte-perfect output when we generate the deltas). While we're here, let's tighten the 5-byte result size used in the "truncated copy parameters" test. This just needs to have enough room to attempt to parse the bogus copy command, meaning 2 is sufficient. Using 5 was arbitrary and just copied from the base size; since those no longer match, it's simply confusing. Let's use a more meaningful number. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-30patch-delta: handle truncated copy parametersLibravatar Jeff King1-1/+1
When we see a delta command instructing us to copy bytes from the base, we have to read the offset and size from the delta stream. We do this without checking whether we're at the end of the stream, meaning we may read past the end of the buffer. In practice this isn't exploitable in any interesting way because: 1. Deltas are always in packfiles, so we have at least a 20-byte trailer that we'll end up reading. 2. The worst case is that we try to perform a nonsense copy from the base object into the result, based on whatever was in the pack stream next. In most cases this will simply fail due to our bounds-checks against the base or the result. But even if you carefully constructed a pack stream for which it succeeds, it wouldn't perform any delta operation that you couldn't have simply included in a non-broken form. But obviously it's poor form to read past the end of the buffer we've been given. Unfortunately there's no easy way to do a single length check, since the number of bytes we need depends on the number of bits set in the initial command byte. So we'll just check each byte as we parse. We can hide the complexity in a macro; it's ugly, but not as ugly as writing out each individual conditional. Signed-off-by: Jeff King <peff@peff.net> Reviewed-by: Nicolas Pitre <nico@fluxnic.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-30patch-delta: consistently report corruptionLibravatar Jann Horn1-0/+30
When applying a delta, if we see an opcode that cannot be fulfilled (e.g., asking to write more bytes than the destination has left), we break out of our parsing loop but don't signal an explicit error. We rely on the sanity check after the loop to see if we have leftover delta bytes or didn't fill our result buffer. This can silently ignore corruption when the delta buffer ends with a bogus command and the destination buffer is already full. Instead, let's jump into the error handler directly when we see this case. Note that the tests also cover the "bad opcode" case, which already handles this correctly. Signed-off-by: Jann Horn <jannh@google.com> Signed-off-by: Jeff King <peff@peff.net> Reviewed-by: Nicolas Pitre <nico@fluxnic.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-30patch-delta: fix oob readLibravatar Jann Horn1-1/+1
If `cmd` is in the range [0x01,0x7f] and `cmd > top-data`, the `memcpy(out, data, cmd)` can copy out-of-bounds data from after `delta_buf` into `dst_buf`. This is not an exploitable bug because triggering the bug increments the `data` pointer beyond `top`, causing the `data != top` sanity check after the loop to trigger and discard the destination buffer - which means that the result of the out-of-bounds read is never used for anything. Signed-off-by: Jann Horn <jannh@google.com> Signed-off-by: Jeff King <peff@peff.net> Reviewed-by: Nicolas Pitre <nico@fluxnic.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-30t5303: test some corrupt deltasLibravatar Jeff King1-0/+59
We don't have any tests that specifically check boundary cases in patch_delta(). It obviously gets exercised by tests which read from packfiles, but it's hard to create packfiles with bogus deltas. So let's cover some obvious boundary cases: 1. commands that overflow the result buffer a. literal content from the delta b. copies from a base 2. commands where the source isn't large enough a. literal content from a truncated delta b. copies that need more bytes than the base has 3. copy commands who parameters are truncated And indeed, we have problems with both 2a and 3. I've marked these both as expect_failure, though note that because they involve reading past the end of a buffer, they will typically only be caught when run under valgrind or ASan. There's one more test here, too, which just applies a basic delta. Since all of the other tests expect failure and we don't otherwise use "test-tool delta" in the test suite, this gives a sanity check that the tool works at all. These are based on an earlier patch by Jann Horn <jannh@google.com>. Signed-off-by: Jeff King <peff@peff.net> Reviewed-by: Nicolas Pitre <nico@fluxnic.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-30test-delta: read input into a heap bufferLibravatar Jeff King1-4/+4
We currently read the input to test-delta by mmap()-ing it. However, memory-checking tools like valgrind and ASan are less able to detect reads/writes past the end of an mmap'd buffer, because the OS is likely to give us extra bytes to pad out the final page size. So instead, let's read into a heap buffer. As a bonus, this also makes it possible to write tests with empty bases, as mmap() will complain about a zero-length map. This is based on a patch by Jann Horn <jannh@google.com> which actually aligned the data at the end of a page, and followed it with another page marked with mprotect(). That would detect problems even without a tool like ASan, but it was significantly more complex and may have introduced portability problems. By comparison, this approach pushes the complexity onto existing memory-checking tools. Note that this could be done even more simply by using strbuf_read_file(), but that would defeat the purpose: strbufs generally overallocate (and at the very least include a trailing NUL which we do not care about), which would defeat most memory checkers. Signed-off-by: Jeff King <peff@peff.net> Reviewed-by: Nicolas Pitre <nico@fluxnic.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-30worktree: delete .git/worktrees if empty after 'remove'Libravatar Eric Sunshine1-0/+12
For cleanliness, "git worktree prune" deletes the .git/worktrees directory if it is empty after pruning is complete. For consistency, make "git worktree remove <path>" likewise delete .git/worktrees if it is empty after the removal. Signed-off-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-30worktree: teach 'remove' to override lock when --force given twiceLibravatar Eric Sunshine1-0/+10
For consistency with "add -f -f" and "move -f -f" which override the lock on a worktree, allow "remove -f -f" to do so, as well, as a convenience. Signed-off-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-30worktree: teach 'move' to override lock when --force given twiceLibravatar Eric Sunshine1-0/+14
For consistency with "add -f -f", which allows a missing but locked worktree path to be re-used, allow "move -f -f" to override a lock, as well, as a convenience. Signed-off-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-30worktree: teach 'add' to respect --force for registered but missing pathLibravatar Eric Sunshine1-1/+12
For safety, "git worktree add <path>" will refuse to add a new worktree at <path> if <path> is already associated with a worktree entry, even if <path> is missing (for instance, has been deleted or resides on non-mounted removable media or network share). The typical way to re-create a worktree at <path> in such a situation is either to prune all "broken" entries ("git worktree prune") or to selectively remove the worktree entry manually ("git worktree remove <path>"). However, neither of these approaches ("prune" nor "remove") is especially convenient, and they may be unsuitable for scripting when a tool merely wants to re-use a worktree if it exists or create it from scratch if it doesn't (much as a tool might use "mkdir -p" to re-use or create a directory). Therefore, teach 'add' to respect --force as a convenient way to re-use a path already associated with a worktree entry if the path is non-existent. For a locked worktree, require --force to be specified twice. Signed-off-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-30worktree: disallow adding same path multiple timesLibravatar Eric Sunshine1-0/+7
A given path should only ever be associated with a single registered worktree. This invariant is enforced by refusing to create a new worktree at a given path if that path already exists. For example: $ git worktree add -q --detach foo $ git worktree add -q --detach foo fatal: 'foo' already exists However, the check can be fooled, and the invariant broken, if the path is missing. Continuing the example: $ rm -fr foo $ git worktree add -q --detach foo $ git worktree list ... eadebfe [master] .../foo eadebfe (detached HEAD) .../foo eadebfe (detached HEAD) This "corruption" leads to the unfortunate situation in which the worktree can not be removed: $ git worktree remove foo fatal: validation failed, cannot remove working tree: '.../foo' does not point back to '.git/worktrees/foo' Nor can the bogus entry be pruned: $ git worktree prune -v $ git worktree list ... eadebfe [master] .../foo eadebfe (detached HEAD) .../foo eadebfe (detached HEAD) without first deleting the worktree directory manually: $ rm -fr foo $ git worktree prune -v Removing .../foo: gitdir file points to non-existent location Removing .../foo1: gitdir file points to non-existent location $ git worktree list ... eadebfe [master] or by manually deleting the worktree entry in .git/worktrees. To address this problem, upgrade "git worktree add" validation to allow worktree creation only if the given path is not already associated with an existing worktree (even if the path itself is non-existent), thus preventing such bogus worktree entries from being created in the first place. Reported-by: Jeff King <peff@peff.net> Signed-off-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-30worktree: don't die() in library function find_worktree()Libravatar Eric Sunshine1-0/+8
Callers don't expect library function find_worktree() to die(); they expect it to return the named worktree if found, or NULL if not. Although find_worktree() itself never invokes die(), it calls real_pathdup() with 'die_on_error' incorrectly set to 'true', thus will die() indirectly if the user-provided path is not to real_pathdup()'s liking. This can be observed, for instance, with any git-worktree command which searches for an existing worktree: $ git worktree unlock foo fatal: 'foo' is not a working tree $ git worktree unlock foo/bar fatal: Invalid path '.../foo': No such file or directory The first error message is the expected one from "git worktree unlock" not finding the specified worktree; the second is from find_worktree() invoking real_pathdup() incorrectly and die()ing prematurely. Aside from the inconsistent error message between the two cases, this bug hasn't otherwise been a serious problem since existing callers all die() anyhow when the worktree can't be found. However, that may not be true of callers added in the future, so fix find_worktree() to avoid die()ing. Signed-off-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-30am: avoid directory rename detection when calling recursive merge machineryLibravatar Elijah Newren1-2/+2
Let's say you have the following three trees, where Base is from one commit behind either master or branch: Base : bar_v1, foo/{file1, file2, file3} branch: bar_v2, foo/{file1, file2}, goo/file3 master: bar_v3, foo/{file1, file2, file3} Using git-am (or am-based rebase) to apply the changes from branch onto master results in the following tree: Result: bar_merged, goo/{file1, file2, file3} This is not what users want; they did not rename foo/ -> goo/, they only renamed one file within that directory. The reason this happens is am constructs fake trees (via build_fake_ancestor()) of the following form: Base_bfa : bar_v1, foo/file3 branch_bfa: bar_v2, goo/file3 Combining these two trees with master's tree: master: bar_v3, foo/{file1, file2, file3}, You can see that merge_recursive_generic() would see branch_bfa as renaming foo/ -> goo/, and master as just adding both foo/file1 and foo/file2. As such, it ends up with goo/{file1, file2, file3} The core problem is that am does not have access to the original trees; it can only construct trees using the blobs involved in the patch. As such, it is not safe to perform directory rename detection within am -3. Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>