summaryrefslogtreecommitdiff
path: root/builtin
AgeCommit message (Collapse)AuthorFilesLines
2020-12-08Merge branch 'ma/grep-init-default'Libravatar Junio C Hamano2-2/+0
Code clean-up. * ma/grep-init-default: MyFirstObjectWalk: drop `init_walken_defaults()` grep: copy struct in one fell swoop grep: use designated initializers for `grep_defaults` grep: don't set up a "default" repo for grep
2020-12-08Merge branch 'js/trace2-session-id'Libravatar Junio C Hamano1-0/+15
The transport layer was taught to optionally exchange the session ID assigned by the trace2 subsystem during fetch/push transactions. * js/trace2-session-id: receive-pack: log received client session ID send-pack: advertise session ID in capabilities upload-pack, serve: log received client session ID fetch-pack: advertise session ID in capabilities transport: log received server session ID serve: advertise session ID in v2 capabilities receive-pack: advertise session ID in v0 capabilities upload-pack: advertise session ID in v0 capabilities trace2: add a public function for getting the SID docs: new transfer.advertiseSID option docs: new capability to advertise session IDs
2020-12-08Merge branch 'ds/maintenance-part-3'Libravatar Junio C Hamano1-2/+3
"git maintenance" command had trouble working in a directory whose pathname contained an ERE metacharacter like '+'. * ds/maintenance-part-3: maintenance: use 'git config --fixed-value'
2020-12-08Merge branch 'ds/config-literal-value'Libravatar Junio C Hamano3-21/+72
Various subcommands of "git config" that takes value_regex learn the "--literal-value" option to take the value_regex option as a literal string. * ds/config-literal-value: config doc: value-pattern is not necessarily a regexp config: implement --fixed-value with --get* config: plumb --fixed-value into config API config: add --fixed-value option, un-implemented t1300: add test for --replace-all with value-pattern t1300: test "set all" mode with value-pattern config: replace 'value_regex' with 'value_pattern' config: convert multi_replace to flags
2020-12-08Merge branch 'ps/update-ref-multi-transaction'Libravatar Junio C Hamano1-1/+14
"git update-ref --stdin" learns to take multiple transactions in a single session. * ps/update-ref-multi-transaction: update-ref: disallow "start" for ongoing transactions p1400: use `git-update-ref --stdin` to test multiple transactions update-ref: allow creation of multiple transactions t1400: avoid touching refs on filesystem
2020-12-03Merge branch 'tb/repack-simplify'Libravatar Junio C Hamano1-99/+54
Simplify the logic to deal with a repack operation that ended up creating the same packfile. * tb/repack-simplify: builtin/repack.c: don't move existing packs out of the way builtin/repack.c: keep track of what pack-objects wrote repack: make "exts" array available outside cmd_repack()
2020-12-03Merge branch 'pb/pull-rebase-recurse-submodules'Libravatar Junio C Hamano1-12/+36
"git pull --rebase --recurse-submodules" checked for local changes in a wrong range and failed to run correctly when it should. * pb/pull-rebase-recurse-submodules: pull: check for local submodule modifications with the right range t5572: describe '--rebase' tests a little more t5572: add notes on a peculiar test pull --rebase: compute rebase arguments in separate function
2020-11-30Merge branch 'sa/credential-store-timeout'Libravatar Junio C Hamano1-2/+6
Multiple "credential-store" backends can race to lock the same file, causing everybody else but one to fail---reattempt locking with some timeout to reduce the rate of the failure. * sa/credential-store-timeout: crendential-store: use timeout when locking file
2020-11-30Merge branch 'km/stash-error-message-fix'Libravatar Junio C Hamano1-1/+1
Error message fix. * km/stash-error-message-fix: stash: add missing space to an error message
2020-11-30Merge branch 'mt/worktree-error-message-fix'Libravatar Junio C Hamano1-2/+2
Fix formulation of an error message with two placeholders in "git worktree add" subcommand. * mt/worktree-error-message-fix: worktree: fix order of arguments in error message
2020-11-30Merge branch 'ab/gc-keep-base-option'Libravatar Junio C Hamano1-4/+4
Fix an option name in "gc" documentation. * ab/gc-keep-base-option: gc: rename keep_base_pack variable for --keep-largest-pack gc docs: change --keep-base-pack to --keep-largest-pack
2020-11-30Merge branch 'js/pull-rebase-use-advise'Libravatar Junio C Hamano1-12/+12
UI improvement. * js/pull-rebase-use-advise: pull: colorize the hint about setting `pull.rebase`
2020-11-25Merge branch 'rs/gc-sort-func-cast-fix'Libravatar Junio C Hamano1-4/+2
Fix broken sorting of maintenance tasks. * rs/gc-sort-func-cast-fix: gc: fix cast in compare_tasks_by_selection()
2020-11-25Merge branch 'jk/4gb-idx'Libravatar Junio C Hamano2-4/+4
The code was not prepared to deal with pack .idx file that is larger than 4GB. * jk/4gb-idx: packfile: detect overflow in .idx file size checks block-sha1: take a size_t length parameter fsck: correctly compute checksums on idx files larger than 4GB use size_t to store pack .idx byte offsets compute pack .idx byte offsets using size_t
2020-11-25Merge branch 'jx/t5411-flake-fix'Libravatar Junio C Hamano1-25/+69
The exchange between receive-pack and proc-receive hook did not carefully check for errors. * jx/t5411-flake-fix: receive-pack: use default version 0 for proc-receive receive-pack: gently write messages to proc-receive t5411: new helper filter_out_user_friendly_and_stable_output
2020-11-25maintenance: use 'git config --fixed-value'Libravatar Derrick Stolee1-2/+3
When a repository's leading directories contain regex metacharacters, the config calls for 'git maintenance register' and 'git maintenance unregister' are not careful enough. Use the new --fixed-value option to direct the config machinery to use exact string matches. This is a more robust option than escaping these arguments in a piecemeal fashion. For the test, require that we are not running on Windows since the '+' and '*' characters are not allowed on that filesystem. Reported-by: Emily Shaffer <emilyshaffer@google.com> Reported-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-11-25config: implement --fixed-value with --get*Libravatar Derrick Stolee1-5/+10
The config builtin does its own regex matching of values for the --get, --get-all, and --get-regexp modes. Plumb the existing 'flags' parameter to the get_value() method so we can initialize the value-pattern argument as a fixed string instead of a regex pattern. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-11-25config: plumb --fixed-value into config APILibravatar Derrick Stolee1-5/+11
The git_config_set_multivar_in_file_gently() and related methods now take a 'flags' bitfield, so add a new bit representing the --fixed-value option from 'git config'. This alters the purpose of the value_pattern parameter to be an exact string match. This requires some initialization changes in git_config_set_multivar_in_file_gently() and a new strcmp() call in the matches() method. The new CONFIG_FLAGS_FIXED_VALUE flag is initialized in builtin/config.c based on the --fixed-value option, and that needs to be updated in several callers. This patch only affects some of the modes of 'git config', and the rest will be completed in the next change. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-11-25config: add --fixed-value option, un-implementedLibravatar Derrick Stolee1-0/+36
The 'git config' builtin takes a 'value-pattern' parameter for several actions. This can cause confusion when expecting exact value matches instead of regex matches, especially when the input string contains metacharacters. While callers can escape the patterns themselves, it would be more friendly to allow an argument to disable the pattern matching in favor of an exact string match. Add a new '--fixed-value' option that does not currently change the behavior. The implementation will be filled in by later changes for each appropriate action. For now, check and test that --fixed-value will abort the command when included with an incompatible action or without a 'value-pattern' argument. The name '--fixed-value' was chosen over something simpler like '--fixed' because some commands allow regular expressions on the key in addition to the value. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-11-25config: replace 'value_regex' with 'value_pattern'Libravatar Derrick Stolee1-6/+6
The 'value_regex' argument in the 'git config' builtin is poorly named, especially related to an upcoming change that allows exact string matches instead of ERE pattern matches. Perform a mostly mechanical change of every instance of 'value_regex' to 'value_pattern' in the codebase. This is only critical for documentation and error messages, but it is best to be consistent inside the codebase, too. For documentation, use 'value-pattern' which is better punctuation. This affects Documentation/git-config.txt and the usage in builtin/config.c, which was already mixed between 'value_regex' and 'value-regex'. I gave some thought to leaving the value_regex variables inside config.c that are regex_t pointers. However, it is probably best to keep the name consistent with the rest of the variables. This does not update the translations inside the po/ directory, as that creates conflicts with ongoing work. The input strings should automatically update through automation, and a few of the output strings currently use "[value_regex]" directly. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-11-25config: convert multi_replace to flagsLibravatar Derrick Stolee3-7/+11
We will extend the flexibility of the config API. Before doing so, let's take an existing 'int multi_replace' parameter and replace it with a new 'unsigned flags' parameter that can take multiple options as a bit field. Update all callers that specified multi_replace to now specify the CONFIG_FLAGS_MULTI_REPLACE flag. To add more clarity, extend the documentation of git_config_set_multivar_in_file() including a clear labeling of its arguments. Other config API methods in config.h require only a change of the final parameter from 'int' to 'unsigned'. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-11-25crendential-store: use timeout when locking fileLibravatar Simão Afonso1-2/+6
When holding the lock for rewriting the credential file, use a timeout to avoid race conditions when the credentials file needs to be updated in parallel. An example would be doing `fetch --all` on a repository with several remotes that need credentials, using parallel fetching. The timeout can be configured using "credentialStore.lockTimeoutMS", defaulting to 1 second. Signed-off-by: Simão Afonso <simao.afonso@powertools-tech.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-11-24stash: add missing space to an error messageLibravatar Kyle Meyer1-1/+1
Restore a space that was lost in 8a0fc8d19d (stash: convert apply to builtin, 2019-02-25). Signed-off-by: Kyle Meyer <kyle@kyleam.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-11-21Merge branch 'en/strmap'Libravatar Junio C Hamano3-65/+11
A specialization of hashmap that uses a string as key has been introduced. Hopefully it will see wider use over time. * en/strmap: shortlog: use strset from strmap.h Use new HASHMAP_INIT macro to simplify hashmap initialization strmap: take advantage of FLEXPTR_ALLOC_STR when relevant strmap: enable allocations to come from a mem_pool strmap: add a strset sub-type strmap: split create_entry() out of strmap_put() strmap: add functions facilitating use as a string->int map strmap: enable faster clearing and reusing of strmaps strmap: add more utility functions strmap: new utility functions hashmap: provide deallocation function names hashmap: introduce a new hashmap_partial_clear() hashmap: allow re-use after hashmap_free() hashmap: adjust spacing to fix argument alignment hashmap: add usage documentation explaining hashmap_free[_entries]()
2020-11-21Merge branch 'jk/rev-parse-end-of-options'Libravatar Junio C Hamano1-45/+55
"git rev-parse" learned the "--end-of-options" to help scripts to safely take a parameter that is supposed to be a revision, e.g. "git rev-parse --verify -q --end-of-options $rev". * jk/rev-parse-end-of-options: rev-parse: handle --end-of-options rev-parse: put all options under the "-" check rev-parse: don't accept options after dashdash
2020-11-21Merge branch 'jc/format-patch-name-max'Libravatar Junio C Hamano1-6/+14
The maximum length of output filenames "git format-patch" creates has become configurable (used to be capped at 64). * jc/format-patch-name-max: format-patch: make output filename configurable
2020-11-21grep: use designated initializers for `grep_defaults`Libravatar Martin Ågren2-2/+0
In 15fabd1bbd ("builtin/grep.c: make configuration callback more reusable", 2012-10-09), we learned to fill a `static struct grep_opt grep_defaults` which we can use as a blueprint for other such structs. At the time, we didn't consider designated initializers to be widely useable, but these days, we do. (See, e.g., cbc0f81d96 ("strbuf: use designated initializers in STRBUF_INIT", 2017-07-10).) Use designated initializers to let the compiler set up the struct and so that we don't need to remember to call `init_grep_defaults()`. Signed-off-by: Martin Ågren <martin.agren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-11-21grep: don't set up a "default" repo for grepLibravatar Martin Ågren2-2/+2
`init_grep_defaults()` fills a `static struct grep_opt grep_defaults`. This struct is then used by `grep_init()` as a blueprint for other such structs. Notably, `grep_init()` takes a `struct repo *` and assigns it into the target struct. As a result, it is unnecessary for us to take a `struct repo *` in `init_grep_defaults()` as well. We assign it into the default struct and never look at it again. And in light of how we return early if we have already set up the default struct, it's not just unnecessary, but is also a bit confusing: If we are called twice and with different repos, is it a bug or a feature that we ignore the second repo? Drop the repo parameter for `init_grep_defaults()`. Signed-off-by: Martin Ågren <martin.agren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-11-21worktree: fix order of arguments in error messageLibravatar Matheus Tavares1-2/+2
`git worktree add` (without --force) errors out when given a path that is already registered as a worktree and the path is missing on disk. But the `cmd` and `path` strings are switched on the error message. Let's fix that. Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br> Reviewed-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-11-21gc: rename keep_base_pack variable for --keep-largest-packLibravatar Ævar Arnfjörð Bjarmason1-4/+4
As noted in an earlier change the keep_base_pack variable name is a relic from an earlier on-list version of ae4e89e549 ("gc: add --keep-largest-pack option", 2018-04-15) before it was renamed to --keep-largest-pack. Let's change the variable name to avoid that confusion, it's easier to read the code if there's a 1=1 mapping between the variable name and option name. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-11-19pull: colorize the hint about setting `pull.rebase`Libravatar Johannes Schindelin1-12/+12
In d18c950a69f (pull: warn if the user didn't say whether to rebase or to merge, 2020-03-09), a new hint was introduced to encourage users to make a conscious decision about whether they want their pull to merge or to rebase by configuring the `pull.rebase` setting. This warning was clearly intended to advise users, but as pointed out in https://lore.kernel.org/git/87ima2rdsm.fsf%40evledraar.gmail.com, it uses `warning()` instead of `advise()`. One consequence is that the advice is not colorized in the same manner as other, similar messages. So let's use `advise()` instead. Pointed-out-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-11-18gc: fix cast in compare_tasks_by_selection()Libravatar René Scharfe1-4/+2
compare_tasks_by_selection() is used with QSORT and gets passed pointers to the elements of "static struct maintenance_task tasks[]". It casts the *addresses* of these passed pointers to element pointers, though, and thus effectively compares some unrelated values from the stack. Fix the casts to actually compare array elements. Detected by USan (make SANITIZE=undefined test). Signed-off-by: René Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-11-18Merge branch 'pb/blame-funcname-range-userdiff'Libravatar Junio C Hamano2-7/+8
"git blame -L :funcname -- path" did not work well for a path for which a userdiff driver is defined. * pb/blame-funcname-range-userdiff: blame: simplify 'setup_blame_bloom_data' interface blame: simplify 'setup_scoreboard' interface blame: enable funcname blaming with userdiff driver line-log: mention both modes in 'blame' and 'log' short help doc: add more pointers to gitattributes(5) for userdiff blame-options.txt: also mention 'funcname' in '-L' description doc: line-range: improve formatting doc: log, gitk: move '-L' description to 'line-range-options.txt'
2020-11-18Merge branch 'en/merge-ort-api-null-impl'Libravatar Junio C Hamano3-4/+42
Preparation for a new merge strategy. * en/merge-ort-api-null-impl: merge,rebase,revert: select ort or recursive by config or environment fast-rebase: demonstrate merge-ort's API via new test-tool command merge-ort-wrappers: new convience wrappers to mimic the old merge API merge-ort: barebones API of new merge strategy with empty implementation
2020-11-18Merge branch 'ds/maintenance-part-3'Libravatar Junio C Hamano2-6/+333
Parts of "git maintenance" to ease writing crontab entries (and other scheduling system configuration) for it. * ds/maintenance-part-3: maintenance: add troubleshooting guide to docs maintenance: use 'incremental' strategy by default maintenance: create maintenance.strategy config maintenance: add start/stop subcommands maintenance: add [un]register subcommands for-each-repo: run subcommands on configured repos maintenance: add --schedule option and config maintenance: optionally skip --auto process
2020-11-18Merge branch 'pw/rebase-i-orig-head'Libravatar Junio C Hamano1-10/+10
"git rebase -i" did not store ORIG_HEAD correctly. * pw/rebase-i-orig-head: rebase -i: simplify get_revision_ranges() rebase -i: use struct object_id when writing state rebase -i: use struct object_id rather than looking up commit rebase -i: stop overwriting ORIG_HEAD buffer
2020-11-18Merge branch 'jk/format-patch-output'Libravatar Junio C Hamano1-15/+22
"git format-patch --output=there" did not work as expected and instead crashed. The option is now supported. * jk/format-patch-output: format-patch: support --output option format-patch: tie file-opening logic to output_directory format-patch: refactor output selection
2020-11-18Merge branch 'jc/line-log-takes-no-pathspec'Libravatar Junio C Hamano1-0/+3
"git log -L<range>:<path>" is documented to take no pathspec, but this was not enforced by the command line option parser, which has been corrected. * jc/line-log-takes-no-pathspec: log: diagnose -L used with pathspec as an error
2020-11-18Merge branch 'rs/empty-reflog-check-fix'Libravatar Junio C Hamano1-14/+13
The code to see if "git stash drop" can safely remove refs/stash has been made more carerful. * rs/empty-reflog-check-fix: stash: simplify reflog emptiness check
2020-11-17builtin/repack.c: don't move existing packs out of the wayLibravatar Taylor Blau1-89/+14
When 'git repack' creates a pack with the same name as any existing pack, it moves the existing one to 'old-pack-xxx.{pack,idx,...}' and then renames the new one into place. Eventually, it would be nice to have 'git repack' allow for writing a multi-pack index at the critical time (after the new packs have been written / moved into place, but before the old ones have been deleted). Guessing that this option might be called '--write-midx', this makes the following situation (where repacks are issued back-to-back without any new objects) impossible: $ git repack -adb $ git repack -adb --write-midx In the second repack, the existing packs are overwritten verbatim with the same rename-to-old sequence. At that point, the current MIDX is invalidated, since it refers to now-missing packs. So that code wants to be run after the MIDX is re-written. But (prior to this patch) the new MIDX can't be written until the new packs are moved into place. So, we have a circular dependency. This is all hypothetical, since no code currently exists to write a MIDX safely during a 'git repack' (the 'GIT_TEST_MULTI_PACK_INDEX' does so unsafely). Putting hypothetical aside, though: why do we need to rename existing packs to be prefixed with 'old-' anyway? This behavior dates all the way back to 2ad47d6 (git-repack: Be careful when updating the same pack as an existing one., 2006-06-25). 2ad47d6 is mainly concerned about a case where a newly written pack would have a different structure than its index. This used to be possible when the pack name was a hash of the set of objects. Under this naming scheme, two packs that store the same set of objects could differ in delta selection, object positioning, or both. If this happened, then any such packs would be unreadable in the instant between copying the new pack and new index (i.e., either the index or pack will be stale depending on the order that they were copied). But since 1190a1a (pack-objects: name pack files after trailer hash, 2013-12-05), this is no longer possible, since pack files are named not after their logical contents (i.e., the set of objects), but by the actual checksum of their contents. So, this old- behavior can safely go, which allows us to avoid our circular dependency above. In addition to avoiding the circular dependency, this patch also makes 'git repack' a lot simpler, since we don't have to deal with failures encountered when renaming existing packs to be prefixed with 'old-'. This patch is mostly limited to removing code paths that deal with the 'old' prefixing, with the exception of files that include the pack's name in their own filename, like .idx, .bitmap, and related files. The exception is that we want to continue to trust what pack-objects wrote. That is, it is not the case that we pretend as if pack-objects didn't write files identical to ones that already exist, but rather that we respect what pack-objects wrote as the source of truth. That cuts two ways: - If pack-objects produced an identical pack to one that already exists with a bitmap, but did not produce a bitmap, we remove the bitmap that already exists. (This behavior is codified in t7700.14). - If pack-objects produced an identical pack to one that already exists, we trust the just-written version of the coresponding .idx, .promisor, and other files over the ones that already exist. This ensures that we use the most up-to-date versions of this files, which is safe even in the face of format changes in, say, the .idx file (which would not be reflected in the .idx file's name). Helped-by: Jeff King <peff@peff.net> Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-11-16pull: check for local submodule modifications with the right rangeLibravatar Philippe Blain1-1/+1
Ever since 'git pull' learned '--recurse-submodules' in a6d7eb2c7a (pull: optionally rebase submodules (remote submodule changes only), 2017-06-23), we check if there are local submodule modifications by checking the revision range 'curr_head --not rebase_fork_point'. The goal of this check is to abort the pull if there are submodule modifications in the local commits being rebased, since this scenario is not supported. However, the actual range of commits being rebased is not 'rebase_fork_point..curr_head', as the logic in 'get_rebase_newbase_and_upstream' reveals, it is 'upstream..curr_head'. If the 'git merge-base --fork-point' invocation in 'get_rebase_fork_point' fails to find a fork point between the current branch and the remote-tracking branch we are pulling from, 'rebase_fork_point' is null and since 4d36f88be7 (submodule: do not pass null OID to setup_revisions, 2018-05-24), 'submodule_touches_in_range' checks 'curr_head' and all its ancestors for submodule modifications. Since it is highly likely that there are submodule modifications in this range (which is in effect the whole history of the current branch), this prevents 'git pull --rebase --recurse-submodules' from succeeding if no fork point exists between the current branch and the remote-tracking branch being pulled. This can happen, for example, when the current branch was forked from a commit which was never recorded in the reflog of the remote-tracking branch we are pulling, as the last two paragraphs of the "Discussion on fork-point mode" section in git-merge-base(1) explain. Fix this bug by passing 'upstream' instead of 'rebase_fork_point' as the 'excl_oid' argument to 'submodule_touches_in_range'. Reported-by: Brice Goglin <bgoglin@free.fr> Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-11-16pull --rebase: compute rebase arguments in separate functionLibravatar Philippe Blain1-11/+35
The function 'run_rebase' is responsible for constructing the command line to be passed to 'git rebase'. This includes both forwarding pass-through options given to 'git pull' as well computing the <newbase> and <upstream> arguments to 'git rebase'. A following commit will need to access the <upstream> argument in 'cmd_pull' to fix a bug with 'git pull --rebase --recurse-submodules'. In order to do so, refactor the code so that the <newbase> and <upstream> commits are computed in a new, separate function, 'get_rebase_newbase_and_upstream'. Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-11-16builtin/repack.c: keep track of what pack-objects wroteLibravatar Taylor Blau1-1/+30
In the subsequent commit, it will become useful to keep track of which metadata files were written by pack-objects. We already do this to an extent with the 'exts' array, which only is used in the context of existing packs. Co-authored-by: Jeff King <peff@peff.net> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-11-16repack: make "exts" array available outside cmd_repack()Libravatar Jeff King1-9/+10
We'll use it in a helper function soon. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-11-16update-ref: disallow "start" for ongoing transactionsLibravatar Patrick Steinhardt1-0/+2
It is currently possible to write multiple "start" commands into git-update-ref(1) for a single session, but none of them except for the first one actually have any effect. Using such nested "start"s may eventually have a sensible effect. One may imagine that it restarts the current transaction, effectively emptying it and creating a new one. It may also allow for creation of nested transactions. But currently, none of these are implemented. Silently ignoring this misuse is making it hard to iterate in the future if "start" is ever going to have meaningful semantics in such a context. This commit thus makes sure to error out in case we see such use. Signed-off-by: Patrick Steinhardt <ps@pks.im> Reviewed-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-11-16update-ref: allow creation of multiple transactionsLibravatar Patrick Steinhardt1-1/+12
While git-update-ref has recently grown commands which allow interactive control of transactions in e48cf33b61 (update-ref: implement interactive transaction handling, 2020-04-02), it is not yet possible to create multiple transactions in a single session. To do so, one currently still needs to invoke the executable multiple times. This commit addresses this shortcoming by allowing the "start" command to create a new transaction if the current transaction has already been either committed or aborted. Signed-off-by: Patrick Steinhardt <ps@pks.im> Reviewed-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-11-16use size_t to store pack .idx byte offsetsLibravatar Jeff King1-3/+3
We sometimes store the offset into a pack .idx file as an "unsigned long", but the mmap'd size of a pack .idx file can exceed 4GB. This is sufficient on LP64 systems like Linux, but will be too small on LLP64 systems like Windows, where "unsigned long" is still only 32 bits. Let's use size_t, which is a better type for an offset into a memory buffer. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-11-16compute pack .idx byte offsets using size_tLibravatar Jeff King1-1/+1
A pack and its matching .idx file are limited to 2^32 objects, because the pack format contains a 32-bit field to store the number of objects. Hence we use uint32_t in the code. But the byte count of even a .idx file can be much larger than that, because it stores at least a hash and an offset for each object. So using SHA-1, a v2 .idx file will cross the 4GB boundary at 153,391,650 objects. This confuses load_idx(), which computes the minimum size like this: unsigned long min_size = 8 + 4*256 + nr*(hashsz + 4 + 4) + hashsz + hashsz; Even though min_size will be big enough on most 64-bit platforms, the actual arithmetic is done as a uint32_t, resulting in a truncation. We actually exceed that min_size, but then we do: unsigned long max_size = min_size; if (nr) max_size += (nr - 1)*8; to account for the variable-sized table. That computation doesn't overflow quite so low, but with the truncation for min_size, we end up with a max_size that is much smaller than our actual size. So we complain that the idx is invalid, and can't find any of its objects. We can fix this case by casting "nr" to a size_t, which will do the multiplication in 64-bits (assuming you're on a 64-bit platform; this will never work on a 32-bit system since we couldn't map the whole .idx anyway). Likewise, we don't have to worry about further additions, because adding a smaller number to a size_t will convert the other side to a size_t. A few notes: - obviously we could just declare "nr" as a size_t in the first place (and likewise, packed_git.num_objects). But it's conceptually a uint32_t because of the on-disk format, and we correctly treat it that way in other contexts that don't need to compute byte offsets (e.g., iterating over the set of objects should and generally does use a uint32_t). Switching to size_t would make all of those other cases look wrong. - it could be argued that the proper type is off_t to represent the file offset. But in practice the .idx file must fit within memory, because we mmap the whole thing. And the rest of the code (including the idx_size variable we're comparing against) uses size_t. - we'll add the same cast to the max_size arithmetic line. Even though we're adding to a larger type, which will convert our result, the multiplication is still done as a 32-bit value and can itself overflow. I didn't check this with my test case, since it would need an even larger pack (~530M objects), but looking at compiler output shows that it works this way. The standard should agree, but I couldn't find anything explicit in 6.3.1.8 ("usual arithmetic conversions"). The case in load_idx() was the most immediate one that I was able to trigger. After fixing it, looking up actual objects (including the very last one in sha1 order) works in a test repo with 153,725,110 objects. That's because bsearch_hash() works with uint32_t entry indices, and the actual byte access: int cmp = hashcmp(table + mi * stride, sha1); is done with "stride" as a size_t, causing the uint32_t "mi" to be promoted to a size_t. This is the way most code will access the index data. However, I audited all of the other byte-wise accesses of packed_git.index_data, and many of the others are suspect (they are similar to the max_size one, where we are adding to a properly sized offset or directly to a pointer, but the multiplication in the sub-expression can overflow). I didn't trigger any of these in practice, but I believe they're potential problems, and certainly adding in the cast is not going to hurt anything here. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-11-11receive-pack: log received client session IDLibravatar Josh Steadmon1-0/+7
When receive-pack receives a session-id capability from the client, log the received session ID via a trace2 data event. Signed-off-by: Josh Steadmon <steadmon@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-11-11receive-pack: advertise session ID in v0 capabilitiesLibravatar Josh Steadmon1-0/+8
When transfer.advertiseSID is true, advertise receive-pack's session ID via the new session-id capability. Signed-off-by: Josh Steadmon <steadmon@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>