summaryrefslogtreecommitdiff
path: root/builtin
AgeCommit message (Collapse)AuthorFilesLines
2021-09-20Merge branch 'js/run-command-close-packs'Libravatar Junio C Hamano7-20/+9
The run-command API has been updated so that the callers can easily ask the file descriptors open for packfiles to be closed immediately before spawning commands that may trigger auto-gc. * js/run-command-close-packs: Close object store closer to spawning child processes run_auto_maintenance(): implicitly close the object store run-command: offer to close the object store before running run-command: prettify the `RUN_COMMAND_*` flags pull: release packs before fetching commit-graph: when closing the graph, also release the slab
2021-09-20Merge branch 'ds/mergies-with-sparse-index'Libravatar Junio C Hamano3-0/+12
Various mergy operations have been prepared to work efficiently with the sparse index. * ds/mergies-with-sparse-index: sparse-index: integrate with cherry-pick and rebase sequencer: ensure full index if not ORT strategy t1092: add cherry-pick, rebase tests merge-ort: expand only for out-of-cone conflicts merge: make sparse-aware with ORT diff: ignore sparse paths in diffstat
2021-09-20Merge branch 'ds/sparse-index-ignored-files'Libravatar Junio C Hamano2-6/+95
In cone mode, the sparse-index code path learned to remove ignored files (like build artifacts) outside the sparse cone, allowing the entire directory outside the sparse cone to be removed, which is especially useful when the sparse patterns change. * ds/sparse-index-ignored-files: sparse-checkout: clear tracked sparse dirs sparse-index: add SPARSE_INDEX_MEMORY_ONLY flag attr: be careful about sparse directories sparse-checkout: create helper methods sparse-index: use WRITE_TREE_MISSING_OK sparse-index: silently return when cache tree fails unpack-trees: fix nested sparse-dir search sparse-index: silently return when not using cone-mode patterns t7519: rewrite sparse index test
2021-09-20Merge branch 'ar/submodule-run-update-procedure'Libravatar Junio C Hamano1-0/+257
Reimplementation of parts of "git submodule" in C continues. * ar/submodule-run-update-procedure: submodule--helper: run update procedures from C
2021-09-20Merge branch 'ab/serve-cleanup'Libravatar Junio C Hamano2-15/+16
Code clean-up around "git serve". * ab/serve-cleanup: upload-pack: document and rename --advertise-refs serve.[ch]: remove "serve_options", split up --advertise-refs code {upload,receive}-pack tests: add --advertise-refs tests serve.c: move version line to advertise_capabilities() serve: move transfer.advertiseSID check into session_id_advertise() serve.[ch]: don't pass "struct strvec *keys" to commands serve: use designated initializers transport: use designated initializers transport: rename "fetch" in transport_vtable to "fetch_refs" serve: mark has_capability() as static
2021-09-20Merge branch 'ar/submodule-add-more'Libravatar Junio C Hamano2-261/+164
More parts of "git submodule add" has been rewritten in C. * ar/submodule-add-more: submodule--helper: rename compute_submodule_clone_url() submodule--helper: remove resolve-relative-url subcommand submodule--helper: remove add-config subcommand submodule--helper: remove add-clone subcommand submodule--helper: convert the bulk of cmd_add() to C dir: libify and export helper functions from clone.c submodule--helper: remove repeated code in sync_submodule() submodule--helper: refactor resolve_relative_url() helper submodule--helper: add options for compute_submodule_clone_url()
2021-09-20Merge branch 'ar/submodule-add-config'Libravatar Junio C Hamano1-0/+129
Large part of "git submodule add" gets rewritten in C. * ar/submodule-add-config: submodule--helper: introduce add-config subcommand
2021-09-20Merge branch 'ab/unbundle-progress'Libravatar Junio C Hamano2-1/+16
Add progress display to "git bundle unbundle". * ab/unbundle-progress: bundle: show progress on "unbundle" index-pack: add --progress-title option bundle API: change "flags" to be "extra_index_pack_args" bundle API: start writing API documentation
2021-09-20Merge branch 'tb/pack-finalize-ordering'Libravatar Junio C Hamano3-30/+35
The order in which various files that make up a single (conceptual) packfile has been reevaluated and straightened up. This matters in correctness, as an incomplete set of files must not be shown to a running Git. * tb/pack-finalize-ordering: pack-objects: rename .idx files into place after .bitmap files pack-write: split up finish_tmp_packfile() function builtin/index-pack.c: move `.idx` files into place last index-pack: refactor renaming in final() builtin/repack.c: move `.idx` files into place last pack-write.c: rename `.idx` files after `*.rev` pack-write: refactor renaming in finish_tmp_packfile() bulk-checkin.c: store checksum directly pack.h: line-wrap the definition of finish_tmp_packfile()
2021-09-20Merge branch 'lh/systemd-timers'Libravatar Junio C Hamano1-71/+512
"git maintenance" scheduler learned to use systemd timers as a possible backend. * lh/systemd-timers: maintenance: add support for systemd timers on Linux maintenance: `git maintenance run` learned `--scheduler=<scheduler>` cache.h: Introduce a generic "xdg_config_home_for(…)" function
2021-09-20Merge branch 'jt/grep-wo-submodule-odb-as-alternate'Libravatar Junio C Hamano1-23/+41
The code to make "git grep" recurse into submodules has been updated to migrate away from the "add submodule's object store as an alternate object store" mechanism (which is suboptimal). * jt/grep-wo-submodule-odb-as-alternate: t7814: show lack of alternate ODB-adding submodule-config: pass repo upon blob config read grep: add repository to OID grep sources grep: allocate subrepos on heap grep: read submodule entry with explicit repo grep: typesafe versions of grep_source_init grep: use submodule-ODB-as-alternate lazy-addition submodule: lazily add submodule ODBs as alternates
2021-09-20Merge branch 'tb/multi-pack-bitmaps'Libravatar Junio C Hamano4-25/+19
The reachability bitmap file used to be generated only for a single pack, but now we've learned to generate bitmaps for history that span across multiple packfiles. * tb/multi-pack-bitmaps: (29 commits) pack-bitmap: drop bitmap_index argument from try_partial_reuse() pack-bitmap: drop repository argument from prepare_midx_bitmap_git() p5326: perf tests for MIDX bitmaps p5310: extract full and partial bitmap tests midx: respect 'GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP' t7700: update to work with MIDX bitmap test knob t5319: don't write MIDX bitmaps in t5319 t5310: disable GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP t0410: disable GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP t5326: test multi-pack bitmap behavior t/helper/test-read-midx.c: add --checksum mode t5310: move some tests to lib-bitmap.sh pack-bitmap: write multi-pack bitmaps pack-bitmap: read multi-pack bitmaps pack-bitmap.c: avoid redundant calls to try_partial_reuse pack-bitmap.c: introduce 'bitmap_is_preferred_refname()' pack-bitmap.c: introduce 'nth_bitmap_object_oid()' pack-bitmap.c: introduce 'bitmap_num_objects()' midx: avoid opening multiple MIDXs when writing midx: close linked MIDXs, avoid leaking memory ...
2021-09-20Merge branch 'ps/fetch-optim'Libravatar Junio C Hamano3-49/+50
Optimize code that handles large number of refs in the "git fetch" code path. * ps/fetch-optim: fetch: avoid second connectivity check if we already have all objects fetch: merge fetching and consuming refs fetch: refactor fetch refs to be more extendable fetch-pack: optimize loading of refs via commit graph connected: refactor iterator to return next object ID directly fetch: avoid unpacking headers in object existence check fetch: speed up lookup of want refs via commit-graph
2021-09-15Merge branch 'jc/prefix-filename-allocates'Libravatar Junio C Hamano1-1/+1
Leakfix. * jc/prefix-filename-allocates: hash-object: prefix_filename() returns allocated memory these days
2021-09-15Merge branch 'so/diff-index-regression-fix'Libravatar Junio C Hamano1-3/+3
Recent "diff -m" changes broke "gitk", which has been corrected. * so/diff-index-regression-fix: diff-index: restore -c/--cc options handling
2021-09-10Merge branch 'rs/show-branch-simplify'Libravatar Junio C Hamano1-10/+5
Code cleanup. * rs/show-branch-simplify: show-branch: simplify rev_is_head()
2021-09-10Merge branch 'ab/retire-advice-config'Libravatar Junio C Hamano12-25/+26
Code clean up to migrate callers from older advice_config[] based API to newer advice_if_enabled() and advice_enabled() API. * ab/retire-advice-config: advice: move advice.graftFileDeprecated squashing to commit.[ch] advice: remove use of global advice_add_embedded_repo advice: remove read uses of most global `advice_` variables advice: add enum variants for missing advice variables
2021-09-10Merge branch 'mk/clone-recurse-submodules'Libravatar Junio C Hamano1-0/+5
After "git clone --recurse-submodules", all submodules are cloned but they are not by default recursed into by other commands. With submodule.stickyRecursiveClone configuration set, submodule.recurse configuration is set to true in a repository created by "clone" with "--recurse-submodules" option. * mk/clone-recurse-submodules: clone: set submodule.recurse=true if submodule.stickyRecursiveClone enabled
2021-09-10Merge branch 'ab/gc-log-rephrase'Libravatar Junio C Hamano1-1/+1
A pathname in an advice message has been made cut-and-paste ready. * ab/gc-log-rephrase: gc: remove trailing dot from "gc.log" line
2021-09-10Merge branch 'tk/fast-export-anonymized-tag-fix'Libravatar Junio C Hamano1-0/+1
The output from "git fast-export", when its anonymization feature is in use, showed an annotated tag incorrectly. * tk/fast-export-anonymized-tag-fix: fast-export: fix anonymized tag using original length
2021-09-10Merge branch 'ab/commit-graph-usage'Libravatar Junio C Hamano2-46/+55
Fixes on usage message from "git commit-graph". * ab/commit-graph-usage: commit-graph: show "unexpected subcommand" error commit-graph: show usage on "commit-graph [write|verify] garbage" commit-graph: early exit to "usage" on !argc multi-pack-index: refactor "goto usage" pattern commit-graph: use parse_options_concat() commit-graph: remove redundant handling of -h commit-graph: define common usage with a macro
2021-09-10Merge branch 'tb/add-objects-in-unpacked-packs-simplify'Libravatar Junio C Hamano1-73/+12
Code simplification with reduced memory usage. * tb/add-objects-in-unpacked-packs-simplify: builtin/pack-objects.c: remove duplicate hash lookup builtin/pack-objects.c: simplify add_objects_in_unpacked_packs() object-store.h: teach for_each_packed_object to ignore kept packs
2021-09-10Merge branch 'ps/fetch-omit-formatting-under-quiet'Libravatar Junio C Hamano1-5/+12
"git fetch --quiet" optimization to avoid useless computation of info that will never be displayed. * ps/fetch-omit-formatting-under-quiet: fetch: skip formatting updated refs with `--quiet`
2021-09-10Merge branch 'js/advise-when-skipping-cherry-picked'Libravatar Junio C Hamano1-0/+1
"git rebase" by default skips changes that are equivalent to commits that are already in the history the branch is rebased onto; give messages when this happens to let the users be aware of skipped commits, and also teach them how to tell "rebase" to keep duplicated changes. * js/advise-when-skipping-cherry-picked: sequencer: advise if skipping cherry-picked commit
2021-09-09pack-objects: rename .idx files into place after .bitmap filesLibravatar Ævar Arnfjörð Bjarmason1-1/+2
In preceding commits the race of renaming .idx files in place before .rev files and other auxiliary files was fixed in pack-write.c's finish_tmp_packfile(), builtin/repack.c's "struct exts", and builtin/index-pack.c's final(). As noted in the change to pack-write.c we left in place the issue of writing *.bitmap files after the *.idx, let's fix that issue. See 7cc8f971085 (pack-objects: implement bitmap writing, 2013-12-21) for commentary at the time when *.bitmap was implemented about how those files are written out, nothing in that commit contradicts what's being done here. Note that this commit and preceding ones only close any race condition with *.idx files being written before their auxiliary files if we're optimistic about our lack of fsync()-ing in this are not tripping us over. See the thread at [1] for a rabbit hole of various discussions about filesystem races in the face of doing and not doing fsync() (and if doing fsync(), not doing it properly). We may want to fsync the containing directory once after renaming the *.idx file into place, but that is outside the scope of this series. 1. https://lore.kernel.org/git/8735qgkvv1.fsf@evledraar.gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-09pack-write: split up finish_tmp_packfile() functionLibravatar Ævar Arnfjörð Bjarmason1-2/+5
Split up the finish_tmp_packfile() function and use the split-up version in pack-objects.c in preparation for moving the step of renaming the *.idx file later as part of a function change. Since the only other caller of finish_tmp_packfile() was in bulk-checkin.c, and it won't be needing a change to its *.idx renaming, provide a thin wrapper for the old function as a static function in that file. If other callers end up needing the simpler version it could be moved back to "pack-write.c" and "pack.h". Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-09builtin/index-pack.c: move `.idx` files into place lastLibravatar Taylor Blau1-2/+2
In a similar spirit as preceding patches to `git repack` and `git pack-objects`, fix the identical problem in `git index-pack`. Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-09index-pack: refactor renaming in final()Libravatar Ævar Arnfjörð Bjarmason1-25/+23
Refactor the renaming in final() into a helper function, this is similar in spirit to a preceding refactoring of finish_tmp_packfile() in pack-write.c. Before e37d0b8730b (builtin/index-pack.c: write reverse indexes, 2021-01-25) it probably wasn't worth it to have this sort of helper, due to the differing "else if" case for "pack" files v.s. "idx" files. But since we've got "rev" as well now, let's do the renaming via a helper, this is both a net decrease in lines, and improves the readability, since we can easily see at a glance that the logic for writing these three types of files is exactly the same, aside from the obviously differing cases of "*final_name" being NULL, and "make_read_only_if_same" being different. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-09builtin/repack.c: move `.idx` files into place lastLibravatar Taylor Blau1-1/+1
In a similar spirit as the previous patch, fix the identical problem from `git repack` (which invokes `pack-objects` with a temporary location for output, and then moves the files into their final locations itself). Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-09pack-write: refactor renaming in finish_tmp_packfile()Libravatar Ævar Arnfjörð Bjarmason1-2/+5
Refactor the renaming in finish_tmp_packfile() into a helper function. The callers are now expected to pass a "name_buffer" ending in "pack-OID." instead of the previous "pack-", we then append "pack", "idx" or "rev" to it. By doing the strbuf_setlen() in rename_tmp_packfile() we reuse the buffer and avoid the repeated allocations we'd get if that function had its own temporary "struct strbuf". This approach of reusing the buffer does make the last user in pack-object.c's write_pack_file() slightly awkward, since we needlessly do a strbuf_setlen() before calling strbuf_release() for consistency. In subsequent changes we'll move that bitmap writing code around, so let's not skip the strbuf_setlen() now. The previous strbuf_reset() idiom originated with 5889271114a (finish_tmp_packfile():use strbuf for pathname construction, 2014-03-03), which in turn was a minimal adjustment of pre-strbuf code added in 0e990530ae (finish_tmp_packfile(): a helper function, 2011-10-28). Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-09sparse-index: integrate with cherry-pick and rebaseLibravatar Derrick Stolee2-0/+9
The hard work was already done with 'git merge' and the ORT strategy. Just add extra tests to see that we get the expected results in the non-conflict cases. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Reviewed-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-09merge: make sparse-aware with ORTLibravatar Derrick Stolee1-0/+3
Allow 'git merge' to operate without expanding a sparse index, at least not immediately. The index still will be expanded in a few cases: 1. If the merge strategy is 'recursive', then we enable command_requires_full_index at the start of the merge_recursive() method. We expect sparse-index users to also have the 'ort' strategy enabled. 2. With the 'ort' strategy, if the merge results in a conflicted file, then we expand the index before updating the working tree. The loop that iterates over the worktree replaces index entries and tracks 'origintal_cache_nr' which can become completely wrong if the index expands in the middle of the operation. This safety valve is important before that loop starts. A later change will focus this to only expand if we indeed have a conflict outside of the sparse-checkout cone. 3. Other merge strategies are executed as a 'git merge-X' subcommand, and those strategies are currently protected with the 'command_requires_full_index' guard. Some test updates are required, including a mistaken 'git checkout -b' that did not specify the base branch, causing merges to be fast-forward merges. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Reviewed-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-09Close object store closer to spawning child processesLibravatar Johannes Schindelin3-16/+8
In many cases where we spawned child processes that _may_ trigger a repack, we explicitly closed the object store first (so that the `repack` process can delete the `.pack` files, which would otherwise not be possible on Windows since files cannot be deleted as long as they as still in use). Wherever possible, we now use the new `close_object_store` bit of the `run_command()` API, to delay closing the object store even further. This makes the code easier to maintain because it is now more obvious that we only release those file handles because of those child processes. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-09run_auto_maintenance(): implicitly close the object storeLibravatar Johannes Schindelin4-5/+0
Before spawning the auto maintenance, we need to make sure that we release all open file handles to all the `.pack` files (and MIDX files and commit-graph files and...) so that the maintenance process has the freedom to delete those files. So far, we did this manually every time before calling `run_auto_maintenance()`. With the new `close_object_store` flag, we can do that implicitly in that function, which is more robust because future callers won't be able to forget to close the object store. Note: this changes behavior slightly, as we previously _always_ closed the object store, but now we only close the object store when actually running the auto maintenance. In practice, this should not matter (if anything, it might speed up operations where auto maintenance is disabled). Suggested-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-08Merge branch 'sg/column-nl'Libravatar Junio C Hamano1-1/+1
The parser for the "--nl" option of "git column" has been corrected. * sg/column-nl: column: fix parsing of the '--nl' option
2021-09-08Merge branch 'rs/branch-allow-deleting-dangling'Libravatar Junio C Hamano1-1/+1
"git branch -D <branch>" used to refuse to remove a broken branch ref that points at a missing commit, which has been corrected. * rs/branch-allow-deleting-dangling: branch: allow deleting dangling branches with --force
2021-09-08Merge branch 'mt/quiet-with-delayed-checkout'Libravatar Junio C Hamano1-1/+1
The delayed checkout code path in "git checkout" etc. were chatty even when --quiet and/or --no-progress options were given. * mt/quiet-with-delayed-checkout: checkout: make delayed checkout respect --quiet and --no-progress
2021-09-08Merge branch 'rs/xopen-reports-open-failures'Libravatar Junio C Hamano11-37/+12
Error diagnostics improvement. * rs/xopen-reports-open-failures: use xopen() to handle fatal open(2) failures xopen: explicitly report creation failures
2021-09-08Merge branch 'js/maintenance-launchctl-fix'Libravatar Junio C Hamano1-21/+70
"git maintenance" scheduler fix for macOS. * js/maintenance-launchctl-fix: maintenance: skip bootout/bootstrap when plist is registered maintenance: create `launchctl` configuration using a lock file
2021-09-08Merge branch 'ab/rebase-fatal-fatal-fix'Libravatar Junio C Hamano1-1/+1
Error message fix. * ab/rebase-fatal-fatal-fix: rebase: emit one "fatal" in "fatal: fatal: <error>"
2021-09-08Merge branch 'ab/ls-remote-packet-trace'Libravatar Junio C Hamano1-0/+2
Debugging aid fix. * ab/ls-remote-packet-trace: ls-remote: set packet_trace_identity(<name>)
2021-09-08pull: release packs before fetchingLibravatar Johannes Schindelin1-0/+2
On Windows, files cannot be removed nor renamed if there are still handles held by a process. To remedy that, we try to release all open handles to any `.pack` file before e.g. repacking (which would want to remove the original `.pack` file(s) after it is done). Since the `read_cache_unmerged()` and/or the `get_oid()` call in `git pull` can cause `.pack` files to be opened, we need to release the open handles before calling `git fetch`: the latter process might want to spawn an auto-gc, which in turn might want to repack the objects. This commit is similar in spirit to 5bdece0d705 (gc/repack: release packs when needed, 2018-12-15). This fixes https://github.com/git-for-windows/git/issues/3336. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-08grep: add repository to OID grep sourcesLibravatar Jonathan Tan1-9/+6
Record the repository whenever an OID grep source is created, and teach the worker threads to explicitly provide the repository when accessing objects. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Reviewed-by: Matheus Tavares <matheus.bernardino@usp.br> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-08grep: allocate subrepos on heapLibravatar Jonathan Tan1-9/+30
Currently, struct repository objects corresponding to submodules are allocated on the stack in grep_submodule(). This currently works because they will not be used once grep_submodule() exits, but a subsequent patch will require these structs to be accessible for longer (perhaps even in another thread). Allocate them on the heap and clear them only at the very end. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Reviewed-by: Matheus Tavares <matheus.bernardino@usp.br> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-08grep: read submodule entry with explicit repoLibravatar Jonathan Tan1-5/+5
Replace an existing parse_object_or_die() call (which implicitly works on the_repository) with a function call that allows a repository to be passed in. There is no such direct equivalent to parse_object_or_die(), but we only need the type of the object, so replace with oid_object_info(). Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Reviewed-by: Emily Shaffer <emilyshaffer@google.com> Reviewed-by: Matheus Tavares <matheus.bernardino@usp.br> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-08grep: typesafe versions of grep_source_initLibravatar Jonathan Tan1-2/+2
grep_source_init() can create "struct grep_source" objects and, depending on the value of the type passed, some void-pointer parameters have different meanings. Because one of these types (GREP_SOURCE_OID) will require an additional parameter in a subsequent patch, take the opportunity to increase clarity and type safety by replacing this function with individual functions for each type. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Reviewed-by: Matheus Tavares <matheus.bernardino@usp.br> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-08grep: use submodule-ODB-as-alternate lazy-additionLibravatar Jonathan Tan1-1/+1
In the parent commit, Git was taught to add submodule ODBs as alternates lazily, but grep does not use this because it computes the path to add directly, not going through add_submodule_odb(). Add an equivalent to add_submodule_odb() that takes the exact ODB path and teach grep to use it. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Reviewed-by: Emily Shaffer <emilyshaffer@google.com> Reviewed-by: Matheus Tavares <matheus.bernardino@usp.br> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-07sparse-checkout: clear tracked sparse dirsLibravatar Derrick Stolee1-0/+94
When changing the scope of a sparse-checkout using cone mode, we might have some tracked directories go out of scope. The current logic removes the tracked files from within those directories, but leaves the ignored files within those directories. This is a bit unexpected to users who have given input to Git saying they don't need those directories anymore. This is something that is new to the cone mode pattern type: the user has explicitly said "I want these directories and _not_ those directories." The typical sparse-checkout patterns more generally apply to "I want files with with these patterns" so it is natural to leave ignored files as they are. This focus on directories in cone mode provides us an opportunity to change the behavior. Leaving these ignored files in the sparse directories makes it impossible to gain performance benefits in the sparse index. When we track into these directories, we need to know if the files are ignored or not, which might depend on the _tracked_ .gitignore file(s) within the sparse directory. This depends on the indexed version of the file, so the sparse directory must be expanded. We must take special care to look for untracked, non-ignored files in these directories before deleting them. We do not want to delete any meaningful work that the users were doing in those directories and perhaps forgot to add and commit before switching sparse-checkout definitions. Since those untracked files might be code files that generated ignored build output, also do not delete any ignored files from these directories in that case. The users can recover their state by resetting their sparse-checkout definition to include that directory and continue. Alternatively, they can see the warning that is presented and delete the directory themselves to regain the performance they expect. By deleting the sparse directories when changing scope (or running 'git sparse-checkout reapply') we regain these performance benefits as if the repository was in a clean state. Since these ignored files are frequently build output or helper files from IDEs, the users should not need the files now that the tracked files are removed. If the tracked files reappear, then they will have newer timestamps than the build artifacts, so the artifacts will need to be regenerated anyway. Use the sparse-index as a data structure in order to find the sparse directories that can be safely deleted. Re-expand the index to a full one if it was full before. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Reviewed-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-07sparse-checkout: create helper methodsLibravatar Derrick Stolee1-6/+1
As we integrate the sparse index into more builtins, we occasionally need to check the sparse-checkout patterns to see if a path is within the sparse-checkout cone. Create some helper methods that help initialize the patterns and check for pattern matching to make this easier. The existing callers of commands like get_sparse_checkout_patterns() use a custom 'struct pattern_list' that is not necessarily the one in the 'struct index_state', so there are not many previous uses that could adopt these helpers. There are just two in builtin/add.c and sparse-index.c that can use path_in_sparse_checkout(). We add a path_in_cone_mode_sparse_checkout() as well that will only return false if the path is outside of the sparse-checkout definition _and_ the sparse-checkout patterns are in cone mode. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Reviewed-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-07hash-object: prefix_filename() returns allocated memory these daysLibravatar Junio C Hamano1-1/+1
Back when a1be47e4 (hash-object: fix buffer reuse with --path in a subdirectory, 2017-03-20) was written, the prefix_filename() helper used a static piece of memory to the caller, making the caller responsible for copying it, if it wants to keep it across another call to the same function. Two callers of the prefix_filename() in hash-object were made to xstrdup() the value obtained from it. But in the same series, when e4da43b1 (prefix_filename: return newly allocated string, 2017-03-20) changed the rule to gave the caller possession of the memory, we forgot to revert one of the xstrdup() changes, allowing the returned value to leak. Signed-off-by: Junio C Hamano <gitster@pobox.com>