summaryrefslogtreecommitdiff
path: root/refs/packed-backend.c
AgeCommit message (Collapse)AuthorFilesLines
2022-02-18Merge branch 'ps/avoid-unnecessary-hook-invocation-with-packed-refs'Libravatar Junio C Hamano1-7/+21
Because a deletion of ref would need to remove it from both the loose ref store and the packed ref store, a delete-ref operation that logically removes one ref may end up invoking ref-transaction hook twice, which has been corrected. * ps/avoid-unnecessary-hook-invocation-with-packed-refs: refs: skip hooks when deleting uncovered packed refs refs: do not execute reference-transaction hook on packing refs refs: demonstrate excessive execution of the reference-transaction hook refs: allow skipping the reference-transaction hook refs: allow passing flags when beginning transactions refs: extract packed_refs_delete_refs() to allow control of transaction
2022-01-17refs: allow passing flags when beginning transactionsLibravatar Patrick Steinhardt1-1/+1
We do not currently have any flags when creating reference transactions, but we'll add one to disable execution of the reference transaction hook in some cases. Allow passing flags to `ref_store_transaction_begin()` to prepare for this change. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-17refs: extract packed_refs_delete_refs() to allow control of transactionLibravatar Patrick Steinhardt1-6/+20
When deleting loose refs, then we also have to delete the refs in the packed backend. This is done by calling `refs_delete_refs()`, which then uses the packed-backend's logic to delete refs. This doesn't allow us to exercise any control over the reference transaction which is being created in the packed backend, which is required in a subsequent commit. Extract a new function `packed_refs_delete_refs()`, which hosts most of the logic to delete refs except for creating the transaction itself. Like this, we can easily create the transaction in the files backend and thus exert more control over it. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-22refs: centralize initialization of the base ref_store.Libravatar Han-Wen Nienhuys1-4/+2
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-22refs: pass gitdir to packed_ref_store_createLibravatar Han-Wen Nienhuys1-4/+5
This is consistent with the calling convention for ref backend creation, and avoids storing ".git/packed-refs" (the name of a regular file) in a variable called ref_store::gitdir. Signed-off-by: Han-Wen Nienhuys <hanwen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-10Merge branch 'hn/create-reflog-simplify'Libravatar Junio C Hamano1-2/+1
A small simplification of API. * hn/create-reflog-simplify: refs: drop force_create argument of create_reflog API
2021-11-29Merge branch 'ab/refs-errno-cleanup'Libravatar Junio C Hamano1-3/+4
The "remainder" of hn/refs-errno-cleanup topic. * ab/refs-errno-cleanup: (21 commits) refs API: post-migration API renaming [2/2] refs API: post-migration API renaming [1/2] refs API: don't expose "errno" in run_transaction_hook() refs API: make expand_ref() & repo_dwim_log() not set errno refs API: make resolve_ref_unsafe() not set errno refs API: make refs_ref_exists() not set errno refs API: make refs_resolve_refdup() not set errno refs tests: ignore ignore errno in test-ref-store helper refs API: ignore errno in worktree.c's find_shared_symref() refs API: ignore errno in worktree.c's add_head_info() refs API: make files_copy_or_rename_ref() et al not set errno refs API: make loose_fill_ref_dir() not set errno refs API: make resolve_gitlink_ref() not set errno refs API: remove refs_read_ref_full() wrapper refs/files: remove "name exist?" check in lock_ref_oid_basic() reflog tests: add --updateref tests refs API: make refs_rename_ref_available() static refs API: make parse_loose_ref_contents() not set errno refs API: make refs_read_raw_ref() not set errno refs API: add a version of refs_resolve_ref_unsafe() with "errno" ...
2021-11-22refs: drop force_create argument of create_reflog APILibravatar Han-Wen Nienhuys1-2/+1
There is only one caller, builtin/checkout.c, and it hardcodes force_create=1. This argument was introduced in abd0cd3a301 (refs: new public ref function: safe_create_reflog, 2015-07-21), which promised to immediately use it in a follow-on commit, but that never happened. Signed-off-by: Han-Wen Nienhuys <hanwen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-25Merge branch 'jt/no-abuse-alternate-odb-for-submodules'Libravatar Junio C Hamano1-3/+10
Follow through the work to use the repo interface to access submodule objects in-process, instead of abusing the alternate object database interface. * jt/no-abuse-alternate-odb-for-submodules: submodule: trace adding submodule ODB as alternate submodule: pass repo to check_has_commit() object-file: only register submodule ODB if needed merge-{ort,recursive}: remove add_submodule_odb() refs: peeling non-the_repository iterators is BUG refs: teach arbitrary repo support to iterators refs: plumb repo into ref stores
2021-10-16refs API: make refs_read_raw_ref() not set errnoLibravatar Han-Wen Nienhuys1-3/+4
Add a "failure_errno" to refs_read_raw_ref(), his allows refs_werrres_ref_unsafe() to pass along its "failure_errno", as a first step before its own callers are migrated to pass it further up the chain. We are leaving out out the refs_read_special_head() in refs_read_raw_ref() for now, as noted in a subsequent commit moving it to "failure_errno" will require some special consideration. Signed-off-by: Han-Wen Nienhuys <hanwen@google.com> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-08refs: peeling non-the_repository iterators is BUGLibravatar Jonathan Tan1-0/+3
There is currently no support for peeling the current ref of an iterator iterating over a non-the_repository ref store, and none is needed. Thus, for now, BUG() if that happens. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-08refs: teach arbitrary repo support to iteratorsLibravatar Jonathan Tan1-2/+4
Note that should_pack_ref() is called when writing refs, which is only supported for the_repository, hence the_repository is hardcoded there. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-08refs: plumb repo into ref storesLibravatar Jonathan Tan1-1/+3
In preparation for the next 2 patches that adds (partial) support for arbitrary repositories to ref iterators, plumb a repository into all ref stores. There are no changes to program logic. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-08-25refs: make errno output explicit for read_raw_ref_fnLibravatar Han-Wen Nienhuys1-4/+4
This makes it explicit how alternative ref backends should report errors in read_raw_ref_fn. read_raw_ref_fn needs to supply a credible errno for a number of cases. These are primarily: 1) The files backend calls read_raw_ref from lock_raw_ref, and uses the resulting error codes to create/remove directories as needed. 2) ENOENT should be translated in a zero OID, optionally with REF_ISBROKEN set, returning the last successfully resolved symref. This is necessary so read_raw_ref("HEAD") on an empty repo returns refs/heads/main (or the default branch du-jour), and we know on which branch to create the first commit. Make this information flow explicit by adding a failure_errno to the signature of read_raw_ref. All errnos from the files backend are still propagated unchanged, even though inspection suggests only ENOTDIR, EISDIR and ENOENT are relevant. Signed-off-by: Han-Wen Nienhuys <hanwen@google.com> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-08-25refs API: remove OID argument to reflog_expire()Libravatar Ævar Arnfjörð Bjarmason1-1/+1
Since the the preceding commit the "oid" parameter to reflog_expire() is always NULL, but it was not cleaned up to reduce the size of the diff. Let's do that subsequent API and documentation cleanup now. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-08-19refs/packet: add missing BUG() invocations to reflog callbacksLibravatar Ævar Arnfjörð Bjarmason1-0/+5
In e0cc8ac8202 (packed_ref_store: make class into a subclass of `ref_store`, 2017-06-23) a die() was added to packed_create_reflog(), but not to any of the other reflog callbacks, let's do that. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-07-16Merge branch 'ab/struct-init'Libravatar Junio C Hamano1-1/+1
Code cleanup around struct_type_init() functions. * ab/struct-init: string-list.h users: change to use *_{nodup,dup}() string-list.[ch]: add a string_list_init_{nodup,dup}() dir.[ch]: replace dir_init() with DIR_INIT *.c *_init(): define in terms of corresponding *_INIT macro *.h: move some *_INIT to designated initializers
2021-07-01string-list.h users: change to use *_{nodup,dup}()Libravatar Ævar Arnfjörð Bjarmason1-1/+1
Change all in-tree users of the string_list_init(LIST, BOOL) API to use string_list_init_{nodup,dup}(LIST) instead. As noted in the preceding commit let's leave the now-unused string_list_init() wrapper in-place for any in-flight users, it can be removed at some later date. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-20refs: make explicit that ref_iterator_peel returns booleanLibravatar Han-Wen Nienhuys1-1/+1
Use -1 as error return value throughout. This removes spurious differences in the GIT_TRACE_REFS output, depending on the ref storage backend active. Before, the cached ref_iterator (but only that iterator!) would return peel_object() output directly. No callers relied on the peel_status values beyond success/failure. All calls to these functions go through peel_iterated_oid(), which returns peel_object() as a fallback, but also squashing the error values. The iteration interface already passes REF_ISSYMREF and REF_ISBROKEN through the flags argument, so the additional error values in enum peel_status provide no value. The ref iteration interface provides a separate peel() function because certain formats (eg. packed-refs and reftable) can store the peeled object next to the tag SHA1. Passing the peeled SHA1 as an optional argument to each_ref_fn maps more naturally to the implementation of ref databases. Changing the code in this way is left for a future refactoring. Signed-off-by: Han-Wen Nienhuys <hanwen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-03-13use CALLOC_ARRAYLibravatar René Scharfe1-2/+2
Add and apply a semantic patch for converting code that open-codes CALLOC_ARRAY to use it instead. It shortens the code and infers the element size automatically. Signed-off-by: René Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-08-19refs: move gitdir into base ref_storeLibravatar Han-Wen Nienhuys1-0/+1
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-08-02pack-refs: always refresh after taking the lock fileLibravatar Sun Chao1-7/+16
When a packed ref is deleted, the whole packed-refs file is rewritten to omit the ref that no longer exists. However if another gc command is running and calls `pack-refs --all` simultaneously, there is a chance that a ref that was just updated lose the newly created commits. Through these steps, losing commits on newly updated refs can be demonstrated: # step 1: compile git without `USE_NSEC` option Some kernel releases do enable it by default while some do not. And if we compile git without `USE_NSEC`, it will be easier demonstrated by the following steps. # step 2: setup a repository and add the first commit git init repo && (cd repo && git config core.logallrefupdates true && git commit --allow-empty -m foo) # step 3: in one terminal, repack the refs repeatedly cd repo && while true do git pack-refs --all done # step 4: in another terminal, simultaneously update the # master with update-ref, and create and delete an # unrelated ref also with update-ref cd repo && while true do us=$(git commit-tree -m foo -p HEAD HEAD^{tree}) && git update-ref refs/heads/newbranch $us && git update-ref refs/heads/master $us && git update-ref -d refs/heads/newbranch && them=$(git rev-parse master) && if test "$them" != "$us" then echo >&2 "lost commit: $us" exit 1 fi # eye candy printf . done Though we have the packed-refs lock file and loose refs lock files to avoid updating conflicts, a ref will lost its newly commits if racy stat-validity of `packed-refs` file happens (which is quite same as the racy-git described in `Documentation/technical/racy-git.txt`), the following specific set of operations demonstrates the problem: 1. Call `pack-refs --all` to pack all the loose refs to packed-refs, and let say the modify time of the packed-refs is DATE_M. 2. Call `update-ref` to update a new commit to master while it is already packed. the old value (let us call it OID_A) remains in the packed-refs file and write the new value (let us call it OID_B) to $GIT_DIR/refs/heads/master. 3. Call `update-ref -d` within the same DATE_M from the 1th step to delete a different ref newbranch which is packed in the packed-refs file. It check newbranch's oid from packed-refs file without locking it. Meanwhile it keeps a snapshot of the packed-refs file in memory and record the file's attributes with the snapshot. The oid of master in the packed-refs's snapshot is OID_A. 4. Call a new `pack-refs --all` to pack the loose refs, the oid of master in packe-refs file is OID_B, and the loose refs $GIT_DIR/refs/heads/master is removed. Let's say the `pack-refs --all` is very quickly done and the new packed-refs file's modify time is still DATE_M, and it has the same file size, even the same inode. 5. 3th step now goes on after checking the newbranch, it begin to rewrite the packed-refs file. After get the lock file of packed-ref file, it checks it's on-disk file attributes with the snapshot, suck as the timestamp, the file size and the inode value. If they are both the same values, and the snapshot is not refreshed. Because the loose ref of master is removed by 4th step, `update-ref -d` will updates the new packed-ref to disk which contains master with the oid OID_A. So now the newly commit OID_B of master is lost. The best path forward is just always refreshing after take the lock file of `packed-refs` file. Traditionally we avoided that because refreshing it implied parsing the whole file. But these days we mmap it, so it really is just an extra open()/mmap() and a quick read of the header. That doesn't seem like an outrageous cost to pay when we're already taking the lock. Signed-off-by: Sun Chao <sunchao9@huawei.com> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Sun Chao <sunchao9@huawei.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-15refs/packed-backend: express constants using the_hash_algoLibravatar brian m. carlson1-7/+7
Switch uses of GIT_SHA1_HEXSZ to use the_hash_algo so that they are appropriate for the any given hash length. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-29convert "oidcmp() != 0" to "!oideq()"Libravatar Jeff King1-1/+1
This is the flip side of the previous two patches: checking for a non-zero oidcmp() can be more strictly expressed as inequality. Like those patches, we write "!= 0" in the coccinelle transformation, which covers by isomorphism the more common: if (oidcmp(E1, E2)) As with the previous two patches, this patch can be achieved almost entirely by running "make coccicheck"; the only differences are manual line-wrap fixes to match the original code. There is one thing to note for anybody replicating this, though: coccinelle 1.0.4 seems to miss the case in builtin/tag.c, even though it's basically the same as all the others. Running with 1.0.7 does catch this, so presumably it's just a coccinelle bug that was fixed in the interim. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-06-01refs/packed-backend.c: close fd of empty fileLibravatar Stefan Beller1-0/+1
Signed-off-by: Stefan Beller <sbeller@google.com> Acked-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-05-06Replace all die("BUG: ...") calls by BUG() onesLibravatar Johannes Schindelin1-8/+8
In d8193743e08 (usage.c: add BUG() function, 2017-05-12), a new macro was introduced to use for reporting bugs instead of die(). It was then subsequently used to convert one single caller in 588a538ae55 (setup_git_env: convert die("BUG") to BUG(), 2017-05-12). The cover letter of the patch series containing this patch (cf 20170513032414.mfrwabt4hovujde2@sigill.intra.peff.net) is not terribly clear why only one call site was converted, or what the plan is for other, similar calls to die() to report bugs. Let's just convert all remaining ones in one fell swoop. This trick was performed by this invocation: sed -i 's/die("BUG: /BUG("/g' $(git grep -l 'die("BUG' \*.c) Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-03-30refs: use chdir_notify to update cached relative pathsLibravatar Jeff King1-0/+3
Commit f57f37e2e1 (files-backend: remove the use of git_path(), 2017-03-26) introduced a regression when a relative $GIT_DIR is used in a working tree: - when we initialize the ref backend, we make a copy of get_git_dir(), which may be relative - later, we may call setup_work_tree() and chdir to the root of the working tree - further calls to the ref code will use the stored git directory, but relative paths will now point to the wrong place The new test in t1501 demonstrates one such instance (the bug causes us to write the ref update to the nonsense "relative/relative/.git"). Since setup_work_tree() now uses chdir_notify, we can just ask it update our relative paths when necessary. Reported-by: Rafael Ascensao <rafa.almas@gmail.com> Helped-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-02-15Merge branch 'kg/packed-ref-cache-fix'Libravatar Junio C Hamano1-51/+55
Avoid mmapping small files while using packed refs (especially ones with zero size, which would cause later munmap() to fail). * kg/packed-ref-cache-fix: packed_ref_cache: don't use mmap() for small files load_contents(): don't try to mmap an empty file packed_ref_iterator_begin(): make optimization more general find_reference_location(): make function safe for empty snapshots create_snapshot(): use `xmemdupz()` rather than a strbuf struct snapshot: store `start` rather than `header_len`
2018-01-24packed_ref_cache: don't use mmap() for small filesLibravatar Kim Gybels1-1/+3
Take a hint from commit ea68b0ce9f8 (hash-object: don't use mmap() for small files, 2010-02-21) and use read() instead of mmap() for small packed-refs files. Signed-off-by: Kim Gybels <kgybels@infogroep.be> Signed-off-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-01-24load_contents(): don't try to mmap an empty fileLibravatar Michael Haggerty1-7/+6
We don't actually create zero-length `packed-refs` files, but they are valid and we should handle them correctly. The old code `xmmap()`ed such files, which led to an error when `munmap()` was called. So, if the `packed-refs` file is empty, leave the snapshot at its zero values and return 0 without trying to read or mmap the file. Returning 0 also makes `create_snapshot()` exit early, which avoids the technically undefined comparison `NULL < NULL`. Reported-by: Kim Gybels <kgybels@infogroep.be> Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-01-24packed_ref_iterator_begin(): make optimization more generalLibravatar Michael Haggerty1-6/+6
We can return an empty iterator not only if the `packed-refs` file is missing, but also if it is empty or if there are no references whose names succeed `prefix`. Optimize away those cases as well by moving the call to `find_reference_location()` higher in the function and checking whether the determined start position is the same as `snapshot->eof`. (This is possible now because the previous commit made `find_reference_location()` robust against empty snapshots.) Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-01-24find_reference_location(): make function safe for empty snapshotsLibravatar Michael Haggerty1-4/+6
This function had two problems if called for an empty snapshot (i.e., `snapshot->start == snapshot->eof == NULL`): * It checked `NULL < NULL`, which is undefined by C (albeit highly unlikely to fail in the real world). * (Assuming the above comparison behaved as expected), it returned NULL when `mustexist` was false, contrary to its docstring. Change the check and fix the docstring. Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-01-24create_snapshot(): use `xmemdupz()` rather than a strbufLibravatar Michael Haggerty1-5/+4
It's lighter weight. Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-01-24struct snapshot: store `start` rather than `header_len`Libravatar Michael Haggerty1-31/+33
Store a pointer to the start of the actual references within the `packed-refs` contents rather than storing the length of the header. This is more convenient for most users of this field. Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-12-06Merge branch 'mh/avoid-rewriting-packed-refs' into maintLibravatar Junio C Hamano1-0/+94
Recent update to the refs infrastructure implementation started rewriting packed-refs file more often than before; this has been optimized again for most trivial cases. * mh/avoid-rewriting-packed-refs: files-backend: don't rewrite the `packed-refs` file unnecessarily t1409: check that `packed-refs` is not rewritten unnecessarily
2017-11-15Merge branch 'mh/tidy-ref-update-flags'Libravatar Junio C Hamano1-9/+9
Code clean-up in refs API implementation. * mh/tidy-ref-update-flags: refs: update some more docs to use "oid" rather than "sha1" write_packed_entry(): take `object_id` arguments refs: rename constant `REF_ISPRUNING` to `REF_IS_PRUNING` refs: rename constant `REF_NODEREF` to `REF_NO_DEREF` refs: tidy up and adjust visibility of the `ref_update` flags ref_transaction_add_update(): remove a check ref_transaction_update(): die on disallowed flags prune_ref(): call `ref_transaction_add_update()` directly files_transaction_prepare(): don't leak flags to packed transaction
2017-11-15Merge branch 'mh/avoid-rewriting-packed-refs'Libravatar Junio C Hamano1-0/+94
Recent update to the refs infrastructure implementation started rewriting packed-refs file more often than before; this has been optimized again for most trivial cases. * mh/avoid-rewriting-packed-refs: files-backend: don't rewrite the `packed-refs` file unnecessarily t1409: check that `packed-refs` is not rewritten unnecessarily
2017-11-06refs: update some more docs to use "oid" rather than "sha1"Libravatar Michael Haggerty1-1/+1
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-11-06write_packed_entry(): take `object_id` argumentsLibravatar Michael Haggerty1-8/+8
Change `write_packed_entry()` to take `struct object_id *` rather than `unsigned char *` arguments. Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-10-30files-backend: don't rewrite the `packed-refs` file unnecessarilyLibravatar Michael Haggerty1-0/+94
Even when we are deleting references, we needn't overwrite the `packed-refs` file if the references that we are deleting only exist as loose references. Implement this optimization as follows: * Add a function `is_packed_transaction_needed()`, which checks whether a given packed-refs transaction actually needs to be carried out (i.e., it returns false if the transaction obviously wouldn't have any effect). This function must be called while holding the `packed-refs` lock to avoid races. * Change `files_transaction_prepare()` to check whether the packed-refs transaction is actually needed. If not, squelch it, but continue holding the `packed-refs` lock until the end of the transaction to avoid races. This fixes a mild regression caused by dc39e09942 (files_ref_store: use a transaction to update packed refs, 2017-09-08). Before that commit, unnecessary rewrites of `packed-refs` were suppressed by `repack_without_refs()`. But the transaction-based writing introduced by that commit didn't perform that optimization. Note that the pre-dc39e09942 code still had to *read* the whole `packed-refs` file to determine that the rewrite could be skipped, so the performance for the cases that the write could be elided was `O(N)` in the number of packed references both before and after dc39e09942. But after that commit the constant factor increased. This commit reimplements the optimization of eliding unnecessary `packed-refs` rewrites. That, plus the fact that since cfa2e29c34 (packed_ref_store: get rid of the `ref_cache` entirely, 2017-03-17) we don't necessarily have to read the whole `packed-refs` file at all, means that deletes of one or a few loose references can now be done with `O(n lg N)` effort, where `n` is the number of loose references being deleted and `N` is the total number of packed references. This commit fixes two tests in t1409. Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-10-16refs: convert read_raw_ref backends to struct object_idLibravatar brian m. carlson1-2/+2
Convert the unsigned char * parameter to struct object_id * for files_read_raw_ref and packed_read_raw_ref. Update the documentation. Switch from using get_sha1_hex and a hard-coded 40 to using parse_oid_hex. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-10-16refs: convert peel_object to struct object_idLibravatar brian m. carlson1-3/+3
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-10-16refs: convert reflog_expire parameter to struct object_idLibravatar brian m. carlson1-1/+1
reflog_expire already used struct object_id internally, but it did not take it as a parameter. Adjust the parameter (and the callers) to pass a pointer to struct object_id instead of a pointer to unsigned char. Remove the temporary inserted earlier as it is no longer required. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-10-05Merge branch 'rs/cleanup-strbuf-users'Libravatar Junio C Hamano1-2/+2
Code clean-up. * rs/cleanup-strbuf-users: graph: use strbuf_addchars() to add spaces use strbuf_addstr() for adding strings to strbufs path: use strbuf_add_real_path()
2017-10-03Merge branch 'mh/mmap-packed-refs'Libravatar Junio C Hamano1-237/+742
Operations that do not touch (majority of) packed refs have been optimized by making accesses to packed-refs file lazy; we no longer pre-parse everything, and an access to a single ref in the packed-refs does not touch majority of irrelevant refs, either. * mh/mmap-packed-refs: (21 commits) packed-backend.c: rename a bunch of things and update comments mmapped_ref_iterator: inline into `packed_ref_iterator` ref_cache: remove support for storing peeled values packed_ref_store: get rid of the `ref_cache` entirely ref_store: implement `refs_peel_ref()` generically packed_read_raw_ref(): read the reference from the mmapped buffer packed_ref_iterator_begin(): iterate using `mmapped_ref_iterator` read_packed_refs(): ensure that references are ordered when read packed_ref_cache: keep the `packed-refs` file mmapped if possible packed-backend.c: reorder some definitions mmapped_ref_iterator_advance(): no peeled value for broken refs mmapped_ref_iterator: add iterator over a packed-refs file packed_ref_cache: remember the file-wide peeling state read_packed_refs(): read references with minimal copying read_packed_refs(): make parsing of the header line more robust read_packed_refs(): only check for a header at the top of the file read_packed_refs(): use mmap to read the `packed-refs` file die_unterminated_line(), die_invalid_line(): new functions packed_ref_cache: add a backlink to the associated `packed_ref_store` prefix_ref_iterator: break when we leave the prefix ...
2017-10-03Merge branch 'sd/branch-copy'Libravatar Junio C Hamano1-0/+8
"git branch" learned "-c/-C" to create a new branch by copying an existing one. * sd/branch-copy: branch: fix "copy" to never touch HEAD branch: add a --copy (-c) option to go with --move (-m) branch: add test for -m renaming multiple config sections config: create a function to format section headers
2017-10-02use strbuf_addstr() for adding strings to strbufsLibravatar René Scharfe1-2/+2
Use strbuf_addstr() instead of strbuf_addf() for adding strings. That's simpler and makes the intent clearer. Patch generated by Coccinelle and contrib/coccinelle/strbuf.cocci; adjusted indentation in refs/packed-backend.c manually. Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-09-25packed-backend.c: rename a bunch of things and update commentsLibravatar Michael Haggerty1-190/+232
We've made huge changes to this file, and some of the old names and comments are no longer very fitting. So rename a bunch of things: * `struct packed_ref_cache` → `struct snapshot` * `acquire_packed_ref_cache()` → `acquire_snapshot()` * `release_packed_ref_buffer()` → `clear_snapshot_buffer()` * `release_packed_ref_cache()` → `release_snapshot()` * `clear_packed_ref_cache()` → `clear_snapshot()` * `struct packed_ref_entry` → `struct snapshot_record` * `cmp_packed_ref_entries()` → `cmp_packed_ref_records()` * `cmp_entry_to_refname()` → `cmp_record_to_refname()` * `sort_packed_refs()` → `sort_snapshot()` * `read_packed_refs()` → `create_snapshot()` * `validate_packed_ref_cache()` → `validate_snapshot()` * `get_packed_ref_cache()` → `get_snapshot()` * Renamed local variables and struct members accordingly. Also update a bunch of comments to reflect the renaming and the accumulated changes that the code has undergone. Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-09-25mmapped_ref_iterator: inline into `packed_ref_iterator`Libravatar Michael Haggerty1-170/+114
Since `packed_ref_iterator` is now delegating to `mmapped_ref_iterator` rather than `cache_ref_iterator` to do the heavy lifting, there is no need to keep the two iterators separate. So "inline" `mmapped_ref_iterator` into `packed_ref_iterator`. This removes a bunch of boilerplate. Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-09-25ref_cache: remove support for storing peeled valuesLibravatar Michael Haggerty1-1/+8
Now that the `packed-refs` backend doesn't use `ref_cache`, there is nobody left who might want to store peeled values of references in `ref_cache`. So remove that feature. Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>