summaryrefslogtreecommitdiff
path: root/t/t3210-pack-refs.sh
AgeCommit message (Collapse)AuthorFilesLines
2015-05-14lock_packed_refs(): allow retries when acquiring the packed-refs lockLibravatar Michael Haggerty1-0/+17
Currently, there is only one attempt to acquire any lockfile, and if the lock is held by another process, the locking attempt fails immediately. This is not such a limitation for loose reference files. First, they don't take long to rewrite. Second, most reference updates have a known "old" value, so if another process is updating a reference at the same moment that we are trying to lock it, then probably the expected "old" value will not longer be valid, and the update will fail anyway. But these arguments do not hold for packed-refs: * The packed-refs file can be large and take significant time to rewrite. * Many references are stored in a single packed-refs file, so it could be that the other process was changing a different reference than the one that we are interested in. Therefore, it is much more likely for there to be spurious lock conflicts in connection to the packed-refs file, resulting in unnecessary command failures. So, if the first attempt to lock the packed-refs file fails, continue retrying for a configurable length of time before giving up. The default timeout is 1 second. Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-09-26Merge branch 'jk/faster-name-conflicts'Libravatar Junio C Hamano1-1/+30
Optimize the check to see if a ref $F can be created by making sure no existing ref has $F/ as its prefix, which especially matters in a repository with a large number of existing refs. * jk/faster-name-conflicts: refs: speed up is_refname_available
2014-09-12refs: speed up is_refname_availableLibravatar Jeff King1-1/+30
Our filesystem ref storage does not allow D/F conflicts; so if "refs/heads/a/b" exists, we do not allow "refs/heads/a" to exist (and vice versa). This falls out naturally for loose refs, where the filesystem enforces the condition. But for packed-refs, we have to make the check ourselves. We do so by iterating over the entire packed-refs namespace and checking whether each name creates a conflict. If you have a very large number of refs, this is quite inefficient, as you end up doing a large number of comparisons with uninteresting bits of the ref tree (e.g., we know that all of "refs/tags" is uninteresting in the example above, yet we check each entry in it). Instead, let's take advantage of the fact that we have the packed refs stored as a trie of ref_entry structs. We can find each component of the proposed refname as we walk through the trie, checking for D/F conflicts as we go. For a refname of depth N (i.e., 4 in the above example), we only have to visit N nodes. And at each visit, we can binary search the M names at that level, for a total complexity of O(N lg M). ("M" is different at each level, of course, but we can take the worst-case "M" as a bound). In a pathological case of fetching 30,000 fresh refs into a repository with 8.5 million refs, this dropped the time to run "git fetch" from tens of minutes to ~30s. This may also help smaller cases in which we check against loose refs (which we do when renaming a ref), as we may avoid a disk access for unrelated loose directories. Note that the tests we add appear at first glance to be redundant with what is already in t3210. However, the early tests are not robust; they are run with reflogs turned on, meaning that we are not actually testing is_refname_available at all! The operations will still fail because the reflogs will hit D/F conflicts in the filesystem. To get a true test, we must turn off reflogs (but we don't want to do so for the entire script, because the point of turning them on was to cover some other cases). Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-08-25pack-refs: prune top-level refs like "refs/foo"Libravatar Jeff King1-0/+7
After we have packed all refs, we prune any loose refs that correspond to what we packed. We do so by first taking a lock with lock_ref_sha1, and then deleting the loose ref file. However, lock_ref_sha1 will refuse to take a lock on any refs that exist at the top-level of the "refs/" directory, and we skip pruning the ref. This is almost certainly not what we want to happen here. The criteria to be pruned should not differ from that to be packed; if a ref makes it to prune_ref, it's because we want it both packed and pruned (if there are refs you do not want to be packed, they should be omitted much earlier by pack_ref_is_possible, which we do in this case if --all is not given). We can fix this by switching to lock_any_ref_for_update. This behaves exactly the same with the exception of this top-level check. Signed-off-by: Jeff King <peff@peff.net> Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu> Reviewed-by: Ronnie Sahlberg <sahlberg@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-05-01repack_without_ref(): silence errors for dangling packed refsLibravatar Michael Haggerty1-1/+1
Stop emitting an error message when deleting a packed reference if we find another dangling packed reference that is overridden by a loose reference. See the previous commit for a longer explanation of the issue. We have to be careful to make sure that the invalid packed reference really *is* overridden by a loose reference; otherwise what we have found is repository corruption, which we *should* report. Please note that this approach is vulnerable to a race condition similar to the race conditions already known to affect packed references [1]: * Process 1 tries to peel packed reference X as part of deleting another packed reference. It discovers that X does not refer to a valid object (because the object that it referred to has been garbage collected). * Process 2 tries to delete reference X. It starts by deleting the loose reference X. * Process 1 checks whether there is a loose reference X. There is not (it has just been deleted by process 2), so process 1 reports a spurious error "X does not point to a valid object!" The worst case seems relatively harmless, and the fix is identical to the fix that will be needed for the other race conditions (namely holding a lock on the packed-refs file during *all* reference deletions), so we leave the cleaning up of all of them as a future project. [1] http://thread.gmane.org/gmane.comp.version-control.git/211956 Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-05-01t3210: test for spurious error messages for dangling packed refsLibravatar Michael Haggerty1-0/+33
A packed reference can be overridden by a loose reference, in which case the packed reference is obsolete and is never used. The object pointed to by such a reference can be garbage collected. Since d66da478f2, this could lead to the emission of a spurious error message: error: refs/heads/master does not point to a valid object! The error is generated by repack_without_ref() if there is an obsolete dangling packed reference in packed-refs when the packed-refs file has to be rewritten due to the deletion of another packed reference. Add a failing test demonstrating this problem and some passing tests of related scenarios. Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-07-07pack-refs: remove newly empty directoriesLibravatar Greg Price1-0/+6
In a large repository which uses directories to organize many refs, "git pack-refs --all --prune" does not improve performance so much as it should, unless we remove all the now-empty directories as well. Signed-off-by: Greg Price <price@ksplice.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-05-31tests: use "test_cmp", not "diff", when verifying the resultLibravatar Gary V. Vaughan1-4/+4
In tests, call test_cmp rather than raw diff where possible (i.e. if the output does not go to a pipe), to allow the use of, say, 'cmp' when the default 'diff -u' is not compatible with a vendor diff. When that is not possible, use $DIFF, as set in GIT-BUILD-OPTIONS. Signed-off-by: Gary V. Vaughan <gary@thewrittenword.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-11-05do not force write of packed refsLibravatar Clemens Buchacher1-0/+7
We force writing a ref if it does not exist. Originally, we only had to look for the ref file to check if it existed. Now we have to look for a packed ref as well. Luckily, resolve_ref already does all the work for us. Signed-off-by: Clemens Buchacher <drizzd@aon.at> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-09-03tests: use "git xyzzy" form (t0000 - t3599)Libravatar Nanako Shiraishi1-2/+2
Converts tests between t0050-t3903. Signed-off-by: Nanako Shiraishi <nanako3@lavabit.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-07-13t/: Use "test_must_fail git" instead of "! git"Libravatar Stephan Beyer1-2/+2
This patch changes every occurrence of "! git" -- with the meaning that a git call has to gracefully fail -- into "test_must_fail git". This is useful to - make sure the test does not fail because of a signal, e.g. SIGSEGV, and - advertise the use of "test_must_fail" for new tests. Signed-off-by: Stephan Beyer <s-beyer@gmx.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-02-01Sane use of test_expect_failureLibravatar Junio C Hamano1-13/+13
Originally, test_expect_failure was designed to be the opposite of test_expect_success, but this was a bad decision. Most tests run a series of commands that leads to the single command that needs to be tested, like this: test_expect_{success,failure} 'test title' ' setup1 && setup2 && setup3 && what is to be tested ' And expecting a failure exit from the whole sequence misses the point of writing tests. Your setup$N that are supposed to succeed may have failed without even reaching what you are trying to test. The only valid use of test_expect_failure is to check a trivial single command that is expected to fail, which is a minority in tests of Porcelain-ish commands. This large-ish patch rewrites all uses of test_expect_failure to use test_expect_success and rewrites the condition of what is tested, like this: test_expect_success 'test title' ' setup1 && setup2 && setup3 && ! this command should fail ' test_expect_failure is redefined to serve as a reminder that that test *should* succeed but due to a known breakage in git it currently does not pass. So if git-foo command should create a file 'bar' but you discovered a bug that it doesn't, you can write a test like this: test_expect_failure 'git-foo should create bar' ' rm -f bar && git foo && test -f bar ' This construct acts similar to test_expect_success, but instead of reporting "ok/FAIL" like test_expect_success does, the outcome is reported as "FIXED/still broken". Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-07-02Rewrite "git-frotz" to "git frotz"Libravatar Junio C Hamano1-43/+43
This uses the remove-dashes target to replace "git-frotz" to "git frotz". Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-01-25Fix seriously broken "git pack-refs"Libravatar Linus Torvalds1-0/+9
Do *NOT* try this on a repository you care about: git pack-refs --all --prune git pack-refs because while the first "pack-refs" does the right thing, the second pack-refs will totally screw you over. This is because the second one tries to pack only tags; we should also pack what are already packed -- otherwise we would lose them. [jc: with an additional test] Signed-off-by: Junio C Hamano <junkio@cox.net>
2007-01-08--prune is now default for 'pack-refs'Libravatar Junio C Hamano1-1/+1
There is no reason not to, really. Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-10-19ref-log: fix D/F conflict coming from deleted refs.Libravatar Junio C Hamano1-0/+2
After deleting a branch l/k, you should be able to create a branch l. Earlier we added remove_empty_directories() on the ref creation side to remove leftover .git/refs/l directory but we also need a matching code to remove .git/logs/refs/l directory. Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-10-08git-pack-refs --allLibravatar Junio C Hamano1-8/+8
This changes 'git-pack-refs' to pack only tags by default. Branches are meant to be updated, either by committing onto it yourself or tracking remote branches, and packed entries can become stale easily, but tags are usually "create once and live forever" and benefit more from packing. Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-10-06Remove bashism from t3210-pack-refs.shLibravatar Dennis Stosberg1-1/+1
This bashism makes the test fail if /bin/sh is not bash. Signed-off-by: Dennis Stosberg <dennis@stosberg.net> Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-10-01Clean up "git-branch.sh" and add remove recursive dir test cases.Libravatar Christian Couder1-0/+27
Now that directory recursive remove works in the core C code, we don't need to do it in "git-branch.sh". Also add test cases to check that directory recursive remove will continue to work. Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-09-27Uncomment test case: git branch c/d should barf if branch c exists.Libravatar Christian Couder1-6/+6
Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-09-27Add pack-refs and show-ref test cases.Libravatar Christian Couder1-0/+70
Some of these test cases are from Junio. One test case is commented out because it doesn't work right now. Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <junkio@cox.net>