summaryrefslogtreecommitdiff
path: root/t/t5304-prune.sh
AgeCommit message (Collapse)AuthorFilesLines
2019-04-19t5304: add a test for pruning with bitmapsLibravatar Jeff King1-0/+8
Commit fde67d6896 (prune: use bitmaps for reachability traversal, 2019-02-13) uses bitmaps for pruning when they're available, but only covers this functionality in the t/perf tests. This makes a kind of sense, since the point is that the behaviour is indistinguishable before and after the patch, just faster. But since the bitmap code path is not exercised at all in the regular test suite, it leaves us open to a regression where the behavior does in fact change. The most thorough way to test that would be running the whole suite with bitmaps enabled. But we don't yet have a way to do that, and anyway it's expensive to do so. Let's at least add a basic test that exercises this path and make sure we prune an object we should (and not one that we shouldn't). That would hopefully catch the most obvious breakages early. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-14t5304: rename "sha1" variables to "oid"Libravatar Jeff King1-12/+12
Let's make the script less jarring to read in a post-sha1 world by using more hash-agnostic variable names. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-14prune: lazily perform reachability traversalLibravatar Jeff King1-0/+12
The general strategy of "git prune" is to do a full reachability walk, then for each loose object see if we found it in our walk. But if we don't have any loose objects, we don't need to do the expensive walk in the first place. This patch postpones that walk until the first time we need to see its results. Note that this is really a specific case of a more general optimization, which is that we could traverse only far enough to find the object under consideration (i.e., stop the traversal when we find it, then pick up again when asked about the next object, etc). That could save us in some instances from having to do a full walk. But it's actually a bit tricky to do with our traversal code, and you'd need to do a full walk anyway if you have even a single unreachable object (which you generally do, if any objects are actually left after running git-repack). So in practice this lazy-load of the full walk catches one easy but common case (i.e., you've just repacked via git-gc, and there's nothing unreachable). The perf script is fairly contrived, but it does show off the improvement: Test HEAD^ HEAD ------------------------------------------------------------------------- 5304.4: prune with no objects 3.66(3.60+0.05) 0.00(0.00+0.00) -100.0% and would let us know if we accidentally regress this optimization. Note also that we need to take special care with prune_shallow(), which relies on us having performed the traversal. So this optimization can only kick in for a non-shallow repository. Since this is easy to get wrong and is not covered by existing tests, let's add an extra test to t5304 that covers this case explicitly. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-21tests: use 'test_must_be_empty' instead of 'test_cmp <empty> <out>'Libravatar SZEDER Gábor1-2/+1
Using 'test_must_be_empty' is shorter and more idiomatic than >empty && test_cmp empty out as it saves the creation of an empty file. Furthermore, sometimes the expected empty file doesn't have such a descriptive name like 'empty', and its creation is far away from the place where it's finally used for comparison (e.g. in 't7600-merge.sh', where two expected empty files are created in the 'setup' test, but are used only about 500 lines later). These cases were found by instrumenting 'test_cmp' to error out the test script when it's used to compare empty files, and then converted manually. Note that even after this patch there still remain a lot of cases where we use 'test_cmp' to check empty files: - Sometimes the expected output is not hard-coded in the test, but 'test_cmp' is used to ensure that two similar git commands produce the same output, and that output happens to be empty, e.g. the test 'submodule update --merge - ignores --merge for new submodules' in 't7406-submodule-update.sh'. - Repetitive common tasks, including preparing the expected results and running 'test_cmp', are often extracted into a helper function, and some of this helper's callsites expect no output. - For the same reason as above, the whole 'test_expect_success' block is within a helper function, e.g. in 't3070-wildmatch.sh'. - Or 'test_cmp' is invoked in a loop, e.g. the test 'cvs update (-p)' in 't9400-git-cvsserver-server.sh'. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-05-08Merge branch 'jc/parseopt-expiry-errors'Libravatar Junio C Hamano1-0/+10
"git gc --prune=nonsense" spent long time repacking and then silently failed when underlying "git prune --expire=nonsense" failed to parse its command line. This has been corrected. * jc/parseopt-expiry-errors: parseopt: handle malformed --expire arguments more nicely gc: do not upcase error message shown with die()
2018-04-23parseopt: handle malformed --expire arguments more nicelyLibravatar Junio C Hamano1-0/+10
A few commands that parse --expire=<time> command line option behave sillily when given nonsense input. For example $ git prune --no-expire Segmentation falut $ git prune --expire=npw; echo $? 129 Both come from parse_opt_expiry_date_cb(). The former is because the function is not prepared to see arg==NULL (for "--no-expire", it is a norm; "--expire" at the end of the command line could be made to pass NULL, if it is told that the argument is optional, but we don't so we do not have to worry about that case). The latter is because it does not check the value returned from the underlying parse_expiry_date(). This seems to be a recent regression introduced while we attempted to avoid spewing the entire usage message when given a correct option but with an invalid value at 3bb0923f ("parse-options: do not show usage upon invalid option value", 2018-03-22). Before that, we didn't fail silently but showed a full usage help (which arguably is not all that better). Also catch this error early when "git gc --prune=<expiration>" is misspelled by doing a dummy parsing before the main body of "gc" that is time consuming even begins. Otherwise, we'd spend time to pack objects and then later have "git prune" first notice the error. Aborting "gc" in the middle that way is not harmful but is ugly and can be avoided. Helped-by: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-03-27t/helper: merge test-chmtime into test-toolLibravatar Nguyễn Thái Ngọc Duy1-8/+8
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-08-24revision.c: --reflog add HEAD reflog from all worktreesLibravatar Nguyễn Thái Ngọc Duy1-0/+16
Note that add_other_reflogs_to_pending() is a bit inefficient, since it scans reflog for all refs of each worktree, including shared refs, so the shared ref's reflog is scanned over and over again. We could update refs API to pass "per-worktree only" flag to avoid that. But long term we should be able to obtain a "per-worktree only" ref store and would need to revert the changes in reflog iteration API. So let's just wait until then. add_reflogs_to_pending() is called by reachable.c so by default "git prune" will examine reflog from all worktrees. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-08-24revision.c: --all adds HEAD from all worktreesLibravatar Nguyễn Thái Ngọc Duy1-0/+12
Unless single_worktree is set, --all now adds HEAD from all worktrees. Since reachable.c code does not use setup_revisions(), we need to call other_head_refs_submodule() explicitly there to have the same effect on "git prune", so that we won't accidentally delete objects needed by some other HEADs. A new FIXME is added because we would need something like int refs_other_head_refs(struct ref_store *, each_ref_fn, cb_data); in addition to other_head_refs() to handle it, which might require int get_submodule_worktrees(const char *submodule, int flags); It could be a separate topic to reduce the scope of this one. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-08-24revision.c: --indexed-objects add objects from all worktreesLibravatar Nguyễn Thái Ngọc Duy1-0/+9
This is the result of single_worktree flag never being set (no way to up until now). To get objects from current index only, set single_worktree. The other add_index_objects_to_pending's caller is mark_reachable_objects() (e.g. "git prune") which also mark objects from all indexes. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-12-28t/t5304-prune.sh: use the $( ... ) construct for command substitutionLibravatar Elia Pinto1-1/+1
The Git CodingGuidelines prefer the $(...) construct for command substitution instead of using the backquotes `...`. The backquoted form is the traditional method for command substitution, and is supported by POSIX. However, all but the simplest uses become complicated quickly. In particular, embedded command substitutions and/or the use of double quotes require careful escaping with the backslash character. The patch was generated by: for _f in $(find . -name "*.sh") do perl -i -pe 'BEGIN{undef $/;} s/`(.+?)`/\$(\1)/smg' "${_f}" done and then carefully proof-read. Signed-off-by: Elia Pinto <gitter.spiros@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-11-04gc: remove garbage .idx files from pack dirLibravatar Doug Kelly1-1/+1
Add a custom report_garbage handler to collect and remove garbage .idx files from the pack directory. Signed-off-by: Doug Kelly <dougk.ff7@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-11-04t5304: test cleaning pack garbageLibravatar Doug Kelly1-0/+21
Pack garbage, noticeably stale .idx files, can be cleaned up during a garbage collection. This tests to ensure such garbage is properly cleaned up. Note that the prior test for checking pack garbage with count-objects left some stale garbage after the test exited. This has also been corrected. Signed-off-by: Doug Kelly <dougk.ff7@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-06-22for_each_packed_object: automatically open pack indexLibravatar Jeff King1-0/+1
When for_each_packed_object is called, we call prepare_packed_git() to make sure we have the actual list of packs. But the latter does not actually open the pack indices, meaning that pack->nr_objects may simply be 0 if the pack has not otherwise been used since the program started. In practice, this didn't come up for the current callers, because they iterate the packed objects only after iterating all reachable objects (so for it to matter you would have to have a pack consisting only of unreachable objects). But it is a dangerous and confusing interface that should be fixed for future callers. Note that we do not end the iteration when a pack cannot be opened, but we do return an error. That lets you complete the iteration even in actively-repacked repository where an .idx file may racily go away, but it also lets callers know that they may not have gotten the complete list (which the current reachability-check caller does care about). We have to tweak one of the prune tests due to the changed return value; an earlier test creates bogus .idx files and does not clean them up. Having to make this tweak is a good thing; it means we will not prune in a broken repository, and the test confirms that we do not negatively impact a more lenient caller, count-objects. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-02-09sha1_file: fix iterating loose alternate objectsLibravatar Jonathon Mah1-0/+8
The string in 'base' contains a path suffix to a specific object; when its value is used, the suffix must either be filled (as in stat_sha1_file, open_sha1_file, check_and_freshen_nonlocal) or cleared (as in prepare_packed_git) to avoid junk at the end. 660c889e (sha1_file: add for_each iterators for loose and packed objects, 2014-10-15) introduced loose_from_alt_odb(), but this did neither and treated 'base' as a complete path to the "base" object directory, instead of a pointer to the "base" of the full path string. The trailing path after 'base' is still initialized to NUL, hiding the bug in some common cases. Additionally the descendent for_each_file_in_obj_subdir() function swallows ENOENT, so an error only shows if the alternate's path was last filled with a valid object (where statting /path/to/existing/00/0bjectfile/00 fails). Signed-off-by: Jonathon Mah <me@JonathonMah.com> Helped-by: Kyle J. McKay <mackyle@gmail.com> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-10-13t5304: use helper to report failure of "test foo = bar"Libravatar Jeff King1-8/+8
For small outputs, we sometimes use: test "$(some_cmd)" = "something we expect" instead of a full test_cmp. The downside of this is that when it fails, there is no output at all from the script. Let's introduce a small helper to make tests easier to debug. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-10-13t5304: use test_path_is_* instead of "test -f"Libravatar Jeff King1-23/+23
This is slightly more robust (checking "! test -f" would not notice a directory of the same name, though that is not likely to happen here). It also makes debugging easier, as the test script will output a message on failure. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-09-03reachable.c: add HEAD to reachability starting commitsLibravatar Max Kirillov1-0/+22
HEAD is not explicitly used as a starting commit for calculating reachability, so if it's detached and reflogs are disabled it may be pruned. Add tests which demonstrate it. Test 'prune: prune former HEAD after checking out branch' also reverts changes to repository. Signed-off-by: Max Kirillov <max@max630.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-03-31code and test: fix misuses of "nor"Libravatar Justin Lebar1-1/+1
Signed-off-by: Justin Lebar <jlebar@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-12-10prune: clean .git/shallow after pruning objectsLibravatar Nguyễn Thái Ngọc Duy1-0/+10
This patch teaches "prune" to remove shallow roots that are no longer reachable from any refs (e.g. when the relevant refs are removed). Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-02-15count-objects: report garbage files in pack directory tooLibravatar Nguyễn Thái Ngọc Duy1-0/+26
prepare_packed_git_one() is modified to allow count-objects to hook a report function to so we don't need to duplicate the pack searching logic in count-objects.c. When report_pack_garbage is NULL, the overhead is insignificant. The garbage is reported with warning() instead of error() in packed garbage case because it's not an error to have garbage. Loose garbage is still reported as errors and will be converted to warnings later. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-04-14t3306,t5304: avoid clock skew issuesLibravatar Michael J Gruber1-1/+2
On systems where the local time and file modification time may be out of sync (e.g. test directory on NFS) t3306 and t5305 can fail because prune compares times such as "now" (client time) with file modification times (server times for remote file systems). I.e., these are spurious test failures. Avoid this by setting the relevant modification times to the local time. Noticed on a system with as little as 2s time skew. Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-02-28prune: honor --expire=neverLibravatar Adam Simpkins1-0/+32
Previously, prune treated an expiration time of 0 to mean that no expire argument was supplied, and everything should be pruned. As a result, "prune --expire=never" would prune all unreachable objects, regardless of their timestamp. prune can be called with --expire=never automatically by gc, when the gc.pruneExpire configuration is set to "never". Signed-off-by: Adam Simpkins <simpkins@facebook.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-09-13test local clone by copyingLibravatar Clemens Buchacher1-21/+33
Test the effect of an earlier change by f7835a2 (preserve mtime of local clone, 2009-09-12) to keep stale loose object files stale in the new repository when a local clone is made by copying files in .git/ directory. Signed-off-by: Clemens Buchacher <drizzd@aon.at> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-02-14gc: make --prune useful again by accepting an optional parameterLibravatar Johannes Schindelin1-0/+38
With this patch, "git gc --no-prune" will not prune any loose (and dangling) object, and "git gc --prune=5.minutes.ago" will prune all loose objects older than 5 minutes. This patch benefitted from suggestions by Thomas Rast and Jan Krï¿œger. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-08-13t5304-prune: adjust file mtime based on system time rather than file mtimeLibravatar Brandon Casey1-4/+4
test-chmtime can adjust the mtime of a file based on the file's mtime, or based on the system time. For files accessed over NFS, the file's mtime is set by the NFS server, and as such may vary a great deal from the NFS client's system time if the clocks of the client and server are out of sync. Since these tests are testing the expire feature of git-prune, an incorrect mtime could cause a file to be expired or not expired incorrectly and produce a test failure. Avoid this NFS pitfall by modifying the calls to test-chmtime so that the mtime is adjusted based on the system time, rather than the file's mtime. Signed-off-by: Brandon Casey <casey@nrlssc.navy.mil> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-07-04Fix executable bits in t/ scriptsLibravatar Junio C Hamano1-0/+0
Pointed out by Ramsay Jones. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-03-27builtin-prune: protect objects listed on the command lineLibravatar Junio C Hamano1-1/+1
Finally, this resurrects the documented behaviour to protect other objects listed on the command line from getting pruned. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-03-27Add tests for git-pruneLibravatar Michele Ballabio1-0/+34
It seems that git prune changed behaviour with respect to revisions added from command line, probably when it became a builtin. Currently, it prints a short usage and exits: instead, it should take those revisions into account and not prune them. So add a couple of test to point this out. We'll be fixing this by switching to parse_options(), so add tests to detect bogus command line parameters as well, to keep ourselves from introducing regressions. Signed-off-by: Michele Ballabio <barra_cuda@katamail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-03-12gc: call "prune --expire 2.weeks.ago" by defaultLibravatar Johannes Schindelin1-0/+49
The only reason we did not call "prune" in git-gc was that it is an inherently dangerous operation: if there is a commit going on, you will prune loose objects that were just created, and are, in fact, needed by the commit object just about to be created. Since it is dangerous, we told users so. That led to many users not even daring to run it when it was actually safe. Besides, they are users, and should not have to remember such details as when to call git-gc with --prune, or to call git-prune directly. Of course, the consequence was that "git gc --auto" gets triggered much more often than we would like, since unreferenced loose objects (such as left-overs from a rebase or a reset --hard) were never pruned. Alas, git-prune recently learnt the option --expire <minimum-age>, which makes it a much safer operation. This allows us to call prune from git-gc, with a grace period of 2 weeks for the unreferenced loose objects (this value was determined in a discussion on the git list as a safe one). If you want to override this grace period, just set the config variable gc.pruneExpire to a different value; an example would be [gc] pruneExpire = 6.months.ago or even "never", if you feel really paranoid. Note that this new behaviour makes "--prune" be a no-op. While adding a test to t5304-prune.sh (since it really tests the implicit call to "prune"), also the original test for "prune --expire" was moved there from t1410-reflog.sh, where it did not belong. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2008-02-11Make git prune remove temporary packs that look like write failuresLibravatar David Steven Tweed1-0/+32
Write errors when repacking (eg, due to out-of-space conditions) can leave temporary packs (and possibly other files beginning with "tmp_") lying around which no existing codepath removes and which aren't obvious to the casual user. These can also be multi-megabyte files wasting noticeable space. Unfortunately there's no way to definitely tell in builtin-prune that a tmp_ file is not being used by a concurrent process, such as a fetch. However, it is documented that pruning should only be done on a quiet repository and --expire is honoured (using code from Johannes Schindelin, along with a test case he wrote) so that its safety is the same as that of loose object pruning. Since they might be signs of a problem (unlike orphaned loose objects) the names of any removed files are printed. Signed-off-by: David Tweed (david.tweed@gmail.com) Acked-by: Nicolas Pitre <nico@cam.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>