summaryrefslogtreecommitdiff
path: root/builtin/gc.c
AgeCommit message (Collapse)AuthorFilesLines
2014-06-16Merge branch 'nd/daemonize-gc'Libravatar Junio C Hamano1-6/+20
"git gc --auto" was recently changed to run in the background to give control back early to the end-user sitting in front of the terminal, but it forgot that housekeeping involving reflogs should be done without other processes competing for accesses to the refs. * nd/daemonize-gc: gc --auto: do not lock refs in the background
2014-05-27gc --auto: do not lock refs in the backgroundLibravatar Nguyễn Thái Ngọc Duy1-6/+20
9f673f9 (gc: config option for running --auto in background - 2014-02-08) puts "gc --auto" in background to reduce user's wait time. Part of the garbage collecting is pack-refs and pruning reflogs. These require locking some refs and may abort other processes trying to lock the same ref. If gc --auto is fired in the middle of a script, gc's holding locks in the background could fail the script, which could never happen before 9f673f9. Keep running pack-refs and "reflog --prune" in foreground to stop parallel ref updates. The remaining background operations (repack, prune and rerere) should not impact running git processes. Reported-by: Adam Borowski <kilobyte@angband.pl> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-04-03Merge branch 'nd/gc-aggressive'Libravatar Junio C Hamano1-1/+7
Allow tweaking the maximum length of the delta-chain produced by "gc --aggressive". * nd/gc-aggressive: environment.c: fix constness for odb_pack_keep() gc --aggressive: make --depth configurable
2014-03-31gc --aggressive: make --depth configurableLibravatar Nguyễn Thái Ngọc Duy1-1/+7
When 1c192f3 (gc --aggressive: make it really aggressive - 2007-12-06) made --depth=250 the default value, it didn't really explain the reason behind, especially the pros and cons of --depth=250. An old mail from Linus below explains it at length. Long story short, --depth=250 is a disk saver and a performance killer. Not everybody agrees on that aggressiveness. Let the user configure it. From: Linus Torvalds <torvalds@linux-foundation.org> Subject: Re: [PATCH] gc --aggressive: make it really aggressive Date: Thu, 6 Dec 2007 08:19:24 -0800 (PST) Message-ID: <alpine.LFD.0.9999.0712060803430.13796@woody.linux-foundation.org> Gmane-URL: http://article.gmane.org/gmane.comp.gcc.devel/94637 On Thu, 6 Dec 2007, Harvey Harrison wrote: > > 7:41:25elapsed 86%CPU Heh. And this is why you want to do it exactly *once*, and then just export the end result for others ;) > -r--r--r-- 1 hharrison hharrison 324094684 2007-12-06 07:26 pack-1d46...pack But yeah, especially if you allow longer delta chains, the end result can be much smaller (and what makes the one-time repack more expensive is the window size, not the delta chain - you could make the delta chains longer with no cost overhead at packing time) HOWEVER. The longer delta chains do make it potentially much more expensive to then use old history. So there's a trade-off. And quite frankly, a delta depth of 250 is likely going to cause overflows in the delta cache (which is only 256 entries in size *and* it's a hash, so it's going to start having hash conflicts long before hitting the 250 depth limit). So when I said "--depth=250 --window=250", I chose those numbers more as an example of extremely aggressive packing, and I'm not at all sure that the end result is necessarily wonderfully usable. It's going to save disk space (and network bandwidth - the delta's will be re-used for the network protocol too!), but there are definitely downsides too, and using long delta chains may simply not be worth it in practice. (And some of it might just want to have git tuning, ie if people think that long deltas are worth it, we could easily just expand on the delta hash, at the cost of some more memory used!) That said, the good news is that working with *new* history will not be affected negatively, and if you want to be _really_ sneaky, there are ways to say "create a pack that contains the history up to a version one year ago, and be very aggressive about those old versions that we still want to have around, but do a separate pack for newer stuff using less aggressive parameters" So this is something that can be tweaked, although we don't really have any really nice interfaces for stuff like that (ie the git delta cache size is hardcoded in the sources and cannot be set in the config file, and the "pack old history more aggressively" involves some manual scripting and knowing how "git pack-objects" works rather than any nice simple command line switch). So the thing to take away from this is: - git is certainly flexible as hell - .. but to get the full power you may need to tweak things - .. happily you really only need to have one person to do the tweaking, and the tweaked end results will be available to others that do not need to know/care. And whether the difference between 320MB and 500MB is worth any really involved tweaking (considering the potential downsides), I really don't know. Only testing will tell. Linus Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-03-28Merge branch 'bp/commit-p-editor'Libravatar Junio C Hamano1-1/+1
When it is not necessary to edit a commit log message (e.g. "git commit -m" is given a message without specifying "-e"), we used to disable the spawning of the editor by overriding GIT_EDITOR, but this means all the uses of the editor, other than to edit the commit log message, are also affected. * bp/commit-p-editor: run-command: mark run_hook_with_custom_index as deprecated merge hook tests: fix and update tests merge: fix GIT_EDITOR override for commit hook commit: fix patch hunk editing with "commit -p -m" test patch hunk editing with "commit -p -m" merge hook tests: use 'test_must_fail' instead of '!' merge hook tests: fix missing '&&' in test
2014-03-18commit: fix patch hunk editing with "commit -p -m"Libravatar Benoit Pierre1-1/+1
Don't change git environment: move the GIT_EDITOR=":" override to the hook command subprocess, like it's already done for GIT_INDEX_FILE. Signed-off-by: Benoit Pierre <benoit.pierre@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-03-05Merge branch 'nd/daemonize-gc'Libravatar Junio C Hamano1-5/+18
Allow running "gc --auto" in the background. * nd/daemonize-gc: gc: config option for running --auto in background daemon: move daemonize() to libgit.a
2014-02-27Merge branch 'ep/varscope'Libravatar Junio C Hamano1-2/+3
Shrink lifetime of variables by moving their definitions to an inner scope where appropriate. * ep/varscope: builtin/gc.c: reduce scope of variables builtin/fetch.c: reduce scope of variable builtin/commit.c: reduce scope of variables builtin/clean.c: reduce scope of variable builtin/blame.c: reduce scope of variables builtin/apply.c: reduce scope of variables bisect.c: reduce scope of variable
2014-02-10gc: config option for running --auto in backgroundLibravatar Nguyễn Thái Ngọc Duy1-5/+18
`gc --auto` takes time and can block the user temporarily (but not any less annoyingly). Make it run in background on systems that support it. The only thing lost with running in background is printouts. But gc output is not really interesting. You can keep it in foreground by changing gc.autodetach. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-01-31builtin/gc.c: reduce scope of variablesLibravatar Elia Pinto1-2/+3
Signed-off-by: Elia Pinto <gitter.spiros@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-01-17Merge branch 'nd/shallow-clone'Libravatar Junio C Hamano1-0/+1
Fetching from a shallow-cloned repository used to be forbidden, primarily because the codepaths involved were not carefully vetted and we did not bother supporting such usage. This attempts to allow object transfer out of a shallow-cloned repository in a controlled way (i.e. the receiver become a shallow repository with truncated history). * nd/shallow-clone: (31 commits) t5537: fix incorrect expectation in test case 10 shallow: remove unused code send-pack.c: mark a file-local function static git-clone.txt: remove shallow clone limitations prune: clean .git/shallow after pruning objects clone: use git protocol for cloning shallow repo locally send-pack: support pushing from a shallow clone via http receive-pack: support pushing to a shallow clone via http smart-http: support shallow fetch/clone remote-curl: pass ref SHA-1 to fetch-pack as well send-pack: support pushing to a shallow clone receive-pack: allow pushes that update .git/shallow connected.c: add new variant that runs with --shallow-file add GIT_SHALLOW_FILE to propagate --shallow-file to subprocesses receive/send-pack: support pushing from a shallow clone receive-pack: reorder some code in unpack() fetch: add --update-shallow to accept refs that update .git/shallow upload-pack: make sure deepening preserves shallow roots fetch: support fetching from a shallow repository clone: support remote shallow repository ...
2014-01-02gc: notice gc processes run by other usersLibravatar Kyle J. McKay1-1/+1
Since 64a99eb4 git gc refuses to run without the --force option if another gc process on the same repository is already running. However, if the repository is shared and user A runs git gc on the repository and while that gc is still running user B runs git gc on the same repository the gc process run by user A will not be noticed and the gc run by user B will go ahead and run. The problem is that the kill(pid, 0) test fails with an EPERM error since user B is not allowed to signal processes owned by user A (unless user B is root). Update the test to recognize an EPERM error as meaning the process exists and another gc should not be run (unless --force is given). Signed-off-by: Kyle J. McKay <mackyle@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-12-10prune: clean .git/shallow after pruning objectsLibravatar Nguyễn Thái Ngọc Duy1-0/+1
This patch teaches "prune" to remove shallow roots that are no longer reachable from any refs (e.g. when the relevant refs are removed). Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-10-30Merge branch 'nd/gc-lock-against-each-other'Libravatar Junio C Hamano1-0/+24
* nd/gc-lock-against-each-other: gc: remove gc.pid file at end of execution
2013-10-18gc: remove gc.pid file at end of executionLibravatar Jonathan Nieder1-0/+24
This file isn't really harmful, but isn't useful either, and can create minor annoyance for the user: * It's confusing, as the presence of a *.pid file often implies that a process is currently running. A user running "ls .git/" and finding this file may incorrectly guess that a "git gc" is currently running. * Leaving this file means that a "git gc" in an already gc-ed repo is no-longer a no-op. A user running "git gc" in a set of repositories, and then synchronizing this set (e.g. rsync -av, unison, ...) will see all the gc.pid files as changed, which creates useless noise. This patch unlinks the file after the garbage collection is done, so that gc.pid is actually present only during execution. Future versions of Git may want to use the information left in the gc.pid file (e.g. for policies like "don't attempt to run a gc if one has already been ran less than X hours ago"). If so, this patch can safely be reverted. For now, let's not bother the users. Explained-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Improved-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-09-04Merge branch 'sb/parseopt-boolean-removal'Libravatar Junio C Hamano1-2/+2
Convert most uses of OPT_BOOLEAN/OPTION_BOOLEAN that can use OPT_BOOL/OPTION_BOOLEAN which have much saner semantics, and turn remaining ones into OPT_SET_INT, OPT_COUNTUP, etc. as necessary. * sb/parseopt-boolean-removal: revert: use the OPT_CMDMODE for parsing, reducing code checkout-index: fix negations of even numbers of -n config parsing options: allow one flag multiple times hash-object: replace stdin parsing OPT_BOOLEAN by OPT_COUNTUP branch, commit, name-rev: ease up boolean conditions checkout: remove superfluous local variable log, format-patch: parsing uses OPT__QUIET Replace deprecated OPT_BOOLEAN by OPT_BOOL Remove deprecated OPTION_BOOLEAN for parsing arguments
2013-08-09gc: reject if another gc is running, unless --force is givenLibravatar Nguyễn Thái Ngọc Duy1-0/+67
This may happen when `git gc --auto` is run automatically, then the user, to avoid wait time, switches to a new terminal, keeps working and `git gc --auto` is started again because the first gc instance has not clean up the repository. This patch tries to avoid multiple gc running, especially in --auto mode. In the worst case, gc may be delayed 12 hours if a daemon reuses the pid stored in gc.pid. kill(pid, 0) support is added to MinGW port so it should work on Windows too. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-08-05Replace deprecated OPT_BOOLEAN by OPT_BOOLLibravatar Stefan Beller1-2/+2
This task emerged from b04ba2bb (parse-options: deprecate OPT_BOOLEAN, 2011-09-27). All occurrences of the respective variables have been reviewed and none of them relied on the counting up mechanism, but all of them were using the variable as a true boolean. This patch does not change semantics of any command intentionally. Signed-off-by: Stefan Beller <stefanbeller@googlemail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-09-27silence git gc --auto --quiet outputLibravatar Tobias Ulmer1-3/+1
When --quiet is requested, gc --auto should not display messages unless there is an error. Signed-off-by: Tobias Ulmer <tobiasu@tmux.org> Acked-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-08-20i18n: gc: mark parseopt strings for translationLibravatar Nguyễn Thái Ngọc Duy1-6/+6
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-04-18gc: use argv-array for sub-commandsLibravatar Jeff King1-45/+33
git-gc executes many sub-commands. The argument list for some of these is constant, but for others we add more arguments at runtime. The latter is implemented by allocating a constant extra number of NULLs, and either using a custom append function, or just referencing unused slots by number. As of commit 7e52f56, which added two new arguments, it is possible to exceed the constant number of slots for "repack" by running "git gc --aggressive", causing "git gc" to die. This patch converts all of the static argv lists to use argv-array. In addition to fixing the overflow caused by 7e52f56, it has a few advantages: 1. We can drop the custom append function (which, incidentally, had an off-by-one error exacerbating the static limit). 2. We can drop the ugly magic numbers used when adding arguments to "prune". 3. Adding further arguments will be easier; you can just add new "push" calls without worrying about increasing any static limits. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-04-11gc: do not explode objects which will be immediately prunedLibravatar Jeff King1-8/+15
When we pack everything into one big pack with "git repack -Ad", any unreferenced objects in to-be-deleted packs are exploded into loose objects, with the intent that they will be examined and possibly cleaned up by the next run of "git prune". Since the exploded objects will receive the mtime of the pack from which they come, if the source pack is old, those loose objects will end up pruned immediately. In that case, it is much more efficient to skip the exploding step entirely for these objects. This patch teaches pack-objects to receive the expiration information and avoid writing these objects out. It also teaches "git gc" to pass the value of gc.pruneexpire to repack (which in turn learns to pass it along to pack-objects) so that this optimization happens automatically during "git gc" and "git gc --auto". Signed-off-by: Jeff King <peff@peff.net> Acked-by: Nicolas Pitre <nico@fluxnic.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-11-07prune: handle --progress/no-progressLibravatar Jeff King1-1/+3
And have "git gc" pass no-progress when quiet. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-06-19builtin/gc.c: add missing newline in messageLibravatar Andreas Schwab1-1/+1
Signed-off-by: Andreas Schwab <schwab@linux-m68k.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-03-09i18n: git-gc "Auto packing the repository" messageLibravatar Ævar Arnfjörð Bjarmason1-7/+7
Split up the "Auto packing the repository" message into quiet and verbose variants to make translation easier. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-03-09i18n: git-gc basic messagesLibravatar Ævar Arnfjörð Bjarmason1-5/+5
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-12-12Merge branch 'jn/git-cmd-h-bypass-setup'Libravatar Junio C Hamano1-0/+3
* jn/git-cmd-h-bypass-setup: update-index -h: show usage even with corrupt index merge -h: show usage even with corrupt index ls-files -h: show usage even with corrupt index gc -h: show usage even with broken configuration commit/status -h: show usage even with broken configuration checkout-index -h: show usage even in an invalid repository branch -h: show usage even in an invalid repository Conflicts: builtin/merge.c
2010-11-15Describe various forms of "be quiet" using OPT__QUIETLibravatar Jonathan Nieder1-1/+1
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-11-15add description parameter to OPT__QUIETLibravatar René Scharfe1-1/+1
Allows better help text to be defined than "be quiet". Also make use of the macro in a place that already had a different description. No object code changes intended. Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-10-22gc -h: show usage even with broken configurationLibravatar Nguyễn Thái Ngọc Duy1-0/+3
Given a request for command-line usage information rather than some more substantial action, the only friendly thing to do is to report the usage information as soon as possible and exit. Without this change, as "git gc" glances over the repository, it can be distracted by the desire to report a malformed configuration file. Noticed while working through reports from Duy's repository access checker. [jn: with rewritten log message and tests] Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-02-22Move 'builtin-*' into a 'builtin/' subdirectoryLibravatar Linus Torvalds1-0/+255
This shrinks the top-level directory a bit, and makes it much more pleasant to use auto-completion on the thing. Instead of [torvalds@nehalem git]$ em buil<tab> Display all 180 possibilities? (y or n) [torvalds@nehalem git]$ em builtin-sh builtin-shortlog.c builtin-show-branch.c builtin-show-ref.c builtin-shortlog.o builtin-show-branch.o builtin-show-ref.o [torvalds@nehalem git]$ em builtin-shor<tab> builtin-shortlog.c builtin-shortlog.o [torvalds@nehalem git]$ em builtin-shortlog.c you get [torvalds@nehalem git]$ em buil<tab> [type] builtin/ builtin.h [torvalds@nehalem git]$ em builtin [auto-completes to] [torvalds@nehalem git]$ em builtin/sh<tab> [type] shortlog.c shortlog.o show-branch.c show-branch.o show-ref.c show-ref.o [torvalds@nehalem git]$ em builtin/sho [auto-completes to] [torvalds@nehalem git]$ em builtin/shor<tab> [type] shortlog.c shortlog.o [torvalds@nehalem git]$ em builtin/shortlog.c which doesn't seem all that different, but not having that annoying break in "Display all 180 possibilities?" is quite a relief. NOTE! If you do this in a clean tree (no object files etc), or using an editor that has auto-completion rules that ignores '*.o' files, you won't see that annoying 'Display all 180 possibilities?' message - it will just show the choices instead. I think bash has some cut-off around 100 choices or something. So the reason I see this is that I'm using an odd editory, and thus don't have the rules to cut down on auto-completion. But you can simulate that by using 'ls' instead, or something similar. Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>