diff options
| -rw-r--r-- | Documentation/config.txt | 9 | ||||
| -rw-r--r-- | Documentation/git-update-index.txt | 67 | ||||
| -rw-r--r-- | builtin/update-index.c | 62 | ||||
| -rw-r--r-- | cache.h | 8 | ||||
| -rw-r--r-- | config.c | 24 | ||||
| -rw-r--r-- | contrib/completion/git-completion.bash | 1 | ||||
| -rw-r--r-- | dir.c | 62 | ||||
| -rw-r--r-- | dir.h | 3 | ||||
| -rw-r--r-- | environment.c | 7 | ||||
| -rw-r--r-- | read-cache.c | 27 | ||||
| -rwxr-xr-x | t/t7063-status-untracked-cache.sh | 89 | ||||
| -rw-r--r-- | test-dump-untracked-cache.c | 4 |
12 files changed, 307 insertions, 56 deletions
diff --git a/Documentation/config.txt b/Documentation/config.txt index 02bcde6bb5..27f02be35e 100644 --- a/Documentation/config.txt +++ b/Documentation/config.txt @@ -308,6 +308,15 @@ core.trustctime:: crawlers and some backup systems). See linkgit:git-update-index[1]. True by default. +core.untrackedCache:: + Determines what to do about the untracked cache feature of the + index. It will be kept, if this variable is unset or set to + `keep`. It will automatically be added if set to `true`. And + it will automatically be removed, if set to `false`. Before + setting it to `true`, you should check that mtime is working + properly on your system. + See linkgit:git-update-index[1]. `keep` by default. + core.checkStat:: Determines which stat fields to match between the index and work tree. The user can set this to 'default' or diff --git a/Documentation/git-update-index.txt b/Documentation/git-update-index.txt index f4e5a85351..c6cbed189c 100644 --- a/Documentation/git-update-index.txt +++ b/Documentation/git-update-index.txt @@ -18,7 +18,7 @@ SYNOPSIS [--[no-]skip-worktree] [--ignore-submodules] [--[no-]split-index] - [--[no-|force-]untracked-cache] + [--[no-|test-|force-]untracked-cache] [--really-refresh] [--unresolve] [--again | -g] [--info-only] [--index-info] [-z] [--stdin] [--index-version <n>] @@ -174,17 +174,30 @@ may not support it yet. --untracked-cache:: --no-untracked-cache:: - Enable or disable untracked cache extension. This could speed - up for commands that involve determining untracked files such - as `git status`. The underlying operating system and file - system must change `st_mtime` field of a directory if files - are added or deleted in that directory. + Enable or disable untracked cache feature. Please use + `--test-untracked-cache` before enabling it. ++ +These options take effect whatever the value of the `core.untrackedCache` +configuration variable (see linkgit:git-config[1]). But a warning is +emitted when the change goes against the configured value, as the +configured value will take effect next time the index is read and this +will remove the intended effect of the option. + +--test-untracked-cache:: + Only perform tests on the working directory to make sure + untracked cache can be used. You have to manually enable + untracked cache using `--untracked-cache` or + `--force-untracked-cache` or the `core.untrackedCache` + configuration variable afterwards if you really want to use + it. If a test fails the exit code is 1 and a message + explains what is not working as needed, otherwise the exit + code is 0 and OK is printed. --force-untracked-cache:: - For safety, `--untracked-cache` performs tests on the working - directory to make sure untracked cache can be used. These - tests can take a few seconds. `--force-untracked-cache` can be - used to skip the tests. + Same as `--untracked-cache`. Provided for backwards + compatibility with older versions of Git where + `--untracked-cache` used to imply `--test-untracked-cache` but + this option would enable the extension unconditionally. \--:: Do not interpret any more arguments as options. @@ -375,6 +388,37 @@ Although this bit looks similar to assume-unchanged bit, its goal is different from assume-unchanged bit's. Skip-worktree also takes precedence over assume-unchanged bit when both are set. +Untracked cache +--------------- + +This cache is meant to speed up commands that involve determining +untracked files such as `git status`. + +This feature works by recording the mtime of the working tree +directories and then omitting reading directories and stat calls +against files in those directories whose mtime hasn't changed. For +this to work the underlying operating system and file system must +change the `st_mtime` field of directories if files in the directory +are added, modified or deleted. + +You can test whether the filesystem supports that with the +`--test-untracked-cache` option. The `--untracked-cache` option used +to implicitly perform that test in older versions of Git, but that's +no longer the case. + +If you want to enable (or disable) this feature, it is easier to use +the `core.untrackedCache` configuration variable (see +linkgit:git-config[1]) than using the `--untracked-cache` option to +`git update-index` in each repository, especially if you want to do so +across all repositories you use, because you can set the configuration +variable to `true` (or `false`) in your `$HOME/.gitconfig` just once +and have it affect all repositories you touch. + +When the `core.untrackedCache` configuration variable is changed, the +untracked cache is added to or removed from the index the next time a +command reads the index; while when `--[no-|force-]untracked-cache` +are used, the untracked cache is immediately added to or removed from +the index. Configuration ------------- @@ -400,6 +444,9 @@ It can be useful when the inode change time is regularly modified by something outside Git (file system crawlers and backup systems use ctime for marking files processed) (see linkgit:git-config[1]). +The untracked cache extension can be enabled by the +`core.untrackedCache` configuration variable (see +linkgit:git-config[1]). SEE ALSO -------- diff --git a/builtin/update-index.c b/builtin/update-index.c index 7c5c143de5..dbc23a46b1 100644 --- a/builtin/update-index.c +++ b/builtin/update-index.c @@ -35,6 +35,15 @@ static int mark_skip_worktree_only; #define UNMARK_FLAG 2 static struct strbuf mtime_dir = STRBUF_INIT; +/* Untracked cache mode */ +enum uc_mode { + UC_UNSPECIFIED = -1, + UC_DISABLE = 0, + UC_ENABLE, + UC_TEST, + UC_FORCE +}; + __attribute__((format (printf, 1, 2))) static void report(const char *fmt, ...) { @@ -121,7 +130,7 @@ static int test_if_untracked_cache_is_supported(void) if (!mkdtemp(mtime_dir.buf)) die_errno("Could not make temporary directory"); - fprintf(stderr, _("Testing ")); + fprintf(stderr, _("Testing mtime in '%s' "), xgetcwd()); atexit(remove_test_directory); xstat_mtime_dir(&st); fill_stat_data(&base, &st); @@ -904,7 +913,7 @@ static int reupdate_callback(struct parse_opt_ctx_t *ctx, int cmd_update_index(int argc, const char **argv, const char *prefix) { int newfd, entries, has_errors = 0, nul_term_line = 0; - int untracked_cache = -1; + enum uc_mode untracked_cache = UC_UNSPECIFIED; int read_from_stdin = 0; int prefix_length = prefix ? strlen(prefix) : 0; int preferred_index_format = 0; @@ -999,8 +1008,10 @@ int cmd_update_index(int argc, const char **argv, const char *prefix) N_("enable or disable split index")), OPT_BOOL(0, "untracked-cache", &untracked_cache, N_("enable/disable untracked cache")), + OPT_SET_INT(0, "test-untracked-cache", &untracked_cache, + N_("test if the filesystem supports untracked cache"), UC_TEST), OPT_SET_INT(0, "force-untracked-cache", &untracked_cache, - N_("enable untracked cache without testing the filesystem"), 2), + N_("enable untracked cache without testing the filesystem"), UC_FORCE), OPT_END() }; @@ -1109,27 +1120,32 @@ int cmd_update_index(int argc, const char **argv, const char *prefix) the_index.split_index = NULL; the_index.cache_changed |= SOMETHING_CHANGED; } - if (untracked_cache > 0) { - struct untracked_cache *uc; - if (untracked_cache < 2) { - setup_work_tree(); - if (!test_if_untracked_cache_is_supported()) - return 1; - } - if (!the_index.untracked) { - uc = xcalloc(1, sizeof(*uc)); - strbuf_init(&uc->ident, 100); - uc->exclude_per_dir = ".gitignore"; - /* should be the same flags used by git-status */ - uc->dir_flags = DIR_SHOW_OTHER_DIRECTORIES | DIR_HIDE_EMPTY_DIRECTORIES; - the_index.untracked = uc; - } - add_untracked_ident(the_index.untracked); - the_index.cache_changed |= UNTRACKED_CHANGED; - } else if (!untracked_cache && the_index.untracked) { - the_index.untracked = NULL; - the_index.cache_changed |= UNTRACKED_CHANGED; + switch (untracked_cache) { + case UC_UNSPECIFIED: + break; + case UC_DISABLE: + if (git_config_get_untracked_cache() == 1) + warning("core.untrackedCache is set to true; " + "remove or change it, if you really want to " + "disable the untracked cache"); + remove_untracked_cache(&the_index); + report(_("Untracked cache disabled")); + break; + case UC_TEST: + setup_work_tree(); + return !test_if_untracked_cache_is_supported(); + case UC_ENABLE: + case UC_FORCE: + if (git_config_get_untracked_cache() == 0) + warning("core.untrackedCache is set to false; " + "remove or change it, if you really want to " + "enable the untracked cache"); + add_untracked_cache(&the_index); + report(_("Untracked cache enabled for '%s'"), get_git_work_tree()); + break; + default: + die("Bug: bad untracked_cache value: %d", untracked_cache); } if (active_cache_changed) { @@ -1624,6 +1624,14 @@ extern int git_config_get_bool(const char *key, int *dest); extern int git_config_get_bool_or_int(const char *key, int *is_bool, int *dest); extern int git_config_get_maybe_bool(const char *key, int *dest); extern int git_config_get_pathname(const char *key, const char **dest); +extern int git_config_get_untracked_cache(void); + +/* + * This is a hack for test programs like test-dump-untracked-cache to + * ensure that they do not modify the untracked cache when reading it. + * Do not use it otherwise! + */ +extern int ignore_untracked_cache_config; struct key_value_info { const char *filename; @@ -1594,6 +1594,30 @@ int git_config_get_pathname(const char *key, const char **dest) return ret; } +int git_config_get_untracked_cache(void) +{ + int val = -1; + const char *v; + + /* Hack for test programs like test-dump-untracked-cache */ + if (ignore_untracked_cache_config) + return -1; + + if (!git_config_get_maybe_bool("core.untrackedcache", &val)) + return val; + + if (!git_config_get_value("core.untrackedcache", &v)) { + if (!strcasecmp(v, "keep")) + return -1; + + error("unknown core.untrackedCache value '%s'; " + "using 'keep' default value", v); + return -1; + } + + return -1; /* default value */ +} + NORETURN void git_die_config_linenr(const char *key, const char *filename, int linenr) { diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash index 15ebba51dc..45ec47f2b1 100644 --- a/contrib/completion/git-completion.bash +++ b/contrib/completion/git-completion.bash @@ -2060,6 +2060,7 @@ _git_config () core.sparseCheckout core.symlinks core.trustctime + core.untrackedCache core.warnAmbiguousRefs core.whitespace core.worktree @@ -1839,31 +1839,67 @@ static const char *get_ident_string(void) return sb.buf; if (uname(&uts) < 0) die_errno(_("failed to get kernel name and information")); - strbuf_addf(&sb, "Location %s, system %s %s %s", get_git_work_tree(), - uts.sysname, uts.release, uts.version); + strbuf_addf(&sb, "Location %s, system %s", get_git_work_tree(), + uts.sysname); return sb.buf; } static int ident_in_untracked(const struct untracked_cache *uc) { - const char *end = uc->ident.buf + uc->ident.len; - const char *p = uc->ident.buf; + /* + * Previous git versions may have saved many NUL separated + * strings in the "ident" field, but it is insane to manage + * many locations, so just take care of the first one. + */ - for (p = uc->ident.buf; p < end; p += strlen(p) + 1) - if (!strcmp(p, get_ident_string())) - return 1; - return 0; + return !strcmp(uc->ident.buf, get_ident_string()); } -void add_untracked_ident(struct untracked_cache *uc) +static void set_untracked_ident(struct untracked_cache *uc) { - if (ident_in_untracked(uc)) - return; + strbuf_reset(&uc->ident); strbuf_addstr(&uc->ident, get_ident_string()); - /* this strbuf contains a list of strings, save NUL too */ + + /* + * This strbuf used to contain a list of NUL separated + * strings, so save NUL too for backward compatibility. + */ strbuf_addch(&uc->ident, 0); } +static void new_untracked_cache(struct index_state *istate) +{ + struct untracked_cache *uc = xcalloc(1, sizeof(*uc)); + strbuf_init(&uc->ident, 100); + uc->exclude_per_dir = ".gitignore"; + /* should be the same flags used by git-status */ + uc->dir_flags = DIR_SHOW_OTHER_DIRECTORIES | DIR_HIDE_EMPTY_DIRECTORIES; + set_untracked_ident(uc); + istate->untracked = uc; + istate->cache_changed |= UNTRACKED_CHANGED; +} + +void add_untracked_cache(struct index_state *istate) +{ + if (!istate->untracked) { + new_untracked_cache(istate); + } else { + if (!ident_in_untracked(istate->untracked)) { + free_untracked_cache(istate->untracked); + new_untracked_cache(istate); + } + } +} + +void remove_untracked_cache(struct index_state *istate) +{ + if (istate->untracked) { + free_untracked_cache(istate->untracked); + istate->untracked = NULL; + istate->cache_changed |= UNTRACKED_CHANGED; + } +} + static struct untracked_cache_dir *validate_untracked_cache(struct dir_struct *dir, int base_len, const struct pathspec *pathspec) @@ -1921,7 +1957,7 @@ static struct untracked_cache_dir *validate_untracked_cache(struct dir_struct *d return NULL; if (!ident_in_untracked(dir->untracked)) { - warning(_("Untracked cache is disabled on this system.")); + warning(_("Untracked cache is disabled on this system or location.")); return NULL; } @@ -307,5 +307,6 @@ void untracked_cache_add_to_index(struct index_state *, const char *); void free_untracked_cache(struct untracked_cache *); struct untracked_cache *read_untracked_extension(const void *data, unsigned long sz); void write_untracked_extension(struct strbuf *out, struct untracked_cache *untracked); -void add_untracked_ident(struct untracked_cache *); +void add_untracked_cache(struct index_state *istate); +void remove_untracked_cache(struct index_state *istate); #endif diff --git a/environment.c b/environment.c index 1cc4aab4ea..6dec9d0403 100644 --- a/environment.c +++ b/environment.c @@ -87,6 +87,13 @@ int auto_comment_line_char; /* Parallel index stat data preload? */ int core_preload_index = 1; +/* + * This is a hack for test programs like test-dump-untracked-cache to + * ensure that they do not modify the untracked cache when reading it. + * Do not use it otherwise! + */ +int ignore_untracked_cache_config; + /* This is set by setup_git_dir_gently() and/or git_default_config() */ char *git_work_tree_cfg; static char *work_tree; diff --git a/read-cache.c b/read-cache.c index 5be7cd1dbf..d9fb78bc55 100644 --- a/read-cache.c +++ b/read-cache.c @@ -1519,6 +1519,28 @@ static void check_ce_order(struct index_state *istate) } } +static void tweak_untracked_cache(struct index_state *istate) +{ + switch (git_config_get_untracked_cache()) { + case -1: /* keep: do nothing */ + break; + case 0: /* false */ + remove_untracked_cache(istate); + break; + case 1: /* true */ + add_untracked_cache(istate); + break; + default: /* unknown value: do nothing */ + break; + } +} + +static void post_read_index_from(struct index_state *istate) +{ + check_ce_order(istate); + tweak_untracked_cache(istate); +} + /* remember to discard_cache() before reading a different cache! */ int do_read_index(struct index_state *istate, const char *path, int must_exist) { @@ -1622,9 +1644,10 @@ int read_index_from(struct index_state *istate, const char *path) return istate->cache_nr; ret = do_read_index(istate, path, 0); + split_index = istate->split_index; if (!split_index || is_null_sha1(split_index->base_sha1)) { - check_ce_order(istate); + post_read_index_from(istate); return ret; } @@ -1642,7 +1665,7 @@ int read_index_from(struct index_state *istate, const char *path) sha1_to_hex(split_index->base_sha1)), sha1_to_hex(split_index->base->sha1)); merge_base_index(istate); - check_ce_order(istate); + post_read_index_from(istate); return ret; } diff --git a/t/t7063-status-untracked-cache.sh b/t/t7063-status-untracked-cache.sh index 0e8d0d42f2..a971884cfd 100755 --- a/t/t7063-status-untracked-cache.sh +++ b/t/t7063-status-untracked-cache.sh @@ -8,10 +8,8 @@ avoid_racy() { sleep 1 } -# It's fine if git update-index returns an error code other than one, -# it'll be caught in the first test. test_lazy_prereq UNTRACKED_CACHE ' - { git update-index --untracked-cache; ret=$?; } && + { git update-index --test-untracked-cache; ret=$?; } && test $ret -ne 1 ' @@ -20,6 +18,10 @@ if ! test_have_prereq UNTRACKED_CACHE; then test_done fi +test_expect_success 'core.untrackedCache is unset' ' + test_must_fail git config --get core.untrackedCache +' + test_expect_success 'setup' ' git init worktree && cd worktree && @@ -32,13 +34,13 @@ test_expect_success 'setup' ' test_expect_success 'untracked cache is empty' ' test-dump-untracked-cache >../actual && - cat >../expect <<EOF && + cat >../expect-empty <<EOF && info/exclude 0000000000000000000000000000000000000000 core.excludesfile 0000000000000000000000000000000000000000 exclude_per_dir .gitignore flags 00000006 EOF - test_cmp ../expect ../actual + test_cmp ../expect-empty ../actual ' cat >../status.expect <<EOF && @@ -508,7 +510,7 @@ EOF test_expect_success 'verify untracked cache dump (sparse/subdirs)' ' test-dump-untracked-cache >../actual && - cat >../expect <<EOF && + cat >../expect-from-test-dump <<EOF && info/exclude 13263c0978fb9fad16b2d580fb800b6d811c3ff0 core.excludesfile 0000000000000000000000000000000000000000 exclude_per_dir .gitignore @@ -527,7 +529,7 @@ file /dtwo/ 0000000000000000000000000000000000000000 recurse check_only valid two EOF - test_cmp ../expect ../actual + test_cmp ../expect-from-test-dump ../actual ' test_expect_success 'test sparse status again with untracked cache and subdir' ' @@ -571,4 +573,77 @@ EOF test_cmp ../status.expect ../status.actual ' +test_expect_success '--no-untracked-cache removes the cache' ' + git update-index --no-untracked-cache && + test-dump-untracked-cache >../actual && + echo "no untracked cache" >../expect-no-uc && + test_cmp ../expect-no-uc ../actual +' + +test_expect_success 'git status does not change anything' ' + git status && + test-dump-untracked-cache >../actual && + test_cmp ../expect-no-uc ../actual +' + +test_expect_success 'setting core.untrackedCache to true and using git status creates the cache' ' + git config core.untrackedCache true && + test-dump-untracked-cache >../actual && + test_cmp ../expect-no-uc ../actual && + git status && + test-dump-untracked-cache >../actual && + test_cmp ../expect-from-test-dump ../actual +' + +test_expect_success 'using --no-untracked-cache does not fail when core.untrackedCache is true' ' + git update-index --no-untracked-cache && + test-dump-untracked-cache >../actual && + test_cmp ../expect-no-uc ../actual && + git update-index --untracked-cache && + test-dump-untracked-cache >../actual && + test_cmp ../expect-empty ../actual +' + +test_expect_success 'setting core.untrackedCache to false and using git status removes the cache' ' + git config core.untrackedCache false && + test-dump-untracked-cache >../actual && + test_cmp ../expect-empty ../actual && + git status && + test-dump-untracked-cache >../actual && + test_cmp ../expect-no-uc ../actual +' + +test_expect_success 'using --untracked-cache does not fail when core.untrackedCache is false' ' + git update-index --untracked-cache && + test-dump-untracked-cache >../actual && + test_cmp ../expect-empty ../actual +' + +test_expect_success 'setting core.untrackedCache to keep' ' + git config core.untrackedCache keep && + git update-index --untracked-cache && + test-dump-untracked-cache >../actual && + test_cmp ../expect-empty ../actual && + git status && + test-dump-untracked-cache >../actual && + test_cmp ../expect-from-test-dump ../actual && + git update-index --no-untracked-cache && + test-dump-untracked-cache >../actual && + test_cmp ../expect-no-uc ../actual && + git update-index --force-untracked-cache && + test-dump-untracked-cache >../actual && + test_cmp ../expect-empty ../actual && + git status && + test-dump-untracked-cache >../actual && + test_cmp ../expect-from-test-dump ../actual +' + +test_expect_success 'test ident field is working' ' + mkdir ../other_worktree && + cp -R done dthree dtwo four three ../other_worktree && + GIT_WORK_TREE=../other_worktree git status 2>../err && + echo "warning: Untracked cache is disabled on this system or location." >../expect && + test_cmp ../expect ../err +' + test_done diff --git a/test-dump-untracked-cache.c b/test-dump-untracked-cache.c index 25d855d98b..0a1c285246 100644 --- a/test-dump-untracked-cache.c +++ b/test-dump-untracked-cache.c @@ -44,6 +44,10 @@ int main(int ac, char **av) { struct untracked_cache *uc; struct strbuf base = STRBUF_INIT; + + /* Hack to avoid modifying the untracked cache when we read it */ + ignore_untracked_cache_config = 1; + setup_git_directory(); if (read_cache() < 0) die("unable to read index file"); |