summaryrefslogtreecommitdiff
path: root/dir.h
AgeCommit message (Collapse)AuthorFilesLines
2018-05-30Merge branch 'bc/object-id'Libravatar Junio C Hamano1-2/+3
Conversion from uchar[20] to struct object_id continues. * bc/object-id: (42 commits) merge-one-file: compute empty blob object ID add--interactive: compute the empty tree value Update shell scripts to compute empty tree object ID sha1_file: only expose empty object constants through git_hash_algo dir: use the_hash_algo for empty blob object ID sequencer: use the_hash_algo for empty tree object ID cache-tree: use is_empty_tree_oid sha1_file: convert cached object code to struct object_id builtin/reset: convert use of EMPTY_TREE_SHA1_BIN builtin/receive-pack: convert one use of EMPTY_TREE_SHA1_HEX wt-status: convert two uses of EMPTY_TREE_SHA1_HEX submodule: convert several uses of EMPTY_TREE_SHA1_HEX sequencer: convert one use of EMPTY_TREE_SHA1_HEX merge: convert empty tree constant to the_hash_algo builtin/merge: switch tree functions to use object_id builtin/am: convert uses of EMPTY_TREE_SHA1_BIN to the_hash_algo sha1-file: add functions for hex empty tree and blob OIDs builtin/receive-pack: avoid hard-coded constants for push certs diff: specify abbreviation size in terms of the_hash_algo upload-pack: replace use of several hard-coded constants ...
2018-05-08Merge branch 'sb/submodule-move-nested'Libravatar Junio C Hamano1-1/+11
Moving a submodule that itself has submodule in it with "git mv" forgot to make necessary adjustment to the nested sub-submodules; now the codepath learned to recurse into the submodules. * sb/submodule-move-nested: submodule: fixup nested submodules after moving the submodule submodule-config: remove submodule_from_cache submodule-config: add repository argument to submodule_from_{name, path} submodule-config: allow submodule_free to handle arbitrary repositories grep: remove "repo" arg from non-supporting funcs submodule.h: drop declaration of connect_work_tree_and_git_dir
2018-05-02dir: convert struct untracked_cache_dir to object_idLibravatar brian m. carlson1-2/+3
Convert the exclude_sha1 member of struct untracked_cache_dir and rename it to exclude_oid. Eliminate several hard-coded integral constants, and update a function name that referred to SHA-1. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-03-29submodule: fixup nested submodules after moving the submoduleLibravatar Stefan Beller1-1/+11
connect_work_tree_and_git_dir is used to connect a submodule worktree with its git directory and vice versa after events that require a reconnection such as moving around the working tree. As submodules can have nested submodules themselves, we'd also want to fix the nested submodules when asked to. Add an option to recurse into the nested submodules and connect them as well. As submodules are identified by their name (which determines their git directory in relation to their superproject's git directory) internally and by their path in the working tree of the superproject, we need to make sure that the mapping of name <-> path is kept intact. We can do that in the git-mv command by writing out the gitmodules file first and then forcing a reload of the submodule config machinery. Signed-off-by: Stefan Beller <sbeller@google.com> Reviewed-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-02-27Merge branch 'nd/fix-untracked-cache-invalidation'Libravatar Junio C Hamano1-1/+1
Some bugs around "untracked cache" feature have been fixed. * nd/fix-untracked-cache-invalidation: dir.c: ignore paths containing .git when invalidating untracked cache dir.c: stop ignoring opendir() error in open_cached_dir() dir.c: fix missing dir invalidation in untracked code dir.c: avoid stat() in valid_cached_dir() status: add a failing test showing a core.untrackedCache bug
2018-02-07dir.c: ignore paths containing .git when invalidating untracked cacheLibravatar Nguyễn Thái Ngọc Duy1-1/+1
read_directory() code ignores all paths named ".git" even if it's not a valid git repository. See treat_path() for details. Since ".git" is basically invisible to read_directory(), when we are asked to invalidate a path that contains ".git", we can safely ignore it because the slow path would not consider it anyway. This helps when fsmonitor is used and we have a real ".git" repo at worktree top. Occasionally .git/index will be updated and if the fsmonitor hook does not filter it, untracked cache is asked to invalidate the path ".git/index". Without this patch, we invalidate the root directory unncessarily, which: - makes read_directory() fall back to slow path for root directory (slower) - makes the index dirty (because UNTR extension is updated). Depending on the index size, writing it down could also be slow. A note about the new "safe_path" knob. Since this new check could be relatively expensive, avoid it when we know it's not needed. If the path comes from the index, it can't contain ".git". If it does contain, we may be screwed up at many more levels, not just this one. Noticed-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-01-30dir: convert struct sha1_stat to use object_idLibravatar Patryk Obara1-6/+6
Convert the declaration of struct sha1_stat. Adjust all usages of this struct and replace hash{clr,cmp,cpy} with oid{clr,cmp,cpy} wherever possible. Rename it to struct oid_stat. Rename static function load_sha1_stat to load_oid_stat. Remove macro EMPTY_BLOB_SHA1_BIN, as it's no longer used. Signed-off-by: Patryk Obara <patryk.obara@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-12-27Merge branch 'jh/object-filtering'Libravatar Junio C Hamano1-0/+3
In preparation for implementing narrow/partial clone, the object walking machinery has been taught a way to tell it to "filter" some objects from enumeration. * jh/object-filtering: rev-list: support --no-filter argument list-objects-filter-options: support --no-filter list-objects-filter-options: fix 'keword' typo in comment pack-objects: add list-objects filtering rev-list: add list-objects filtering support list-objects: filter objects in traverse_commit_list oidset: add iterator methods to oidset oidmap: add oidmap iterator methods dir: allow exclusions from blob in addition to file
2017-11-22dir: allow exclusions from blob in addition to fileLibravatar Jeff Hostetler1-0/+3
Refactor add_excludes() to separate the reading of the exclude file into a buffer and the parsing of the buffer into exclude_list items. Add add_excludes_from_blob_to_list() to allow an exclude file be specified with an OID without assuming a local worktree or index exists. Refactor read_skip_worktree_file_from_index() and add do_read_blob() to eliminate duplication of preliminary processing of blob contents. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Reviewed-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-11-21Merge branch 'bp/fsmonitor'Libravatar Junio C Hamano1-0/+2
We learned to talk to watchman to speed up "git status" and other operations that need to see which paths have been modified. * bp/fsmonitor: fsmonitor: preserve utf8 filenames in fsmonitor-watchman log fsmonitor: read entirety of watchman output fsmonitor: MINGW support for watchman integration fsmonitor: add a performance test fsmonitor: add a sample integration script for Watchman fsmonitor: add test cases for fsmonitor extension split-index: disable the fsmonitor extension when running the split index test fsmonitor: add a test tool to dump the index extension update-index: add fsmonitor support to update-index ls-files: Add support in ls-files to display the fsmonitor valid bit fsmonitor: add documentation for the fsmonitor extension. fsmonitor: teach git to optionally utilize a file system monitor to speed up detecting new or changed files. update-index: add a new --force-write-index option preload-index: add override to enable testing preload-index bswap: add 64 bit endianness helper get_be64
2017-10-31status: add option to show ignored files differentlyLibravatar Jameson Miller1-1/+2
Teach the status command more flexibility in how ignored files are reported. Currently, the reporting of ignored files and untracked files are linked. You cannot control how ignored files are reported independently of how untracked files are reported (i.e. `all` vs `normal`). This makes it impossible to show untracked files with the `all` option, but show ignored files with the `normal` option. This work 1) adds the ability to control the reporting of ignored files independently of untracked files and 2) introduces the concept of status reporting ignored paths that explicitly match an ignored pattern. There are 2 benefits to these changes: 1) if a consumer needs all untracked files but not all ignored files, there is a performance benefit to not scanning all contents of an ignored directory and 2) returning ignored files that explicitly match a path allow a consumer to make more informed decisions about when a status result might be stale. This commit implements --ignored=matching with --untracked-files=all. The following commit will implement --ignored=matching with --untracked=files=normal. As an example of where this flexibility could be useful is that our application (Visual Studio) runs the status command and presents the output. It shows all untracked files individually (e.g. using the '--untracked-files==all' option), and would like to know about which paths are ignored. It uses information about ignored paths to make decisions about when the status result might have changed. Additionally, many projects place build output into directories inside a repository's working directory (e.g. in "bin/" and "obj/" directories). Normal usage is to explicitly ignore these 2 directory names in the .gitignore file (rather than or in addition to the *.obj pattern).If an application could know that these directories are explicitly ignored, it could infer that all contents are ignored as well and make better informed decisions about files in these directories. It could infer that any changes under these paths would not affect the output of status. Additionally, there can be a significant performance benefit by avoiding scanning through ignored directories. When status is set to report matching ignored files, it has the following behavior. Ignored files and directories that explicitly match an exclude pattern are reported. If an ignored directory matches an exclude pattern, then the path of the directory is returned. If a directory does not match an exclude pattern, but all of its contents are ignored, then the contained files are reported instead of the directory. Signed-off-by: Jameson Miller <jamill@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-10-01fsmonitor: teach git to optionally utilize a file system monitor to speed up ↵Libravatar Ben Peart1-0/+2
detecting new or changed files. When the index is read from disk, the fsmonitor index extension is used to flag the last known potentially dirty index entries. The registered core.fsmonitor command is called with the time the index was last updated and returns the list of files changed since that time. This list is used to flag any additional dirty cache entries and untracked cache directories. We can then use this valid state to speed up preload_index(), ie_match_stat(), and refresh_cache_ent() as they do not need to lstat() files to detect potential changes for those entries marked CE_FSMONITOR_VALID. In addition, if the untracked cache is turned on valid_cached_dir() can skip checking directories for new or changed files as fsmonitor will invalidate the cache only for those directories that have been identified as having potential changes. To keep the CE_FSMONITOR_VALID state accurate during git operations; when git updates a cache entry to match the current state on disk, it will now set the CE_FSMONITOR_VALID bit. Inversely, anytime git changes a cache entry, the CE_FSMONITOR_VALID bit is cleared and the corresponding untracked cache directory is marked invalid. Signed-off-by: Ben Peart <benpeart@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-22Merge branch 'pc/dir-count-slashes'Libravatar Junio C Hamano1-0/+3
Three instances of the same helper function have been consolidated to one. * pc/dir-count-slashes: dir: create function count_slashes()
2017-06-12dir: create function count_slashes()Libravatar Prathamesh Chavan1-0/+3
Similar functions exist in apply.c and builtin/show-branch.c for counting the number of slashes in a string. Also in the later patches, we introduce a third caller for the same. Hence, we unify it now by cleaning the existing functions and declaring a common function count_slashes in dir.h and implementing it in dir.c to remove this code duplication. Mentored-by: Christian Couder <christian.couder@gmail.com> Mentored-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Prathamesh Chavan <pc44800@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-02Merge branch 'sl/clean-d-ignored-fix'Libravatar Junio C Hamano1-1/+5
"git clean -d" used to clean directories that has ignored files, even though the command should not lose ignored ones without "-x". "git status --ignored" did not list ignored and untracked files without "-uall". These have been corrected. * sl/clean-d-ignored-fix: clean: teach clean -d to preserve ignored paths dir: expose cmp_name() and check_contains() dir: hide untracked contents of untracked dirs dir: recurse into untracked dirs for ignored files t7061: status --ignored should search untracked dirs t7300: clean -d should skip dirs with ignored files
2017-05-22dir: expose cmp_name() and check_contains()Libravatar Samuel Lijin1-0/+3
We want to use cmp_name() and check_contains() (which both compare `struct dir_entry`s, the former in terms of the sort order, the latter in terms of whether one lexically contains another) outside of dir.c, so we have to (1) change their linkage and (2) rename them as appropriate for the global namespace. The second is achieved by renaming cmp_name() to cmp_dir_entry() and check_contains() to check_dir_entry_contains(). Signed-off-by: Samuel Lijin <sxlijin@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-22dir: hide untracked contents of untracked dirsLibravatar Samuel Lijin1-1/+2
When we taught read_directory_recursive() to recurse into untracked directories in search of ignored files given DIR_SHOW_IGNORED_TOO, that had the side effect of teaching it to collect the untracked contents of untracked directories. It doesn't always make sense to return these, though (we do need them for `clean -d`), so we introduce a flag (DIR_KEEP_UNTRACKED_CONTENTS) to control whether or not read_directory() strips dir->entries of the untracked contents of untracked dirs. We also introduce check_contains() to check if one dir_entry corresponds to a path which contains the path corresponding to another dir_entry. This also fixes known breakages in t7061, since status --ignored now searches untracked directories for ignored files. Signed-off-by: Samuel Lijin <sxlijin@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-06dir: convert fill_directory to take an indexLibravatar Brandon Williams1-1/+3
Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-06dir: convert read_directory to take an indexLibravatar Brandon Williams1-1/+3
Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-06dir: convert is_excluded to take an indexLibravatar Brandon Williams1-1/+4
Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-06dir: convert add_excludes to take an indexLibravatar Brandon Williams1-1/+1
Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-06dir: convert is_excluded_from_list to take an indexLibravatar Brandon Williams1-2/+4
Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-06dir: convert dir_add* to take an indexLibravatar Brandon Williams1-1/+3
Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-12submodule: add absorb-git-dir functionLibravatar Stefan Beller1-0/+3
When a submodule has its git dir inside the working dir, the submodule support for checkout that we plan to add in a later patch will fail. Add functionality to migrate the git directory to be absorbed into the superprojects git directory. The newly added code in this patch is structured such that other areas of Git can also make use of it. The code in the submodule--helper is a mere wrapper and option parser for the function `absorb_git_dir_into_superproject`, that takes care of embedding the submodules git directory into the superprojects git dir. That function makes use of the more abstract function for this use case `relocate_gitdir`, which can be used by e.g. the worktree code eventually to move around a git directory. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-12move connect_work_tree_and_git_dir to dir.hLibravatar Stefan Beller1-0/+1
That function was primarily used by submodule code, but the function itself is not inherently about submodules. In the next patch we'll introduce relocate_git_dir, which can be used by worktrees as well, so find a neutral middle ground in dir.h. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-10-10ls-files: add pathspec matching for submodulesLibravatar Brandon Williams1-0/+4
Pathspecs can be a bit tricky when trying to apply them to submodules. The main challenge is that the pathspecs will be with respect to the superproject and not with respect to paths in the submodule. The approach this patch takes is to pass in the identical pathspec from the superproject to the submodule in addition to the submodule-prefix, which is the path from the root of the superproject to the submodule, and then we can compare an entry in the submodule prepended with the submodule-prefix to the pathspec in order to determine if there is a match. This patch also permits the pathspec logic to perform a prefix match against submodules since a pathspec could refer to a file inside of a submodule. Due to limitations in the wildmatch logic, a prefix match is only done literally. If any wildcard character is encountered we'll simply punt and produce a false positive match. More accurate matching will be done once inside the submodule. This is due to the superproject not knowing what files could exist in the submodule. Signed-off-by: Brandon Williams <bmwill@google.com> Reviewed-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-25Merge branch 'mh/split-under-lock'Libravatar Junio C Hamano1-0/+23
Further preparatory work on the refs API before the pluggable backend series can land. * mh/split-under-lock: (33 commits) lock_ref_sha1_basic(): only handle REF_NODEREF mode commit_ref_update(): remove the flags parameter lock_ref_for_update(): don't resolve symrefs lock_ref_for_update(): don't re-read non-symbolic references refs: resolve symbolic refs first ref_transaction_update(): check refname_is_safe() at a minimum unlock_ref(): move definition higher in the file lock_ref_for_update(): new function add_update(): initialize the whole ref_update verify_refname_available(): adjust constness in declaration refs: don't dereference on rename refs: allow log-only updates delete_branches(): use resolve_refdup() ref_transaction_commit(): correctly report close_ref() failure ref_transaction_create(): disallow recursive pruning refs: make error messages more consistent lock_ref_sha1_basic(): remove unneeded local variable read_raw_ref(): move docstring to header file read_raw_ref(): improve docstring read_raw_ref(): rename symref argument to referent ...
2016-05-23Merge branch 'nd/worktree-various-heads'Libravatar Junio C Hamano1-2/+2
The experimental "multiple worktree" feature gains more safety to forbid operations on a branch that is checked out or being actively worked on elsewhere, by noticing that e.g. it is being rebased. * nd/worktree-various-heads: branch: do not rename a branch under bisect or rebase worktree.c: check whether branch is bisected in another worktree wt-status.c: split bisect detection out of wt_status_get_state() worktree.c: check whether branch is rebased in another worktree worktree.c: avoid referencing to worktrees[i] multiple times wt-status.c: make wt_status_check_rebase() work on any worktree wt-status.c: split rebase detection out of wt_status_get_state() path.c: refactor and add worktree_git_path() worktree.c: mark current worktree worktree.c: make find_shared_symref() return struct worktree * worktree.c: store "id" instead of "git_dir" path.c: add git_common_path() and strbuf_git_common_path() dir.c: rename str(n)cmp_icase to fspath(n)cmp
2016-05-05remove_dir_recursively(): add docstringLibravatar Michael Haggerty1-0/+23
Add a docstring for the remove_dir_recursively() function and the REMOVE_DIR_* flags that can be passed to it. Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
2016-04-22dir.c: rename str(n)cmp_icase to fspath(n)cmpLibravatar Nguyễn Thái Ngọc Duy1-2/+2
These functions compare two paths that are taken from file system. Depending on the running file system, paths may need to be compared case-sensitively or not, and maybe even something else in future. The current names do not convey that well. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-04-22dir.c: remove dead function fnmatch_icase()Libravatar Nguyễn Thái Ngọc Duy1-1/+0
It was largely replaced by fnmatch_icase_mem() and its last use was in 84b8b5d (remove match_pathspec() in favor of match_pathspec_depth() - 2013-07-14). Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-04-14Merge branch 'ss/exc-flag-is-a-collection-of-bits' into maintLibravatar Junio C Hamano1-4/+4
Code clean-up. * ss/exc-flag-is-a-collection-of-bits: dir: store EXC_FLAG_* values in unsigned integers
2016-04-06Merge branch 'ss/exc-flag-is-a-collection-of-bits'Libravatar Junio C Hamano1-4/+4
Code clean-up. * ss/exc-flag-is-a-collection-of-bits: dir: store EXC_FLAG_* values in unsigned integers
2016-03-18Revert "Merge branch 'nd/exclusion-regression-fix'"Libravatar Junio C Hamano1-3/+0
This reverts commit 5e57f9c3dfe7dd44a1b56bb5b3327d7a1356ec7c, reversing changes made to e79112d21024beb997951381db21a70b087d459d. We will be postponing nd/exclusion-regression-fix topic to later cycle.
2016-03-01dir: store EXC_FLAG_* values in unsigned integersLibravatar Saurav Sachidanand1-4/+4
The values defined by the macro EXC_FLAG_* (1, 4, 8, 16) are stored in fields of the structs "pattern" and "exclude", some functions arguments and a local variable. None of these uses its most significant bit in any special way and there is no good reason to use a signed integer for them. And while we're at it, document "flags" of "exclude" to explicitly state the values it's supposed to take on. Signed-off-by: Saurav Sachidanand <sauravsachidanand@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-02-15dir.c: support marking some patterns already matchedLibravatar Nguyễn Thái Ngọc Duy1-0/+3
Given path "a" and the pattern "a", it's matched. But if we throw path "a/b" to pattern "a", the code fails to realize that if "a" matches "a" then "a/b" should also be matched. When the pattern is matched the first time, we can mark it "sticky", so that all files and dirs inside the matched path also matches. This is a simpler solution than modify all match scenarios to fix that. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-01-25dir: simplify untracked cache "ident" fieldLibravatar Christian Couder1-1/+0
It is not a good idea to compare kernel versions and disable the untracked cache if it changes, as people may upgrade and still want the untracked cache to work. So let's just compare work tree locations and kernel name to decide if we should disable it. Also storing many locations in the ident field and comparing to any of them can be dangerous if GIT_WORK_TREE is used with different values. So let's just store one location, the location of the current work tree. The downside is that untracked cache can only be used by one type of OS for now. Exporting a git repo to different clients via a network to e.g. Linux and Windows means that only one can use the untracked cache. If the location changed in the ident field and we still want an untracked cache, let's delete the cache and recreate it. Note that if an untracked cache has been created by a previous Git version, then the kernel version is stored in the ident field. As we now compare with just the kernel name the comparison will fail and the untracked cache will be disabled until it's recreated. Helped-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-01-25dir: add remove_untracked_cache()Libravatar Christian Couder1-0/+1
Factor out code into remove_untracked_cache(), which will be used in a later commit. Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-01-25dir: add {new,add}_untracked_cache()Libravatar Christian Couder1-0/+1
Factor out code into new_untracked_cache() and add_untracked_cache(), which will be used in later commits. Helped-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-05-26Merge branch 'nd/untracked-cache'Libravatar Junio C Hamano1-0/+82
Teach the index to optionally remember already seen untracked files to speed up "git status" in a working tree with tons of cruft. * nd/untracked-cache: (24 commits) git-status.txt: advertisement for untracked cache untracked cache: guard and disable on system changes mingw32: add uname() t7063: tests for untracked cache update-index: test the system before enabling untracked cache update-index: manually enable or disable untracked cache status: enable untracked cache untracked-cache: temporarily disable with $GIT_DISABLE_UNTRACKED_CACHE untracked cache: mark index dirty if untracked cache is updated untracked cache: print stats with $GIT_TRACE_UNTRACKED_STATS untracked cache: avoid racy timestamps read-cache.c: split racy stat test to a separate function untracked cache: invalidate at index addition or removal untracked cache: load from UNTR index extension untracked cache: save to an index extension ewah: add convenient wrapper ewah_serialize_strbuf() untracked cache: don't open non-existent .gitignore untracked cache: mark what dirs should be recursed/saved untracked cache: record/validate dir mtime and reuse cached output untracked cache: make a wrapper around {open,read,close}dir() ...
2015-03-26Merge branch 'jc/report-path-error-to-dir'Libravatar Junio C Hamano1-0/+1
Code clean-up. * jc/report-path-error-to-dir: report_path_error(): move to dir.c
2015-03-24report_path_error(): move to dir.cLibravatar Junio C Hamano1-0/+1
The expected call sequence is for the caller to use match_pathspec() repeatedly on a set of pathspecs, accumulating the "hits" in a separate array, and then call this function to diagnose a pathspec that never matched anything, as that can indicate a typo from the command line, e.g. "git commit Maekfile". Many builtin commands use this function from builtin/ls-files.c, which is not a very healthy arrangement. ls-files might have been the first command to feel the need for such a helper, but the need is shared by everybody who uses the "match and then report" pattern. Move it to dir.c where match_pathspec() is defined. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-03-12untracked cache: guard and disable on system changesLibravatar Nguyễn Thái Ngọc Duy1-0/+2
If the user enables untracked cache, then - move worktree to an unsupported filesystem - or simply upgrade OS - or move the whole (portable) disk from one machine to another - or access a shared fs from another machine there's no guarantee that untracked cache can still function properly. Record the worktree location and OS footprint in the cache. If it changes, err on the safe side and disable the cache. The user can 'update-index --untracked-cache' again to make sure all conditions are met. This adds a new requirement that setup_git_directory* must be called before read_cache() because we need worktree location by then, or the cache is dropped. This change does not cover all bases, you can fool it if you try hard. The point is to stop accidents. Helped-by: Eric Sunshine <sunshine@sunshineco.com> Helped-by: brian m. carlson <sandals@crustytoothpaste.net> Helped-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-03-12untracked cache: invalidate at index addition or removalLibravatar Nguyễn Thái Ngọc Duy1-0/+4
Ideally we should implement untracked_cache_remove_from_index() and untracked_cache_add_to_index() so that they update untracked cache right away instead of invalidating it and wait for read_directory() next time to deal with it. But that may need some more work in unpack-trees.c. So stay simple as the first step. The new call in add_index_entry_with_check() may look strange because new calls usually stay close to cache_tree_invalidate_path(). We do it a bit later than c_t_i_p() in this function because if it's about replacing the entry with the same name, we don't care (but cache-tree does). Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-03-12untracked cache: load from UNTR index extensionLibravatar Nguyễn Thái Ngọc Duy1-0/+2
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-03-12untracked cache: save to an index extensionLibravatar Nguyễn Thái Ngọc Duy1-0/+1
Helped-by: Stefan Beller <sbeller@google.com> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-03-12untracked cache: mark what dirs should be recursed/savedLibravatar Nguyễn Thái Ngọc Duy1-1/+2
If we redo this thing in a functional style, we would have one struct untracked_dir as input tree and another as output. The input is used for verification. The output is a brand new tree, reflecting current worktree. But that means recreate a lot of dir nodes even if a lot could be shared between input and output trees in good cases. So we go with the messy but efficient way, combining both input and output trees into one. We need a way to know which node in this combined tree belongs to the output. This is the purpose of this "recurse" flag. "valid" bit can't be used for this because it's about data of the node except the subdirs. When we invalidate a directory, we want to keep cached data of the subdirs intact even though we don't really know what subdir still exists (yet). Then we check worktree to see what actual subdir remains on disk. Those will have 'recurse' bit set again. If cached data for those are still valid, we may be able to avoid computing exclude files for them. Those subdirs that are deleted will have 'recurse' remained clear and their 'valid' bits do not matter. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-03-12untracked cache: record/validate dir mtime and reuse cached outputLibravatar Nguyễn Thái Ngọc Duy1-0/+2
The main readdir loop in read_directory_recursive() is replaced with a new one that checks if cached results of a directory is still valid. If a file is added or removed from the index, the containing directory is invalidated (but not its subdirs). If directory's mtime is changed, the same happens. If a .gitignore is updated, the containing directory and all subdirs are invalidated recursively. If dir_struct#flags or other conditions change, the cache is ignored. If a directory is invalidated, we opendir/readdir/closedir and run the exclude machinery on that directory listing as usual. If untracked cache is also enabled, we'll update the cache along the way. If a directory is validated, we simply pull the untracked listing out from the cache. The cache also records the list of direct subdirs that we have to recurse in. Fully excluded directories are seen as "untracked files". In the best case when no dirs are invalidated, read_directory() becomes a series of stat(dir), open(.gitignore), fstat(), read(), close() and optionally hash_sha1_file() For comparison, standard read_directory() is a sequence of opendir(), readdir(), open(.gitignore), fstat(), read(), close(), the expensive last_exclude_matching() and closedir(). We already try not to open(.gitignore) if we know it does not exist, so open/fstat/read/close sequence does not apply to every directory. The sequence could be reduced further, as noted in prep_exclude() in another patch. So in theory, the entire best-case read_directory sequence could be reduced to a series of stat() and nothing else. This is not a silver bullet approach. When you compile a C file, for example, the old .o file is removed and a new one with the same name created, effectively invalidating the containing directory's cache (but not its subdirectories). If your build process touches every directory, this cache adds extra overhead for nothing, so it's a good idea to separate generated files from tracked files.. Editors may use the same strategy for saving files. And of course you're out of luck running your repo on an unsupported filesystem and/or operating system. Helped-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-03-12untracked cache: initial untracked cache validationLibravatar Nguyễn Thái Ngọc Duy1-0/+4
Make sure the starting conditions and all global exclude files are good to go. If not, either disable untracked cache completely, or wipe out the cache and start fresh. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-03-12untracked cache: record .gitignore information and dir hierarchyLibravatar Nguyễn Thái Ngọc Duy1-0/+60
The idea is if we can capture all input and (non-rescursive) output of read_directory_recursive(), and can verify later that all the input is the same, then the second r_d_r() should produce the same output as in the first run. The requirement for this to work is stat info of a directory MUST change if an entry is added to or removed from that directory (and should not change often otherwise). If your OS and filesystem do not meet this requirement, untracked cache is not for you. Most file systems on *nix should be fine. On Windows, NTFS is fine while FAT may not be [1] even though FAT on Linux seems to be fine. The list of input of r_d_r() is in the big comment block in dir.h. In short, the output of a directory (not counting subdirs) mainly depends on stat info of the directory in question, all .gitignore leading to it and the check_only flag when r_d_r() is called recursively. This patch records all this info (and the output) as r_d_r() runs. Two hash_sha1_file() are required for $GIT_DIR/info/exclude and core.excludesfile unless their stat data matches. hash_sha1_file() is only needed when .gitignore files in the worktree are modified, otherwise their SHA-1 in index is used (see the previous patch). We could store stat data for .gitignore files so we don't have to rehash them if their content is different from index, but I think .gitignore files are rarely modified, so not worth extra cache data (and hashing penalty read-cache.c:verify_hdr(), as we will be storing this as an index extension). The implication is, if you change .gitignore, you better add it to the index soon or you lose all the benefit of untracked cache because a modified .gitignore invalidates all subdirs recursively. This is especially bad for .gitignore at root. This cached output is about untracked files only, not ignored files because the number of tracked files is usually small, so small cache overhead, while the number of ignored files could go really high (e.g. *.o files mixing with source code). [1] "Description of NTFS date and time stamps for files and folders" http://support.microsoft.com/kb/299648 Helped-by: Torsten Bögershausen <tboegi@web.de> Helped-by: David Turner <dturner@twopensource.com> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>