summaryrefslogtreecommitdiff
path: root/dir.c
AgeCommit message (Collapse)AuthorFilesLines
2018-06-18Merge branch 'jk/ewah-bounds-check'Libravatar Junio C Hamano1-1/+2
The code to read compressed bitmap was not careful to avoid reading past the end of the file, which has been corrected. * jk/ewah-bounds-check: ewah: adjust callers of ewah_read_mmap() ewah_read_mmap: bounds-check mmap reads
2018-06-18ewah: adjust callers of ewah_read_mmap()Libravatar Jeff King1-1/+2
The return value of ewah_read_mmap() is now an ssize_t, since we could (in theory) process up to 32GB of data. This would never happen in practice, but a corrupt or malicious .bitmap or index file could convince us to do so. Let's make sure that we don't stuff the value into an int, which would cause us to incorrectly move our pointer forward. We'd always move too little, since negative values are used for reporting errors. So the worst case is just that we end up reporting a corrupt file, not an out-of-bounds read. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-05-30Merge branch 'bc/object-id'Libravatar Junio C Hamano1-12/+13
Conversion from uchar[20] to struct object_id continues. * bc/object-id: (42 commits) merge-one-file: compute empty blob object ID add--interactive: compute the empty tree value Update shell scripts to compute empty tree object ID sha1_file: only expose empty object constants through git_hash_algo dir: use the_hash_algo for empty blob object ID sequencer: use the_hash_algo for empty tree object ID cache-tree: use is_empty_tree_oid sha1_file: convert cached object code to struct object_id builtin/reset: convert use of EMPTY_TREE_SHA1_BIN builtin/receive-pack: convert one use of EMPTY_TREE_SHA1_HEX wt-status: convert two uses of EMPTY_TREE_SHA1_HEX submodule: convert several uses of EMPTY_TREE_SHA1_HEX sequencer: convert one use of EMPTY_TREE_SHA1_HEX merge: convert empty tree constant to the_hash_algo builtin/merge: switch tree functions to use object_id builtin/am: convert uses of EMPTY_TREE_SHA1_BIN to the_hash_algo sha1-file: add functions for hex empty tree and blob OIDs builtin/receive-pack: avoid hard-coded constants for push certs diff: specify abbreviation size in terms of the_hash_algo upload-pack: replace use of several hard-coded constants ...
2018-05-29Sync with Git 2.17.1Libravatar Junio C Hamano1-1/+1
* maint: (25 commits) Git 2.17.1 Git 2.16.4 Git 2.15.2 Git 2.14.4 Git 2.13.7 fsck: complain when .gitmodules is a symlink index-pack: check .gitmodules files with --strict unpack-objects: call fsck_finish() after fscking objects fsck: call fsck_finish() after fscking objects fsck: check .gitmodules content fsck: handle promisor objects in .gitmodules check fsck: detect gitmodules files fsck: actually fsck blob data fsck: simplify ".git" check index-pack: make fsck error message more specific verify_path: disallow symlinks in .gitmodules update-index: stat updated files earlier verify_dotfile: mention case-insensitivity in comment verify_path: drop clever fallthrough skip_prefix: add case-insensitive variant ...
2018-05-22Sync with Git 2.16.4Libravatar Junio C Hamano1-1/+1
* maint-2.16: Git 2.16.4 Git 2.15.2 Git 2.14.4 Git 2.13.7 verify_path: disallow symlinks in .gitmodules update-index: stat updated files earlier verify_dotfile: mention case-insensitivity in comment verify_path: drop clever fallthrough skip_prefix: add case-insensitive variant is_{hfs,ntfs}_dotgitmodules: add tests is_ntfs_dotgit: match other .git files is_hfs_dotgit: match other .git files is_ntfs_dotgit: use a size_t for traversing string submodule-config: verify submodule names as paths
2018-05-08Merge branch 'sb/submodule-move-nested'Libravatar Junio C Hamano1-3/+57
Moving a submodule that itself has submodule in it with "git mv" forgot to make necessary adjustment to the nested sub-submodules; now the codepath learned to recurse into the submodules. * sb/submodule-move-nested: submodule: fixup nested submodules after moving the submodule submodule-config: remove submodule_from_cache submodule-config: add repository argument to submodule_from_{name, path} submodule-config: allow submodule_free to handle arbitrary repositories grep: remove "repo" arg from non-supporting funcs submodule.h: drop declaration of connect_work_tree_and_git_dir
2018-05-02dir: use the_hash_algo for empty blob object IDLibravatar brian m. carlson1-1/+1
To ensure that we are hash algorithm agnostic, use the_hash_algo to look up the object ID for the empty blob instead of using the empty_tree_oid variable. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-05-02dir: convert struct untracked_cache_dir to object_idLibravatar brian m. carlson1-11/+12
Convert the exclude_sha1 member of struct untracked_cache_dir and rename it to exclude_oid. Eliminate several hard-coded integral constants, and update a function name that referred to SHA-1. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-03-29submodule: fixup nested submodules after moving the submoduleLibravatar Stefan Beller1-3/+57
connect_work_tree_and_git_dir is used to connect a submodule worktree with its git directory and vice versa after events that require a reconnection such as moving around the working tree. As submodules can have nested submodules themselves, we'd also want to fix the nested submodules when asked to. Add an option to recurse into the nested submodules and connect them as well. As submodules are identified by their name (which determines their git directory in relation to their superproject's git directory) internally and by their path in the working tree of the superproject, we need to make sure that the mapping of name <-> path is kept intact. We can do that in the git-mv command by writing out the gitmodules file first and then forcing a reload of the submodule config machinery. Signed-off-by: Stefan Beller <sbeller@google.com> Reviewed-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-03-14sha1_file: convert read_sha1_file to struct object_idLibravatar brian m. carlson1-1/+1
Convert read_sha1_file to take a pointer to struct object_id and rename it read_object_file. Do the same for read_sha1_file_extended. Convert one use in grep.c to use the new function without any other code change, since the pointer being passed is a void pointer that is already initialized with a pointer to struct object_id. Update the declaration and definitions of the modified functions, and apply the following semantic patch to convert the remaining callers: @@ expression E1, E2, E3; @@ - read_sha1_file(E1.hash, E2, E3) + read_object_file(&E1, E2, E3) @@ expression E1, E2, E3; @@ - read_sha1_file(E1->hash, E2, E3) + read_object_file(E1, E2, E3) @@ expression E1, E2, E3, E4; @@ - read_sha1_file_extended(E1.hash, E2, E3, E4) + read_object_file_extended(&E1, E2, E3, E4) @@ expression E1, E2, E3, E4; @@ - read_sha1_file_extended(E1->hash, E2, E3, E4) + read_object_file_extended(E1, E2, E3, E4) Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-03-08Merge branch 'bp/untracked-cache-noflush'Libravatar Junio C Hamano1-2/+13
Writing out the index file when the only thing that changed in it is the untracked cache information is often wasteful, and this has been optimized out. * bp/untracked-cache-noflush: untracked cache: use git_env_bool() not getenv() for customization dir.c: don't flag the index as dirty for changes to the untracked cache
2018-02-28untracked cache: use git_env_bool() not getenv() for customizationLibravatar Junio C Hamano1-2/+12
GIT_DISABLE_UNTRACKED_CACHE and GIT_TEST_UNTRACKED_CACHE are only sensed for their presense by using getenv(); use git_env_bool() instead so that GIT_DISABLE_UNTRACKED_CACHE=false would work as naïvely expected. Also rename GIT_TEST_UNTRACKED_CACHE to GIT_FORCE_UNTRACKED_CACHE to express what it does more honestly. Forcing its use may be one useful thing to do while testing the feature, but testing does not have to be the only use of the knob. While at it, avoid repeated calls to git_env_bool() by capturing the return value from the first call in a static variable. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-02-27Merge branch 'nd/fix-untracked-cache-invalidation'Libravatar Junio C Hamano1-14/+27
Some bugs around "untracked cache" feature have been fixed. * nd/fix-untracked-cache-invalidation: dir.c: ignore paths containing .git when invalidating untracked cache dir.c: stop ignoring opendir() error in open_cached_dir() dir.c: fix missing dir invalidation in untracked code dir.c: avoid stat() in valid_cached_dir() status: add a failing test showing a core.untrackedCache bug
2018-02-15Merge branch 'nd/trace-index-ops'Libravatar Junio C Hamano1-0/+2
* nd/trace-index-ops: trace: measure where the time is spent in the index-heavy operations
2018-02-15Merge branch 'po/object-id'Libravatar Junio C Hamano1-54/+50
Conversion from uchar[20] to struct object_id continues. * po/object-id: sha1_file: rename hash_sha1_file_literally sha1_file: convert write_loose_object to object_id sha1_file: convert force_object_loose to object_id sha1_file: convert write_sha1_file to object_id notes: convert write_notes_tree to object_id notes: convert combine_notes_* to object_id commit: convert commit_tree* to object_id match-trees: convert splice_tree to object_id cache: clear whole hash buffer with oidclr sha1_file: convert hash_sha1_file to object_id dir: convert struct sha1_stat to use object_id sha1_file: convert pretend_sha1_file to object_id
2018-02-07dir.c: ignore paths containing .git when invalidating untracked cacheLibravatar Nguyễn Thái Ngọc Duy1-4/+6
read_directory() code ignores all paths named ".git" even if it's not a valid git repository. See treat_path() for details. Since ".git" is basically invisible to read_directory(), when we are asked to invalidate a path that contains ".git", we can safely ignore it because the slow path would not consider it anyway. This helps when fsmonitor is used and we have a real ".git" repo at worktree top. Occasionally .git/index will be updated and if the fsmonitor hook does not filter it, untracked cache is asked to invalidate the path ".git/index". Without this patch, we invalidate the root directory unncessarily, which: - makes read_directory() fall back to slow path for root directory (slower) - makes the index dirty (because UNTR extension is updated). Depending on the index size, writing it down could also be slow. A note about the new "safe_path" knob. Since this new check could be relatively expensive, avoid it when we know it's not needed. If the path comes from the index, it can't contain ".git". If it does contain, we may be screwed up at many more levels, not just this one. Noticed-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-02-05dir.c: don't flag the index as dirty for changes to the untracked cacheLibravatar Ben Peart1-1/+2
The untracked cache saves its current state in the UNTR index extension. Currently, _any_ change to that state causes the index to be flagged as dirty and written out to disk. Unfortunately, the cost to write out the index can exceed the savings gained by using the untracked cache. Since it is a cache that can be updated from the current state of the working directory, there is no functional requirement that the index be written out for every change to the untracked cache. Update the untracked cache logic so that it no longer forces the index to be written to disk except in the case where the extension is being turned on or off. When some other git command requires the index to be written to disk, the untracked cache will take advantage of that to save it's updated state as well. This results in a performance win when looked at over common sequences of git commands (ie such as a status followed by add, commit, etc). After this patch, all the logic to track statistics for the untracked cache could be removed as it is only used by debug tracing used to debug the untracked cache. Signed-off-by: Ben Peart <benpeart@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-02-02trace: measure where the time is spent in the index-heavy operationsLibravatar Nguyễn Thái Ngọc Duy1-0/+2
All the known heavy code blocks are measured (except object database access). This should help identify if an optimization is effective or not. An unoptimized git-status would give something like below: 0.001791141 s: read cache ... 0.004011363 s: preload index 0.000516161 s: refresh index 0.003139257 s: git command: ... 'status' '--porcelain=2' 0.006788129 s: diff-files 0.002090267 s: diff-index 0.001885735 s: initialize name hash 0.032013138 s: read directory 0.051781209 s: git command: './git' 'status' Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-02-02dir.c: stop ignoring opendir() error in open_cached_dir()Libravatar Nguyễn Thái Ngọc Duy1-1/+6
A follow-up to the recently fixed bugs in the untracked invalidation. If opendir() fails it should show a warning, perhaps this should die, but if this ever happens the error is probably recoverable for the user, and dying would just make things worse. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-01-30sha1_file: convert hash_sha1_file to object_idLibravatar Patryk Obara1-2/+2
Convert the declaration and definition of hash_sha1_file to use struct object_id and adjust all function calls. Rename this function to hash_object_file. Signed-off-by: Patryk Obara <patryk.obara@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-01-30dir: convert struct sha1_stat to use object_idLibravatar Patryk Obara1-54/+50
Convert the declaration of struct sha1_stat. Adjust all usages of this struct and replace hash{clr,cmp,cpy} with oid{clr,cmp,cpy} wherever possible. Rename it to struct oid_stat. Rename static function load_sha1_stat to load_oid_stat. Remove macro EMPTY_BLOB_SHA1_BIN, as it's no longer used. Signed-off-by: Patryk Obara <patryk.obara@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-01-24dir.c: fix missing dir invalidation in untracked codeLibravatar Nguyễn Thái Ngọc Duy1-8/+14
Let's start with how create a new directory cache after the last one becomes invalid (e.g. because its dir mtime has changed...). In open_cached_dir(): 1. We start out with valid_cached_dir() returning false, which should call invalidate_directory() to put a directory state back to initial state, no untracked entries (untracked_nr zero), no sub directory traversal (dirs[].recurse zero). 2. Since the cache cannot be used, we go the slow path opendir() and go through items one by one via readdir(). All the directories on disk will be added back to the cache (if not already exist in dirs[]) and its flag "recurse" gets changed to one to note that it's part of the cached dir travesal next time. 3. By the time we reach close_cached_dir() we should have a good subdir list in dirs[]. Those with "recurse" flag set are the ones present in the on-disk directory. The directory is now marked "valid". Next time read_directory() is called, since the directory is marked valid, it will skip readdir(), go fast path and traverse through dirs[] array instead. Steps one and two need some tight cooperation. If a subdir is removed, readdir() will not find it and of course we cannot examine/invalidate it. To make sure removed directories on disk are gone from the cache, step one must make sure recurse flag of all subdirs are zero. But that's not true. If "valid" flag is already false, there is a chance we go straight to the end of valid_cached_dir() without calling invalidate_directory(). Or we fail to meet the "if (untracked-valid)" condition and skip over the invalidate_directory(). After step 3, we mark the cache valid. Any stale subdir with incorrect recurse flag becomes a real subdir next time we traverse the directory using dirs[] array. We could avoid this by making sure invalidate_directory() is always called (therefore dirs[].recurse cleared) at the beginning of open_cached_dir(). Which is what this patch does. As to how we get into this situation, the key in the test is this command git checkout master where "one/file" is replaced with "one" in the index. This index update triggers untracked_cache_invalidate_path(), which clears valid flag of the root directory while keeping "recurse" flag on the subdir "one" on. On the next git-status, we go through steps 1-3 above and save an incorrect cache on disk. The second git-status blindly follows the bad cache data and shows the problem. This is arguably because of a bad design where "recurse" flag plays double roles: whether a directory should be saved on disk, and whether it is part of a directory traversal. We need to keep recurse flag set at "checkout master" because of the first role: we need to keep subdir caches (dir "two" for example has not been touched at all, no reason to throw its cache away). As long as we make sure to ignore/reset "recurse" flag at the beginning of a directory traversal, we're good. But maybe eventually we should separate these two roles. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-01-24dir.c: avoid stat() in valid_cached_dir()Libravatar Nguyễn Thái Ngọc Duy1-1/+1
stat() may follow a symlink and return stat data of the link's target instead of the link itself. We are concerned about the link itself. It's kind of hard to demonstrate the bug. I think when path->buf is a symlink, we most likely find that its target's stat data does not match our cached one, which means we ignore the cache and fall back to slow path. This is performance issue, not correctness (though we could still catch it by verifying test-dump-untracked-cache. The less unlikely case is, link target stat data matches the cached version and we incorrectly go fast path, ignoring real data on disk. A test for this may involve manipulating stat data, which may be not portable. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-01-22Use MOVE_ARRAYLibravatar SZEDER Gábor1-2/+2
Use the helper macro MOVE_ARRAY to move arrays. This is shorter and safer, as it automatically infers the size of elements. Patch generated by Coccinelle and contrib/coccinelle/array.cocci in Travis CI's static analysis build job. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-12-27Merge branch 'jh/object-filtering'Libravatar Junio C Hamano1-31/+101
In preparation for implementing narrow/partial clone, the object walking machinery has been taught a way to tell it to "filter" some objects from enumeration. * jh/object-filtering: rev-list: support --no-filter argument list-objects-filter-options: support --no-filter list-objects-filter-options: fix 'keword' typo in comment pack-objects: add list-objects filtering rev-list: add list-objects filtering support list-objects: filter objects in traverse_commit_list oidset: add iterator methods to oidset oidmap: add oidmap iterator methods dir: allow exclusions from blob in addition to file
2017-11-22dir: allow exclusions from blob in addition to fileLibravatar Jeff Hostetler1-31/+101
Refactor add_excludes() to separate the reading of the exclude file into a buffer and the parsing of the buffer into exclude_list items. Add add_excludes_from_blob_to_list() to allow an exclude file be specified with an OID without assuming a local worktree or index exists. Refactor read_skip_worktree_file_from_index() and add do_read_blob() to eliminate duplication of preliminary processing of blob contents. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Reviewed-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-11-21Merge branch 'bp/fsmonitor'Libravatar Junio C Hamano1-10/+17
We learned to talk to watchman to speed up "git status" and other operations that need to see which paths have been modified. * bp/fsmonitor: fsmonitor: preserve utf8 filenames in fsmonitor-watchman log fsmonitor: read entirety of watchman output fsmonitor: MINGW support for watchman integration fsmonitor: add a performance test fsmonitor: add a sample integration script for Watchman fsmonitor: add test cases for fsmonitor extension split-index: disable the fsmonitor extension when running the split index test fsmonitor: add a test tool to dump the index extension update-index: add fsmonitor support to update-index ls-files: Add support in ls-files to display the fsmonitor valid bit fsmonitor: add documentation for the fsmonitor extension. fsmonitor: teach git to optionally utilize a file system monitor to speed up detecting new or changed files. update-index: add a new --force-write-index option preload-index: add override to enable testing preload-index bswap: add 64 bit endianness helper get_be64
2017-11-13Merge branch 'jm/status-ignored-files-list'Libravatar Junio C Hamano1-2/+42
The set of paths output from "git status --ignored" was tied closely with its "--untracked=<mode>" option, but now it can be controlled more flexibly. Most notably, a directory that is ignored because it is listed to be ignored in the ignore/exclude mechanism can be handled differently from a directory that ends up to be ignored only because all files in it are ignored. * jm/status-ignored-files-list: status: test ignored modes status: document options to show matching ignored files status: report matching ignored and normal untracked status: add option to show ignored files differently
2017-11-06Merge branch 'bc/object-id'Libravatar Junio C Hamano1-4/+4
Conversion from uchar[20] to struct object_id continues. * bc/object-id: (25 commits) refs/files-backend: convert static functions to object_id refs: convert read_raw_ref backends to struct object_id refs: convert peel_object to struct object_id refs: convert resolve_ref_unsafe to struct object_id worktree: convert struct worktree to object_id refs: convert resolve_gitlink_ref to struct object_id Convert remaining callers of resolve_gitlink_ref to object_id sha1_file: convert index_path and index_fd to struct object_id refs: convert reflog_expire parameter to struct object_id refs: convert read_ref_at to struct object_id refs: convert peel_ref to struct object_id builtin/pack-objects: convert to struct object_id pack-bitmap: convert traverse_bitmap_commit_list to object_id refs: convert dwim_log to struct object_id builtin/reflog: convert remaining unsigned char uses to object_id refs: convert dwim_ref and expand_ref to struct object_id refs: convert read_ref and read_ref_full to object_id refs: convert resolve_refdup and refs_resolve_refdup to struct object_id Convert check_connected to use struct object_id refs: update ref transactions to use struct object_id ...
2017-11-06Merge branch 'js/submodule-in-excluded'Libravatar Junio C Hamano1-1/+1
"git status --ignored -u" did not stop at a working tree of a separate project that is embedded in an ignored directory and listed files in that other project, instead of just showing the directory itself as ignored. * js/submodule-in-excluded: status: do not get confused by submodules in excluded directories
2017-10-31status: report matching ignored and normal untrackedLibravatar Jameson Miller1-2/+18
Teach status command to handle `--ignored=matching` with `--untracked-files=normal` Signed-off-by: Jameson Miller <jamill@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-10-31status: add option to show ignored files differentlyLibravatar Jameson Miller1-0/+24
Teach the status command more flexibility in how ignored files are reported. Currently, the reporting of ignored files and untracked files are linked. You cannot control how ignored files are reported independently of how untracked files are reported (i.e. `all` vs `normal`). This makes it impossible to show untracked files with the `all` option, but show ignored files with the `normal` option. This work 1) adds the ability to control the reporting of ignored files independently of untracked files and 2) introduces the concept of status reporting ignored paths that explicitly match an ignored pattern. There are 2 benefits to these changes: 1) if a consumer needs all untracked files but not all ignored files, there is a performance benefit to not scanning all contents of an ignored directory and 2) returning ignored files that explicitly match a path allow a consumer to make more informed decisions about when a status result might be stale. This commit implements --ignored=matching with --untracked-files=all. The following commit will implement --ignored=matching with --untracked=files=normal. As an example of where this flexibility could be useful is that our application (Visual Studio) runs the status command and presents the output. It shows all untracked files individually (e.g. using the '--untracked-files==all' option), and would like to know about which paths are ignored. It uses information about ignored paths to make decisions about when the status result might have changed. Additionally, many projects place build output into directories inside a repository's working directory (e.g. in "bin/" and "obj/" directories). Normal usage is to explicitly ignore these 2 directory names in the .gitignore file (rather than or in addition to the *.obj pattern).If an application could know that these directories are explicitly ignored, it could infer that all contents are ignored as well and make better informed decisions about files in these directories. It could infer that any changes under these paths would not affect the output of status. Additionally, there can be a significant performance benefit by avoiding scanning through ignored directories. When status is set to report matching ignored files, it has the following behavior. Ignored files and directories that explicitly match an exclude pattern are reported. If an ignored directory matches an exclude pattern, then the path of the directory is returned. If a directory does not match an exclude pattern, but all of its contents are ignored, then the contained files are reported instead of the directory. Signed-off-by: Jameson Miller <jamill@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-10-26status: do not get confused by submodules in excluded directoriesLibravatar Johannes Schindelin1-1/+1
We meticulously pass the `exclude` flag to the `treat_directory()` function so that we can indicate that files in it are excluded rather than untracked when recursing. But we did not yet treat submodules the same way. Because of that, `git status --ignored --untracked` with a submodule `submodule` in a gitignored `tracked/` would show the submodule in the "Untracked files" section, e.g. On branch master Untracked files: (use "git add <file>..." to include in what will be committed) tracked/submodule/ Ignored files: (use "git add -f <file>..." to include in what will be committed) tracked/submodule/initial.t Instead, we would want it to show the submodule in the "Ignored files" section: On branch master Ignored files: (use "git add -f <file>..." to include in what will be committed) tracked/submodule/ Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-10-16refs: convert resolve_gitlink_ref to struct object_idLibravatar brian m. carlson1-2/+2
Convert the declaration and definition of resolve_gitlink_ref to use struct object_id and apply the following semantic patch: @@ expression E1, E2, E3; @@ - resolve_gitlink_ref(E1, E2, E3.hash) + resolve_gitlink_ref(E1, E2, &E3) @@ expression E1, E2, E3; @@ - resolve_gitlink_ref(E1, E2, E3->hash) + resolve_gitlink_ref(E1, E2, E3) Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-10-16Convert remaining callers of resolve_gitlink_ref to object_idLibravatar brian m. carlson1-4/+4
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-10-01fsmonitor: teach git to optionally utilize a file system monitor to speed up ↵Libravatar Ben Peart1-10/+17
detecting new or changed files. When the index is read from disk, the fsmonitor index extension is used to flag the last known potentially dirty index entries. The registered core.fsmonitor command is called with the time the index was last updated and returns the list of files changed since that time. This list is used to flag any additional dirty cache entries and untracked cache directories. We can then use this valid state to speed up preload_index(), ie_match_stat(), and refresh_cache_ent() as they do not need to lstat() files to detect potential changes for those entries marked CE_FSMONITOR_VALID. In addition, if the untracked cache is turned on valid_cached_dir() can skip checking directories for new or changed files as fsmonitor will invalidate the cache only for those directories that have been identified as having potential changes. To keep the CE_FSMONITOR_VALID state accurate during git operations; when git updates a cache entry to match the current state on disk, it will now set the CE_FSMONITOR_VALID bit. Inversely, anytime git changes a cache entry, the CE_FSMONITOR_VALID bit is cleared and the corresponding untracked cache directory is marked invalid. Signed-off-by: Ben Peart <benpeart@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-09-19Improve performance of git status --ignoredLibravatar Jameson Miller1-6/+41
Improve the performance of the directory listing logic when it wants to list non-empty ignored directories. In order to show non-empty ignored directories, the existing logic will recursively iterate through all contents of an ignored directory. This change introduces the optimization to stop iterating through the contents once it finds the first file. This can have a significant improvement in 'git status --ignored' performance in repositories with a large number of files in ignored directories. For an example of the performance difference on an example repository with 196,000 files in 400 ignored directories: | Command | Time (s) | | -------------------------- | --------- | | git status | 1.2 | | git status --ignored (old) | 3.9 | | git status --ignored (new) | 1.4 | Signed-off-by: Jameson Miller <jamill@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-08-23Merge branch 'rs/stat-data-unaligned-reads-fix' into maintLibravatar Junio C Hamano1-23/+27
Code clean-up. * rs/stat-data-unaligned-reads-fix: dir: support platforms that require aligned reads
2017-08-11Merge branch 'rs/stat-data-unaligned-reads-fix'Libravatar Junio C Hamano1-23/+27
Code clean-up. * rs/stat-data-unaligned-reads-fix: dir: support platforms that require aligned reads
2017-07-17dir: support platforms that require aligned readsLibravatar René Scharfe1-23/+27
The untracked cache is stored on disk by concatenating its memory structures without any padding. Consequently some of the structs are not aligned at a particular boundary when the whole extension is read back in one go. That's only OK on platforms without strict alignment requirements, or for byte-aligned data like strings or hash values. Decode struct ondisk_untracked_cache carefully from the extension blob by using explicit pointer arithmetic with offsets, avoiding alignment issues. Use char pointers for passing stat_data objects to stat_data_from_disk(), and use memcpy(3) in that function to get the contents into a properly aligned struct, then perform the byte-order adjustment in place there. Found with Clang's UBSan. Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-07-10Merge branch 'ab/wildmatch'Libravatar Junio C Hamano1-5/+3
Minor code cleanup. * ab/wildmatch: wildmatch: remove unused wildopts parameter
2017-06-24Merge branch 'ab/free-and-null'Libravatar Junio C Hamano1-6/+3
A common pattern to free a piece of memory and assign NULL to the pointer that used to point at it has been replaced with a new FREE_AND_NULL() macro. * ab/free-and-null: *.[ch] refactoring: make use of the FREE_AND_NULL() macro coccinelle: make use of the "expression" FREE_AND_NULL() rule coccinelle: add a rule to make "expression" code use FREE_AND_NULL() coccinelle: make use of the "type" FREE_AND_NULL() rule coccinelle: add a rule to make "type" code use FREE_AND_NULL() git-compat-util: add a FREE_AND_NULL() wrapper around free(ptr); ptr = NULL
2017-06-24Merge branch 'bw/config-h'Libravatar Junio C Hamano1-0/+1
Fix configuration codepath to pay proper attention to commondir that is used in multi-worktree situation, and isolate config API into its own header file. * bw/config-h: config: don't implicitly use gitdir or commondir config: respect commondir setup: teach discover_git_directory to respect the commondir config: don't include config.h by default config: remove git_config_iter config: create config.h
2017-06-24Merge branch 'bw/ls-files-sans-the-index'Libravatar Junio C Hamano1-1/+1
Code clean-up. * bw/ls-files-sans-the-index: ls-files: factor out tag calculation ls-files: factor out debug info into a function ls-files: convert show_files to take an index ls-files: convert show_ce_entry to take an index ls-files: convert prune_cache to take an index ls-files: convert ce_excluded to take an index ls-files: convert show_ru_info to take an index ls-files: convert show_other_files to take an index ls-files: convert show_killed_files to take an index ls-files: convert write_eolinfo to take an index ls-files: convert overlay_tree_on_cache to take an index tree: convert read_tree to take an index parameter convert: convert renormalize_buffer to take an index convert: convert convert_to_git to take an index convert: convert convert_to_git_filter_fd to take an index convert: convert crlf_to_git to take an index convert: convert get_cached_convert_stats_ascii to take an index
2017-06-23wildmatch: remove unused wildopts parameterLibravatar Ævar Arnfjörð Bjarmason1-5/+3
Remove the unused wildopts placeholder struct from being passed to all wildmatch() invocations, or rather remove all the boilerplate NULL parameters. This parameter was added back in commit 9b3497cab9 ("wildmatch: rename constants and update prototype", 2013-01-01) as a placeholder for future use. Over 4 years later nothing has made use of it, let's just remove it. It can be added in the future if we find some reason to start using such a parameter. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-22Merge branch 'pc/dir-count-slashes'Libravatar Junio C Hamano1-0/+9
Three instances of the same helper function have been consolidated to one. * pc/dir-count-slashes: dir: create function count_slashes()
2017-06-16coccinelle: make use of the "type" FREE_AND_NULL() ruleLibravatar Ævar Arnfjörð Bjarmason1-6/+3
Apply the result of the just-added coccinelle rule. This manually excludes a few occurrences, mostly things that resulted in many FREE_AND_NULL() on one line, that'll be manually fixed in a subsequent change. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-15config: don't include config.h by defaultLibravatar Brandon Williams1-0/+1
Stop including config.h by default in cache.h. Instead only include config.h in those files which require use of the config system. Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-13Merge branch 'nd/fopen-errors'Libravatar Junio C Hamano1-3/+3
We often try to open a file for reading whose existence is optional, and silently ignore errors from open/fopen; report such errors if they are not due to missing files. * nd/fopen-errors: mingw_fopen: report ENOENT for invalid file names mingw: verify that paths are not mistaken for remote nicknames log: fix memory leak in open_next_file() rerere.c: move error_errno() closer to the source system call print errno when reporting a system call error wrapper.c: make warn_on_inaccessible() static wrapper.c: add and use fopen_or_warn() wrapper.c: add and use warn_on_fopen_errors() config.mak.uname: set FREAD_READS_DIRECTORIES for Darwin, too config.mak.uname: set FREAD_READS_DIRECTORIES for Linux and FreeBSD clone: use xfopen() instead of fopen() use xfopen() in more places git_fopen: fix a sparse 'not declared' warning
2017-06-13Merge branch 'jc/noent-notdir'Libravatar Junio C Hamano1-1/+1
Our code often opens a path to an optional file, to work on its contents when we can successfully open it. We can ignore a failure to open if such an optional file does not exist, but we do want to report a failure in opening for other reasons (e.g. we got an I/O error, or the file is there, but we lack the permission to open). The exact errors we need to ignore are ENOENT (obviously) and ENOTDIR (less obvious). Instead of repeating comparison of errno with these two constants, introduce a helper function to do so. * jc/noent-notdir: treewide: use is_missing_file_error() where ENOENT and ENOTDIR are checked compat-util: is_missing_file_error()