summaryrefslogtreecommitdiff
path: root/Documentation/technical
AgeCommit message (Collapse)AuthorFilesLines
2017-11-21Merge branch 'bp/fsmonitor'Libravatar Junio C Hamano1-0/+19
We learned to talk to watchman to speed up "git status" and other operations that need to see which paths have been modified. * bp/fsmonitor: fsmonitor: preserve utf8 filenames in fsmonitor-watchman log fsmonitor: read entirety of watchman output fsmonitor: MINGW support for watchman integration fsmonitor: add a performance test fsmonitor: add a sample integration script for Watchman fsmonitor: add test cases for fsmonitor extension split-index: disable the fsmonitor extension when running the split index test fsmonitor: add a test tool to dump the index extension update-index: add fsmonitor support to update-index ls-files: Add support in ls-files to display the fsmonitor valid bit fsmonitor: add documentation for the fsmonitor extension. fsmonitor: teach git to optionally utilize a file system monitor to speed up detecting new or changed files. update-index: add a new --force-write-index option preload-index: add override to enable testing preload-index bswap: add 64 bit endianness helper get_be64
2017-10-31status: document options to show matching ignored filesLibravatar Jameson Miller1-4/+23
Signed-off-by: Jameson Miller <jamill@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-10-11Merge branch 'hn/string-list-doc'Libravatar Junio C Hamano1-1/+1
Docfix. * hn/string-list-doc: api-argv-array.txt: remove broken link to string-list API
2017-10-11Merge branch 'jr/hash-migration-plan-doc'Libravatar Junio C Hamano1-0/+797
Lay out plans for weaning us off of SHA-1. * jr/hash-migration-plan-doc: technical doc: add a design doc for hash function transition
2017-10-06api-argv-array.txt: remove broken link to string-list APILibravatar Todd Zullinger1-1/+1
In 4f665f2cf3 (string-list.h: move documentation from Documentation/api/ into header, 2017-09-26) the string-list API documentation was moved to string-list.h. The argv-array API documentation may follow a similar course in the future. Until then, prevent the broken link from making it to the end-user documentation. Signed-off-by: Todd Zullinger <tmz@pobox.com> Reviewed-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-10-01fsmonitor: add documentation for the fsmonitor extension.Libravatar Ben Peart1-0/+19
This includes the core.fsmonitor setting, the fsmonitor integration hook, and the fsmonitor index extension. Also add documentation for the new fsmonitor options to ls-files and update-index. Signed-off-by: Ben Peart <benpeart@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-09-28technical doc: add a design doc for hash function transitionLibravatar Jonathan Nieder1-0/+797
This document describes what a transition to a new hash function for Git would look like. Add it to Documentation/technical/ as the plan of record so that future changes can be recorded as patches. Also-by: Brandon Williams <bmwill@google.com> Also-by: Jonathan Tan <jonathantanmy@google.com> Also-by: Stefan Beller <sbeller@google.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-09-27string-list.h: move documentation from Documentation/api/ into headerLibravatar Han-Wen Nienhuys1-209/+0
This mirrors commit 'bdfdaa497 ("strbuf.h: integrate api-strbuf.txt documentation, 2015-01-16") which did the same for strbuf.h: * API documentation uses /** */ to set it apart from other comments. * Function names were stripped from the comments. * Ordering of the header was adjusted to follow the one from the text file. * Edited some existing comments from string-list.h for consistency. Signed-off-by: Han-Wen Nienhuys <hanwen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-09-19Merge branch 'ma/remove-config-maybe-bool'Libravatar Junio C Hamano1-4/+0
Finishing touches to a recent topic. * ma/remove-config-maybe-bool: config: remove git_config_maybe_bool
2017-09-19Merge branch 'nd/prune-in-worktree'Libravatar Junio C Hamano1-5/+2
"git gc" and friends when multiple worktrees are used off of a single repository did not consider the index and per-worktree refs of other worktrees as the root for reachability traversal, making objects that are in use only in other worktrees to be subject to garbage collection. * nd/prune-in-worktree: refs.c: reindent get_submodule_ref_store() refs.c: remove fallback-to-main-store code get_submodule_ref_store() rev-list: expose and document --single-worktree revision.c: --reflog add HEAD reflog from all worktrees files-backend: make reflog iterator go through per-worktree reflog revision.c: --all adds HEAD from all worktrees refs: remove dead for_each_*_submodule() refs.c: move for_each_remote_ref_submodule() to submodule.c revision.c: use refs_for_each*() instead of for_each_*_submodule() refs: add refs_head_ref() refs: move submodule slash stripping code to get_submodule_ref_store refs.c: refactor get_submodule_ref_store(), share common free block revision.c: --indexed-objects add objects from all worktrees revision.c: refactor add_index_objects_to_pending() refs.c: use is_dir_sep() in resolve_gitlink_ref() revision.h: new flag in struct rev_info wrt. worktree-related refs
2017-09-10Merge branch 'ma/up-to-date'Libravatar Junio C Hamano2-3/+3
Message and doc updates. * ma/up-to-date: treewide: correct several "up-to-date" to "up to date" Documentation/user-manual: update outdated example output
2017-09-07config: remove git_config_maybe_boolLibravatar Martin Ågren1-4/+0
The function was deprecated in commit 89576613 ("treewide: deprecate git_config_maybe_bool, use git_parse_maybe_bool", 2017-08-07) and has no users. Signed-off-by: Martin Ågren <martin.agren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-08-24refs: remove dead for_each_*_submodule()Libravatar Nguyễn Thái Ngọc Duy1-5/+2
These are used in revision.c. After the last patch they are replaced with the refs_ version. Delete them. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-08-24Merge branch 'rs/object-id'Libravatar Junio C Hamano1-3/+3
Conversion from uchar[20] to struct object_id continues. * rs/object-id: tree-walk: convert fill_tree_descriptor() to object_id
2017-08-23treewide: correct several "up-to-date" to "up to date"Libravatar Martin Ågren2-3/+3
Follow the Oxford style, which says to use "up-to-date" before the noun, but "up to date" after it. Don't change plumbing (specifically send-pack.c, but transport.c (git push) also has the same string). This was produced by grepping for "up-to-date" and "up to date". It turned out we only had to edit in one direction, removing the hyphens. Fix a typo in Documentation/git-diff-index.txt while we're there. Reported-by: Jeffrey Manian <jeffrey.manian@gmail.com> Reported-by: STEVEN WHITE <stevencharleswhitevoices@gmail.com> Signed-off-by: Martin Ågren <martin.agren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-08-22Merge branch 'ma/parse-maybe-bool'Libravatar Junio C Hamano1-0/+4
Code clean-up. * ma/parse-maybe-bool: parse_decoration_style: drop unused argument `var` treewide: deprecate git_config_maybe_bool, use git_parse_maybe_bool config: make git_{config,parse}_maybe_bool equivalent config: introduce git_parse_maybe_bool_text t5334: document that git push --signed=1 does not work Doc/git-{push,send-pack}: correct --sign= to --signed=
2017-08-14tree-walk: convert fill_tree_descriptor() to object_idLibravatar René Scharfe1-3/+3
All callers of fill_tree_descriptor() have been converted to object_id already, so convert that function as well. As a nice side-effect we get rid of NULL checks in tree-diff.c, as fill_tree_descriptor() already does them for us. Helped-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Rene Scharfe <l.s.r@web.de> Reviewed-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-08-11Merge branch 'ma/pager-per-subcommand-action'Libravatar Junio C Hamano1-73/+0
The "tag.pager" configuration variable was useless for those who actually create tag objects, as it interfered with the use of an editor. A new mechanism has been introduced for commands to enable pager depending on what operation is being carried out to fix this, and then "git tag -l" is made to run pager by default. * ma/pager-per-subcommand-action: git.c: ignore pager.* when launching builtin as dashed external tag: change default of `pager.tag` to "on" tag: respect `pager.tag` in list-mode only t7006: add tests for how git tag paginates git.c: provide setup_auto_pager() git.c: let builtins opt for handling `pager.foo` themselves builtin.h: take over documentation from api-builtin.txt
2017-08-07treewide: deprecate git_config_maybe_bool, use git_parse_maybe_boolLibravatar Martin Ågren1-0/+4
The only difference between these is that the former takes an argument `name` which it ignores completely. Still, the callers are quite careful to provide reasonable values for it. Once in-flight topics have landed, we should be able to remove git_config_maybe_bool. In the meantime, document it as deprecated in the technical documentation. While at it, document git_parse_maybe_bool. Signed-off-by: Martin Ågren <martin.agren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-08-03builtin.h: take over documentation from api-builtin.txtLibravatar Martin Ågren1-73/+0
Delete Documentation/technical/api-builtin.txt and move its content into builtin.h. Format it as a comment. Remove a '+' which was needed when the information was formatted for AsciiDoc. Similarly, change "::" to ":". Document SUPPORT_SUPER_PREFIX, thereby bringing the documentation up to date with the available flags. While at it, correct '3 more things to do' to '4 more things to do'. Signed-off-by: Martin Ågren <martin.agren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-07-26Documentation: migrate sub-process docs to headerLibravatar Jonathan Tan1-59/+0
Move the documentation for the sub-process API from a separate txt file to its header file. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-30hashmap: migrate documentation from Documentation/technical into headerLibravatar Stefan Beller1-309/+0
While at it, clarify the use of `key`, `keydata`, `entry_or_key` as well as documenting the new data pointer for the compare function. Rework the example. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-02Merge branch 'sl/clean-d-ignored-fix'Libravatar Junio C Hamano1-0/+6
"git clean -d" used to clean directories that has ignored files, even though the command should not lose ignored ones without "-x". "git status --ignored" did not list ignored and untracked files without "-uall". These have been corrected. * sl/clean-d-ignored-fix: clean: teach clean -d to preserve ignored paths dir: expose cmp_name() and check_contains() dir: hide untracked contents of untracked dirs dir: recurse into untracked dirs for ignored files t7061: status --ignored should search untracked dirs t7300: clean -d should skip dirs with ignored files
2017-05-30Merge branch 'bp/sub-process-convert-filter'Libravatar Junio C Hamano1-0/+59
Code from "conversion using external process" codepath has been extracted to a separate sub-process.[ch] module. * bp/sub-process-convert-filter: convert: update subprocess_read_status() to not die on EOF sub-process: move sub-process functions into separate files convert: rename reusable sub-process functions convert: update generic functions to only use generic data structures convert: separate generic structures and variables from the filter specific ones convert: split start_multi_file_filter() into two separate functions pkt-line: annotate packet_writel with LAST_ARG_MUST_BE_NULL convert: move packet_write_line() into pkt-line as packet_writel() pkt-line: add packet_read_line_gently() pkt-line: fix packet_read_line() to handle len < 0 errors convert: remove erroneous tests for errno == EPIPE
2017-05-23Merge branch 'jt/push-options-doc'Libravatar Junio C Hamano1-6/+26
The receive-pack program now makes sure that the push certificate records the same set of push options used for pushing. * jt/push-options-doc: receive-pack: verify push options in cert docs: correct receive.advertisePushOptions default
2017-05-22dir: hide untracked contents of untracked dirsLibravatar Samuel Lijin1-0/+6
When we taught read_directory_recursive() to recurse into untracked directories in search of ignored files given DIR_SHOW_IGNORED_TOO, that had the side effect of teaching it to collect the untracked contents of untracked directories. It doesn't always make sense to return these, though (we do need them for `clean -d`), so we introduce a flag (DIR_KEEP_UNTRACKED_CONTENTS) to control whether or not read_directory() strips dir->entries of the untracked contents of untracked dirs. We also introduce check_contains() to check if one dir_entry corresponds to a path which contains the path corresponding to another dir_entry. This also fixes known breakages in t7061, since status --ignored now searches untracked directories for ignored files. Signed-off-by: Samuel Lijin <sxlijin@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-16Merge branch 'js/larger-timestamps'Libravatar Junio C Hamano1-4/+4
Some platforms have ulong that is smaller than time_t, and our historical use of ulong for timestamp would mean they cannot represent some timestamp that the platform allows. Invent a separate and dedicated timestamp_t (so that we can distingiuish timestamps and a vanilla ulongs, which along is already a good move), and then declare uintmax_t is the type to be used as the timestamp_t. * js/larger-timestamps: archive-tar: fix a sparse 'constant too large' warning use uintmax_t for timestamps date.c: abort if the system time cannot handle one of our timestamps timestamp_t: a new data type for timestamps PRItime: introduce a new "printf format" for timestamps parse_timestamp(): specify explicitly where we parse timestamps t0006 & t5000: skip "far in the future" test when time_t is too limited t0006 & t5000: prepare for 64-bit timestamps ref-filter: avoid using `unsigned long` for catch-all data type
2017-05-15sub-process: move sub-process functions into separate filesLibravatar Ben Peart1-0/+59
Move the sub-proces functions into sub-process.h/c. Add documentation for the new module in Documentation/technical/api-sub-process.txt Signed-off-by: Ben Peart <benpeart@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-10receive-pack: verify push options in certLibravatar Jonathan Tan1-6/+26
In commit f6a4e61 ("push: accept push options", 2016-07-14), send-pack was taught to include push options both within the signed cert (if the push is a signed push) and outside the signed cert; however, receive-pack ignores push options within the cert, only handling push options outside the cert. Teach receive-pack, in the case that push options are provided for a signed push, to verify that the push options both within the cert and outside the cert are consistent. This sets in stone the requirement that send-pack redundantly send its push options in 2 places, but I think that this is better than the alternatives. Sending push options only within the cert is backwards-incompatible with existing Git servers (which read push options only from outside the cert), and sending push options only outside the cert means that the push options are not signed for. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-04-27timestamp_t: a new data type for timestampsLibravatar Johannes Schindelin1-4/+4
Git's source code assumes that unsigned long is at least as precise as time_t. Which is incorrect, and causes a lot of problems, in particular where unsigned long is only 32-bit (notably on Windows, even in 64-bit versions). So let's just use a more appropriate data type instead. In preparation for this, we introduce the new `timestamp_t` data type. By necessity, this is a very, very large patch, as it has to replace all timestamps' data type in one go. As we will use a data type that is not necessarily identical to `time_t`, we need to be very careful to use `time_t` whenever we interact with the system functions, and `timestamp_t` everywhere else. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-04-23Merge branch 'jt/fetch-pack-error-reporting'Libravatar Junio C Hamano1-1/+6
"git fetch-pack" was not prepared to accept ERR packet that the upload-pack can send with a human-readable error message. It showed the packet contents with ERR prefix, so there was no data loss, but it was redundant to say "ERR" in an error message. * jt/fetch-pack-error-reporting: fetch-pack: show clearer error message upon ERR
2017-04-19Merge branch 'bc/object-id'Libravatar Junio C Hamano1-22/+22
Conversion from unsigned char [40] to struct object_id continues. * bc/object-id: Documentation: update and rename api-sha1-array.txt Rename sha1_array to oid_array Convert sha1_array_for_each_unique and for_each_abbrev to object_id Convert sha1_array_lookup to take struct object_id Convert remaining callers of sha1_array_lookup to object_id Make sha1_array_append take a struct object_id * sha1-array: convert internal storage for struct sha1_array to object_id builtin/pull: convert to struct object_id submodule: convert check_for_new_submodule_commits to object_id sha1_name: convert disambiguate_hint_fn to take object_id sha1_name: convert struct disambiguate_state to object_id test-sha1-array: convert most code to struct object_id parse-options-cb: convert sha1_array_append caller to struct object_id fsck: convert init_skiplist to struct object_id builtin/receive-pack: convert portions to struct object_id builtin/pull: convert portions to struct object_id builtin/diff: convert to struct object_id Convert GIT_SHA1_RAWSZ used for allocation to GIT_MAX_RAWSZ Convert GIT_SHA1_HEXSZ used for allocation to GIT_MAX_HEXSZ Define new hash-size constants for allocating memory
2017-04-17fetch-pack: show clearer error message upon ERRLibravatar Jonathan Tan1-1/+6
Currently, fetch-pack prints a confusing error message ("expected ACK/NAK") when the server it's communicating with sends a pkt-line starting with "ERR". Replace it with a less confusing error message. Also update the documentation describing the fetch-pack/upload-pack protocol (pack-protocol.txt) to indicate that "ERR" can be sent in the place of "ACK" or "NAK". In practice, this has been done for quite some time by other Git implementations (e.g. JGit sends "want $id not valid") and by Git itself (since commit bdb31ea: "upload-pack: report "not our ref" to client", 2017-02-23) whenever a "want" line references an object that it does not have. (This is uncommon, but can happen if a repository is garbage-collected during a negotiation.) Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Reviewed-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-31Documentation: update and rename api-sha1-array.txtLibravatar brian m. carlson1-22/+22
Since the structure and functions have changed names, update the code examples and the documentation. Rename the file to match the new name of the API. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-28Merge branch 'jh/memihash-opt'Libravatar Junio C Hamano1-0/+22
The name-hash used for detecting paths that are different only in cases (which matter on case insensitive filesystems) has been optimized to take advantage of multi-threading when it makes sense. * jh/memihash-opt: name-hash: add test-lazy-init-name-hash to .gitignore name-hash: add perf test for lazy_init_name_hash name-hash: add test-lazy-init-name-hash name-hash: perf improvement for lazy_init_name_hash hashmap: document memihash_cont, hashmap_disallow_rehash api hashmap: add disallow_rehash setting hashmap: allow memihash computation to be continued name-hash: specify initial size for istate.dir_hash table
2017-03-23hashmap: document memihash_cont, hashmap_disallow_rehash apiLibravatar Jeff Hostetler1-0/+22
Document memihash_cont() and hashmap_disallow_rehash() in Documentation/technical/api-hashmap.txt. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-02-27Merge branch 'bw/attr'Libravatar Junio C Hamano1-30/+56
The gitattributes machinery is being taught to work better in a multi-threaded environment. * bw/attr: (27 commits) attr: reformat git_attr_set_direction() function attr: push the bare repo check into read_attr() attr: store attribute stack in attr_check structure attr: tighten const correctness with git_attr and match_attr attr: remove maybe-real, maybe-macro from git_attr attr: eliminate global check_all_attr array attr: use hashmap for attribute dictionary attr: change validity check for attribute names to use positive logic attr: pass struct attr_check to collect_some_attrs attr: retire git_check_attrs() API attr: convert git_check_attrs() callers to use the new API attr: convert git_all_attrs() to use "struct attr_check" attr: (re)introduce git_check_attr() and struct attr_check attr: rename function and struct related to checking attributes attr.c: outline the future plans by heavily commenting Documentation: fix a typo attr.c: add push_stack() helper attr: support quoting pathname patterns in C style attr.c: plug small leak in parse_attr_line() attr.c: tighten constness around "git_attr" structure ...
2017-02-27Merge branch 'jk/describe-omit-some-refs'Libravatar Junio C Hamano1-0/+5
"git describe" and "git name-rev" have been taught to take more than one refname patterns to restrict the set of refs to base their naming output on, and also learned to take negative patterns to name refs not to be used for naming via their "--exclude" option. * jk/describe-omit-some-refs: describe: teach describe negative pattern matches describe: teach --match to accept multiple patterns name-rev: add support to exclude refs by pattern match name-rev: extend --refs to accept multiple patterns doc: add documentation for OPT_STRING_LIST
2017-02-01attr: retire git_check_attrs() APILibravatar Junio C Hamano1-30/+56
Since nobody uses the old API, make it file-scope static, and update the documentation to describe the new API. Signed-off-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-01-31Merge branch 'sb/in-core-index-doc'Libravatar Junio C Hamano1-21/+0
Documentation and in-code comments updates. * sb/in-core-index-doc: documentation: retire unfinished documentation cache.h: document add_[file_]to_index cache.h: document remove_index_entry_at cache.h: document index_name_pos
2017-01-31Merge branch 'jk/clear-delta-base-cache-fix'Libravatar Junio C Hamano1-1/+3
A crashing bug introduced in v2.11 timeframe has been found (it is triggerable only in fast-import) and fixed. * jk/clear-delta-base-cache-fix: clear_delta_base_cache(): don't modify hashmap while iterating
2017-01-19doc: add documentation for OPT_STRING_LISTLibravatar Jacob Keller1-0/+5
Commit c8ba16391655 ("parse-options: add OPT_STRING_LIST helper", 2011-06-09) added the OPT_STRING_LIST as a way to accumulate a repeated list of strings. However, this was not documented in the api-parse-options documentation. Add documentation now so that future developers may learn of its existence. Signed-off-by: Jacob Keller <jacob.keller@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-01-19documentation: retire unfinished documentationLibravatar Stefan Beller1-21/+0
When looking for documentation for a specific function, you may be tempted to run git -C Documentation grep index_name_pos only to find the file technical/api-in-core-index.txt, which doesn't help for understanding the given function. It would be better to not find these functions in the documentation, such that people directly dive into the code instead. In the previous patches we have documented * index_name_pos() * remove_index_entry_at() * add_[file_]to_index() in cache.h We already have documentation for: * add_index_entry() * read_index() Which leaves us with a TODO for: * cache -> the_index macros * refresh_index() * discard_index() * ie_match_stat() and ie_modified(); how they are different and when to use which. * write_index() that was renamed to write_locked_index * cache_tree_invalidate_path() * cache_tree_update() Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-01-19clear_delta_base_cache(): don't modify hashmap while iteratingLibravatar Jeff King1-1/+3
On Thu, Jan 19, 2017 at 03:03:46PM +0100, Ulrich Spörlein wrote: > > I suspect the patch below may fix things for you. It works around it by > > walking over the lru list (either is fine, as they both contain all > > entries, and since we're clearing everything, we don't care about the > > order). > > Confirmed. With the patch applied, I can import the whole 55G in one go > without any crashes or aborts. Thanks much! Thanks. Here it is rolled up with a commit message. -- >8 -- Subject: clear_delta_base_cache(): don't modify hashmap while iterating Removing entries while iterating causes fast-import to access an already-freed `struct packed_git`, leading to various confusing errors. What happens is that clear_delta_base_cache() drops the whole contents of the cache by iterating over the hashmap, calling release_delta_base_cache() on each entry. That function removes the item from the hashmap. The hashmap code may then shrink the table, but the hashmap_iter struct retains an offset from the old table. As a result, the next call to hashmap_iter_next() may claim that the iteration is done, even though some items haven't been visited. The only caller of clear_delta_base_cache() is fast-import, which wants to clear the cache because it is discarding the packed_git struct for its temporary pack. So by failing to remove all of the entries, we still have references to the freed packed_git. To make things even more confusing, this doesn't seem to trigger with the test suite, because it depends on complexities like the size of the hash table, which entries got cleared, whether we try to access them before they're evicted from the cache, etc. So I've been able to identify the problem with large imports like freebsd's svn import, or a fast-export of linux.git. But nothing that would be reasonable to run as part of the normal test suite. We can fix this easily by iterating over the lru linked list instead of the hashmap. They both contain the same entries, and we can use the "safe" variant of the list iterator, which exists for exactly this case. Let's also add a warning to the hashmap API documentation to reduce the chances of getting bit by this again. Reported-by: Ulrich Spörlein <uqs@freebsd.org> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-01-18Merge branch 'bw/pathspec-cleanup'Libravatar Junio C Hamano1-2/+0
Code clean-up in the pathspec API. * bw/pathspec-cleanup: pathspec: rename prefix_pathspec to init_pathspec_item pathspec: small readability changes pathspec: create strip submodule slash helpers pathspec: create parse_element_magic helper pathspec: create parse_long_magic function pathspec: create parse_short_magic function pathspec: factor global magic into its own function pathspec: simpler logic to prefix original pathspec elements pathspec: always show mnemonic and name in unsupported_magic pathspec: remove unused variable from unsupported_magic pathspec: copy and free owned memory pathspec: remove the deprecated get_pathspec function ls-tree: convert show_recursive to use the pathspec struct interface dir: convert fill_directory to use the pathspec struct interface dir: remove struct path_simplify mv: remove use of deprecated 'get_pathspec()'
2017-01-08pathspec: remove the deprecated get_pathspec functionLibravatar Brandon Williams1-2/+0
Now that all callers of the old 'get_pathspec' interface have been migrated to use the new pathspec struct interface it can be removed from the codebase. Since there are no more users of the '_raw' field in the pathspec struct it can also be removed. This patch also removes the old functionality of modifying the const char **argv array that was passed into parse_pathspec. Instead the constructed 'match' string (which is a pathspec element with the prefix prepended) is only stored in its corresponding pathspec_item entry. Signed-off-by: Brandon Williams <bmwill@google.com> Reviewed-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-11-22submodule-config: clarify parsing of null_sha1 elementLibravatar Stefan Beller1-2/+5
Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com> Reviewed-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-11-22submodule-config: rename commit_sha1 to treeish_nameLibravatar Stefan Beller1-4/+5
It is also possible to pass in any treeish name to lookup a submodule config. Make it clear by naming the variables accordingly. Looking up a submodule config by tree hash will come in handy in a later patch. Signed-off-by: Stefan Beller <sbeller@google.com> Reviewed-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-10-10Merge branch 'nd/shallow-deepen'Libravatar Junio C Hamano2-1/+28
The existing "git fetch --depth=<n>" option was hard to use correctly when making the history of an existing shallow clone deeper. A new option, "--deepen=<n>", has been added to make this easier to use. "git clone" also learned "--shallow-since=<date>" and "--shallow-exclude=<tag>" options to make it easier to specify "I am interested only in the recent N months worth of history" and "Give me only the history since that version". * nd/shallow-deepen: (27 commits) fetch, upload-pack: --deepen=N extends shallow boundary by N commits upload-pack: add get_reachable_list() upload-pack: split check_unreachable() in two, prep for get_reachable_list() t5500, t5539: tests for shallow depth excluding a ref clone: define shallow clone boundary with --shallow-exclude fetch: define shallow boundary with --shallow-exclude upload-pack: support define shallow boundary by excluding revisions refs: add expand_ref() t5500, t5539: tests for shallow depth since a specific date clone: define shallow clone boundary based on time with --shallow-since fetch: define shallow boundary with --shallow-since upload-pack: add deepen-since to cut shallow repos based on time shallow.c: implement a generic shallow boundary finder based on rev-list fetch-pack: use a separate flag for fetch in deepening mode fetch-pack.c: mark strings for translating fetch-pack: use a common function for verbose printing fetch-pack: use skip_prefix() instead of starts_with() upload-pack: move rev-list code out of check_non_tip() upload-pack: make check_non_tip() clean things up on error upload-pack: tighten number parsing at "deepen" lines ...
2016-09-26sha1_array: let callbacks interrupt iterationLibravatar Jeff King1-2/+6
The callbacks for iterating a sha1_array must have a void return. This is unlike our usual for_each semantics, where a callback may interrupt iteration and have its value propagated. Let's switch it to the usual form, which will enable its use in more places (e.g., where we are replacing an existing iteration with a different data structure). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>