summaryrefslogtreecommitdiff
path: root/builtin
AgeCommit message (Collapse)AuthorFilesLines
2016-08-03Merge branch 'jk/push-progress'Libravatar Junio C Hamano5-17/+128
"git push" and "git clone" learned to give better progress meters to the end user who is waiting on the terminal. * jk/push-progress: receive-pack: send keepalives during quiet periods receive-pack: turn on connectivity progress receive-pack: relay connectivity errors to sideband receive-pack: turn on index-pack resolving progress index-pack: add flag for showing delta-resolution progress clone: use a real progress meter for connectivity check check_connected: add progress flag check_connected: relay errors to alternate descriptor check_everything_connected: use a struct with named options check_everything_connected: convert to argv_array rev-list: add optional progress reporting check_everything_connected: always pass --quiet to rev-list
2016-08-03Merge branch 'jk/parse-options-concat'Libravatar Junio C Hamano1-9/+4
Users of the parse_options_concat() API function need to allocate extra slots in advance and fill them with OPT_END() when they want to decide the set of supported options dynamically, which makes the code error-prone and hard to read. This has been corrected by tweaking the API to allocate and return a new copy of "struct option" array. * jk/parse-options-concat: parse_options: allocate a new array when concatenating
2016-08-03Merge branch 'sb/push-options'Libravatar Junio C Hamano2-16/+82
"git push" learned to accept and pass extra options to the receiving end so that hooks can read and react to them. * sb/push-options: add a test for push options push: accept push options receive-pack: implement advertising and receiving push options push options: {pre,post}-receive hook learns about push options
2016-08-03Merge branch 'os/no-verify-skips-commit-msg-too'Libravatar Junio C Hamano1-1/+1
"git commit --help" said "--no-verify" is only about skipping the pre-commit hook, and failed to say that it also skipped the commit-msg hook. * os/no-verify-skips-commit-msg-too: commit: describe that --no-verify skips the commit-msg hook in the help text
2016-07-28Merge branch 'nd/pack-ofs-4gb-limit'Libravatar Junio C Hamano4-21/+27
"git pack-objects" and "git index-pack" mostly operate with off_t when talking about the offset of objects in a packfile, but there were a handful of places that used "unsigned long" to hold that value, leading to an unintended truncation. * nd/pack-ofs-4gb-limit: fsck: use streaming interface for large blobs in pack pack-objects: do not truncate result in-pack object size on 32-bit systems index-pack: correct "offset" type in unpack_entry_data() index-pack: report correct bad object offsets even if they are large index-pack: correct "len" type in unpack_data() sha1_file.c: use type off_t* for object_info->disk_sizep pack-objects: pass length to check_pack_crc() without truncation
2016-07-28Merge branch 'nd/worktree-lock'Libravatar Junio C Hamano1-0/+66
"git worktree prune" protected worktrees that are marked as "locked" by creating a file in a known location. "git worktree" command learned a dedicated command pair to create and remove such a file, so that the users do not have to do this with editor. * nd/worktree-lock: worktree.c: find_worktree() search by path suffix worktree: add "unlock" command worktree: add "lock" command worktree.c: add is_worktree_locked() worktree.c: add is_main_worktree() worktree.c: add find_worktree()
2016-07-26commit: describe that --no-verify skips the commit-msg hook in the help textLibravatar Orgad Shaneh1-1/+1
This brings the short help in line with the documentation. Signed-off-by: Orgad Shaneh <orgads@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-25Merge branch 'mh/blame-worktree'Libravatar Junio C Hamano1-1/+9
"git blame file" allowed the lineage of lines in the uncommitted, unadded contents of "file" to be inspected, but it refused when "file" did not appear in the current commit. When "file" was created by renaming an existing file (but the change has not been committed), this restriction was unnecessarily tight. * mh/blame-worktree: t/t8003-blame-corner-cases.sh: Use here documents blame: allow to blame paths freshly added to the index
2016-07-25Merge branch 'js/fsck-name-object'Libravatar Junio C Hamano1-18/+62
When "git fsck" reports a broken link (e.g. a tree object contains a blob that does not exist), both containing object and the object that is referred to were reported with their 40-hex object names. The command learned the "--name-objects" option to show the path to the containing object from existing refs (e.g. "HEAD~24^2:file.txt"). * js/fsck-name-object: fsck: optionally show more helpful info for broken links fsck: give the error function a chance to see the fsck_options fsck_walk(): optionally name objects on the go fsck: refactor how to describe objects
2016-07-25Merge branch 'rs/rm-strbuf-optim'Libravatar Junio C Hamano1-2/+3
The use of strbuf in "git rm" to build filename to remove was a bit suboptimal, which has been fixed. * rs/rm-strbuf-optim: rm: reuse strbuf for all remove_dir_recursively() calls
2016-07-25Merge branch 'mh/ref-iterators'Libravatar Junio C Hamano2-7/+3
The API to iterate over all the refs (i.e. for_each_ref(), etc.) has been revamped. * mh/ref-iterators: for_each_reflog(): reimplement using iterators dir_iterator: new API for iterating over a directory tree for_each_reflog(): don't abort for bad references do_for_each_ref(): reimplement using reference iteration refs: introduce an iterator interface ref_resolves_to_object(): new function entry_resolves_to_object(): rename function from ref_resolves_to_object() get_ref_cache(): only create an instance if there is a submodule remote rm: handle symbolic refs correctly delete_refs(): add a flags argument refs: use name "prefix" consistently do_for_each_ref(): move docstring to the header file refs: remove unnecessary "extern" keywords
2016-07-25Merge branch 'mh/split-under-lock'Libravatar Junio C Hamano1-8/+11
Further preparatory work on the refs API before the pluggable backend series can land. * mh/split-under-lock: (33 commits) lock_ref_sha1_basic(): only handle REF_NODEREF mode commit_ref_update(): remove the flags parameter lock_ref_for_update(): don't resolve symrefs lock_ref_for_update(): don't re-read non-symbolic references refs: resolve symbolic refs first ref_transaction_update(): check refname_is_safe() at a minimum unlock_ref(): move definition higher in the file lock_ref_for_update(): new function add_update(): initialize the whole ref_update verify_refname_available(): adjust constness in declaration refs: don't dereference on rename refs: allow log-only updates delete_branches(): use resolve_refdup() ref_transaction_commit(): correctly report close_ref() failure ref_transaction_create(): disallow recursive pruning refs: make error messages more consistent lock_ref_sha1_basic(): remove unneeded local variable read_raw_ref(): move docstring to header file read_raw_ref(): improve docstring read_raw_ref(): rename symref argument to referent ...
2016-07-20receive-pack: send keepalives during quiet periodsLibravatar Jeff King2-1/+72
After a client has sent us the complete pack, we may spend some time processing the data and running hooks. If the client asked us to be quiet, receive-pack won't send any progress data during the index-pack or connectivity-check steps. And hooks may or may not produce their own progress output. In these cases, the network connection is totally silent from both ends. Git itself doesn't care about this (it will wait forever), but other parts of the system (e.g., firewalls, load-balancers, etc) might hang up the connection. So we'd like to send some sort of keepalive to let the network and the client side know that we're still alive and processing. We can use the same trick we did in 05e9515 (upload-pack: send keepalive packets during pack computation, 2013-09-08). Namely, we will send an empty sideband data packet every `N` seconds that we do not relay any stderr data over the sideband channel. As with 05e9515, this means that we won't bother sending keepalives when there's actual progress data, but will kick in when it has been disabled (or if there is a lull in the progress data). The concept is simple, but the details are subtle enough that they need discussing here. Before the client sends us the pack, we don't want to do any keepalives. We'll have sent our ref advertisement, and we're waiting for them to send us the pack (and tell us that they support sidebands at all). While we're receiving the pack from the client (or waiting for it to start), there's no need for keepalives; it's up to them to keep the connection active by sending data. Moreover, it would be wrong for us to do so. When we are the server in the smart-http protocol, we must treat our connection as half-duplex. So any keepalives we send while receiving the pack would potentially be buffered by the webserver. Not only does this make them useless (since they would not be delivered in a timely manner), but it could actually cause a deadlock if we fill up the buffer with keepalives. (It wouldn't be wrong to send keepalives in this phase for a full-duplex connection like ssh; it's simply pointless, as it is the client's responsibility to speak). As soon as we've gotten all of the pack data, then the client is waiting for us to speak, and we should start keepalives immediately. From here until the end of the connection, we send one any time we are not otherwise sending data. But there's a catch. Receive-pack doesn't know the moment we've gotten all the data. It passes the descriptor to index-pack, who reads all of the data, and then starts resolving the deltas. We have to communicate that back. To make this work, we instruct the sideband muxer to enable keepalives in three phases: 1. In the beginning, not at all. 2. While reading from index-pack, wait for a signal indicating end-of-input, and then start them. 3. Afterwards, always. The signal from index-pack in phase 2 has to come over the stderr channel which the muxer is reading. We can't use an extra pipe because the portable run-command interface only gives us stderr and stdout. Stdout is already used to pass the .keep filename back to receive-pack. We could also send a signal there, but then we would find out about it in the main thread. And the keepalive needs to be done by the async muxer thread (since it's the one writing sideband data back to the client). And we can't reliably signal the async thread from the main thread, because the async code sometimes uses threads and sometimes uses forked processes. Therefore the signal must come over the stderr channel, where it may be interspersed with other random human-readable messages from index-pack. This patch makes the signal a single NUL byte. This is easy to parse, should not appear in any normal stderr output, and we don't have to worry about any timing issues (like seeing half the signal bytes in one read(), and half in a subsequent one). This is a bit ugly, but it's simple to code and should work reliably. Another option would be to stop using an async thread for muxing entirely, and just poll() both stderr and stdout of index-pack from the main thread. This would work for index-pack (because we aren't doing anything useful in the main thread while it runs anyway). But it would make the connectivity check and the hook muxers much more complicated, as they need to simultaneously feed the sub-programs while reading their stderr. The index-pack phase is the only one that needs this signaling, so it could simply behave differently than the other two. That would mean having two separate implementations of copy_to_sideband (and the keepalive code), though. And it still doesn't get rid of the signaling; it just means we can write a nicer message like "END_OF_INPUT" or something on stdout, since we don't have to worry about separating it from the stderr cruft. One final note: this signaling trick is only done with index-pack, not with unpack-objects. There's no point in doing it for the latter, because by definition it only kicks in for a small number of objects, where keepalives are not as useful (and this conveniently lets us avoid duplicating the implementation). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-20receive-pack: turn on connectivity progressLibravatar Jeff King1-0/+1
When we receive a large push, the server side of the connection may spend a lot of time (30s or more for a full push of linux.git) walking the object graph without producing any output. Let's give the user some indication that we're actually working. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-20receive-pack: relay connectivity errors to sidebandLibravatar Jeff King1-1/+17
If the connectivity check encounters a problem when receiving a push, the error output goes to receive-pack's stderr, whose destination depends on the protocol used (ssh tends to send it to the user, though without a "remote" prefix; http will generally eat it in the server's error log). The information should consistently go back to the user, as there is a reasonable chance their client is buggy and generating a bad pack. We can do so by muxing it over the sideband as we do with other sub-process stderr. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-20receive-pack: turn on index-pack resolving progressLibravatar Jeff King1-0/+2
When we receive a large push, the server side may have to spend a lot of CPU processing the incoming packfile. During the "receiving" phase, we are typically network bound, and the client is writing its own progress to the user. But during the delta resolution phase, we may spend minutes (e.g., for a full push of linux.git) without making any indication to the user that the connection has not hung. Let's ask index-pack to produce progress output for this phase (unless the client asked us to be quiet, of course). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-20index-pack: add flag for showing delta-resolution progressLibravatar Jeff King1-1/+4
The index-pack command has two progress meters: one for "receiving objects", and one for "resolving deltas". You get neither by default, or both with "-v". But for a push through receive-pack, we would want only the "resolving deltas" phase, _not_ the "receiving objects" progress. There are two reasons for this. One is simply that existing clients are already printing "writing objects" progress at the same time. Arguably "receiving" from the far end is more useful, because it tells you what has actually gotten there, as opposed to what might be stuck in a buffer somewhere between the client and server. But that would require a protocol extension to tell clients not to print their progress. Possible, but complexity for little gain. The second reason is much more important. In a full-duplex connection like git-over-ssh, we can print progress while the pack is incoming, and it will immediately get to the client. But for a half-duplex connection like git-over-http, we should not say anything until we have received the full request. Anything we write is subject to being stuck in a buffer by the webserver. Worse, we can end up in a deadlock if that buffer fills up. So our best bet is to avoid writing anything that isn't a small fixed size until we've received the full pack. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-20clone: use a real progress meter for connectivity checkLibravatar Jeff King1-4/+1
Because the initial connectivity check for a cloned repository can be slow, 0781aa4 (clone: let the user know when check_everything_connected is run, 2013-05-03) added a "fake" progress meter; we simply say "Checking connectivity" when it starts, and "done" at the end, with nothing between. Since check_connected() now knows how to do a real progress meter, we can drop our fake one and use that one instead. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-20check_everything_connected: use a struct with named optionsLibravatar Jeff King3-11/+15
The number of variants of check_everything_connected has grown over the years, so that the "real" function takes several possibly-zero, possibly-NULL arguments. We hid the complexity behind some wrapper functions, but this doesn't scale well when we want to add new options. If we add more wrapper variants to handle the new options, then we can get a combinatorial explosion when those options might be used together (right now nobody wants to use both "shallow" and "transport" together, so we get by with just a few wrappers). If instead we add new parameters to each function, each of which can have a default value, then callers who want the defaults end up with confusing invocations like: check_everything_connected(fn, 0, data, -1, 0, NULL); where it is unclear which parameter is which (and every caller needs updated when we add new options). Instead, let's add a struct to hold all of the optional parameters. This is a little more verbose for the callers (who have to declare the struct and fill it in), but it makes their code much easier to follow, because every option is named as it is set (and unused options do not have to be mentioned at all). Note that we could also stick the iteration function and its callback data into the option struct, too. But since those are required for each call, by avoiding doing so, we can let very simple callers just pass "NULL" for the options and not worry about the struct at all. While we're touching each site, let's also rename the function to check_connected(). The existing name was quite long, and not all of the wrappers even used the full name. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-20rev-list: add optional progress reportingLibravatar Jeff King1-0/+17
It's easy to ask rev-list to do a traversal that may takes many seconds (e.g., by calling "--objects --all"). In theory you can monitor its progress by the output you get to stdout, but this isn't always easy. Some operations, like "--count", don't make any output until the end. And some callers, like check_everything_connected(), are using it just for the error-checking of the traversal, and throw away stdout entirely. This patch adds a "--progress" option which can be used to give some eye-candy for a user waiting for a long traversal. This is just a rev-list option and not a regular traversal option, because it needs cooperation from the callbacks in builtin/rev-list.c to do the actual count. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-19Merge branch 'js/am-call-theirs-theirs-in-fallback-3way'Libravatar Junio C Hamano1-8/+8
One part of "git am" had an oddball helper function that called stuff from outside "his" as opposed to calling what we have "ours", which was not gender-neutral and also inconsistent with the rest of the system where outside stuff is usuall called "theirs" in contrast to "ours". * js/am-call-theirs-theirs-in-fallback-3way: am: counteract gender bias
2016-07-19Merge branch 'jk/write-file'Libravatar Junio C Hamano4-60/+17
General code clean-up around a helper function to write a single-liner to a file. * jk/write-file: branch: use write_file_buf instead of write_file use write_file_buf where applicable write_file: add format attribute write_file: add pointer+len variant write_file: use xopen write_file: drop "gently" form branch: use non-gentle write_file for branch description am: ignore return value of write_file() config: fix bogus fd check when setting up default config
2016-07-19Merge branch 'jk/printf-format'Libravatar Junio C Hamano1-1/+1
Code clean-up to avoid using a variable string that compilers may feel untrustable as printf-style format given to write_file() helper function. * jk/printf-format: commit.c: remove print_commit_list() avoid using sha1_to_hex output as printf format walker: let walker_say take arbitrary formats
2016-07-19Merge branch 'nd/fetch-ref-summary'Libravatar Junio C Hamano1-40/+157
Improve the look of the way "git fetch" reports what happened to each ref that was fetched. * nd/fetch-ref-summary: fetch: reduce duplicate in ref update status lines with placeholder fetch: align all "remote -> local" output fetch: change flag code for displaying tag update and deleted ref fetch: refactor ref update status formatting code git-fetch.txt: document fetch output
2016-07-19Merge branch 'bc/cocci'Libravatar Junio C Hamano6-30/+29
Conversion from unsigned char sha1[20] to struct object_id continues. * bc/cocci: diff: convert prep_temp_blob() to struct object_id merge-recursive: convert merge_recursive_generic() to object_id merge-recursive: convert leaf functions to use struct object_id merge-recursive: convert struct merge_file_info to object_id merge-recursive: convert struct stage_data to use object_id diff: rename struct diff_filespec's sha1_valid member diff: convert struct diff_filespec to struct object_id coccinelle: apply object_id Coccinelle transformations coccinelle: convert hashcpy() with null_sha1 to hashclr() contrib/coccinelle: add basic Coccinelle transforms hex: add oid_to_hex_r()
2016-07-19Merge branch 'js/log-to-diffopt-file'Libravatar Junio C Hamano2-44/+58
The commands in the "log/diff" family have had an FILE* pointer in the data structure they pass around for a long time, but some codepaths used to always write to the standard output. As a preparatory step to make "git format-patch" available to the internal callers, these codepaths have been updated to consistently write into that FILE* instead. * js/log-to-diffopt-file: mingw: fix the shortlog --output=<file> test diff: do not color output when --color=auto and --output=<file> is given t4211: ensure that log respects --output=<file> shortlog: respect the --output=<file> setting format-patch: use stdout directly format-patch: avoid freopen() format-patch: explicitly switch off color when writing to files shortlog: support outputting to streams other than stdout graph: respect the diffopt.file setting line-log: respect diffopt's configured output file stream log-tree: respect diffopt's configured output file stream log: prepare log/log-tree to reuse the diffopt.close_file attribute
2016-07-19Merge branch 'dk/blame-move-no-reason-for-1-line-context'Libravatar Junio C Hamano1-4/+3
"git blame -M" missed a single line that was moved within the file. * dk/blame-move-no-reason-for-1-line-context: blame: require 0 context lines while finding moved lines with -M
2016-07-18fsck: optionally show more helpful info for broken linksLibravatar Johannes Schindelin1-4/+38
When reporting broken links between commits/trees/blobs, it would be quite helpful at times if the user would be told how the object is supposed to be reachable. With the new --name-objects option, git-fsck will try to do exactly that: name the objects in a way that shows how they are reachable. For example, when some reflog got corrupted and a blob is missing that should not be, the user might want to remove the corresponding reflog entry. This option helps them find that entry: `git fsck` will now report something like this: broken link from tree b5eb6ff... (refs/stash@{<date>}~37:) to blob ec5cf80... Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-18blame: allow to blame paths freshly added to the indexLibravatar Mike Hommey1-1/+9
When blaming files, changes in the work tree are taken into account and displayed as being "Not Committed Yet". However, when blaming a file that is not known to the current HEAD, git blame fails with `no such path 'foo' in HEAD`, even when the file was git add'ed. Allowing such a blame is useful when the new file added to the index (not yet committed) was created by renaming an existing file. It also is useful when the new file was created from pieces already in HEAD, moved or copied from other files and blaming with copy detection (i.e. "-C"). Signed-off-by: Mike Hommey <mh@glandium.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-18fsck: give the error function a chance to see the fsck_optionsLibravatar Johannes Schindelin1-1/+2
We will need this in the next commit, where fsck will be taught to optionally name the objects when reporting issues about them. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-18fsck: refactor how to describe objectsLibravatar Johannes Schindelin1-14/+23
In many places, we refer to objects via their SHA-1s. Let's abstract that into a function. For the moment, it does nothing else than what we did previously: print out the 40-digit hex string. But that will change over the course of the next patches. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-14push: accept push optionsLibravatar Stefan Beller1-3/+18
This implements everything that is required on the client side to make use of push options from the porcelain push command. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-14receive-pack: implement advertising and receiving push optionsLibravatar Stefan Beller1-0/+30
The pre/post receive hook may be interested in more information from the user. This information can be transmitted when both client and server support the "push-options" capability, which when used is a phase directly after update commands ended by a flush pkt. Similar to the atomic option, the server capability can be disabled via the `receive.advertisePushOptions` config variable. While documenting this, fix a nit in the `receive.advertiseAtomic` wording. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-14push options: {pre,post}-receive hook learns about push optionsLibravatar Stefan Beller1-13/+34
The environment variable GIT_PUSH_OPTION_COUNT is set to the number of push options sent, and GIT_PUSH_OPTION_{0,1,..} is set to the transmitted option. The code is not executed as the push options are set to NULL, nor is the new capability advertised. There was some discussion back and forth how to present these push options to the user as there are some ways to do it: Keep all options in one environment variable ============================================ + easiest way to implement in Git - This would make things hard to parse correctly in the hook. Put the options in files instead, filenames are in GIT_PUSH_OPTION_FILES ====================================== + After a discussion about environment variables and shells, we may not want to put user data into an environment variable (see [1] for example). + We could transmit binaries, i.e. we're not bound to C strings as we are when using environment variables to the user. + Maybe easier to parse than constructing environment variable names GIT_PUSH_OPTION_{0,1,..} yourself - cleanup of the temporary files is hard to do reliably - we have race conditions with multiple clients pushing, hence we'd need to use mkstemp. That's not too bad, but still. Use environment variables, but restrict to key/value pairs ========================================================== (When the user pushes a push option `foo=bar`, we'd GIT_PUSH_OPTION_foo=bar) + very easy to parse for a simple model of push options - it's not sufficient for more elaborate models, e.g. it doesn't allow doubles (e.g. cc=reviewer@email) Present the options in different environment variables ====================================================== (This is implemented) * harder to parse as a user, but we have a sample hook for that. - doesn't allow binary files + allows the same option twice, i.e. is not restrictive about options, except for binary files. + doesn't clutter a remote directory with (possibly stale) temporary files As we first want to focus on getting simple strings to work reliably, we go with the last option for now. If we want to do transmission of binaries later, we can just attach a 'side-channel', e.g. "any push option that contains a '\0' is put into a file instead of the environment variable and we'd have new GIT_PUSH_OPTION_FILES, GIT_PUSH_OPTION_FILENAME_{0,1,..} environment variables". [1] 'Shellshock' https://lwn.net/Articles/614218/ Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-13Merge branch 'nd/ita-cleanup'Libravatar Junio C Hamano1-2/+2
Git does not know what the contents in the index should be for a path added with "git add -N" yet, so "git grep --cached" should not show hits (or show lack of hits, with -L) in such a path, but that logic does not apply to "git grep", i.e. searching in the working tree files. But we did so by mistake, which has been corrected. * nd/ita-cleanup: grep: fix grepping for "intent to add" files t7810-grep.sh: fix a whitespace inconsistency t7810-grep.sh: fix duplicated test name
2016-07-13Merge branch 'ew/gc-auto-pack-limit-fix'Libravatar Junio C Hamano1-1/+1
"gc.autoPackLimit" when set to 1 should not trigger a repacking when there is only one pack, but the code counted poorly and did so. * ew/gc-auto-pack-limit-fix: gc: fix off-by-one error with gc.autoPackLimit
2016-07-13Merge branch 'va/i18n-even-more'Libravatar Junio C Hamano9-46/+48
More markings of messages for i18n, with updates to various tests to pass GETTEXT_POISON tests. One patch from the original submission dropped due to conflicts with jk/upload-pack-hook, which is still in flux. * va/i18n-even-more: (38 commits) t5541: become resilient to GETTEXT_POISON i18n: branch: mark comment when editing branch description for translation i18n: unmark die messages for translation i18n: submodule: escape shell variables inside eval_gettext i18n: submodule: join strings marked for translation i18n: init-db: join message pieces i18n: remote: allow translations to reorder message i18n: remote: mark URL fallback text for translation i18n: standardise messages i18n: sequencer: add period to error message i18n: merge: change command option help to lowercase i18n: merge: mark messages for translation i18n: notes: mark options for translation i18n: notes: mark strings for translation i18n: transport-helper.c: change N_() call to _() i18n: bisect: mark strings for translation t5523: use test_i18ngrep for negation t4153: fix negated test_i18ngrep call t9003: become resilient to GETTEXT_POISON tests: unpack-trees: update to use test_i18n* functions ...
2016-07-13fsck: use streaming interface for large blobs in packLibravatar Nguyễn Thái Ngọc Duy1-0/+4
For blobs, we want to make sure the on-disk data is not corrupted (i.e. can be inflated and produce the expected SHA-1). Blob content is opaque, there's nothing else inside to check for. For really large blobs, we may want to avoid unpacking the entire blob in memory, just to check whether it produces the same SHA-1. On 32-bit systems, we may not have enough virtual address space for such memory allocation. And even on 64-bit where it's not a problem, allocating a lot more memory could result in kicking other parts of systems to swap file, generating lots of I/O and slowing everything down. For this particular operation, not unpacking the blob and letting check_sha1_signature, which supports streaming interface, do the job is sufficient. check_sha1_signature() is not shown in the diff, unfortunately. But if will be called when "data_valid && !data" is false. We will call the callback function "fn" with NULL as "data". The only callback of this function is fsck_obj_buffer(), which does not touch "data" at all if it's a blob. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-13pack-objects: do not truncate result in-pack object size on 32-bit systemsLibravatar Nguyễn Thái Ngọc Duy1-7/+8
A typical diff will not show what's going on and you need to see full functions. The core code is like this, at the end of of write_one() e->idx.offset = *offset; size = write_object(f, e, *offset); if (!size) { e->idx.offset = recursing; return WRITE_ONE_BREAK; } written_list[nr_written++] = &e->idx; /* make sure off_t is sufficiently large not to wrap */ if (signed_add_overflows(*offset, size)) die("pack too large for current definition of off_t"); *offset += size; Here we can see that the in-pack object size is returned by write_object (or indirectly by write_reuse_object). And it's used to calculate object offsets, which end up in the pack index file, generated at the end. If "size" overflows (on 32-bit sytems, unsigned long is 32-bit while off_t can be 64-bit), we got wrong offsets and produce incorrect .idx file, which may make it look like the .pack file is corrupted. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-13index-pack: correct "offset" type in unpack_entry_data()Libravatar Nguyễn Thái Ngọc Duy1-1/+1
unpack_entry_data() receives an off_t value from unpack_raw_entry(), which could be larger than unsigned long on 32-bit systems with large file support. Correct the type so truncation does not happen. This only affects bad object reporting though. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-13index-pack: report correct bad object offsets even if they are largeLibravatar Nguyễn Thái Ngọc Duy1-3/+4
Use the right type for offsets in this case, off_t, which makes a difference on 32-bit systems with large file support, and change formatting code accordingly. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-13index-pack: correct "len" type in unpack_data()Libravatar Nguyễn Thái Ngọc Duy1-7/+7
On 32-bit systems with large file support, one entry could be larger than 4GB and overflow "len". Correct it so we can unpack a full entry. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-13sha1_file.c: use type off_t* for object_info->disk_sizepLibravatar Nguyễn Thái Ngọc Duy1-2/+2
This field, filled by sha1_object_info() contains the on-disk size of an object, which could go over 4GB limit of unsigned long on 32-bit systems. Use off_t for it instead and update all callers. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-12rm: reuse strbuf for all remove_dir_recursively() callsLibravatar René Scharfe1-2/+3
Don't throw the memory allocated for remove_dir_recursively() away after a single call, use it for the other entries as well instead. Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-12pack-objects: pass length to check_pack_crc() without truncationLibravatar Nguyễn Thái Ngọc Duy1-1/+1
On 32 bit systems with large file support, unsigned long is 32-bit while the two offsets in the subtraction expression (pack-objects has the exact same expression as in sha1_file.c but not shown in diff) are in 64-bit. If an in-pack object is larger than 2^32 len/datalen is truncated and we get a misleading "error: bad packed object CRC for ..." as a result. Use off_t for len and datalen. check_pack_crc() already accepts this argument as off_t and can deal with 4+ GB. Noticed-by: Christoph Michelbach <michelbach94@gmail.com> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-11Merge branch 'mj/log-show-signature-conf'Libravatar Junio C Hamano1-0/+6
"git log" learns log.showSignature configuration variable, and a command line option "--no-show-signature" to countermand it. * mj/log-show-signature-conf: log: add log.showSignature configuration variable log: add "--no-show-signature" command line option t4202: refactor test
2016-07-11Merge branch 'js/find-commit-subject-ignore-leading-blanks'Libravatar Junio C Hamano2-2/+2
A helper function that takes the contents of a commit object and finds its subject line did not ignore leading blank lines, as is commonly done by other codepaths. Make it ignore leading blank lines to match. * js/find-commit-subject-ignore-leading-blanks: reset --hard: skip blank lines when reporting the commit subject sequencer: use skip_blank_lines() to find the commit subject commit -C: skip blank lines at the beginning of the message commit.c: make find_commit_subject() more robust pretty: make the skip_blank_lines() function public
2016-07-11Merge branch 'sb/submodule-clone-retry'Libravatar Junio C Hamano1-7/+59
"git submodule update" that drives many "git clone" could eventually hit flaky servers/network conditions on one of the submodules; the command learned to retry the attempt. * sb/submodule-clone-retry: submodule update: continue when a clone fails submodule--helper: initial clone learns retry logic
2016-07-08worktree: add "unlock" commandLibravatar Nguyễn Thái Ngọc Duy1-0/+28
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-08worktree: add "lock" commandLibravatar Nguyễn Thái Ngọc Duy1-0/+38
Helped-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>